The effect of the synaptic plasticity on the dynamics of a large neural network is studied. Our approach is analytic but inspired by the data, both simulated and experimental. We explain formation of the small strongly connected assemblies within a dynamical network following Hebb's rule. Also, we find the conditions for the synchrony effect in the stochastic network in the absence of large synchronized input.