In this paper, the existence and uniqueness of the equilibrium point and stability of the cellular neural networks (CNNs) with time-varying delays are analyzed and proved. Several global exponential stability conditions of the neural networks are obtained by the delay differential inequality and matrix measures approach. The obtained results are extensions of the earlier literature. The approach used in this paper is also suitable for delayed Hopfield neural networks and delayed bi-directional associative memory neural networks whose activation functions are often nondifferentiable or unbounded. Two simulation examples in comparison to previous results in literature are shown to check the theory in this paper.