Convolutional Neural Networks (CNNs) and its variants are increasingly used across wide domain of applications achieving high performance measures. For high performance, application specific CNN architecture is required, hence the need for network architecture search (NAS) becomes essential. This paper proposes a hybrid evolutionary approach for network architecture search (HyENAS), and targets convolution class of neural networks. One of the significant contribution of this technique is to completely evolve the high performance network by simultaneously finding network structures and their corresponding parameters. An elegant string representation has been proposed which efficiently represents the network. The concept of sparse block evolving requisite layer wise features for dense network is deployed. This permits the network to dynamically evolve for a specific application. In comparison to the other state‐of‐art methods, the high performance of the proposed HyENAS approach is demonstrated across various benchmark data sets belonging to the domain of malariology, oncology, neurology, ophthalmology, and genomics. Further, to deploy the proposed model on lower hardware specification devices, another salient feature of the HyENAS technique is to seamlessly sift out the simpler network architecture with comparable accuracy.