In this study, we introduce and investigate a class of neural architectures of Polynomial Neural Networks (PNNs), discuss a comprehensive design methodology and carry out a series of numeric experiments. Two kinds of PNN architectures, namely a basic PNN and a modified PNN architecture are discussed. Each of them comes with two types such as the generic and the advanced type. The essence of the design procedure dwells on the Group Method of Data Handling. PNN is a flexible neural architecture whose structure is developed through learning. In particular, the number of layers of the PNN is not fixed in advance but becomes dynamically meaning that the network grows over the training period. In this sense, PNN is a self-organizing network. A comparative analysis shows that the proposed PNN are models with higher accuracy than other fuzzy models.