The optimization design for a complex nonlinear system is difficult to be achieved by using the traditional methods. This paper presents an optimization design method for nonlinear system by using an improved neural network. The proposed approach is a combination of error back-propagation neural network (BPNN), principal component analysis (PCA) and genetic algorithms (GAs). PCA is employed to reduce the dimension and de-noise for the learning matrix of BPNN model. A combination of GAs and the BPNN model is used to find the most appropriate linking weight with its global search feature. As a demonstration example, the optimization method for selecting the material and dimension parameters for a QFN package is explored. Firstly, in order to search for the optimal parameter combination, the well-trained network model, which includes a nonlinear function of the input parameters and corresponding outputs, is considered as an observation tool to select optimal parameter size as to reduce the J-integral value of interface cracking in the packaging device. Secondly, the optimal parameter combinations are selected for the device after verification. The results from the optimization design show that the well-trained PCA-GA-BPNN model using the proposed approach can be used in the optimization design of the microelectronics packaging device to reduce the interface delamination problems.