The automatic recognition of hidden defects plays a key role in the structure integrity and healthy monitoring (SIHM) system. The article proposes a novel method that combines the Hilbert Huang transform (HHT) and hybrid support vector machine-particle swarm optimization (Hybrid SVM-PSO) model for recognizing hidden defects during using pulsed eddy current (PEC) testing. The proposed approach uses the HHT algorithm to extract feature from PEC response signal to indicate various defects, and the first step of HHT, ensemble empirical mode decomposition (EEMD) method, is improved in order to obtain accurate intrinsic mode function components (IMFs). The defects are recognized by the proposed Hybrid SVM-PSO model, which is composed of classification module and optimization module. The optimization module made up of PSO method is used to configure the key parameters of classification module realized by SVM. A set of experimental data collected from a self-made rail specimen is used to validate the proposed approach. The results show that the proposed method based on the HHT and hybrid SVM-PSO model is beneficial to improve the automatic recognition of hidden defects.