Owning the characteristics of concealment and complexity etc., it is a great challenge to quickly and accurately identify the faults for computer numerical control (CNC) machine. The conventional neural network based fault diagnosis algorithms are not able to deal with human knowledge, while the fuzzy system based fault diagnosis methods face the problems of poor self-learning, poor self-organization etc. In fact, the above two kinds of methods can make up for each other's shortcomings. Thus, this paper presents a new fault diagnosis method based on the combination of fuzzy logic and RBF neural network. Further, a modified particle swarm algorithm is proposed for to optimize the structure parameters of fuzzy neural network. Thus, the fault diagnosis model of CNC machine spindle servo system with the improved particle swarm optimized fuzzy neural network is established. The experiment results show that the proposed method has higher fault identification accuracy and stronger generalization ability, compared with the RBF neural network and the standard particle swarm optimized fuzzy neural network.