Facial expression plays a key role in non-verbal face-to-face communication. In this paper, we present a method based on feature difference matrix and QNNs to recognize facial expression from single static images. Firstly, we divide expression image into several expression feature blocks (eyebrows block, eyes block, mouth block) which contain more discriminant information for each facial expression. And then, feature difference matrix is obtained by subtracting respectively neutral expression block from above feature blocks. Finally, a QNNs (Quantum Neural Networks) classifier was used for expression classification from feature difference matrix. The proposed algorithm is tested in the Japanese female facial expression database. The experimental results show that our approach achieves excellent performance in terms of recognition rate and recognition reliability.