In this paper, we propose a positive face expression recognition method based on the image around the mouth. We also estimate the direction of the face of a person to examine whether or not he/she has interest toward this side. In the proposed method, estimation of the position and the direction of a face is realized by using particle filter and FAST operator. The image around the mouth is detected based on the estimated face position. The feature is calculated by Gabor Filter and the facial expression is recognized by support vector machine. In the recognition of positive expression like smile or laugh, we confirmed the effectiveness of the proposed method.