We consider in this paper that the use of personal attributes can improve robustness and reliability of the facial authentication. We describe a new technique for authentication system which based on the integration of a human face-based authentication. Face recognition with variant illumination and expression is a challenging problem. An Improved Census Transform (ICT) takes 3×3 nonlinear filter window into use to transform the original facial images into pattern images. The ICT deals well with illumination variation. A chaos neural network based facial expression recognition algorithm was proposed with the detailed pattern image chaos modulation method and training method. The algorithm was tested on PC(P4, 1.8G,256M DDR), as to 50×50 pixels single facial image, the recognition time was less than 20 ms and the correct percent is 91% more. We integrated the above existing approaches and developed the authentication system based on facial expression recognition.