International statistics shows that large numbers of road accidents are caused due to driver fatigue. In this paper the authors tried to detect the fatigue of a person who is working under some stressful conditions like Driving. The principle of the proposed system is based on the facial image analysis for warning the driver of fatigue to prevent traffic accidents. The facial images of the driver are taken by CCD camera which is installed on the dashboard in front of the driver. After detecting the drivers' face, features like eyes and mouth are extracted and their states are estimated. A fuzzy logic algorithm and an inference are proposed to determine the level of fatigue and warn the driver accordingly. Various visual cues typically characterizing the level of alertness of a person are extracted and systematically combined to infer the fatigue level. The experimental results indicated that the proposed expert system is effective in increasing safety while driving.