One of the first steps in most facial expression and facial analysis systems is the localization of prominent facial feature points. In this paper we present a novel approach for facial feature point detection using Simplified Gabor Wavelets (SGW). The classifier is built in cascades, where each stage of the cascade is a Gentle-AdaBoost trained classifier. In addition, we suggest a confidence based weighted grouping of multi-detected feature points to enhance accuracy. We have trained and tested our algorithm with a shuffled mix of four available labeled databases with more than 700 individuals. Our experimental results achieve approximately 82% detection rate in average, which is a considerable result, since the databases contain not only frontal faces.