In this paper, a robust traffic sign recognition system is introduced for driver assistance applications and/or autonomous cars. The system incorporates two major operations, traffic sign detection and classification. The sign detection is based on color segmentation and incorporates hue detection, morphological filter and labeling. A nearest neighbor classifier is introduced for sign classification. The training features are extracted by SURF algorithm. Three feature extraction strategies are compared to find an optimal feature database for training. The proposed system benefits from the SURF algorithm, which achieves invariance to the rotated, skewed and occluded signs. Extensive experimental results show detection accuracy reaching up to 97.54%.