Development of smart cities has grasped much attention in research community and industry as well. Smart healthcare, communication, infrastructure are required for the development of smart cities. Security is one of the major concern in the development of smart cities. Automatic surveillance helps in boosting security in multiple areas like traffic, hospitals, schools, and industries etc. Video camera and Global Positioning System (GPS) based monitoring are one of the key parts of it. Filtering or classification of infrequent or anomalous activities in traffic data help to understand the flow of movements in monitoring area. Video based surveillance involves the extraction of object trajectories from videos and then analyzing them to spot unusual behavior of objects to secure area under surveillance. In this paper, we propose an efficient approach for the classification of object trajectories using the combination of Fuzzy C-Means (FCM) clustering technique and Support Vector Machine (SVM). The features extracted from FCM are then classified using SVM classifier. The approach has been tested on two publicly available datasets, namely, CROSS [12] and T11 [18]. Accuracies of 90.37% and 87.29% have been recorded on CROSS and T11 datasets, respectively. The combined approach outperforms the traditional SVM based classification on these datasets.