This paper presents an object-based fusion of hyperspectral data with LIDAR data for efficient classification of urban areas. Image segmentation is performed on the features extracted from hyperspectral data at multiple levels in a hierarchical way for utilizing spatial information at various scales. Additional information on the classes is derived from the LIDAR data to aid in the classification process. While fusing hyperspectral data with the LIDAR data, the proposed approach is resilient to small misregistration of images as we only consider the information with maximum overlap based on the median operator. The final classification is performed using the random forest classifier.