This paper presents a novel solution to the occlusion handling problem in pedestrian tracking using labeled random finite set theory. The occlusion handling module uses motion and color cues of tracked targets to recover target labels after occlusion. An effective algorithm is also proposed for false alarm detection and removal which is designed based on tracked targets features such as, overlap ratio, size similarity and the time of track initialization of the tracked targets. We implement our solution using sequential Monte Carlo method, and compare it with state-of-the-art visual tracking methods. The results show that the proposed algorithm perform favorably in terms of various standard performance metrics.