This paper presents a Bayesian approach for human segmentation in infrared video sequences. To overcome the limitations of background modeling in dealing with pixel-wise processing, our background model is combined with clustering cue in a maximum a posterior probability (MAP)-MRF framework. This can not only enable us to exploit the spatial and temporal coherence to maintain the continuity of our segmentation, but also takes the interdependence of feature and segmentation field into consideration. Experimental results for several infrared video sequences are provided to demonstrate the effectiveness of the proposed approach.