Pedestrian detection and counting is an important topic in developing an intelligent surveillance system. In this work, we propose a vision-based system for detecting pedestrians in an image. Be robust to crowded scenes and adapt to incomplete foreground from background subtraction algorithm, expectation maximization (EM) algorithm is applied to impose the constraint of body part for achieving successful detection. A well-known dataset called CAVIAR is used to validate the effectiveness of the proposed method.