This paper describes the passenger management system based on the face recognition technology for intelligent transport vehicles. In the mass transport vehicles such as a public transport bus, a subway and a train, a lot of people get on and off at the same time, thus the speed of face detection is the most important issue for the development of intelligent vehicle system which can identify passengers. In this paper, the AdaBoost method utilizing Local Binary Pattern (LBP) histogram is employed for face detection system. And the cascade structure is also introduced in order to improve the detection speed.