In this paper, we propose a Partial Least Squares based sub-window search method for pedestrian detection, by which the detection speed can be improved effectively while maintaining high detection accuracy. Firstly, a sparse search is implemented to find all the possible locations containing parts of a pedestrian. Then a pre-learned Partial Least Squares regression model is applied to estimate the displacements of the subwindows to guide them towards the approximate locations of the pedestrians. Finally, we conduct a dense search around the approximate locations to obtain the exact locations of the pedestrians. Experiments on the INRIA dataset demonstrate that our method greatly reduces the number of search windows, which leads to much fewer feature extraction in the detection phase. Thus, it is about 10 times faster than the sliding window method with a jump step of 8 × 8.