Background subtraction is a widely used method for moving object detection in computer vision. It is usually applied in video surveillance systems. There are two major kinds of background subtraction approaches: pixel-based and block based. Yet there are three problems that can not be simultaneously solved by either method: the robustness to illumination changes, the effectiveness in suppressing shadows, and the smoothness of foregroundpsilas boundary. In order to solve these problems, a pixel-wise local information-based background subtraction method is proposed in this paper. In the proposed method, Gabor filters are performed to extract the spatial feature vectors for each pixel from the source image sequence. Then, the spatial feature vectors are modeled by Gaussian Mixture Model, and then moving objects are detected. Experiments show the validity of the proposed method.