This paper proposes an effective method for video object segmentation without background construction. In the proposed method, the coarse foreground extraction and fine foreground extraction are obtained using the motion information, edge information, and gradient-variation information which are first evaluated by two successive frames. Next, the video object is extracted using the horizontal/vertical filling scheme based on the coarse foreground extraction and fine foreground extraction. Finally, video object refining is used to obtain the more accurate video object. Experimental results show that the proposed method has good performance in sensitivity, specificity, and spatial accuracy.