In this paper, we propose a method for improving the parametric eigenspace method by automatically removing backgrounds in an input image. The region of a target object is extracted by fitting multiple ellipses to the image and the outer regions around the object are removed as background. The combination of multiple ellipses can flexibly represent various shapes of the target object. In addition, efficient ellipse fitting is realized by using foreground probability which is calculated from a learning image set. It has been shown that the recognition success rate is 67.8% to 100.0% by experiments using 180 actual images including various kinds of complicated backgrounds. This result means that our method has 43% higher performance compared with the original parametric eigenspace method.