A new method has been proposed in this paper to estimate the camera rotation angles and automatically extract the main contour from a simple building image. In this algorithm, the morphological sieves of small-scale were applied to construct the simple building world Bayesian model. After estimating the maximum a posterior of camera angles and the position of vanishing points, the larger scale morphological sieves were used associated with the pixels grouping result to get the main parallelepiped contour. The algorithm described in this paper requires only a single input image and does not involve processing stages such as edge detection and Hough grouping. The model we design is simple and effective. The experimental results show that estimating the grid we constructed can obtain the relative rotation accurately and extract the main border of the parallelepiped.