Classification of halftone image is one of the important methods to resolve the optimal reconstruction problems of halftone image. Novel region covariance descriptor is presented in this paper for classification of halftone image. A set of pre-defined templates are proposed to convolute with the Fourier spectrum of halftone image to acquire covariance matrices. Bayesian classification on Riemannian manifolds is presented as classifier of halftone images. In experiments, our method has lower classification error rate than other five classic methods. Our experimental results show the proposed method is effective.