In this paper, we propose a super resolution method based on linear regression in different middle-frequency texture categories. We benefit from the hypothesis that the mapping from middle-frequency manifold to high-frequency manifold is similar locally, and use simple linear regression method to learn mapping functions in different area of middle-frequency manifold. Different from previous works, our method only uses the database to learn the mapping functions in different categories in the training phase, then we just need to save these mapping functions instead of a huge external database to get the missing details. Some experiments are used to confirm the effectiveness and efficiency of our method as well as our hypothesis.