Image super-resolution reconstruction is the current research focus and important field of digital image processing, and widely applied in public security, medical diagnosis, etc. Inspired by the idea of granular computing, we explore the granular computing models and algorithms of image super-resolution reconstruction integrating theories and methods of computer science, mathematics, and cognitive psychology. The research includes the following aspects. (1) The granulation method of image is proposed to transform the image space into granularity space. (2) The join operator and the meet operator between two granules are designed for the fuzzy inclusion measure mu and sigma between two granules, are used to realize the transformation between two granularity spaces with different granularities, to obtain the prior knowledge to guide the design of image super-resolution reconstruction algorithms. (3) According to up–down and bottom-up computing models, the granular computing algorithms of image super-resolution reconstruction are designed to realize the transformation from granularity space to image space in terms of the prior knowledge. Research can be summarized for learning process and reconstruction process, the learning process obtains the prior knowledge of image by granulation and forms granular computing model, and the reconstruction process reconstructs the high-resolution image of the given low-resolution image by granulation and the obtained prior knowledge. The feasibility of the preliminary study was verified by experiments. The research aims to build image super-resolution reconstruction models and algorithms satisfying human cognition.