Our team has developed the digital panoramic system named PanoExpert VI based on linear CCD and fisheye lens, which can acquire the high resolution panorama with the full surrounding in one scan. However, the image includes significant distortions. So, this paper describes the work how to correct the distortion. We research on calibration of fisheye lens' barrel distortion and cylindrical deviations due to linear CCD on base of analysis of current distortion. Then for it's unique, a new model must be established for panoramic image processing, and we bring forward the anti-plane view model based on hemispherical circle and cylindrical image surface. Experiments have proved the precision of the model and the panorama can be calibrated satisfactorily.