Omnidirectional vision system has wide application prospects in maritime surveillance applications for its advantages of unified imaging, 360-degree view and rotation invariant. But it has weak support in techniques of target detection. With the complexity of sea-sky background and the specificity of omnidirectional image, this paper is carrying out the small targets detection on the sea based on omnidirectional image sequences. A method for omnidirectional circular sea-sky line extraction is presented based on orientations of the gradient, then an algorithm for small target detection around the sea-sky line area is proposed based on gradient threshold. The result of the experimental shows the efficiency and advantage of the method in this paper.