An outlier mining based algorithm is presented to detect dim target in sea and sky background infrared image sequences. Firstly, according to the characteristic that dim target intensity is not correlation with background, viewed the target pixels as outliers; and then used an outlier mining algorithm based on local reachability density (lrd) to detect the candidate target pixels; finally, target moving continuity and trajectory consistency is used for tracking the targets trajectory. Algorithm is different from the traditional DBT algorithms that based on clutter suppression and binarization; it detected the target pixels from image data matrix directly. Algorithm computationally simple and lose less information. The experiment shows that the algorithm can detect the dim moving target effectively and accurately estimate its trajectory.