This paper proposes an optimized cross array passive acoustic source localization method based on Crosspower-Spectrum Phase. This method including several optimized strategies successfully improves the real-time performance and reliability of localization in applications. Firstly, according to the characteristics of the cross correlation function, some criterions are designed to eliminate the abnormal time delay value resulting from the unstable data, as well as reduce the time consumption on the invalid computation. Secondly, interpolating method is used to increase the accuracy of the time delay to some extent. Thirdly, due to the non-uniform distribution of the TDOA estimation accuracy, TDOAs calculated by the cross array will be chosen in different areas, in order to obtain the source bearing estimate with higher accuracy. Compared with beamforming method with comparable accuracy and verified in DSP experiments, the proposed method in this paper is proved to have a better real-time performance and reliability, which make it applicable to the real-time localization system.