Limited by the available location space, the spacing between the adjacent elements of the ship-borne HF ground wave radar is closer than that of shore-based HF radar. Therefore, the spatial resolution is not high and the mutual coupling is heavy. To improve this situation, in this paper, a modified particle filter for the suboptimal selection of the interpolation matrix and the estimate of the mutual coupling coefficient matrix is proposed. Firstly, the shipborne conformal antenna array consists of a quarter circle array with N elements and a linear uniform array with N2 elements. Secondly, a global optimization is introduced, which has heavy computational load. To alleviate the computational load, a modified particle filter is used for the estimation of the mutual coupling matrix, which would increase the accuracy of the estimates. Simulation results demonstrate that the proposed approach has similar spatial spectra with the result of the linear array and its estimates are more accurate. Also, it is more robust.