Infrared (IR) image segmentation is a key technology of automatic target recognition technology (ATR), which strongly affects the image analysis results. As a global threshold method, the maximum entropy method is often applied in multi-threshold image segmentation, which can get ideal result. However, its time-consuming computation is often an obstacle in real time application systems. In this paper, cultural algorithm (CA) is used to optimize the maximum entropy function of image. The optimal threshold values can be successfully found through the cooperation between the population space and the belief space of CA. It is realized successfully in the process of solving the maximum entropy problem. It shows that the method proposed in this paper can get better solutions in shorter time and has better performance in stabilization and convergence than the other same-type methods. And, the segmentation results of IR image are ideal. The experimental results show that the method is feasible and effective.