Magnetic resonance imaging (MRI) can aid in assessing post-ablation scar formation. Automatic segmentation of left atrium (LA) offers great benefits for an accurate statistical assessment of LA region. However, how to robustly segment LA is still remaining as a challenging task for its high anatomical variability. In this paper, a robust segmentation method that exploits semantic information from different parts is proposed. The semantic correlation is exploited by the K Nearest Neighbor (KNN) search from corpus images with Convolutional Neural Network (CNN) features, which can be regarded as our main contribution. We propose a graph model to fuse semantic cues and eliminate accidental factors. Meanwhile, to optimize segmentation results, a super pixel voting method is also proposed. Experiments on public datasets of MRI image demonstrate the validity and accuracy of our semantic segmentation.