We introduce a 3D segmentation framework which uses principal shapes. The probabilistic energy function of the method is defined based on intensity, tissue type, and location information of the structures using a multiple atlas method. For intensity information, nonparametric probability density function is used which considers intensity relation of different structures. To find a local minimum of the energy function, a two-step optimization strategy is used. In the first step, shape parameters are optimized based on the analytic derivatives of the energy function. In the second step, shapes of the structures are fine-tuned using a level set method. The proposed method is shown to be superior to some popular methods in the literature using a dataset of 64 patients with mesial temporal lobe epilepsy. In addition, the method can be used for lateralization with accuracy close to that of manual segmentation.