In this paper, we present two methods to improve the performance of landmark detection algorithms that are designed to detect individual landmarks. We focus on the landmark configuration module that takes the output of the individual landmark detectors and searches for a configuration of optimal landmark locations based on appropriate shape constraints. We design two configuration search approaches: (i) a multivariate conditional Gaussian-based model, and (ii) a MRF-based formulation with higher-order potentials. We evaluated the performance of our proposed methods using several state-of-the-art detectors, and consistently obtained improved performance.