This paper presents a new approach for real-world gender recognition, where images are captured under uncontrolled environments with various poses, illuminations and expressions. While a large number of gender recognition methods have been introduced in recent years, most of them describe each image in a single feature space or simple combination of multiple individual spaces, which can not be powerful enough to alleviate the noise in real-world scenarios. To address this, we propose exploring multiple order local binary patterns (MOLBP) as features for learning, and develop a localized multi-boost learning (LMBL) algorithm to combine the different features for classification. Experimental results show that the proposed algorithm outperforms state-of-the-art methods in two real-world datasets.