In this paper, we first propose a scheme to learn with automatically aligned objects for image categorization. To fulfil this purpose, the learning is formulated into a multiple instance learning problem with bags. Each bag is a virtually generated image set from an original image. The virtually generated image set covers multiple possible object locations, scales and views. Secondly, we propose a novel adaptive learning method to take advantage of the sharing information among different categories in image categorization. The method models the sharing information between one category and the other categories by learning a classifier of the current category from perturbed pre-learned classifiers from other categories. And the adaptive learning is realized in the multiple kernel learning framework. Finally, to align the images from one category and share the information among different categories simultaneously, the proposed multiple instance learning scheme and the proposed adaptive learning method are integrated into one learning framework. A new formulation called Adaptive Multiple Instance Learning (A-MIL) is derived. The optimization method is provided. To evaluate the proposed method, comprehensive experiments are conducted on two well-known public image categorization datasets (Caltech101 and 15Scenes). The experimental results show that the proposed method outperforms the state-of-the-arts in classification accuracy.