In manufacturing industry there is a need for an adaptable automated visual inspection (AVI) system that can be used for different inspection tasks under different operation condition without requiring excessive retuning or retraining. This paper proposes an adaptable AVI scheme using an efficient and effective online learning approach. The AVI scheme uses a novel inspection model that consists of the two sub-models for localization and verification. In the AVI scheme, the region localization module is implemented by using a template-matching technique to locate the subject to be inspected based on the localization sub-mode. The defect detection module is realized by using the representative features obtained from the feature extraction module and executing the verification sub-model built in the model training module. A support vector machine (SVM) based online learning algorithm is proposed for training and updating the verification sub-model. In the case studies, the adaptable AVI scheme demonstrated its promising performances with respect to the training efficiency and inspection accuracy. The expected outcome of this research will be beneficial to the manufacturing industry.