In this paper we present a robust object tracking approach which is suitable for real-time image processing on mobile devices. Challenging mobile environments render traditional color-based tracking methods useless. Many online learning tracking methods are too computationally complex to be used for real-time mobile applications, which only have access to limited computational resource and memory storage. The proposed method takes advantage of local feature to deal with rapid camera motion, and employs an online feature updating scheme to cope with variation in object appearances. The method is also computationally lightweight, being able to support real-time image processing on mobile devices.