This paper reports on the design of a vision-based exercise monitoring system. The system aims to promote well-being by making exercise sessions enjoyable experiences, either through real-time interaction and instructions proposed to the user, or via experience sharing or group gaming with peers in a virtual community. The use of avatars is explored as means of representation of the user’s exercise movements or appearance, and the system employs user-centric approaches in visual processing, behavior modeling via history data accumulation, and user feedback to learn the user’s appreciation. A preliminary user survey study has been conducted to explore the avatar appreciations across different types of social contexts.