Young children need their parents' love and care but their parents are not always available to tell them stories. This motivates our proposal of a Smart Teddy Bear, a vision-based story teller, to assist young children in their social-emotional growth. When a young child wants to listen to a story, he or she simply opens any page in that book before a Smart Teddy Bear and the system automatically recognizes which book is of interest and plays the corresponding audio story. Our proposed system is based on the Bag-of-Words model. We propose to eliminate too small visual features and to downscale an image up to 1/16 of its original size to remove up to 70% of unnecessary visual features. Furthermore we ensemble results of multiple lightweight SVM classifiers to boost the overall accuracy. Experimental results show that our system achieves the accuracy of 99.33% with 40 books containing 1369 different pages. We propose the Smart Teddy Bears help children enjoy audio books in a new exciting way through attractive interactions with their lovely toys.