Recent progress in sensing, networking and data management has led to a wealth of valuable information. The challenge is to extract meaningful knowledge from such data produced at an astonishing rate. Unlike batch learning algorithms designed under the assumptions that data is static and its volume is small (and manageable), incremental algorithms can rapidly update their models to incorporate new information (on a sample-by-sample basis). In this paper we propose a new incremental instance-based learning algorithm which presents good properties in terms of multi-class support, complexity, scalability and interpretability. The Incremental Hypersphere Classifier (IHC) is tested in well-known benchmarks yielding good classification performance results. Additionally, it can be used as an instance selection method since it preserves class boundary samples.