In this paper, a new approach to face recognition is presented in which not only a classifier but also a feature space is learned incrementally to adapt to a chunk of training samples. A benefit of this type of incremental learning is that the search for useful features and the learning of an optimal decision boundary are carried out in an online fashion. To implement this idea, chunk incremental principal component analysis (IPCA) and resource allocating network with long-term memory are effectively combined. Using chunk IPCA, a feature space is updated by rotating its eigen-axes and increasing the dimensions to adapt to a chunk of given training samples. In the experiments, the proposed incremental learning model is evaluated over a self-compiled face image database. As the result, we verify that the proposed model works well without serious forgetting and the test performance is improved as the learning stages proceed.