This paper presents a novel approach for activity recognition from accelerometer data. Existing approaches usually extract hand-crafted features that are used as input for classifiers. However, hand-crafted features are data dependent and could not be generalized for different application domains. To overcome these limitations, our approach relies on matrix factorization for dimensionality reduction and deep learning algorithm such as a stacked auto-encoder to automatically learn suitable features, which will be then fed into a softmax classifier for classification. Our approach has potential advantages over existing approaches in terms of automatic feature extraction and generalization across different application domains. The proposed approach is validated using extensive experiments on various publicly available datasets. We empirically demonstrate that our proposed approach accurately discriminates between human activities and performs better than several state-of-the-art approaches.