Mimicking the human brain to achieve human-level cognition performance has been a core challenge in artificial intelligence research for decades. Humans are very efficient in capturing the most important information while being exposed to a plethora of different stimuli, a capability that is used to represent and understand their surroundings in a concise fashion. Machine learning research has made considerable progress toward cloning such a human capability with innovative techniques like deep learning, feature learning, incremental learning, and so on. In this article, an overview of the mainstream brain-inspired architectures and research directions proposed over the past decade is provided. In addition, a novel architecture exploiting the strengths of the current methods is proposed. Preliminary results demonstrate that it is able to achieve state-of-the-art results in a more efficient way.