Advent of Matrix Theory has greatly aided and simplified the analysis for variety of signal processing algorithms. It has been proven that matrix notation is convenient for representation of signals and to perform operations on them. Many problems such as signal modeling, Wiener filtering and spectrum estimation require finding the solution or solutions to a set of linear equations. Some of the common matrix related operations include transpose, triangularization, determinant calculation, eigen value decomposition and matrix inversion. Most of these operations are computationally intensive and have been difficult to implement on real time systems and therefore are not pursued much in VLSI design. In this paper we present a highly hardware efficient and simple memory based novel architecture implementing widely established Gauss Jordan technique for finding matrix inverse. First triangularization of the matrix is done which on further processing calculates the inverse matrix.