The aim of this article is twofold. In the first place, we describe a special non-abelian group of block upper triangular matrices, and verify that choosing properly certain parameters, the order of the subgroup generated by one of these matrices can be as large as needed. Secondly, we propose and implement a new key exchange scheme based on these primitives. The security of the proposed system is based on discrete logarithm problem although a non-abelian group of matrices is used. The primary advantadge of this scheme is that no prime numbers are used and the efficiency is guaranteed by the use of a quick exponentiation algorithm for this group of matrices.