Let and be positive integers with being odd, for and , the exponential sum is studied systematically in this paper, where , is a Galois ring, is the Teichmüller set of and is the trace function from the Galois ring to . Through the discussions on the solutions of certain equations and the newly developed theory of -valued quadratic forms, the distribution of the exponential sum is completely determined. As its applications, we can determine the Lee weight and Hamming weight distributions of a class of codes over and the correlation distribution of a quaternary sequence family , respectively. Furthermore, the Hamming weight distributions of the binary codes obtained from under the most significant bit (MSB) and Gray maps are also determined. For the MSB map sequences of , the nontrivial maximal correlation value is given and the correlation distribution is determined for the Gray map sequences of . It should be noted that the distribution of the exponential sum for the case is obtained for the first time, and then the corresponding codes and sequences are novel.