Fundamental and high-order phonon-phonon interactions dominate the far infrared absorption and dispersion spectrum of most semiconductors and dielectrics. We present the detailed investigation of the temperature dependence of the dielectric function of the semiconductor GaSe in the frequency range below 4 THz, between 100 K and 310 K, using terahertz time-domain spectroscopy (THz-TDS) technique. Combined with the Fourier translation infrared (FTIR) spectroscopy, the complete infrared absorption spectra, including both sum and difference two-phonon absorption, are obtained. From the dielectric function we determine the blue-shift of the fundamental phonon frequency Epsila at 0.59 THz with the decrease of the temperature. Furthermore, our experimental data enable assignment of the two-phonon terahertz absorption to difference combinations of fundamental phonon mode at critical points in the Brillouin zone center.