The shielding effectiveness of a rectangular enclosure with rectangular apertures is investigated using the mode-matching method. With this method, the generalized scattering matrix and optimum topological method are utilized. In modeling the aperture penetration, the waveguide modal approximation for an incident plane wave is proposed based on the mode-matching method. In this method, the incident plane wave is replaced with the dominant mode of the virtual rectangular waveguide. The electromagnetic fields of the virtual rectangular waveguide and the rectangular aperture are matched by using the mode-matching method. The proposed method is verified by comparing to the results obtained by the CST simulation. It is shown that the proposed method can be applied to the arbitrary rectangular waveguides regardless of the size, thickness, and orientation of the aperture and the polarization and incidence angle of the incident wave.