The modal analysis is needed for the design of photonic waveguide devices to determine propagation constants of guided modes and their field pattern. Rigorous methods based on finite-difference or finite-element methods for the approximation of wave equations rely on the solution of eigenvalue problems. Imaginary-distance beam propagation methods require the solution of matrix equations of high order. The computation time needed for both approaches can be tedious. A modal analysis method based on a beam propagation scheme is proposed, which allows an efficient computation of field patterns and propagation constants, as the resulting algorithm is based on simple matrix vector multiplications. This allows the choice of a large number of discretization points and a dense discretization. The beam propagation operator is designed to include the eigenvalue spectrum of corresponding guided modes only. For this reason, the method can be interpreted as a subdomain propagation method. The concept will be validated by utilizing a single-mode and multimode rib waveguide.