This paper proposes a new regularized adaptive windowed Lomb periodogram (RAWLP) method for time-frequency analysis of non-stationary power signals. It extends the conventional Lomb periodogram by estimating the periodogram locally using the weighted least-squares (WLS) estimator. Instead of employing one constant window in WLS, variable window bandwidth is adaptively selected by the intersection of confidence intervals (ICI) method to achieve a better tradeoff between time resolution and frequency resolution. Furthermore, regularization techniques are incorporated in the AWLP to further improve its performance by reducing the variance of the estimator. Simulation results show that the proposed RAWLP method has superior performance over windowed Lomb periodogram with one constant bandwidth for estimating the harmonic and interharmonic frequencies in power systems.