Robust stability conditions are investigated for an adaptive regulation scheme which is capable of tuning a time-varying controller to mitigate the effects of unknown disturbances. Coprime factorization of the plant and the controller are used to tune the controller while allowing parameter uncertainty within the model. While the stability conditions are applicable to general time-varying parameterizations of the controller, it is shown that the conditions boil down to those of time-invariant controllers if the adaptation is parameterized with an FIR filter. Under these conditions, stability analysis is formulated via a type of small-gain theorem in a time-varying sense. In addition, it is shown that by filtering parameter updates of the FIR filter, one can reduce parameter update volatility while maintaining stability. Application of the results to model of a fourth order plant with an uncertain resonance frequency serves as an illustration of the proposed stability criteria.