The problem of state estimation with stochastic uncertainties in the initial state, model noise, and measurement noise is considered using the restricted risk Bayes approach. It is assumed that the a priori distributions of these quantities are not perfectly known, but that some information about them may be available. While offering robustness, the restricted risk Bayes approach incorporates the available a priori information to give less conservative state estimators than the Gamma-minimax approach popular in the literature. When attention is restricted to linear estimators based on a quadratic loss function, a systematic method to derive restricted risk Bayes estimators is proposed. Applying to the filtering problem, the restricted risk Bayes approach provides us with a robust method to calibrate the Kalman filter (KF), considering the presence of stochastic uncertainties. This method is illustrated with a target tracking example and a wireless channel tracking example for which the Bayes, minimax, and restricted risk Bayes estimators are derived and their performance is compared.