Relations between deterministic (e.g. variational or PDE based methods) and Bayesian inference have been known for a long time. However, a classification of deterministic approaches into those methods which can be handled within a Bayesian framework and those with no such statistical counterpart is still missing in literature. After providing such taxonomy, we present a Bayesian framework for embedding the former ones into a statistical context allowing to equip them with advantages of probabilistic estimation theory. A stochastic point of view allows us (1) to learn influence functions and derivative filter, (2) adapt diffusion and regularization approaches to changes in the image characteristics (e.g. varying noise levels), and (3) to estimate error bounds on the solution. For the latter ones we present alternative learning schemes also allowing their parameters to be related to the image statistics such that hand tuning becomes dispensable. We demonstrate that a statistical point of view on diffusion and regularization schemes leads to image denoising performances comparable with state of the art Markov random field approaches while being computationally much more effective.