Constructing efficient and accurate parametrizations of subgrid‐scale processes is a central area of interest in the numerical modelling of geophysical fluids. Using a modified version of the two‐level Lorenz '96 model, we present here a proof of concept of a scale‐adaptive parametrization constructed using statistical mechanical arguments. By suitable use of the Ruelle response theory and the Mori–Zwanzig projection method, it is possible to derive explicitly a parametrization for the fast variables that translates into deterministic, stochastic and non‐Markovian extra terms in the equations of motion for the variables of interest. We show that our approach is computationally parsimonious and has great flexibility, as it is explicitly scale‐adaptive, and we prove that it is competitive compared with empirical ad‐hoc approaches. While the parametrization proposed here is universal and can easily be adapted analytically to changes in parameter values by a simple rescaling procedure, the parametrization constructed with the ad‐hoc approach needs to be recomputed each time the parameters of the systems are changed. The price we pay for the higher flexibility of the method proposed here is having a lower accuracy in each individual case.