In Large-Eddy Simulation (LES), scalar fluctuations are decomposed into a resolved part and a complementary Sub-Grid Scale (SGS) part. Accordingly, it is usually assumed that the scalar energy contained in these two parts sum up, so that the time average of the scalar energy equals the time average of the resolved part of the scalar energy to which the time average of the SGS scalar variance is added. Conditions are discussed under which an additional residual term must be added to close this scalar energy budget. For this residual term to stay at a moderate level, the LES filter must be small enough compared to the integral length-scale of the scalar field, a condition that is verified from a canonical manufactured turbulent scalar solution. A mesh-quality criterion is derived from these observations and the minimum Reynolds number that a Direct Numerical Simulation (DNS) should feature for SGS scalar variance to be accurately studied from a priori filtering is obtained as a corollary.