Results obtained from two accredited laboratories that regularly conduct ASTM E108 fire tests showed serious discrepancies. The source of the problem was identified to be errors in the ASTM E108 standard instructions for calibration of the fire test apparatus. The standard requires that the apparatus calibration achieve three criteria—two quantitative and one qualitative criteria. The quantitative calibration criteria include measurement of a specified wind speed across the calibration deck surface and measurement of a specified flame temperature at the leading edge of the calibration deck. The qualitative criterion is the specification of a required flame shape above the calibration deck surface that is determined by observation rather than measurement. From an engineering control standpoint, this problem has only two independent variables (air flow rate and fuel flow rate), so there can be no more than two control points, but the standard specifies three (air flow rate, flame temperature, and flame size). This is an example of an over‐constrained problem where a unique solution cannot be obtained, which satisfies all the constraints. It is suggested that the test standard be revised to eliminate this conflict. It is further suggested that the ASTM E108 calibration procedure be revised to control the burner heat release rate instead of controlling the flame temperature and flame size.