Outcome studies report the percentage of clinically significant outcomes; however, the reliability of these classifications is unclear. The current study explored the extent to which inconsistencies arise in classifying patient outcomes using five clinical significance calculation methods and three outcome measures. Adult inpatients (N = 2,676) treated for depression completed the three outcome measures pre‐ and post‐treatment. Their outcomes were classified as recovered, improved, unchanged, or deteriorated using selected clinical significance calculation methods and outcome measures. The choice of outcome measure used in calculating clinical significance had a greater impact on the classification than the choice of clinical significance calculation method. Guidelines for reporting recovery rates are presented to ensure that appropriate conclusions are drawn.