Psychometric theory allows interindividual comparisons by scaling differences among subjects with respect to some psychological attribute. The most widely used psychometric scaling method is classical test theory. The properties of classical test theory pain scores are limited to the observed range of pain scores in a given sample, to the specific conditions in a given sample that are the source of pain (e.g. surgery vs. cancer), and to a particular pain survey. The reliability and meaning of classical test theory scores differ for subjects who have higher or lower amounts of pain, have different painful conditions, or are given a different pain survey. Thus, classical test theory pain scores cannot be used to compare dissimilar patient groups or painful conditions, especially if different pain surveys are used. A different psychometric scaling method, item response theory, produces scores with properties that are not limited to an observed score range, specific conditions, or a particular pain survey, and may thus be better for making such comparisons. To compare the two psychometric methods, data were obtained from 335 subjects who rated their clinical pain using 15 words. The psychometric scaling methods were compared using standardized residuals (data fit), standard errors of measurement (score precision), and a graphical plot of predicted scores against scale scores (bias). The item response theory scores demonstrated better data fit and less bias than did the classical test theory scores. In addition to superior psychometric properties, item response theory scores have several other important theoretical and practical advantages.