False confidence in intervals: Inaccuracies in reporting confidence intervals

Confidence intervals often are recommended as a means of communicating the extent to which individual test scores may be influenced by measurement error. However, test manuals and assessment texts vary widely in their recommendations about how confidence intervals should be constructed, and several contain misinterpretations of classical test theory. The most widely used procedure for constructing confidence intervals misrepresents the likely distribution of true scores, and confidence intervals constructed with it will be inaccurate, especially when extreme scores are involved. The various procedures for constructing confidence intervals that have been suggested in measurement texts are examined in relation to their approximation to the most accurate procedure that uses the estimated true score as the center of the confidence interval and the standard error of estimate to determine the width. In addition, the problems of applying these procedures to norm‐referenced scores are discussed—an issue that has been largely ignored in the assessment literature and that leads to further misinterpretations of confidence intervals.