Research on the statistical control of rating error is reviewed. It is shown that current approaches have been criticized on operational grounds. A new approach for statistically controlling rater errors is proposed that avoids problems inherent in past research by using an index of rater error that is theoretically independent of actual performance. Results indicated that partialing out the effects of this extraneous response bias from performance ratings enhanced discriminant validity and slightly attenuated convergent validity (relative to the uncontrolled analysis). In contrast, partialing out overall effectiveness led to substantial reductions in both convergent and discriminant validities.
ASJC Scopus subject areas
- Developmental and Educational Psychology
- Applied Psychology
- Applied Mathematics