A new method for the statistical control of rating error in performance ratings

Brendan D. Bannister; Angelo J. Kinicki; Peter Hom; Angelo S. Denisi

doi:10.1177/001316448704700305

A new method for the statistical control of rating error in performance ratings

Brendan D. Bannister, Angelo J. Kinicki, Peter Hom, Angelo S. Denisi

Management and Entrepreneurship

Research output: Contribution to journal › Article › peer-review

5 Scopus citations

Abstract

Research on the statistical control of rating error is reviewed. It is shown that current approaches have been criticized on operational grounds. A new approach for statistically controlling rater errors is proposed that avoids problems inherent in past research by using an index of rater error that is theoretically independent of actual performance. Results indicated that partialing out the effects of this extraneous response bias from performance ratings enhanced discriminant validity and slightly attenuated convergent validity (relative to the uncontrolled analysis). In contrast, partialing out overall effectiveness led to substantial reductions in both convergent and discriminant validities.

Original language	English (US)
Pages (from-to)	583-596
Number of pages	14
Journal	Educational and Psychological Measurement
Volume	47
Issue number	3
DOIs	https://doi.org/10.1177/001316448704700305
State	Published - Sep 1987

ASJC Scopus subject areas

Education
Developmental and Educational Psychology
Applied Psychology
Applied Mathematics

Access to Document

10.1177/001316448704700305

Cite this

@article{f8b0ce1e7acf43de8c0ae60914bbd0c8,

title = "A new method for the statistical control of rating error in performance ratings",

abstract = "Research on the statistical control of rating error is reviewed. It is shown that current approaches have been criticized on operational grounds. A new approach for statistically controlling rater errors is proposed that avoids problems inherent in past research by using an index of rater error that is theoretically independent of actual performance. Results indicated that partialing out the effects of this extraneous response bias from performance ratings enhanced discriminant validity and slightly attenuated convergent validity (relative to the uncontrolled analysis). In contrast, partialing out overall effectiveness led to substantial reductions in both convergent and discriminant validities.",

author = "Bannister, {Brendan D.} and Kinicki, {Angelo J.} and Peter Hom and Denisi, {Angelo S.}",

year = "1987",

month = sep,

doi = "10.1177/001316448704700305",

language = "English (US)",

volume = "47",

pages = "583--596",

journal = "Educational and Psychological Measurement",

issn = "0013-1644",

publisher = "SAGE Publications Inc.",

number = "3",

}

TY - JOUR

T1 - A new method for the statistical control of rating error in performance ratings

AU - Bannister, Brendan D.

AU - Kinicki, Angelo J.

AU - Hom, Peter

AU - Denisi, Angelo S.

PY - 1987/9

Y1 - 1987/9

N2 - Research on the statistical control of rating error is reviewed. It is shown that current approaches have been criticized on operational grounds. A new approach for statistically controlling rater errors is proposed that avoids problems inherent in past research by using an index of rater error that is theoretically independent of actual performance. Results indicated that partialing out the effects of this extraneous response bias from performance ratings enhanced discriminant validity and slightly attenuated convergent validity (relative to the uncontrolled analysis). In contrast, partialing out overall effectiveness led to substantial reductions in both convergent and discriminant validities.

AB - Research on the statistical control of rating error is reviewed. It is shown that current approaches have been criticized on operational grounds. A new approach for statistically controlling rater errors is proposed that avoids problems inherent in past research by using an index of rater error that is theoretically independent of actual performance. Results indicated that partialing out the effects of this extraneous response bias from performance ratings enhanced discriminant validity and slightly attenuated convergent validity (relative to the uncontrolled analysis). In contrast, partialing out overall effectiveness led to substantial reductions in both convergent and discriminant validities.

UR - http://www.scopus.com/inward/record.url?scp=84973743766&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84973743766&partnerID=8YFLogxK

U2 - 10.1177/001316448704700305

DO - 10.1177/001316448704700305

M3 - Article

AN - SCOPUS:84973743766

SN - 0013-1644

VL - 47

SP - 583

EP - 596

JO - Educational and Psychological Measurement

JF - Educational and Psychological Measurement

IS - 3

ER -

A new method for the statistical control of rating error in performance ratings

Abstract

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this