Development and evaluation of shortcut-derived behaviorally anchored rating scales

Cecilia H. Champion, Samuel B. Green, William I. Sauser

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

The purpose of the present study was to evaluate a shortcut method for developing Behaviorally Anchored Rating Scales (BARS). The developmental method included two novel steps: (a) the performance of judgmental cluster analysis to define scales and (b) the direct placement of behavioral incidents on the scales rather than the use of the retranslation step. Two groups of six students each developed shortcut BARS through using an instructional booklet. The BARS were developed to evaluate teaching performance of college instructors. The sets of shortcut scales constructed by the two groups and a traditionally developed BARS were then distributed randomly to students in nine classes who were asked to evaluate both their class instructors and the instructors in their immediately preceding class. The shortcut scales, in comparison with the traditional BARS, were slightly more reliable. Both sets of scales had approximately the same degree of leniency error. The shortcut scales were in general somewhat less adequate in terms of halo error and discriminant validity, but were almost equivalent on these characteristics after the scales with very low reliabilities had been dropped. The results did not support the use of the judgmental cluster analysis, but did indicate that the retranslation step may not be necessary to develop BARS.

Original languageEnglish (US)
Pages (from-to)29-41
Number of pages13
JournalEducational and Psychological Measurement
Volume48
Issue number1
DOIs
StatePublished - 1988
Externally publishedYes

Fingerprint

Cluster analysis
rating scale
Cluster Analysis
Students
Pamphlets
Evaluation
evaluation
instructor
Teaching
cluster analysis
performance
incident
Evaluate
Group
student
Discriminant
Placement
Immediately

ASJC Scopus subject areas

  • Developmental and Educational Psychology
  • Applied Psychology
  • Applied Mathematics
  • Education

Cite this

Development and evaluation of shortcut-derived behaviorally anchored rating scales. / Champion, Cecilia H.; Green, Samuel B.; Sauser, William I.

In: Educational and Psychological Measurement, Vol. 48, No. 1, 1988, p. 29-41.

Research output: Contribution to journalArticle

Champion, Cecilia H. ; Green, Samuel B. ; Sauser, William I. / Development and evaluation of shortcut-derived behaviorally anchored rating scales. In: Educational and Psychological Measurement. 1988 ; Vol. 48, No. 1. pp. 29-41.
@article{ae89cb2a18b64f6297d1dd0027f7436e,
title = "Development and evaluation of shortcut-derived behaviorally anchored rating scales",
abstract = "The purpose of the present study was to evaluate a shortcut method for developing Behaviorally Anchored Rating Scales (BARS). The developmental method included two novel steps: (a) the performance of judgmental cluster analysis to define scales and (b) the direct placement of behavioral incidents on the scales rather than the use of the retranslation step. Two groups of six students each developed shortcut BARS through using an instructional booklet. The BARS were developed to evaluate teaching performance of college instructors. The sets of shortcut scales constructed by the two groups and a traditionally developed BARS were then distributed randomly to students in nine classes who were asked to evaluate both their class instructors and the instructors in their immediately preceding class. The shortcut scales, in comparison with the traditional BARS, were slightly more reliable. Both sets of scales had approximately the same degree of leniency error. The shortcut scales were in general somewhat less adequate in terms of halo error and discriminant validity, but were almost equivalent on these characteristics after the scales with very low reliabilities had been dropped. The results did not support the use of the judgmental cluster analysis, but did indicate that the retranslation step may not be necessary to develop BARS.",
author = "Champion, {Cecilia H.} and Green, {Samuel B.} and Sauser, {William I.}",
year = "1988",
doi = "10.1177/001316448804800105",
language = "English (US)",
volume = "48",
pages = "29--41",
journal = "Educational and Psychological Measurement",
issn = "0013-1644",
publisher = "SAGE Publications Inc.",
number = "1",

}

TY - JOUR

T1 - Development and evaluation of shortcut-derived behaviorally anchored rating scales

AU - Champion, Cecilia H.

AU - Green, Samuel B.

AU - Sauser, William I.

PY - 1988

Y1 - 1988

N2 - The purpose of the present study was to evaluate a shortcut method for developing Behaviorally Anchored Rating Scales (BARS). The developmental method included two novel steps: (a) the performance of judgmental cluster analysis to define scales and (b) the direct placement of behavioral incidents on the scales rather than the use of the retranslation step. Two groups of six students each developed shortcut BARS through using an instructional booklet. The BARS were developed to evaluate teaching performance of college instructors. The sets of shortcut scales constructed by the two groups and a traditionally developed BARS were then distributed randomly to students in nine classes who were asked to evaluate both their class instructors and the instructors in their immediately preceding class. The shortcut scales, in comparison with the traditional BARS, were slightly more reliable. Both sets of scales had approximately the same degree of leniency error. The shortcut scales were in general somewhat less adequate in terms of halo error and discriminant validity, but were almost equivalent on these characteristics after the scales with very low reliabilities had been dropped. The results did not support the use of the judgmental cluster analysis, but did indicate that the retranslation step may not be necessary to develop BARS.

AB - The purpose of the present study was to evaluate a shortcut method for developing Behaviorally Anchored Rating Scales (BARS). The developmental method included two novel steps: (a) the performance of judgmental cluster analysis to define scales and (b) the direct placement of behavioral incidents on the scales rather than the use of the retranslation step. Two groups of six students each developed shortcut BARS through using an instructional booklet. The BARS were developed to evaluate teaching performance of college instructors. The sets of shortcut scales constructed by the two groups and a traditionally developed BARS were then distributed randomly to students in nine classes who were asked to evaluate both their class instructors and the instructors in their immediately preceding class. The shortcut scales, in comparison with the traditional BARS, were slightly more reliable. Both sets of scales had approximately the same degree of leniency error. The shortcut scales were in general somewhat less adequate in terms of halo error and discriminant validity, but were almost equivalent on these characteristics after the scales with very low reliabilities had been dropped. The results did not support the use of the judgmental cluster analysis, but did indicate that the retranslation step may not be necessary to develop BARS.

UR - http://www.scopus.com/inward/record.url?scp=84973731687&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84973731687&partnerID=8YFLogxK

U2 - 10.1177/001316448804800105

DO - 10.1177/001316448804800105

M3 - Article

VL - 48

SP - 29

EP - 41

JO - Educational and Psychological Measurement

JF - Educational and Psychological Measurement

SN - 0013-1644

IS - 1

ER -