TY - JOUR
T1 - Different teacher-level effectiveness estimates, different results
T2 - inter-model concordance across six generalized value-added models (VAMs)
AU - Sloat, Edward
AU - Beardsley, Audrey
AU - Holloway, Jessica
N1 - Publisher Copyright:
© 2018, Springer Nature B.V.
PY - 2018/11/1
Y1 - 2018/11/1
N2 - In this study, researchers compared the concordance of teacher-level effectiveness ratings derived via six common generalized value-added model (VAM) approaches including a (1) student growth percentile (SGP) model, (2) value-added linear regression model (VALRM), (3) value-added hierarchical linear model (VAHLM), (4) simple difference (gain) score model, (5) rubric-based performance level (growth) model, and (6) simple criterion (percent passing) model. The study sample included fourth to sixth grade teachers employed in a large, suburban school district who taught the same sets of students, at the same time, and for whom a consistent set of achievement measures and background variables were available. Findings indicate that ratings significantly and substantively differed depending upon the methodological approach used. Findings, accordingly, bring into question the validity of the inferences based on such estimates, especially when high-stakes decisions are made about teachers as based on estimates measured via different, albeit popular methods across different school districts and states.
AB - In this study, researchers compared the concordance of teacher-level effectiveness ratings derived via six common generalized value-added model (VAM) approaches including a (1) student growth percentile (SGP) model, (2) value-added linear regression model (VALRM), (3) value-added hierarchical linear model (VAHLM), (4) simple difference (gain) score model, (5) rubric-based performance level (growth) model, and (6) simple criterion (percent passing) model. The study sample included fourth to sixth grade teachers employed in a large, suburban school district who taught the same sets of students, at the same time, and for whom a consistent set of achievement measures and background variables were available. Findings indicate that ratings significantly and substantively differed depending upon the methodological approach used. Findings, accordingly, bring into question the validity of the inferences based on such estimates, especially when high-stakes decisions are made about teachers as based on estimates measured via different, albeit popular methods across different school districts and states.
KW - Teacher accountability
KW - Teacher effectiveness
KW - Teacher evaluation
KW - Teacher quality
KW - Validity
KW - Value-added models
UR - http://www.scopus.com/inward/record.url?scp=85050908415&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85050908415&partnerID=8YFLogxK
U2 - 10.1007/s11092-018-9283-7
DO - 10.1007/s11092-018-9283-7
M3 - Article
AN - SCOPUS:85050908415
SN - 1874-8597
VL - 30
SP - 367
EP - 397
JO - Educational Assessment, Evaluation and Accountability
JF - Educational Assessment, Evaluation and Accountability
IS - 4
ER -