Multidimensional Assessment of Value Added by Teachers to Real-World Outcomes

Jennifer Broatch, Sharon Lohr

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

Measuring teacher effectiveness is challenging since no direct estimate exists; teacher effectiveness can be measured only indirectly through student responses. Traditional value-added assessment (VAA) models generally attempt to estimate the value that an individual teacher adds to students' knowledge as measured by scores on successive administrations of a standardized test. Such responses, however, do not reflect the long-term contribution of a teacher to real-world student outcomes such as graduation, and cannot be used in most university settings where standardized tests are not given. In this paper, the authors develop a multiresponse approach to VAA models that allows responses to be either continuous or categorical. This approach leads to multidimensional estimates of value added by teachers and allows the correlations among those dimensions to be explored. The authors derive sufficient conditions for maximum likelihood estimators to be consistent and asymptotically normally distributed. The authors then demonstrate how to use SAS software to calculate estimates. The models are applied to university data from 2001 to 2008 on calculus instruction and graduation in a science or engineering field.

Original languageEnglish (US)
Pages (from-to)256-277
Number of pages22
JournalJournal of Educational and Behavioral Statistics
Volume37
Issue number2
DOIs
StatePublished - Apr 2012

Fingerprint

value added
teacher
university
student
engineering
instruction
science
Values

Keywords

  • binary responses
  • generalized linear mixed models
  • multivariate mixed models
  • random effects
  • value-added assessment models

ASJC Scopus subject areas

  • Education
  • Social Sciences (miscellaneous)

Cite this

Multidimensional Assessment of Value Added by Teachers to Real-World Outcomes. / Broatch, Jennifer; Lohr, Sharon.

In: Journal of Educational and Behavioral Statistics, Vol. 37, No. 2, 04.2012, p. 256-277.

Research output: Contribution to journalArticle

@article{8d6c9ab6434c480dbcd4021699d300f7,
title = "Multidimensional Assessment of Value Added by Teachers to Real-World Outcomes",
abstract = "Measuring teacher effectiveness is challenging since no direct estimate exists; teacher effectiveness can be measured only indirectly through student responses. Traditional value-added assessment (VAA) models generally attempt to estimate the value that an individual teacher adds to students' knowledge as measured by scores on successive administrations of a standardized test. Such responses, however, do not reflect the long-term contribution of a teacher to real-world student outcomes such as graduation, and cannot be used in most university settings where standardized tests are not given. In this paper, the authors develop a multiresponse approach to VAA models that allows responses to be either continuous or categorical. This approach leads to multidimensional estimates of value added by teachers and allows the correlations among those dimensions to be explored. The authors derive sufficient conditions for maximum likelihood estimators to be consistent and asymptotically normally distributed. The authors then demonstrate how to use SAS software to calculate estimates. The models are applied to university data from 2001 to 2008 on calculus instruction and graduation in a science or engineering field.",
keywords = "binary responses, generalized linear mixed models, multivariate mixed models, random effects, value-added assessment models",
author = "Jennifer Broatch and Sharon Lohr",
year = "2012",
month = "4",
doi = "10.3102/1076998610396900",
language = "English (US)",
volume = "37",
pages = "256--277",
journal = "Journal of Educational and Behavioral Statistics",
issn = "1076-9986",
publisher = "SAGE Publications Inc.",
number = "2",

}

TY - JOUR

T1 - Multidimensional Assessment of Value Added by Teachers to Real-World Outcomes

AU - Broatch, Jennifer

AU - Lohr, Sharon

PY - 2012/4

Y1 - 2012/4

N2 - Measuring teacher effectiveness is challenging since no direct estimate exists; teacher effectiveness can be measured only indirectly through student responses. Traditional value-added assessment (VAA) models generally attempt to estimate the value that an individual teacher adds to students' knowledge as measured by scores on successive administrations of a standardized test. Such responses, however, do not reflect the long-term contribution of a teacher to real-world student outcomes such as graduation, and cannot be used in most university settings where standardized tests are not given. In this paper, the authors develop a multiresponse approach to VAA models that allows responses to be either continuous or categorical. This approach leads to multidimensional estimates of value added by teachers and allows the correlations among those dimensions to be explored. The authors derive sufficient conditions for maximum likelihood estimators to be consistent and asymptotically normally distributed. The authors then demonstrate how to use SAS software to calculate estimates. The models are applied to university data from 2001 to 2008 on calculus instruction and graduation in a science or engineering field.

AB - Measuring teacher effectiveness is challenging since no direct estimate exists; teacher effectiveness can be measured only indirectly through student responses. Traditional value-added assessment (VAA) models generally attempt to estimate the value that an individual teacher adds to students' knowledge as measured by scores on successive administrations of a standardized test. Such responses, however, do not reflect the long-term contribution of a teacher to real-world student outcomes such as graduation, and cannot be used in most university settings where standardized tests are not given. In this paper, the authors develop a multiresponse approach to VAA models that allows responses to be either continuous or categorical. This approach leads to multidimensional estimates of value added by teachers and allows the correlations among those dimensions to be explored. The authors derive sufficient conditions for maximum likelihood estimators to be consistent and asymptotically normally distributed. The authors then demonstrate how to use SAS software to calculate estimates. The models are applied to university data from 2001 to 2008 on calculus instruction and graduation in a science or engineering field.

KW - binary responses

KW - generalized linear mixed models

KW - multivariate mixed models

KW - random effects

KW - value-added assessment models

UR - http://www.scopus.com/inward/record.url?scp=84859395442&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84859395442&partnerID=8YFLogxK

U2 - 10.3102/1076998610396900

DO - 10.3102/1076998610396900

M3 - Article

AN - SCOPUS:84859395442

VL - 37

SP - 256

EP - 277

JO - Journal of Educational and Behavioral Statistics

JF - Journal of Educational and Behavioral Statistics

SN - 1076-9986

IS - 2

ER -