Building validity evidence for scores on a state-wide alternate assessment: A contrasting groups, multimethod approach

Stephen Elliott, Elizabeth Compton, Andrew T. Roach

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

The relationships between ratings on the Idaho Alternate Assessment (IAA) for 116 students with significant disabilities and corresponding ratings for the same students on two norm-referenced teacher rating scales were examined to gain evidence about the validity of resulting IAA scores. To contextualize these findings, another group of 54 students who had disabilities, but were not officially eligible for the alternate assessment also was assessed. Evidence to support the validity of the inferences about IAA scores was mixed, yet promising. Specifically, the relationship among the reading, language arts, and mathematics achievement level ratings on the IAA and the concurrent scores on the ACES-Academic Skills scales for the eligible students varied across grade clusters, but in general were moderate. These findings provided evidence that IAA scales measure skills indicative of the state's content standards. This point was further reinforced by moderate to high correlations between the IAA and Idaho State Achievement Test (ISAT) for the not eligible students. Additional evidence concerning the valid use of the IAA was provided by logistic regression results that the scores do an excellent job of differentiating students who were eligible from those not eligible to participate in an alternate assessment. The collective evidence for the validity of the IAA scores suggests it is a promising assessment for NCLB accountability of students with significant disabilities. The methods of establishing this evidence have the potential to advance validation efforts of other states' alternate assessments.

Original languageEnglish (US)
Pages (from-to)30-43
Number of pages14
JournalEducational Measurement: Issues and Practice
Volume26
Issue number2
DOIs
StatePublished - Jun 2007
Externally publishedYes

Fingerprint

evidence
Group
student
disability
rating
teacher rating
achievement test
rating scale
logistics
mathematics
art
regression
responsibility
language

Keywords

  • Alternate assessment
  • Contrasting groups validity design
  • Inclusive assessment

ASJC Scopus subject areas

  • Education

Cite this

Building validity evidence for scores on a state-wide alternate assessment : A contrasting groups, multimethod approach. / Elliott, Stephen; Compton, Elizabeth; Roach, Andrew T.

In: Educational Measurement: Issues and Practice, Vol. 26, No. 2, 06.2007, p. 30-43.

Research output: Contribution to journalArticle

@article{e8539237896d42ca8bca4e067f3911dc,
title = "Building validity evidence for scores on a state-wide alternate assessment: A contrasting groups, multimethod approach",
abstract = "The relationships between ratings on the Idaho Alternate Assessment (IAA) for 116 students with significant disabilities and corresponding ratings for the same students on two norm-referenced teacher rating scales were examined to gain evidence about the validity of resulting IAA scores. To contextualize these findings, another group of 54 students who had disabilities, but were not officially eligible for the alternate assessment also was assessed. Evidence to support the validity of the inferences about IAA scores was mixed, yet promising. Specifically, the relationship among the reading, language arts, and mathematics achievement level ratings on the IAA and the concurrent scores on the ACES-Academic Skills scales for the eligible students varied across grade clusters, but in general were moderate. These findings provided evidence that IAA scales measure skills indicative of the state's content standards. This point was further reinforced by moderate to high correlations between the IAA and Idaho State Achievement Test (ISAT) for the not eligible students. Additional evidence concerning the valid use of the IAA was provided by logistic regression results that the scores do an excellent job of differentiating students who were eligible from those not eligible to participate in an alternate assessment. The collective evidence for the validity of the IAA scores suggests it is a promising assessment for NCLB accountability of students with significant disabilities. The methods of establishing this evidence have the potential to advance validation efforts of other states' alternate assessments.",
keywords = "Alternate assessment, Contrasting groups validity design, Inclusive assessment",
author = "Stephen Elliott and Elizabeth Compton and Roach, {Andrew T.}",
year = "2007",
month = "6",
doi = "10.1111/j.1745-3992.2007.00092.x",
language = "English (US)",
volume = "26",
pages = "30--43",
journal = "Educational Measurement: Issues and Practice",
issn = "0731-1745",
publisher = "Wiley-Blackwell",
number = "2",

}

TY - JOUR

T1 - Building validity evidence for scores on a state-wide alternate assessment

T2 - A contrasting groups, multimethod approach

AU - Elliott, Stephen

AU - Compton, Elizabeth

AU - Roach, Andrew T.

PY - 2007/6

Y1 - 2007/6

N2 - The relationships between ratings on the Idaho Alternate Assessment (IAA) for 116 students with significant disabilities and corresponding ratings for the same students on two norm-referenced teacher rating scales were examined to gain evidence about the validity of resulting IAA scores. To contextualize these findings, another group of 54 students who had disabilities, but were not officially eligible for the alternate assessment also was assessed. Evidence to support the validity of the inferences about IAA scores was mixed, yet promising. Specifically, the relationship among the reading, language arts, and mathematics achievement level ratings on the IAA and the concurrent scores on the ACES-Academic Skills scales for the eligible students varied across grade clusters, but in general were moderate. These findings provided evidence that IAA scales measure skills indicative of the state's content standards. This point was further reinforced by moderate to high correlations between the IAA and Idaho State Achievement Test (ISAT) for the not eligible students. Additional evidence concerning the valid use of the IAA was provided by logistic regression results that the scores do an excellent job of differentiating students who were eligible from those not eligible to participate in an alternate assessment. The collective evidence for the validity of the IAA scores suggests it is a promising assessment for NCLB accountability of students with significant disabilities. The methods of establishing this evidence have the potential to advance validation efforts of other states' alternate assessments.

AB - The relationships between ratings on the Idaho Alternate Assessment (IAA) for 116 students with significant disabilities and corresponding ratings for the same students on two norm-referenced teacher rating scales were examined to gain evidence about the validity of resulting IAA scores. To contextualize these findings, another group of 54 students who had disabilities, but were not officially eligible for the alternate assessment also was assessed. Evidence to support the validity of the inferences about IAA scores was mixed, yet promising. Specifically, the relationship among the reading, language arts, and mathematics achievement level ratings on the IAA and the concurrent scores on the ACES-Academic Skills scales for the eligible students varied across grade clusters, but in general were moderate. These findings provided evidence that IAA scales measure skills indicative of the state's content standards. This point was further reinforced by moderate to high correlations between the IAA and Idaho State Achievement Test (ISAT) for the not eligible students. Additional evidence concerning the valid use of the IAA was provided by logistic regression results that the scores do an excellent job of differentiating students who were eligible from those not eligible to participate in an alternate assessment. The collective evidence for the validity of the IAA scores suggests it is a promising assessment for NCLB accountability of students with significant disabilities. The methods of establishing this evidence have the potential to advance validation efforts of other states' alternate assessments.

KW - Alternate assessment

KW - Contrasting groups validity design

KW - Inclusive assessment

UR - http://www.scopus.com/inward/record.url?scp=34248182019&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34248182019&partnerID=8YFLogxK

U2 - 10.1111/j.1745-3992.2007.00092.x

DO - 10.1111/j.1745-3992.2007.00092.x

M3 - Article

AN - SCOPUS:34248182019

VL - 26

SP - 30

EP - 43

JO - Educational Measurement: Issues and Practice

JF - Educational Measurement: Issues and Practice

SN - 0731-1745

IS - 2

ER -