A Stepwise Test Characteristic Curve Method to Detect Item Parameter Drift

Rui Guo, Yi Zheng, Hua Hua Chang

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

An important assumption of item response theory is item parameter invariance. Sometimes, however, item parameters are not invariant across different test administrations due to factors other than sampling error; this phenomenon is termed item parameter drift. Several methods have been developed to detect drifted items. However, most of the existing methods were designed to detect drifts in individual items, which may not be adequate for test characteristic curve-based linking or equating. One example is the item response theory-based true score equating, whose goal is to generate a conversion table to relate number-correct scores on two forms based on their test characteristic curves. This article introduces a stepwise test characteristic curve method to detect item parameter drift iteratively based on test characteristic curves without needing to set any predetermined critical values. Comparisons are made between the proposed method and two existing methods under the three-parameter logistic item response model through simulation and real data analysis. Results show that the proposed method produces a small difference in test characteristic curves between administrations, an accurate conversion table, and a good classification of drifted and nondrifted items and at the same time keeps a large amount of linking items.

Original languageEnglish (US)
Pages (from-to)280-300
Number of pages21
JournalJournal of Educational Measurement
Volume52
Issue number3
DOIs
StatePublished - Sep 1 2015

Fingerprint

sampling error
Selection Bias
data analysis
logistics
simulation

ASJC Scopus subject areas

  • Education
  • Applied Psychology
  • Developmental and Educational Psychology
  • Psychology (miscellaneous)

Cite this

A Stepwise Test Characteristic Curve Method to Detect Item Parameter Drift. / Guo, Rui; Zheng, Yi; Chang, Hua Hua.

In: Journal of Educational Measurement, Vol. 52, No. 3, 01.09.2015, p. 280-300.

Research output: Contribution to journalArticle

@article{a1dc17f4f27c45099eaee803c51b3870,
title = "A Stepwise Test Characteristic Curve Method to Detect Item Parameter Drift",
abstract = "An important assumption of item response theory is item parameter invariance. Sometimes, however, item parameters are not invariant across different test administrations due to factors other than sampling error; this phenomenon is termed item parameter drift. Several methods have been developed to detect drifted items. However, most of the existing methods were designed to detect drifts in individual items, which may not be adequate for test characteristic curve-based linking or equating. One example is the item response theory-based true score equating, whose goal is to generate a conversion table to relate number-correct scores on two forms based on their test characteristic curves. This article introduces a stepwise test characteristic curve method to detect item parameter drift iteratively based on test characteristic curves without needing to set any predetermined critical values. Comparisons are made between the proposed method and two existing methods under the three-parameter logistic item response model through simulation and real data analysis. Results show that the proposed method produces a small difference in test characteristic curves between administrations, an accurate conversion table, and a good classification of drifted and nondrifted items and at the same time keeps a large amount of linking items.",
author = "Rui Guo and Yi Zheng and Chang, {Hua Hua}",
year = "2015",
month = "9",
day = "1",
doi = "10.1111/jedm.12077",
language = "English (US)",
volume = "52",
pages = "280--300",
journal = "Journal of Educational Measurement",
issn = "0022-0655",
publisher = "Wiley-Blackwell",
number = "3",

}

TY - JOUR

T1 - A Stepwise Test Characteristic Curve Method to Detect Item Parameter Drift

AU - Guo, Rui

AU - Zheng, Yi

AU - Chang, Hua Hua

PY - 2015/9/1

Y1 - 2015/9/1

N2 - An important assumption of item response theory is item parameter invariance. Sometimes, however, item parameters are not invariant across different test administrations due to factors other than sampling error; this phenomenon is termed item parameter drift. Several methods have been developed to detect drifted items. However, most of the existing methods were designed to detect drifts in individual items, which may not be adequate for test characteristic curve-based linking or equating. One example is the item response theory-based true score equating, whose goal is to generate a conversion table to relate number-correct scores on two forms based on their test characteristic curves. This article introduces a stepwise test characteristic curve method to detect item parameter drift iteratively based on test characteristic curves without needing to set any predetermined critical values. Comparisons are made between the proposed method and two existing methods under the three-parameter logistic item response model through simulation and real data analysis. Results show that the proposed method produces a small difference in test characteristic curves between administrations, an accurate conversion table, and a good classification of drifted and nondrifted items and at the same time keeps a large amount of linking items.

AB - An important assumption of item response theory is item parameter invariance. Sometimes, however, item parameters are not invariant across different test administrations due to factors other than sampling error; this phenomenon is termed item parameter drift. Several methods have been developed to detect drifted items. However, most of the existing methods were designed to detect drifts in individual items, which may not be adequate for test characteristic curve-based linking or equating. One example is the item response theory-based true score equating, whose goal is to generate a conversion table to relate number-correct scores on two forms based on their test characteristic curves. This article introduces a stepwise test characteristic curve method to detect item parameter drift iteratively based on test characteristic curves without needing to set any predetermined critical values. Comparisons are made between the proposed method and two existing methods under the three-parameter logistic item response model through simulation and real data analysis. Results show that the proposed method produces a small difference in test characteristic curves between administrations, an accurate conversion table, and a good classification of drifted and nondrifted items and at the same time keeps a large amount of linking items.

UR - http://www.scopus.com/inward/record.url?scp=84940886297&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940886297&partnerID=8YFLogxK

U2 - 10.1111/jedm.12077

DO - 10.1111/jedm.12077

M3 - Article

AN - SCOPUS:84940886297

VL - 52

SP - 280

EP - 300

JO - Journal of Educational Measurement

JF - Journal of Educational Measurement

SN - 0022-0655

IS - 3

ER -