Examinee Judgments of Changes in Item Difficulty: Implications for Item Review in Computerized Adaptive Testing

Steven L. Wise, Sara J. Finney, Craig K. Enders, Sharon A. Freeman, Donald D. Severance

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

We examined the degree to which providing item review on a computerized adaptive test (CAT) could be used by examinees to artificially inflate their scores. Kingsbury (1996) described a strategy in which examinees could use the changes in item difficulty during a CAT to determine which items' answers are incorrect and should be changed during item review. The results of our first 2 studies suggest that examinees are not highly proficient at discriminating item difficulty - a skill needed for a successful application of the Kingsbury strategy. In the third study, we compared the Kingsbury strategy - which examinees would use only for guessed items - to a generalized strategy used for all sequential item pairs. The Kingsbury strategy yielded a small average score gain, whereas the generalized strategy yielded an average score loss. These results suggest that only the Kingsbury strategy is likely to enable examinees to successfully inflate their test scores.

Original languageEnglish (US)
Pages (from-to)185-198
Number of pages14
JournalApplied Measurement in Education
Volume12
Issue number2
StatePublished - 1999
Externally publishedYes

ASJC Scopus subject areas

  • Psychology (miscellaneous)
  • Developmental and Educational Psychology
  • Education

Cite this

Wise, S. L., Finney, S. J., Enders, C. K., Freeman, S. A., & Severance, D. D. (1999). Examinee Judgments of Changes in Item Difficulty: Implications for Item Review in Computerized Adaptive Testing. Applied Measurement in Education, 12(2), 185-198.

Examinee Judgments of Changes in Item Difficulty : Implications for Item Review in Computerized Adaptive Testing. / Wise, Steven L.; Finney, Sara J.; Enders, Craig K.; Freeman, Sharon A.; Severance, Donald D.

In: Applied Measurement in Education, Vol. 12, No. 2, 1999, p. 185-198.

Research output: Contribution to journalArticle

Wise, SL, Finney, SJ, Enders, CK, Freeman, SA & Severance, DD 1999, 'Examinee Judgments of Changes in Item Difficulty: Implications for Item Review in Computerized Adaptive Testing', Applied Measurement in Education, vol. 12, no. 2, pp. 185-198.
Wise, Steven L. ; Finney, Sara J. ; Enders, Craig K. ; Freeman, Sharon A. ; Severance, Donald D. / Examinee Judgments of Changes in Item Difficulty : Implications for Item Review in Computerized Adaptive Testing. In: Applied Measurement in Education. 1999 ; Vol. 12, No. 2. pp. 185-198.
@article{77a560b1aba74d798f8acda5ad429b97,
title = "Examinee Judgments of Changes in Item Difficulty: Implications for Item Review in Computerized Adaptive Testing",
abstract = "We examined the degree to which providing item review on a computerized adaptive test (CAT) could be used by examinees to artificially inflate their scores. Kingsbury (1996) described a strategy in which examinees could use the changes in item difficulty during a CAT to determine which items' answers are incorrect and should be changed during item review. The results of our first 2 studies suggest that examinees are not highly proficient at discriminating item difficulty - a skill needed for a successful application of the Kingsbury strategy. In the third study, we compared the Kingsbury strategy - which examinees would use only for guessed items - to a generalized strategy used for all sequential item pairs. The Kingsbury strategy yielded a small average score gain, whereas the generalized strategy yielded an average score loss. These results suggest that only the Kingsbury strategy is likely to enable examinees to successfully inflate their test scores.",
author = "Wise, {Steven L.} and Finney, {Sara J.} and Enders, {Craig K.} and Freeman, {Sharon A.} and Severance, {Donald D.}",
year = "1999",
language = "English (US)",
volume = "12",
pages = "185--198",
journal = "Applied Measurement in Education",
issn = "0895-7347",
publisher = "Routledge",
number = "2",

}

TY - JOUR

T1 - Examinee Judgments of Changes in Item Difficulty

T2 - Implications for Item Review in Computerized Adaptive Testing

AU - Wise, Steven L.

AU - Finney, Sara J.

AU - Enders, Craig K.

AU - Freeman, Sharon A.

AU - Severance, Donald D.

PY - 1999

Y1 - 1999

N2 - We examined the degree to which providing item review on a computerized adaptive test (CAT) could be used by examinees to artificially inflate their scores. Kingsbury (1996) described a strategy in which examinees could use the changes in item difficulty during a CAT to determine which items' answers are incorrect and should be changed during item review. The results of our first 2 studies suggest that examinees are not highly proficient at discriminating item difficulty - a skill needed for a successful application of the Kingsbury strategy. In the third study, we compared the Kingsbury strategy - which examinees would use only for guessed items - to a generalized strategy used for all sequential item pairs. The Kingsbury strategy yielded a small average score gain, whereas the generalized strategy yielded an average score loss. These results suggest that only the Kingsbury strategy is likely to enable examinees to successfully inflate their test scores.

AB - We examined the degree to which providing item review on a computerized adaptive test (CAT) could be used by examinees to artificially inflate their scores. Kingsbury (1996) described a strategy in which examinees could use the changes in item difficulty during a CAT to determine which items' answers are incorrect and should be changed during item review. The results of our first 2 studies suggest that examinees are not highly proficient at discriminating item difficulty - a skill needed for a successful application of the Kingsbury strategy. In the third study, we compared the Kingsbury strategy - which examinees would use only for guessed items - to a generalized strategy used for all sequential item pairs. The Kingsbury strategy yielded a small average score gain, whereas the generalized strategy yielded an average score loss. These results suggest that only the Kingsbury strategy is likely to enable examinees to successfully inflate their test scores.

UR - http://www.scopus.com/inward/record.url?scp=0033469621&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0033469621&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:0033469621

VL - 12

SP - 185

EP - 198

JO - Applied Measurement in Education

JF - Applied Measurement in Education

SN - 0895-7347

IS - 2

ER -