Dimensionality in Compensatory MIRT When Complex Structure Exists: Evaluation of DETECT and NOHARM

Dubravka Svetina, Roy Levy

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

This study investigated the effect of complex structure on dimensionality assessment in compensatory multidimensional item response models using DETECT- and NOHARM-based methods. The performance was evaluated via the accuracy of identifying the correct number of dimensions and the ability to accurately recover item groupings using a simple matching similarity (SM) coefficient. The DETECT-based methods yielded higher proportion correct than the NOHARM-based methods in two- and three-dimensional conditions, especially when correlations were ≤.60, data exhibited ≤30% complexity, and sample size was 1,000. As the complexity increased and the sample size decreased, the performance of the methods typically diminished. The NOHARM-based methods were either equally successful or better in recovering item groupings than the DETECT-based methods and were mostly affected by complexity levels. The DETECT-based methods were affected largely by the test length, such that with the increase of the number of items, SM coefficients would decrease substantially.

Original languageEnglish (US)
Pages (from-to)398-420
Number of pages23
JournalJournal of Experimental Education
Volume84
Issue number2
DOIs
StatePublished - Apr 2 2016

Fingerprint

evaluation
Sample Size
grouping
performance
ability

Keywords

  • complex structure
  • dimensionality assessment
  • factor analysis
  • item response theory
  • simulation studies

ASJC Scopus subject areas

  • Education
  • Developmental and Educational Psychology

Cite this

Dimensionality in Compensatory MIRT When Complex Structure Exists : Evaluation of DETECT and NOHARM. / Svetina, Dubravka; Levy, Roy.

In: Journal of Experimental Education, Vol. 84, No. 2, 02.04.2016, p. 398-420.

Research output: Contribution to journalArticle

@article{49f8002b3e1f4ca4803f8c28a2c39759,
title = "Dimensionality in Compensatory MIRT When Complex Structure Exists: Evaluation of DETECT and NOHARM",
abstract = "This study investigated the effect of complex structure on dimensionality assessment in compensatory multidimensional item response models using DETECT- and NOHARM-based methods. The performance was evaluated via the accuracy of identifying the correct number of dimensions and the ability to accurately recover item groupings using a simple matching similarity (SM) coefficient. The DETECT-based methods yielded higher proportion correct than the NOHARM-based methods in two- and three-dimensional conditions, especially when correlations were ≤.60, data exhibited ≤30{\%} complexity, and sample size was 1,000. As the complexity increased and the sample size decreased, the performance of the methods typically diminished. The NOHARM-based methods were either equally successful or better in recovering item groupings than the DETECT-based methods and were mostly affected by complexity levels. The DETECT-based methods were affected largely by the test length, such that with the increase of the number of items, SM coefficients would decrease substantially.",
keywords = "complex structure, dimensionality assessment, factor analysis, item response theory, simulation studies",
author = "Dubravka Svetina and Roy Levy",
year = "2016",
month = "4",
day = "2",
doi = "10.1080/00220973.2015.1048845",
language = "English (US)",
volume = "84",
pages = "398--420",
journal = "Journal of Experimental Education",
issn = "0022-0973",
publisher = "Routledge",
number = "2",

}

TY - JOUR

T1 - Dimensionality in Compensatory MIRT When Complex Structure Exists

T2 - Evaluation of DETECT and NOHARM

AU - Svetina, Dubravka

AU - Levy, Roy

PY - 2016/4/2

Y1 - 2016/4/2

N2 - This study investigated the effect of complex structure on dimensionality assessment in compensatory multidimensional item response models using DETECT- and NOHARM-based methods. The performance was evaluated via the accuracy of identifying the correct number of dimensions and the ability to accurately recover item groupings using a simple matching similarity (SM) coefficient. The DETECT-based methods yielded higher proportion correct than the NOHARM-based methods in two- and three-dimensional conditions, especially when correlations were ≤.60, data exhibited ≤30% complexity, and sample size was 1,000. As the complexity increased and the sample size decreased, the performance of the methods typically diminished. The NOHARM-based methods were either equally successful or better in recovering item groupings than the DETECT-based methods and were mostly affected by complexity levels. The DETECT-based methods were affected largely by the test length, such that with the increase of the number of items, SM coefficients would decrease substantially.

AB - This study investigated the effect of complex structure on dimensionality assessment in compensatory multidimensional item response models using DETECT- and NOHARM-based methods. The performance was evaluated via the accuracy of identifying the correct number of dimensions and the ability to accurately recover item groupings using a simple matching similarity (SM) coefficient. The DETECT-based methods yielded higher proportion correct than the NOHARM-based methods in two- and three-dimensional conditions, especially when correlations were ≤.60, data exhibited ≤30% complexity, and sample size was 1,000. As the complexity increased and the sample size decreased, the performance of the methods typically diminished. The NOHARM-based methods were either equally successful or better in recovering item groupings than the DETECT-based methods and were mostly affected by complexity levels. The DETECT-based methods were affected largely by the test length, such that with the increase of the number of items, SM coefficients would decrease substantially.

KW - complex structure

KW - dimensionality assessment

KW - factor analysis

KW - item response theory

KW - simulation studies

UR - http://www.scopus.com/inward/record.url?scp=84954026382&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84954026382&partnerID=8YFLogxK

U2 - 10.1080/00220973.2015.1048845

DO - 10.1080/00220973.2015.1048845

M3 - Article

AN - SCOPUS:84954026382

VL - 84

SP - 398

EP - 420

JO - Journal of Experimental Education

JF - Journal of Experimental Education

SN - 0022-0973

IS - 2

ER -