Evaluating evidence for invariant items: A Bayes factor applied to testing measurement invariance in IRT models

Josine Verhagen, Roy Levy, Roger E. Millsap, Jean Paul Fox

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

When comparing test or questionnaire scores between groups, an important assumption is that the questionnaire or test items are measurement invariant: that they measure the underlying construct in the same way in each group. The main goal of tests for measurement invariance is to establish whether support exists for the null hypothesis of invariance. Bayesian hypothesis testing enables researchers to investigate this null hypothesis, where evidence in favor of invariance is quantified using the Bayes factor. A Bayes factor for the investigation of measurement invariance assumptions of test items for randomly selected groups was developed by Verhagen and Fox (2013a). For specific groups or measurement occasions, a different Bayes factor test is proposed here, which directly evaluates item parameter differences between groups. This test is compared to recently developed frequentist measurement invariance tests based on the Wald test in a simulation study. The close-comparison with the Wald-test performance validates the proposed Bayes factor and shows the advantages of the additional information given by the Bayes factor. Both tests are applied to the investigation of measurement invariance of a geometry test (CBASE) to illustrate the use of the Bayes factor test for measurement invariance.

Original languageEnglish (US)
JournalJournal of Mathematical Psychology
DOIs
StateAccepted/In press - 2015

Fingerprint

Measurement Invariance
Bayes Factor
Invariance
Testing
Invariant
Research Personnel
Wald Test
Model
Null hypothesis
Questionnaire
Evidence
Surveys and Questionnaires
Hypothesis Testing
Geometry

Keywords

  • Bayes factor
  • Bayesian hypothesis testing
  • Bayesian modeling
  • IRT
  • Latent variable models
  • Measurement invariance

ASJC Scopus subject areas

  • Psychology(all)
  • Applied Mathematics

Cite this

Evaluating evidence for invariant items : A Bayes factor applied to testing measurement invariance in IRT models. / Verhagen, Josine; Levy, Roy; Millsap, Roger E.; Fox, Jean Paul.

In: Journal of Mathematical Psychology, 2015.

Research output: Contribution to journalArticle

@article{454fb050b18745f2b901e86a9fed1a76,
title = "Evaluating evidence for invariant items: A Bayes factor applied to testing measurement invariance in IRT models",
abstract = "When comparing test or questionnaire scores between groups, an important assumption is that the questionnaire or test items are measurement invariant: that they measure the underlying construct in the same way in each group. The main goal of tests for measurement invariance is to establish whether support exists for the null hypothesis of invariance. Bayesian hypothesis testing enables researchers to investigate this null hypothesis, where evidence in favor of invariance is quantified using the Bayes factor. A Bayes factor for the investigation of measurement invariance assumptions of test items for randomly selected groups was developed by Verhagen and Fox (2013a). For specific groups or measurement occasions, a different Bayes factor test is proposed here, which directly evaluates item parameter differences between groups. This test is compared to recently developed frequentist measurement invariance tests based on the Wald test in a simulation study. The close-comparison with the Wald-test performance validates the proposed Bayes factor and shows the advantages of the additional information given by the Bayes factor. Both tests are applied to the investigation of measurement invariance of a geometry test (CBASE) to illustrate the use of the Bayes factor test for measurement invariance.",
keywords = "Bayes factor, Bayesian hypothesis testing, Bayesian modeling, IRT, Latent variable models, Measurement invariance",
author = "Josine Verhagen and Roy Levy and Millsap, {Roger E.} and Fox, {Jean Paul}",
year = "2015",
doi = "10.1016/j.jmp.2015.06.005",
language = "English (US)",
journal = "Journal of Mathematical Psychology",
issn = "0022-2496",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Evaluating evidence for invariant items

T2 - A Bayes factor applied to testing measurement invariance in IRT models

AU - Verhagen, Josine

AU - Levy, Roy

AU - Millsap, Roger E.

AU - Fox, Jean Paul

PY - 2015

Y1 - 2015

N2 - When comparing test or questionnaire scores between groups, an important assumption is that the questionnaire or test items are measurement invariant: that they measure the underlying construct in the same way in each group. The main goal of tests for measurement invariance is to establish whether support exists for the null hypothesis of invariance. Bayesian hypothesis testing enables researchers to investigate this null hypothesis, where evidence in favor of invariance is quantified using the Bayes factor. A Bayes factor for the investigation of measurement invariance assumptions of test items for randomly selected groups was developed by Verhagen and Fox (2013a). For specific groups or measurement occasions, a different Bayes factor test is proposed here, which directly evaluates item parameter differences between groups. This test is compared to recently developed frequentist measurement invariance tests based on the Wald test in a simulation study. The close-comparison with the Wald-test performance validates the proposed Bayes factor and shows the advantages of the additional information given by the Bayes factor. Both tests are applied to the investigation of measurement invariance of a geometry test (CBASE) to illustrate the use of the Bayes factor test for measurement invariance.

AB - When comparing test or questionnaire scores between groups, an important assumption is that the questionnaire or test items are measurement invariant: that they measure the underlying construct in the same way in each group. The main goal of tests for measurement invariance is to establish whether support exists for the null hypothesis of invariance. Bayesian hypothesis testing enables researchers to investigate this null hypothesis, where evidence in favor of invariance is quantified using the Bayes factor. A Bayes factor for the investigation of measurement invariance assumptions of test items for randomly selected groups was developed by Verhagen and Fox (2013a). For specific groups or measurement occasions, a different Bayes factor test is proposed here, which directly evaluates item parameter differences between groups. This test is compared to recently developed frequentist measurement invariance tests based on the Wald test in a simulation study. The close-comparison with the Wald-test performance validates the proposed Bayes factor and shows the advantages of the additional information given by the Bayes factor. Both tests are applied to the investigation of measurement invariance of a geometry test (CBASE) to illustrate the use of the Bayes factor test for measurement invariance.

KW - Bayes factor

KW - Bayesian hypothesis testing

KW - Bayesian modeling

KW - IRT

KW - Latent variable models

KW - Measurement invariance

UR - http://www.scopus.com/inward/record.url?scp=84939530364&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84939530364&partnerID=8YFLogxK

U2 - 10.1016/j.jmp.2015.06.005

DO - 10.1016/j.jmp.2015.06.005

M3 - Article

AN - SCOPUS:84939530364

JO - Journal of Mathematical Psychology

JF - Journal of Mathematical Psychology

SN - 0022-2496

ER -