The Thorny Relation Between Measurement Quality and Fit Index Cutoffs in Latent Variable Models

Daniel McNeish, Ji An, Gregory R. Hancock

Research output: Contribution to journalArticle

39 Citations (Scopus)

Abstract

Latent variable modeling is a popular and flexible statistical framework. Concomitant with fitting latent variable models is assessment of how well the theoretical model fits the observed data. Although firm cutoffs for these fit indexes are often cited, recent statistical proofs and simulations have shown that these fit indexes are highly susceptible to measurement quality. For instance, a root mean square error of approximation (RMSEA) value of 0.06 (conventionally thought to indicate good fit) can actually indicate poor fit with poor measurement quality (e.g., standardized factors loadings of around 0.40). Conversely, an RMSEA value of 0.20 (conventionally thought to indicate very poor fit) can indicate acceptable fit with very high measurement quality (standardized factor loadings around 0.90). Despite the wide-ranging effect on applications of latent variable models, the high level of technical detail involved with this phenomenon has curtailed the exposure of these important findings to empirical researchers who are employing these methods. This article briefly reviews these methodological studies in minimal technical detail and provides a demonstration to easily quantify the large influence measurement quality has on fit index values and how greatly the cutoffs would change if they were derived under an alternative level of measurement quality. Recommendations for best practice are also discussed.

Original languageEnglish (US)
Pages (from-to)43-52
Number of pages10
JournalJournal of Personality Assessment
Volume100
Issue number1
DOIs
StatePublished - Jan 2 2018
Externally publishedYes

Fingerprint

Practice Guidelines
Theoretical Models
Research Personnel

ASJC Scopus subject areas

  • Clinical Psychology
  • Psychiatry and Mental health
  • Health, Toxicology and Mutagenesis

Cite this

The Thorny Relation Between Measurement Quality and Fit Index Cutoffs in Latent Variable Models. / McNeish, Daniel; An, Ji; Hancock, Gregory R.

In: Journal of Personality Assessment, Vol. 100, No. 1, 02.01.2018, p. 43-52.

Research output: Contribution to journalArticle

@article{3dd9f5ffbd2e49f6884e8b326c305c3d,
title = "The Thorny Relation Between Measurement Quality and Fit Index Cutoffs in Latent Variable Models",
abstract = "Latent variable modeling is a popular and flexible statistical framework. Concomitant with fitting latent variable models is assessment of how well the theoretical model fits the observed data. Although firm cutoffs for these fit indexes are often cited, recent statistical proofs and simulations have shown that these fit indexes are highly susceptible to measurement quality. For instance, a root mean square error of approximation (RMSEA) value of 0.06 (conventionally thought to indicate good fit) can actually indicate poor fit with poor measurement quality (e.g., standardized factors loadings of around 0.40). Conversely, an RMSEA value of 0.20 (conventionally thought to indicate very poor fit) can indicate acceptable fit with very high measurement quality (standardized factor loadings around 0.90). Despite the wide-ranging effect on applications of latent variable models, the high level of technical detail involved with this phenomenon has curtailed the exposure of these important findings to empirical researchers who are employing these methods. This article briefly reviews these methodological studies in minimal technical detail and provides a demonstration to easily quantify the large influence measurement quality has on fit index values and how greatly the cutoffs would change if they were derived under an alternative level of measurement quality. Recommendations for best practice are also discussed.",
author = "Daniel McNeish and Ji An and Hancock, {Gregory R.}",
year = "2018",
month = "1",
day = "2",
doi = "10.1080/00223891.2017.1281286",
language = "English (US)",
volume = "100",
pages = "43--52",
journal = "Journal of Personality Assessment",
issn = "0022-3891",
publisher = "Taylor and Francis Ltd.",
number = "1",

}

TY - JOUR

T1 - The Thorny Relation Between Measurement Quality and Fit Index Cutoffs in Latent Variable Models

AU - McNeish, Daniel

AU - An, Ji

AU - Hancock, Gregory R.

PY - 2018/1/2

Y1 - 2018/1/2

N2 - Latent variable modeling is a popular and flexible statistical framework. Concomitant with fitting latent variable models is assessment of how well the theoretical model fits the observed data. Although firm cutoffs for these fit indexes are often cited, recent statistical proofs and simulations have shown that these fit indexes are highly susceptible to measurement quality. For instance, a root mean square error of approximation (RMSEA) value of 0.06 (conventionally thought to indicate good fit) can actually indicate poor fit with poor measurement quality (e.g., standardized factors loadings of around 0.40). Conversely, an RMSEA value of 0.20 (conventionally thought to indicate very poor fit) can indicate acceptable fit with very high measurement quality (standardized factor loadings around 0.90). Despite the wide-ranging effect on applications of latent variable models, the high level of technical detail involved with this phenomenon has curtailed the exposure of these important findings to empirical researchers who are employing these methods. This article briefly reviews these methodological studies in minimal technical detail and provides a demonstration to easily quantify the large influence measurement quality has on fit index values and how greatly the cutoffs would change if they were derived under an alternative level of measurement quality. Recommendations for best practice are also discussed.

AB - Latent variable modeling is a popular and flexible statistical framework. Concomitant with fitting latent variable models is assessment of how well the theoretical model fits the observed data. Although firm cutoffs for these fit indexes are often cited, recent statistical proofs and simulations have shown that these fit indexes are highly susceptible to measurement quality. For instance, a root mean square error of approximation (RMSEA) value of 0.06 (conventionally thought to indicate good fit) can actually indicate poor fit with poor measurement quality (e.g., standardized factors loadings of around 0.40). Conversely, an RMSEA value of 0.20 (conventionally thought to indicate very poor fit) can indicate acceptable fit with very high measurement quality (standardized factor loadings around 0.90). Despite the wide-ranging effect on applications of latent variable models, the high level of technical detail involved with this phenomenon has curtailed the exposure of these important findings to empirical researchers who are employing these methods. This article briefly reviews these methodological studies in minimal technical detail and provides a demonstration to easily quantify the large influence measurement quality has on fit index values and how greatly the cutoffs would change if they were derived under an alternative level of measurement quality. Recommendations for best practice are also discussed.

UR - http://www.scopus.com/inward/record.url?scp=85014574396&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85014574396&partnerID=8YFLogxK

U2 - 10.1080/00223891.2017.1281286

DO - 10.1080/00223891.2017.1281286

M3 - Article

C2 - 28631976

AN - SCOPUS:85014574396

VL - 100

SP - 43

EP - 52

JO - Journal of Personality Assessment

JF - Journal of Personality Assessment

SN - 0022-3891

IS - 1

ER -