Should We Use F-Tests for Model Fit Instead of Chi-Square in Overidentified Structural Equation Models?

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Debate continues about whether the likelihood ratio test (TML) or goodness-of-fit indices are most appropriate for assessing data-model fit in structural equation models. Though potential advantages and disadvantages of these methods with large samples are often discussed, shortcomings concomitant with smaller samples are not. This article aims to (a) highlight the broader small sample issues with both approaches to data-model fit assessment, (b) note that what constitutes a small sample is common in empirical studies (approximately 20% to 50% in review studies, depending on the definition of “small”), and (c) more widely introduce F-tests as a desirable alternative than the traditional TML tests, small-sample corrections, or goodness-of-fit indices with smaller samples. Both goodness-of-fit indices and comparing TML to a chi-square distribution at smaller samples leads to overrejection of well-fitting models. Simulations and example analyses show that F-tests yield more desirable statistical properties—with or without normality—than standard approaches like chi-square tests or goodness-of-fit indices with smaller samples, roughly defined as N < 200 or N: df < 3.

Original languageEnglish (US)
JournalOrganizational Research Methods
DOIs
StateAccepted/In press - Jan 1 2018

Fingerprint

Data structures
Structural equation model
Small sample
Goodness of fit

Keywords

  • factor analysis
  • measurement models
  • multivariate analysis
  • quantitative research
  • structural equation modeling

ASJC Scopus subject areas

  • Decision Sciences(all)
  • Strategy and Management
  • Management of Technology and Innovation

Cite this

@article{8c716156105f4e3dae6399dd7f798a7f,
title = "Should We Use F-Tests for Model Fit Instead of Chi-Square in Overidentified Structural Equation Models?",
abstract = "Debate continues about whether the likelihood ratio test (TML) or goodness-of-fit indices are most appropriate for assessing data-model fit in structural equation models. Though potential advantages and disadvantages of these methods with large samples are often discussed, shortcomings concomitant with smaller samples are not. This article aims to (a) highlight the broader small sample issues with both approaches to data-model fit assessment, (b) note that what constitutes a small sample is common in empirical studies (approximately 20{\%} to 50{\%} in review studies, depending on the definition of “small”), and (c) more widely introduce F-tests as a desirable alternative than the traditional TML tests, small-sample corrections, or goodness-of-fit indices with smaller samples. Both goodness-of-fit indices and comparing TML to a chi-square distribution at smaller samples leads to overrejection of well-fitting models. Simulations and example analyses show that F-tests yield more desirable statistical properties—with or without normality—than standard approaches like chi-square tests or goodness-of-fit indices with smaller samples, roughly defined as N < 200 or N: df < 3.",
keywords = "factor analysis, measurement models, multivariate analysis, quantitative research, structural equation modeling",
author = "Daniel McNeish",
year = "2018",
month = "1",
day = "1",
doi = "10.1177/1094428118809495",
language = "English (US)",
journal = "Organizational Research Methods",
issn = "1094-4281",
publisher = "SAGE Publications Inc.",

}

TY - JOUR

T1 - Should We Use F-Tests for Model Fit Instead of Chi-Square in Overidentified Structural Equation Models?

AU - McNeish, Daniel

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Debate continues about whether the likelihood ratio test (TML) or goodness-of-fit indices are most appropriate for assessing data-model fit in structural equation models. Though potential advantages and disadvantages of these methods with large samples are often discussed, shortcomings concomitant with smaller samples are not. This article aims to (a) highlight the broader small sample issues with both approaches to data-model fit assessment, (b) note that what constitutes a small sample is common in empirical studies (approximately 20% to 50% in review studies, depending on the definition of “small”), and (c) more widely introduce F-tests as a desirable alternative than the traditional TML tests, small-sample corrections, or goodness-of-fit indices with smaller samples. Both goodness-of-fit indices and comparing TML to a chi-square distribution at smaller samples leads to overrejection of well-fitting models. Simulations and example analyses show that F-tests yield more desirable statistical properties—with or without normality—than standard approaches like chi-square tests or goodness-of-fit indices with smaller samples, roughly defined as N < 200 or N: df < 3.

AB - Debate continues about whether the likelihood ratio test (TML) or goodness-of-fit indices are most appropriate for assessing data-model fit in structural equation models. Though potential advantages and disadvantages of these methods with large samples are often discussed, shortcomings concomitant with smaller samples are not. This article aims to (a) highlight the broader small sample issues with both approaches to data-model fit assessment, (b) note that what constitutes a small sample is common in empirical studies (approximately 20% to 50% in review studies, depending on the definition of “small”), and (c) more widely introduce F-tests as a desirable alternative than the traditional TML tests, small-sample corrections, or goodness-of-fit indices with smaller samples. Both goodness-of-fit indices and comparing TML to a chi-square distribution at smaller samples leads to overrejection of well-fitting models. Simulations and example analyses show that F-tests yield more desirable statistical properties—with or without normality—than standard approaches like chi-square tests or goodness-of-fit indices with smaller samples, roughly defined as N < 200 or N: df < 3.

KW - factor analysis

KW - measurement models

KW - multivariate analysis

KW - quantitative research

KW - structural equation modeling

UR - http://www.scopus.com/inward/record.url?scp=85058969914&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058969914&partnerID=8YFLogxK

U2 - 10.1177/1094428118809495

DO - 10.1177/1094428118809495

M3 - Article

AN - SCOPUS:85058969914

JO - Organizational Research Methods

JF - Organizational Research Methods

SN - 1094-4281

ER -