Evaluation of structural equation modeling estimates of reliability for scales with ordered categorical items

Yanyun Yang, Samuel B. Green

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

Different methods are reviewed for estimating reliability based on a single administration of a test consisting of ordered categorical items. The nonlinear SEM reliability coefficient appears more flexible than other methods because it is designed to assess the reliability of summed scores for ordered categorical items through the modeling of the item relationships. However, this method assumes that the variables underlying the ordered categorical items are normally distributed. We evaluated nonlinear and linear SEM reliability coefficients using a Monte Carlo study by manipulating factor structure, distribution of the continuous items, number of ordered categories, distribution of the ordered categorical items, and sample size. The nonlinear SEM reliability coefficient performed as well as or better than the linear SEM reliability coefficient in most conditions and was relatively robust to modest violations of the normality assumption. The nonlinear SEM reliability coefficient was substantially biased when categorical item distributions were extremely skewed and the skewness differed markedly across items. Bias in these coefficients may be diagnosable in practice by assessing model fit.

Original languageEnglish (US)
Pages (from-to)23-34
Number of pages12
JournalMethodology
Volume11
Issue number1
DOIs
StatePublished - 2015

Fingerprint

evaluation
Sample Size
distribution structure
normality
trend

Keywords

  • Nonnormality
  • Reliability
  • Robustness
  • Structural equation modeling

ASJC Scopus subject areas

  • Psychology(all)
  • Social Sciences(all)

Cite this

Evaluation of structural equation modeling estimates of reliability for scales with ordered categorical items. / Yang, Yanyun; Green, Samuel B.

In: Methodology, Vol. 11, No. 1, 2015, p. 23-34.

Research output: Contribution to journalArticle

@article{35ccb1f479ba441e917bd080cab0e971,
title = "Evaluation of structural equation modeling estimates of reliability for scales with ordered categorical items",
abstract = "Different methods are reviewed for estimating reliability based on a single administration of a test consisting of ordered categorical items. The nonlinear SEM reliability coefficient appears more flexible than other methods because it is designed to assess the reliability of summed scores for ordered categorical items through the modeling of the item relationships. However, this method assumes that the variables underlying the ordered categorical items are normally distributed. We evaluated nonlinear and linear SEM reliability coefficients using a Monte Carlo study by manipulating factor structure, distribution of the continuous items, number of ordered categories, distribution of the ordered categorical items, and sample size. The nonlinear SEM reliability coefficient performed as well as or better than the linear SEM reliability coefficient in most conditions and was relatively robust to modest violations of the normality assumption. The nonlinear SEM reliability coefficient was substantially biased when categorical item distributions were extremely skewed and the skewness differed markedly across items. Bias in these coefficients may be diagnosable in practice by assessing model fit.",
keywords = "Nonnormality, Reliability, Robustness, Structural equation modeling",
author = "Yanyun Yang and Green, {Samuel B.}",
year = "2015",
doi = "10.1027/1614-2241/a000087",
language = "English (US)",
volume = "11",
pages = "23--34",
journal = "Methodology",
issn = "1614-1881",
publisher = "Hogrefe Publishing",
number = "1",

}

TY - JOUR

T1 - Evaluation of structural equation modeling estimates of reliability for scales with ordered categorical items

AU - Yang, Yanyun

AU - Green, Samuel B.

PY - 2015

Y1 - 2015

N2 - Different methods are reviewed for estimating reliability based on a single administration of a test consisting of ordered categorical items. The nonlinear SEM reliability coefficient appears more flexible than other methods because it is designed to assess the reliability of summed scores for ordered categorical items through the modeling of the item relationships. However, this method assumes that the variables underlying the ordered categorical items are normally distributed. We evaluated nonlinear and linear SEM reliability coefficients using a Monte Carlo study by manipulating factor structure, distribution of the continuous items, number of ordered categories, distribution of the ordered categorical items, and sample size. The nonlinear SEM reliability coefficient performed as well as or better than the linear SEM reliability coefficient in most conditions and was relatively robust to modest violations of the normality assumption. The nonlinear SEM reliability coefficient was substantially biased when categorical item distributions were extremely skewed and the skewness differed markedly across items. Bias in these coefficients may be diagnosable in practice by assessing model fit.

AB - Different methods are reviewed for estimating reliability based on a single administration of a test consisting of ordered categorical items. The nonlinear SEM reliability coefficient appears more flexible than other methods because it is designed to assess the reliability of summed scores for ordered categorical items through the modeling of the item relationships. However, this method assumes that the variables underlying the ordered categorical items are normally distributed. We evaluated nonlinear and linear SEM reliability coefficients using a Monte Carlo study by manipulating factor structure, distribution of the continuous items, number of ordered categories, distribution of the ordered categorical items, and sample size. The nonlinear SEM reliability coefficient performed as well as or better than the linear SEM reliability coefficient in most conditions and was relatively robust to modest violations of the normality assumption. The nonlinear SEM reliability coefficient was substantially biased when categorical item distributions were extremely skewed and the skewness differed markedly across items. Bias in these coefficients may be diagnosable in practice by assessing model fit.

KW - Nonnormality

KW - Reliability

KW - Robustness

KW - Structural equation modeling

UR - http://www.scopus.com/inward/record.url?scp=84924616760&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84924616760&partnerID=8YFLogxK

U2 - 10.1027/1614-2241/a000087

DO - 10.1027/1614-2241/a000087

M3 - Article

VL - 11

SP - 23

EP - 34

JO - Methodology

JF - Methodology

SN - 1614-1881

IS - 1

ER -