TY - JOUR
T1 - Challenging Conventional Wisdom for Multivariate Statistical Models With Small Samples
AU - McNeish, Daniel
N1 - Publisher Copyright:
© 2017, © 2017 AERA.
PY - 2017/12/1
Y1 - 2017/12/1
N2 - In education research, small samples are common because of financial limitations, logistical challenges, or exploratory studies. With small samples, statistical principles on which researchers rely do not hold, leading to trust issues with model estimates and possible replication issues when scaling up. Researchers are generally aware of such pitfalls and often attempt to tailor their analyses to accommodate small samples. Despite well-intentioned efforts, conventional statistical axioms do not always translate to best practice with small samples, and common recommendations can, counterintuitively, exacerbate small sample problems. In this article, we overview the landscape of small sample techniques and note why conventionally recommended approaches can fail with small samples while also suggesting lesser known alternatives that tend to perform better in statistical research but are not widely adopted in education research. Topics include bootstrapping, latent variable model fit, and Bayesian methods for multilevel, latent variable, and growth models. Simulated and real-data examples are interspersed throughout.
AB - In education research, small samples are common because of financial limitations, logistical challenges, or exploratory studies. With small samples, statistical principles on which researchers rely do not hold, leading to trust issues with model estimates and possible replication issues when scaling up. Researchers are generally aware of such pitfalls and often attempt to tailor their analyses to accommodate small samples. Despite well-intentioned efforts, conventional statistical axioms do not always translate to best practice with small samples, and common recommendations can, counterintuitively, exacerbate small sample problems. In this article, we overview the landscape of small sample techniques and note why conventionally recommended approaches can fail with small samples while also suggesting lesser known alternatives that tend to perform better in statistical research but are not widely adopted in education research. Topics include bootstrapping, latent variable model fit, and Bayesian methods for multilevel, latent variable, and growth models. Simulated and real-data examples are interspersed throughout.
KW - Bayes
KW - growth model
KW - model fit
KW - multilevel model
KW - small sample
UR - http://www.scopus.com/inward/record.url?scp=85032937357&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85032937357&partnerID=8YFLogxK
U2 - 10.3102/0034654317727727
DO - 10.3102/0034654317727727
M3 - Article
AN - SCOPUS:85032937357
SN - 0034-6543
VL - 87
SP - 1117
EP - 1151
JO - Review of Educational Research
JF - Review of Educational Research
IS - 6
ER -