Small Sample Methods for Multilevel Modeling: A Colloquial Elucidation of REML and the Kenward-Roger Correction

Research output: Contribution to journalArticle

26 Citations (Scopus)

Abstract

Studies on small sample properties of multilevel models have become increasingly prominent in the methodological literature in response to the frequency with which small sample data appear in empirical studies. Simulation results generally recommend that empirical researchers employ restricted maximum likelihood estimation (REML) with a Kenward-Roger correction with small samples in frequentist contexts to minimize small sample bias in estimation and to prevent inflation of Type-I error rates. However, simulation studies focus on recommendations for best practice, and there is little to no explanation of why traditional maximum likelihood (ML) breaks down with smaller samples, what differentiates REML from ML, or how the Kenward-Roger correction remedies lingering small sample issues. Due to the complexity of these methods, most extant descriptions are highly mathematical and are intended to prove that the methods improve small sample performance as intended. Thus, empirical researchers have documentation that these methods are advantageous but still lack resources to help understand what the methods actually do and why they are needed. This tutorial explains why ML falters with small samples, how REML circumvents some issues, and how Kenward-Roger works. We do so without equations or derivations to support more widespread understanding and use of these valuable methods.

Original languageEnglish (US)
Pages (from-to)661-670
Number of pages10
JournalMultivariate Behavioral Research
Volume52
Issue number5
DOIs
StatePublished - Sep 3 2017

Fingerprint

Restricted Estimation
Multilevel Modeling
Restricted Maximum Likelihood
Maximum Likelihood Estimation
Small Sample
Maximum Likelihood
Research Personnel
Economic Inflation
Practice Guidelines
Documentation
Multilevel Models
Type I Error Rate
Elucidation
Best Practice
Differentiate
Inflation
Empirical Study
Breakdown
Recommendations
Simulation Study

Keywords

  • explanation
  • Kenward-Roger
  • mixed model
  • Restricted maximum likelihood
  • tutorial

ASJC Scopus subject areas

  • Statistics and Probability
  • Experimental and Cognitive Psychology
  • Arts and Humanities (miscellaneous)

Cite this

@article{33bf9f3b38f242cc9aa253e8156d1abb,
title = "Small Sample Methods for Multilevel Modeling: A Colloquial Elucidation of REML and the Kenward-Roger Correction",
abstract = "Studies on small sample properties of multilevel models have become increasingly prominent in the methodological literature in response to the frequency with which small sample data appear in empirical studies. Simulation results generally recommend that empirical researchers employ restricted maximum likelihood estimation (REML) with a Kenward-Roger correction with small samples in frequentist contexts to minimize small sample bias in estimation and to prevent inflation of Type-I error rates. However, simulation studies focus on recommendations for best practice, and there is little to no explanation of why traditional maximum likelihood (ML) breaks down with smaller samples, what differentiates REML from ML, or how the Kenward-Roger correction remedies lingering small sample issues. Due to the complexity of these methods, most extant descriptions are highly mathematical and are intended to prove that the methods improve small sample performance as intended. Thus, empirical researchers have documentation that these methods are advantageous but still lack resources to help understand what the methods actually do and why they are needed. This tutorial explains why ML falters with small samples, how REML circumvents some issues, and how Kenward-Roger works. We do so without equations or derivations to support more widespread understanding and use of these valuable methods.",
keywords = "explanation, Kenward-Roger, mixed model, Restricted maximum likelihood, tutorial",
author = "Daniel McNeish",
year = "2017",
month = "9",
day = "3",
doi = "10.1080/00273171.2017.1344538",
language = "English (US)",
volume = "52",
pages = "661--670",
journal = "Multivariate Behavioral Research",
issn = "0027-3171",
publisher = "Psychology Press Ltd",
number = "5",

}

TY - JOUR

T1 - Small Sample Methods for Multilevel Modeling

T2 - A Colloquial Elucidation of REML and the Kenward-Roger Correction

AU - McNeish, Daniel

PY - 2017/9/3

Y1 - 2017/9/3

N2 - Studies on small sample properties of multilevel models have become increasingly prominent in the methodological literature in response to the frequency with which small sample data appear in empirical studies. Simulation results generally recommend that empirical researchers employ restricted maximum likelihood estimation (REML) with a Kenward-Roger correction with small samples in frequentist contexts to minimize small sample bias in estimation and to prevent inflation of Type-I error rates. However, simulation studies focus on recommendations for best practice, and there is little to no explanation of why traditional maximum likelihood (ML) breaks down with smaller samples, what differentiates REML from ML, or how the Kenward-Roger correction remedies lingering small sample issues. Due to the complexity of these methods, most extant descriptions are highly mathematical and are intended to prove that the methods improve small sample performance as intended. Thus, empirical researchers have documentation that these methods are advantageous but still lack resources to help understand what the methods actually do and why they are needed. This tutorial explains why ML falters with small samples, how REML circumvents some issues, and how Kenward-Roger works. We do so without equations or derivations to support more widespread understanding and use of these valuable methods.

AB - Studies on small sample properties of multilevel models have become increasingly prominent in the methodological literature in response to the frequency with which small sample data appear in empirical studies. Simulation results generally recommend that empirical researchers employ restricted maximum likelihood estimation (REML) with a Kenward-Roger correction with small samples in frequentist contexts to minimize small sample bias in estimation and to prevent inflation of Type-I error rates. However, simulation studies focus on recommendations for best practice, and there is little to no explanation of why traditional maximum likelihood (ML) breaks down with smaller samples, what differentiates REML from ML, or how the Kenward-Roger correction remedies lingering small sample issues. Due to the complexity of these methods, most extant descriptions are highly mathematical and are intended to prove that the methods improve small sample performance as intended. Thus, empirical researchers have documentation that these methods are advantageous but still lack resources to help understand what the methods actually do and why they are needed. This tutorial explains why ML falters with small samples, how REML circumvents some issues, and how Kenward-Roger works. We do so without equations or derivations to support more widespread understanding and use of these valuable methods.

KW - explanation

KW - Kenward-Roger

KW - mixed model

KW - Restricted maximum likelihood

KW - tutorial

UR - http://www.scopus.com/inward/record.url?scp=85024407861&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85024407861&partnerID=8YFLogxK

U2 - 10.1080/00273171.2017.1344538

DO - 10.1080/00273171.2017.1344538

M3 - Article

C2 - 28715244

AN - SCOPUS:85024407861

VL - 52

SP - 661

EP - 670

JO - Multivariate Behavioral Research

JF - Multivariate Behavioral Research

SN - 0027-3171

IS - 5

ER -