Explanation of Two Anomalous Results in Statistical Mediation Analysis

Matthew S. Fritz, Aaron B. Taylor, David Mackinnon

Research output: Contribution to journalArticle

138 Citations (Scopus)

Abstract

Previous studies of different methods of testing mediation models have consistently found two anomalous results. The first result is elevated Type I error rates for the bias-corrected and accelerated bias-corrected bootstrap tests not found in nonresampling tests or in resampling tests that did not include a bias correction. This is of special concern as the bias-corrected bootstrap is often recommended and used due to its higher statistical power compared with other tests. The second result is statistical power reaching an asymptote far below 1.0 and in some conditions even declining slightly as the size of the relationship between X and M, a, increased. Two computer simulations were conducted to examine these findings in greater detail. Results from the first simulation found that the increased Type I error rates for the bias-corrected and accelerated bias-corrected bootstrap are a function of an interaction between the size of the individual paths making up the mediated effect and the sample size, such that elevated Type I error rates occur when the sample size is small and the effect size of the nonzero path is medium or larger. Results from the second simulation found that stagnation and decreases in statistical power as a function of the effect size of the a path occurred primarily when the path between M and Y, b, was small. Two empirical mediation examples are provided using data from a steroid prevention and health promotion program aimed at high school football players (Athletes Training and Learning to Avoid Steroids; Goldberg et al., 1996), one to illustrate a possible Type I error for the bias-corrected bootstrap test and a second to illustrate a loss in power related to the size of a. Implications of these findings are discussed.

Original languageEnglish (US)
Pages (from-to)61-87
Number of pages27
JournalMultivariate Behavioral Research
Volume47
Issue number1
DOIs
StatePublished - Feb 2012

Fingerprint

Mediation
Sample Size
Anomalous
Steroids
Statistical Power
Football
Type I Error Rate
Health Promotion
Athletes
Computer Simulation
Bootstrap Test
Path
Effect Size
Learning
Bootstrap
Asymptote
Bias Correction
Type I error
Resampling
Simulation

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Statistics and Probability
  • Arts and Humanities (miscellaneous)

Cite this

Explanation of Two Anomalous Results in Statistical Mediation Analysis. / Fritz, Matthew S.; Taylor, Aaron B.; Mackinnon, David.

In: Multivariate Behavioral Research, Vol. 47, No. 1, 02.2012, p. 61-87.

Research output: Contribution to journalArticle

Fritz, Matthew S. ; Taylor, Aaron B. ; Mackinnon, David. / Explanation of Two Anomalous Results in Statistical Mediation Analysis. In: Multivariate Behavioral Research. 2012 ; Vol. 47, No. 1. pp. 61-87.
@article{5f59731925454c889a6521adfdb3ce4e,
title = "Explanation of Two Anomalous Results in Statistical Mediation Analysis",
abstract = "Previous studies of different methods of testing mediation models have consistently found two anomalous results. The first result is elevated Type I error rates for the bias-corrected and accelerated bias-corrected bootstrap tests not found in nonresampling tests or in resampling tests that did not include a bias correction. This is of special concern as the bias-corrected bootstrap is often recommended and used due to its higher statistical power compared with other tests. The second result is statistical power reaching an asymptote far below 1.0 and in some conditions even declining slightly as the size of the relationship between X and M, a, increased. Two computer simulations were conducted to examine these findings in greater detail. Results from the first simulation found that the increased Type I error rates for the bias-corrected and accelerated bias-corrected bootstrap are a function of an interaction between the size of the individual paths making up the mediated effect and the sample size, such that elevated Type I error rates occur when the sample size is small and the effect size of the nonzero path is medium or larger. Results from the second simulation found that stagnation and decreases in statistical power as a function of the effect size of the a path occurred primarily when the path between M and Y, b, was small. Two empirical mediation examples are provided using data from a steroid prevention and health promotion program aimed at high school football players (Athletes Training and Learning to Avoid Steroids; Goldberg et al., 1996), one to illustrate a possible Type I error for the bias-corrected bootstrap test and a second to illustrate a loss in power related to the size of a. Implications of these findings are discussed.",
author = "Fritz, {Matthew S.} and Taylor, {Aaron B.} and David Mackinnon",
year = "2012",
month = "2",
doi = "10.1080/00273171.2012.640596",
language = "English (US)",
volume = "47",
pages = "61--87",
journal = "Multivariate Behavioral Research",
issn = "0027-3171",
publisher = "Psychology Press Ltd",
number = "1",

}

TY - JOUR

T1 - Explanation of Two Anomalous Results in Statistical Mediation Analysis

AU - Fritz, Matthew S.

AU - Taylor, Aaron B.

AU - Mackinnon, David

PY - 2012/2

Y1 - 2012/2

N2 - Previous studies of different methods of testing mediation models have consistently found two anomalous results. The first result is elevated Type I error rates for the bias-corrected and accelerated bias-corrected bootstrap tests not found in nonresampling tests or in resampling tests that did not include a bias correction. This is of special concern as the bias-corrected bootstrap is often recommended and used due to its higher statistical power compared with other tests. The second result is statistical power reaching an asymptote far below 1.0 and in some conditions even declining slightly as the size of the relationship between X and M, a, increased. Two computer simulations were conducted to examine these findings in greater detail. Results from the first simulation found that the increased Type I error rates for the bias-corrected and accelerated bias-corrected bootstrap are a function of an interaction between the size of the individual paths making up the mediated effect and the sample size, such that elevated Type I error rates occur when the sample size is small and the effect size of the nonzero path is medium or larger. Results from the second simulation found that stagnation and decreases in statistical power as a function of the effect size of the a path occurred primarily when the path between M and Y, b, was small. Two empirical mediation examples are provided using data from a steroid prevention and health promotion program aimed at high school football players (Athletes Training and Learning to Avoid Steroids; Goldberg et al., 1996), one to illustrate a possible Type I error for the bias-corrected bootstrap test and a second to illustrate a loss in power related to the size of a. Implications of these findings are discussed.

AB - Previous studies of different methods of testing mediation models have consistently found two anomalous results. The first result is elevated Type I error rates for the bias-corrected and accelerated bias-corrected bootstrap tests not found in nonresampling tests or in resampling tests that did not include a bias correction. This is of special concern as the bias-corrected bootstrap is often recommended and used due to its higher statistical power compared with other tests. The second result is statistical power reaching an asymptote far below 1.0 and in some conditions even declining slightly as the size of the relationship between X and M, a, increased. Two computer simulations were conducted to examine these findings in greater detail. Results from the first simulation found that the increased Type I error rates for the bias-corrected and accelerated bias-corrected bootstrap are a function of an interaction between the size of the individual paths making up the mediated effect and the sample size, such that elevated Type I error rates occur when the sample size is small and the effect size of the nonzero path is medium or larger. Results from the second simulation found that stagnation and decreases in statistical power as a function of the effect size of the a path occurred primarily when the path between M and Y, b, was small. Two empirical mediation examples are provided using data from a steroid prevention and health promotion program aimed at high school football players (Athletes Training and Learning to Avoid Steroids; Goldberg et al., 1996), one to illustrate a possible Type I error for the bias-corrected bootstrap test and a second to illustrate a loss in power related to the size of a. Implications of these findings are discussed.

UR - http://www.scopus.com/inward/record.url?scp=84857247519&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84857247519&partnerID=8YFLogxK

U2 - 10.1080/00273171.2012.640596

DO - 10.1080/00273171.2012.640596

M3 - Article

AN - SCOPUS:84857247519

VL - 47

SP - 61

EP - 87

JO - Multivariate Behavioral Research

JF - Multivariate Behavioral Research

SN - 0027-3171

IS - 1

ER -