Sample-Size Planning for More Accurate Statistical Power: A Method Adjusting Sample Effect Sizes for Publication Bias and Uncertainty

Samantha F. Anderson, Ken Kelley, Scott E. Maxwell

Research output: Contribution to journalArticlepeer-review

90 Scopus citations

Abstract

The sample size necessary to obtain a desired level of statistical power depends in part on the population value of the effect size, which is, by definition, unknown. A common approach to sample-size planning uses the sample effect size from a prior study as an estimate of the population value of the effect to be detected in the future study. Although this strategy is intuitively appealing, effect-size estimates, taken at face value, are typically not accurate estimates of the population effect size because of publication bias and uncertainty. We show that the use of this approach often results in underpowered studies, sometimes to an alarming degree. We present an alternative approach that adjusts sample effect sizes for bias and uncertainty, and we demonstrate its effectiveness for several experimental designs. Furthermore, we discuss an open-source R package, BUCSS, and user-friendly Web applications that we have made available to researchers so that they can easily implement our suggested methods.

Original languageEnglish (US)
Pages (from-to)1547-1562
Number of pages16
JournalPsychological Science
Volume28
Issue number11
DOIs
StatePublished - Nov 1 2017
Externally publishedYes

Keywords

  • effect size
  • methodology
  • publication bias
  • sample size
  • statistical power

ASJC Scopus subject areas

  • Psychology(all)

Fingerprint Dive into the research topics of 'Sample-Size Planning for More Accurate Statistical Power: A Method Adjusting Sample Effect Sizes for Publication Bias and Uncertainty'. Together they form a unique fingerprint.

Cite this