Sample Size Planning for Replication Studies: The Devil Is in the Design

Research output: Contribution to journalArticlepeer-review

Abstract

Replication is central to scientific progress. Because of widely reported replication failures, replication has received increased attention in psychology, sociology, education, management, and related fields in recent years. Replication studies have generally been assessed dichotomously, designated either a “success” or “failure” based entirely on the outcome of a null hypothesis significance test (i.e., p <.05 or p >.05, respectively). However, alternative definitions of success depend on researchers’ goals for the replication. Previous work on alternative definitions for success has focused on the analysis phase of replication. However, the design of the replication is also important, as emphasized with the adage, “an ounce of prevention is better than a pound of cure.” One critical component of design often ignored or oversimplified in replication studies is sample size planning, indeed, the details here are crucial. Sample size planning for replication studies should correspond to the method by which success will be evaluated. Researchers have received little guidance, some of which is misguided, on sample size planning for replication goals other than the aforementioned dichotomous null hypothesis significance testing approach. In this article, we describe four different replication goals. Then, we formalize sample size planning methods for each of the four goals.

Original languageEnglish (US)
JournalPsychological Methods
DOIs
StateAccepted/In press - 2022

Keywords

  • Accuracy in parameter estimation
  • Meta-analysis
  • Replication
  • Sample size
  • Statistical power

ASJC Scopus subject areas

  • Psychology (miscellaneous)

Fingerprint

Dive into the research topics of 'Sample Size Planning for Replication Studies: The Devil Is in the Design'. Together they form a unique fingerprint.

Cite this