TY - JOUR
T1 - Sample Size Planning for Replication Studies
T2 - The Devil Is in the Design
AU - Anderson, Samantha F.
AU - Kelley, Ken
N1 - Publisher Copyright:
© 2022. American Psychological Association
PY - 2022
Y1 - 2022
N2 - Replication is central to scientific progress. Because of widely reported replication failures, replication has received increased attention in psychology, sociology, education, management, and related fields in recent years. Replication studies have generally been assessed dichotomously, designated either a “success” or “failure” based entirely on the outcome of a null hypothesis significance test (i.e., p <.05 or p >.05, respectively). However, alternative definitions of success depend on researchers’ goals for the replication. Previous work on alternative definitions for success has focused on the analysis phase of replication. However, the design of the replication is also important, as emphasized with the adage, “an ounce of prevention is better than a pound of cure.” One critical component of design often ignored or oversimplified in replication studies is sample size planning, indeed, the details here are crucial. Sample size planning for replication studies should correspond to the method by which success will be evaluated. Researchers have received little guidance, some of which is misguided, on sample size planning for replication goals other than the aforementioned dichotomous null hypothesis significance testing approach. In this article, we describe four different replication goals. Then, we formalize sample size planning methods for each of the four goals.
AB - Replication is central to scientific progress. Because of widely reported replication failures, replication has received increased attention in psychology, sociology, education, management, and related fields in recent years. Replication studies have generally been assessed dichotomously, designated either a “success” or “failure” based entirely on the outcome of a null hypothesis significance test (i.e., p <.05 or p >.05, respectively). However, alternative definitions of success depend on researchers’ goals for the replication. Previous work on alternative definitions for success has focused on the analysis phase of replication. However, the design of the replication is also important, as emphasized with the adage, “an ounce of prevention is better than a pound of cure.” One critical component of design often ignored or oversimplified in replication studies is sample size planning, indeed, the details here are crucial. Sample size planning for replication studies should correspond to the method by which success will be evaluated. Researchers have received little guidance, some of which is misguided, on sample size planning for replication goals other than the aforementioned dichotomous null hypothesis significance testing approach. In this article, we describe four different replication goals. Then, we formalize sample size planning methods for each of the four goals.
KW - Accuracy in parameter estimation
KW - Meta-analysis
KW - Replication
KW - Sample size
KW - Statistical power
UR - http://www.scopus.com/inward/record.url?scp=85135575746&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85135575746&partnerID=8YFLogxK
U2 - 10.1037/met0000520
DO - 10.1037/met0000520
M3 - Article
AN - SCOPUS:85135575746
SN - 1082-989X
JO - Psychological Methods
JF - Psychological Methods
ER -