There's more than one way to conduct a replication study: Beyond statistical significance

Samantha F. Anderson, Scott E. Maxwell

Research output: Contribution to journalArticle

49 Scopus citations

Abstract

As the field of psychology struggles to trust published findings, replication research has begun to become more of a priority to both scientists and journals. With this increasing emphasis placed on reproducibility, it is essential that replication studies be capable of advancing the field. However, we argue that many researchers have been only narrowly interpreting the meaning of replication, with studies being designed with a simple statistically significant or nonsignificant results framework in mind. Although this interpretation may be desirable in some cases, we develop a variety of additional "replication goals" that researchers could consider when planning studies. Even if researchers are aware of these goals, we show that they are rarely used in practice-as results are typically analyzed in a manner only appropriate to a simple significance test. We discuss each goal conceptually, explain appropriate analysis procedures, and provide 1 or more examples to illustrate these analyses in practice. We hope that these various goals will allow researchers to develop a more nuanced understanding of replication that can be flexible enough to answer the various questions that researchers might seek to understand.

Original languageEnglish (US)
Pages (from-to)1-12
Number of pages12
JournalPsychological Methods
Volume21
Issue number1
DOIs
StatePublished - Mar 1 2016

Keywords

  • Confidence interval
  • Data analysis
  • Effect size
  • Equivalence test
  • Replication

ASJC Scopus subject areas

  • Psychology (miscellaneous)

Fingerprint Dive into the research topics of 'There's more than one way to conduct a replication study: Beyond statistical significance'. Together they form a unique fingerprint.

  • Cite this