Learning from the past: Approaches for reproducibility in computational neuroscience

Sharon Crook, Andrew P. Davison, Hans E. Plesser

Research output: Chapter in Book/Report/Conference proceedingChapter

21 Citations (Scopus)

Abstract

Reproducible experiments are the cornerstone of science: Only observations that can be independently confirmed enter the body of scientific knowledge. Computational science should excel in reproducibility, as simulations on digital computers avoid many of the small variations that are beyond the control of the experimental biologist or physicist. However, in reality, computational science has its own challenges for reproducibility: Many computational scientists find it difficult to reproduce results published in the literature, and many authors have met problems replicating even the figures in their own papers. We present a distinction between different levels of replicability and reproducibility of findings in computational neuroscience. We also demonstrate that simulations of neural models can be highly sensitive to numerical details, and conclude that often it is futile to expect exact replicability of simulation results across simulator software packages. Thus, the computational neuroscience community needs to discuss how to define successful reproduction of simulation studies. Any investigation of failures to reproduce published results will benefit significantly from the ability to track the provenance of the original results. We present tools and best practices developed over the past 2 decades that facilitate provenance tracking and model sharing.

Original languageEnglish (US)
Title of host publication20 Years of Computational Neuroscience
PublisherSpringer New York
Pages73-102
Number of pages30
ISBN (Electronic)9781461414247
ISBN (Print)9781461414230
DOIs
StatePublished - Jan 1 2013

Fingerprint

Neurosciences
Learning
Aptitude
Practice Guidelines
Reproducibility of Results
Reproduction
Software

ASJC Scopus subject areas

  • Medicine(all)
  • Neuroscience(all)

Cite this

Crook, S., Davison, A. P., & Plesser, H. E. (2013). Learning from the past: Approaches for reproducibility in computational neuroscience. In 20 Years of Computational Neuroscience (pp. 73-102). Springer New York. https://doi.org/10.1007/978-1-4614-1424-7_4

Learning from the past : Approaches for reproducibility in computational neuroscience. / Crook, Sharon; Davison, Andrew P.; Plesser, Hans E.

20 Years of Computational Neuroscience. Springer New York, 2013. p. 73-102.

Research output: Chapter in Book/Report/Conference proceedingChapter

Crook, S, Davison, AP & Plesser, HE 2013, Learning from the past: Approaches for reproducibility in computational neuroscience. in 20 Years of Computational Neuroscience. Springer New York, pp. 73-102. https://doi.org/10.1007/978-1-4614-1424-7_4
Crook S, Davison AP, Plesser HE. Learning from the past: Approaches for reproducibility in computational neuroscience. In 20 Years of Computational Neuroscience. Springer New York. 2013. p. 73-102 https://doi.org/10.1007/978-1-4614-1424-7_4
Crook, Sharon ; Davison, Andrew P. ; Plesser, Hans E. / Learning from the past : Approaches for reproducibility in computational neuroscience. 20 Years of Computational Neuroscience. Springer New York, 2013. pp. 73-102
@inbook{cea2513640d74684a318544b5d8e9938,
title = "Learning from the past: Approaches for reproducibility in computational neuroscience",
abstract = "Reproducible experiments are the cornerstone of science: Only observations that can be independently confirmed enter the body of scientific knowledge. Computational science should excel in reproducibility, as simulations on digital computers avoid many of the small variations that are beyond the control of the experimental biologist or physicist. However, in reality, computational science has its own challenges for reproducibility: Many computational scientists find it difficult to reproduce results published in the literature, and many authors have met problems replicating even the figures in their own papers. We present a distinction between different levels of replicability and reproducibility of findings in computational neuroscience. We also demonstrate that simulations of neural models can be highly sensitive to numerical details, and conclude that often it is futile to expect exact replicability of simulation results across simulator software packages. Thus, the computational neuroscience community needs to discuss how to define successful reproduction of simulation studies. Any investigation of failures to reproduce published results will benefit significantly from the ability to track the provenance of the original results. We present tools and best practices developed over the past 2 decades that facilitate provenance tracking and model sharing.",
author = "Sharon Crook and Davison, {Andrew P.} and Plesser, {Hans E.}",
year = "2013",
month = "1",
day = "1",
doi = "10.1007/978-1-4614-1424-7_4",
language = "English (US)",
isbn = "9781461414230",
pages = "73--102",
booktitle = "20 Years of Computational Neuroscience",
publisher = "Springer New York",
address = "United States",

}

TY - CHAP

T1 - Learning from the past

T2 - Approaches for reproducibility in computational neuroscience

AU - Crook, Sharon

AU - Davison, Andrew P.

AU - Plesser, Hans E.

PY - 2013/1/1

Y1 - 2013/1/1

N2 - Reproducible experiments are the cornerstone of science: Only observations that can be independently confirmed enter the body of scientific knowledge. Computational science should excel in reproducibility, as simulations on digital computers avoid many of the small variations that are beyond the control of the experimental biologist or physicist. However, in reality, computational science has its own challenges for reproducibility: Many computational scientists find it difficult to reproduce results published in the literature, and many authors have met problems replicating even the figures in their own papers. We present a distinction between different levels of replicability and reproducibility of findings in computational neuroscience. We also demonstrate that simulations of neural models can be highly sensitive to numerical details, and conclude that often it is futile to expect exact replicability of simulation results across simulator software packages. Thus, the computational neuroscience community needs to discuss how to define successful reproduction of simulation studies. Any investigation of failures to reproduce published results will benefit significantly from the ability to track the provenance of the original results. We present tools and best practices developed over the past 2 decades that facilitate provenance tracking and model sharing.

AB - Reproducible experiments are the cornerstone of science: Only observations that can be independently confirmed enter the body of scientific knowledge. Computational science should excel in reproducibility, as simulations on digital computers avoid many of the small variations that are beyond the control of the experimental biologist or physicist. However, in reality, computational science has its own challenges for reproducibility: Many computational scientists find it difficult to reproduce results published in the literature, and many authors have met problems replicating even the figures in their own papers. We present a distinction between different levels of replicability and reproducibility of findings in computational neuroscience. We also demonstrate that simulations of neural models can be highly sensitive to numerical details, and conclude that often it is futile to expect exact replicability of simulation results across simulator software packages. Thus, the computational neuroscience community needs to discuss how to define successful reproduction of simulation studies. Any investigation of failures to reproduce published results will benefit significantly from the ability to track the provenance of the original results. We present tools and best practices developed over the past 2 decades that facilitate provenance tracking and model sharing.

UR - http://www.scopus.com/inward/record.url?scp=85010065988&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85010065988&partnerID=8YFLogxK

U2 - 10.1007/978-1-4614-1424-7_4

DO - 10.1007/978-1-4614-1424-7_4

M3 - Chapter

AN - SCOPUS:85010065988

SN - 9781461414230

SP - 73

EP - 102

BT - 20 Years of Computational Neuroscience

PB - Springer New York

ER -