What Would It Take to Change an Inference? Using Rubin's Causal Model to Interpret the Robustness of Causal Inferences

Kenneth A. Frank, Spiro Maroulis, Minh Q. Duong, Benjamin M. Kelcey

    Research output: Contribution to journalArticlepeer-review

    144 Scopus citations

    Abstract

    We contribute to debate about causal inferences in educational research in two ways. First, we quantify how much bias there must be in an estimate to invalidate an inference. Second, we utilize Rubin's causal model to interpret the bias necessary to invalidate an inference in terms of sample replacement. We apply our analysis to an inference of a positive effect of Open Court Curriculum on reading achievement from a randomized experiment, and an inference of a negative effect of kindergarten retention on reading achievement from an observational study. We consider details of our framework, and then discuss how our approach informs judgment of inference relative to study design. We conclude with implications for scientific discourse.

    Original languageEnglish (US)
    Pages (from-to)437-460
    Number of pages24
    JournalEducational Evaluation and Policy Analysis
    Volume35
    Issue number4
    DOIs
    StatePublished - Dec 2013

    Keywords

    • Rubin's causal model
    • causal inference
    • observational studies
    • sensitivity analysis

    ASJC Scopus subject areas

    • Education

    Fingerprint

    Dive into the research topics of 'What Would It Take to Change an Inference? Using Rubin's Causal Model to Interpret the Robustness of Causal Inferences'. Together they form a unique fingerprint.

    Cite this