Hypothetical case replacement can be used to quantify the robustness of trial results

Kenneth A. Frank, Qinyun Lin, Spiro Maroulis, Anna S. Mueller, Ran Xu, Joshua M. Rosenberg, Christopher S. Hayter, Ramy A. Mahmoud, Marynia Kolak, Thomas Dietz, Lixin Zhang

Research output: Contribution to journalArticlepeer-review

Abstract

Objectives: We apply a general case replacement framework for quantifying the robustness of causal inferences to characterize the uncertainty of findings from clinical trials. Study design and setting: We express the robustness of inferences as the amount of data that must be replaced to change the conclusion and relate this to the fragility of trial results used for dichotomous outcomes. We illustrate our approach in the context of an RCT of hydroxychloroquine on pneumonia in COVID-19 patients and a cumulative meta-analysis of the effect of antihypertensive treatments on stroke. Results: We developed the Robustness of an Inference to Replacement (RIR), which quantifies how many treatment cases with positive outcomes would have to be replaced with hypothetical patients who did not receive a treatment to change an inference. The RIR addresses known limitations of the Fragility Index by accounting for the observed rates of outcomes. It can be used for varying thresholds for inference, including clinical importance. Conclusion: Because the RIR expresses uncertainty in terms of patient experiences, it is more relatable to stakeholders than P-values alone. It helps identify when results are statistically significant, but conclusions are not robust, while considering the rareness of events in the underlying data.

Original languageEnglish (US)
Pages (from-to)150-159
Number of pages10
JournalJournal of Clinical Epidemiology
Volume134
DOIs
StatePublished - Jun 2021

Keywords

  • Case replacement
  • Clinical importance
  • Fragility
  • Randomized controlled trials
  • Robustness of findings
  • Statistical significance

ASJC Scopus subject areas

  • Epidemiology

Fingerprint

Dive into the research topics of 'Hypothetical case replacement can be used to quantify the robustness of trial results'. Together they form a unique fingerprint.

Cite this