Training calibration-based counterfactual explainers for deep learning models in medical image analysis

Jayaraman J. Thiagarajan, Kowshik Thopalli, Deepta Rajan, Pavan Turaga

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


The rapid adoption of artificial intelligence methods in healthcare is coupled with the critical need for techniques to rigorously introspect models and thereby ensure that they behave reliably. This has led to the design of explainable AI techniques that uncover the relationships between discernible data signatures and model predictions. In this context, counterfactual explanations that synthesize small, interpretable changes to a given query while producing desired changes in model predictions have become popular. This under-constrained, inverse problem is vulnerable to introducing irrelevant feature manipulations, particularly when the model’s predictions are not well-calibrated. Hence, in this paper, we propose the TraCE (training calibration-based explainers) technique, which utilizes a novel uncertainty-based interval calibration strategy for reliably synthesizing counterfactuals. Given the wide-spread adoption of machine-learned solutions in radiology, our study focuses on deep models used for identifying anomalies in chest X-ray images. Using rigorous empirical studies, we demonstrate the superiority of TraCE explanations over several state-of-the-art baseline approaches, in terms of several widely adopted evaluation metrics. Our findings show that TraCE can be used to obtain a holistic understanding of deep models by enabling progressive exploration of decision boundaries, to detect shortcuts, and to infer relationships between patient attributes and disease severity.

Original languageEnglish (US)
Article number597
JournalScientific reports
Issue number1
StatePublished - Dec 2022
Externally publishedYes

ASJC Scopus subject areas

  • General


Dive into the research topics of 'Training calibration-based counterfactual explainers for deep learning models in medical image analysis'. Together they form a unique fingerprint.

Cite this