Appropriately estimating the standardized average treatment effect with missing data: A simulation and primer

Research output: Contribution to journalArticlepeer-review


Reporting standardized effects in randomized treatment studies aids interpretation and facilitates future meta-analyses and policy considerations. However, when outcome data are missing, achieving an unbiased, accurate estimate of the standardized average treatment effect, sATE, can pose challenges even for those with general knowledge of missing data handling, given that the sATE is a ratio of a mean difference to a (within-group) standard deviation. Under both homogeneity and heterogeneity of variance, a Monte Carlo simulation study was conducted to compare missing data handling strategies in terms of bias and accuracy in the sATE, under specific missingness patterns plausible for randomized pretest posttest studies. Within two broad missing data handling approaches, maximum likelihood and multiple imputation, modeling choices were thoroughly investigated including the analysis model, variance estimator, imputation algorithm, and method of pooling results across imputed datasets. Results demonstrated that although the sATE can be estimated with little bias using either maximum likelihood or multiple imputation, particular attention should be paid to the model and variance estimator, especially at smaller sample sizes (i.e., N = 50). Differences in accuracy were driven by differences in bias. To improve estimation of the sATE in practice, recommendations and a software demonstration are provided. Moreover, a pedagogical explanation of the causes of bias, described separately for the numerator and denominator of the sATE is provided, demonstrating visually how and why bias occurs with certain methods.

Original languageEnglish (US)
JournalBehavior Research Methods
StateAccepted/In press - 2022
Externally publishedYes


  • Average treatment effect
  • Maximum likelihood
  • Missing data
  • Multiple imputation
  • Standardized mean difference

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Developmental and Educational Psychology
  • Arts and Humanities (miscellaneous)
  • Psychology (miscellaneous)
  • Psychology(all)


Dive into the research topics of 'Appropriately estimating the standardized average treatment effect with missing data: A simulation and primer'. Together they form a unique fingerprint.

Cite this