Redesigning Implementation Measurement for Monitoring and Quality Improvement in Community Delivery Settings

Cady Berkel, Carlos G. Gallo, Irwin Sandler, Anne Mauricio, Justin D. Smith, C. Hendricks Brown

Research output: Contribution to journalArticle

Abstract

The field of prevention has established the potential to promote child adjustment across a wide array of outcomes. However, when evidence-based prevention programs have been delivered at scale in community settings, declines in implementation and outcomes have resulted. Maintaining high quality implementation is a critical challenge for the field. We describe steps towards the development of a practical system to monitor and support the high-quality implementation of evidence-based prevention programs in community settings. Research on the implementation of an evidence-based parenting program for divorcing families called the “New Beginnings Program” serves as an illustration of the promise of such a system. As a first step, we describe a multidimensional theoretical model of implementation that links aspects of program delivery with improvements in participant outcomes. We then describe research on the measurement of each of these implementation dimensions and test their relations to intended program outcomes. As a third step, we develop approaches to the assessment of these implementation constructs that are feasible to use in community settings and to establish their reliability and validity. We focus on the application of machine learning algorithms and web-based data collection systems to assess implementation and provide support for high quality delivery and positive outcomes. Examples are presented to demonstrate that valid and reliable measures can be collected using these methods. Finally, we envision how these measures can be used to develop an unobtrusive system to monitor implementation and provide feedback and support in real time to maintain high quality implementation and program outcomes.

Original languageEnglish (US)
JournalJournal of Primary Prevention
DOIs
StateAccepted/In press - Jan 1 2019

Fingerprint

Quality Improvement
Social Adjustment
Parenting
Research
Information Systems
Reproducibility of Results
Theoretical Models

Keywords

  • Evidence-based programs
  • Implementation
  • Measurement
  • Parenting
  • Pragmatic measures
  • Technology

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health

Cite this

Redesigning Implementation Measurement for Monitoring and Quality Improvement in Community Delivery Settings. / Berkel, Cady; Gallo, Carlos G.; Sandler, Irwin; Mauricio, Anne; Smith, Justin D.; Brown, C. Hendricks.

In: Journal of Primary Prevention, 01.01.2019.

Research output: Contribution to journalArticle

@article{33965d7f4de54157a39270e971a801f5,
title = "Redesigning Implementation Measurement for Monitoring and Quality Improvement in Community Delivery Settings",
abstract = "The field of prevention has established the potential to promote child adjustment across a wide array of outcomes. However, when evidence-based prevention programs have been delivered at scale in community settings, declines in implementation and outcomes have resulted. Maintaining high quality implementation is a critical challenge for the field. We describe steps towards the development of a practical system to monitor and support the high-quality implementation of evidence-based prevention programs in community settings. Research on the implementation of an evidence-based parenting program for divorcing families called the “New Beginnings Program” serves as an illustration of the promise of such a system. As a first step, we describe a multidimensional theoretical model of implementation that links aspects of program delivery with improvements in participant outcomes. We then describe research on the measurement of each of these implementation dimensions and test their relations to intended program outcomes. As a third step, we develop approaches to the assessment of these implementation constructs that are feasible to use in community settings and to establish their reliability and validity. We focus on the application of machine learning algorithms and web-based data collection systems to assess implementation and provide support for high quality delivery and positive outcomes. Examples are presented to demonstrate that valid and reliable measures can be collected using these methods. Finally, we envision how these measures can be used to develop an unobtrusive system to monitor implementation and provide feedback and support in real time to maintain high quality implementation and program outcomes.",
keywords = "Evidence-based programs, Implementation, Measurement, Parenting, Pragmatic measures, Technology",
author = "Cady Berkel and Gallo, {Carlos G.} and Irwin Sandler and Anne Mauricio and Smith, {Justin D.} and Brown, {C. Hendricks}",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/s10935-018-00534-z",
language = "English (US)",
journal = "Journal of Primary Prevention",
issn = "0278-095X",
publisher = "Kluwer Academic/Human Sciences Press Inc.",

}

TY - JOUR

T1 - Redesigning Implementation Measurement for Monitoring and Quality Improvement in Community Delivery Settings

AU - Berkel, Cady

AU - Gallo, Carlos G.

AU - Sandler, Irwin

AU - Mauricio, Anne

AU - Smith, Justin D.

AU - Brown, C. Hendricks

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The field of prevention has established the potential to promote child adjustment across a wide array of outcomes. However, when evidence-based prevention programs have been delivered at scale in community settings, declines in implementation and outcomes have resulted. Maintaining high quality implementation is a critical challenge for the field. We describe steps towards the development of a practical system to monitor and support the high-quality implementation of evidence-based prevention programs in community settings. Research on the implementation of an evidence-based parenting program for divorcing families called the “New Beginnings Program” serves as an illustration of the promise of such a system. As a first step, we describe a multidimensional theoretical model of implementation that links aspects of program delivery with improvements in participant outcomes. We then describe research on the measurement of each of these implementation dimensions and test their relations to intended program outcomes. As a third step, we develop approaches to the assessment of these implementation constructs that are feasible to use in community settings and to establish their reliability and validity. We focus on the application of machine learning algorithms and web-based data collection systems to assess implementation and provide support for high quality delivery and positive outcomes. Examples are presented to demonstrate that valid and reliable measures can be collected using these methods. Finally, we envision how these measures can be used to develop an unobtrusive system to monitor implementation and provide feedback and support in real time to maintain high quality implementation and program outcomes.

AB - The field of prevention has established the potential to promote child adjustment across a wide array of outcomes. However, when evidence-based prevention programs have been delivered at scale in community settings, declines in implementation and outcomes have resulted. Maintaining high quality implementation is a critical challenge for the field. We describe steps towards the development of a practical system to monitor and support the high-quality implementation of evidence-based prevention programs in community settings. Research on the implementation of an evidence-based parenting program for divorcing families called the “New Beginnings Program” serves as an illustration of the promise of such a system. As a first step, we describe a multidimensional theoretical model of implementation that links aspects of program delivery with improvements in participant outcomes. We then describe research on the measurement of each of these implementation dimensions and test their relations to intended program outcomes. As a third step, we develop approaches to the assessment of these implementation constructs that are feasible to use in community settings and to establish their reliability and validity. We focus on the application of machine learning algorithms and web-based data collection systems to assess implementation and provide support for high quality delivery and positive outcomes. Examples are presented to demonstrate that valid and reliable measures can be collected using these methods. Finally, we envision how these measures can be used to develop an unobtrusive system to monitor implementation and provide feedback and support in real time to maintain high quality implementation and program outcomes.

KW - Evidence-based programs

KW - Implementation

KW - Measurement

KW - Parenting

KW - Pragmatic measures

KW - Technology

UR - http://www.scopus.com/inward/record.url?scp=85060211484&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060211484&partnerID=8YFLogxK

U2 - 10.1007/s10935-018-00534-z

DO - 10.1007/s10935-018-00534-z

M3 - Article

JO - Journal of Primary Prevention

JF - Journal of Primary Prevention

SN - 0278-095X

ER -