Adherence and delivery

implementation quality and program outcomes for the seventh-grade keepin' it REAL program

Jonathan Pettigrew, John W. Graham, Michelle Miller-Day, Michael L. Hecht, Janice L. Krieger, YoungJu Shin

Research output: Contribution to journalArticle

25 Citations (Scopus)

Abstract

Poor implementation quality (IQ) is known to reduce program effects making it important to consider IQ for evaluation and dissemination of prevention programs. However, less is known about the ways specific implementation variables relate to outcomes. In this study, two versions of keepin' it REAL, a seventh-grade drug prevention intervention, were implemented in 78 classrooms in 25 schools in rural districts in Pennsylvania and Ohio. IQ was measured through observational coding of 276 videos. IQ variables included adherence to the curriculum, teacher engagement (attentiveness, enthusiasm, seriousness, clarity, positivity), student engagement (attention, participation), and a global rating of teacher delivery quality. Factor analysis showed that teacher engagement, student engagement, and delivery quality formed one factor, which was labeled delivery. A second factor was adherence to the curriculum. Self-report student surveys measured substance use, norms (beliefs about prevalence and acceptability of use), and efficacy (beliefs about one's ability to refuse substance offers) at two waves (pretest, immediate posttest). Mixed model regression analysis which accounted for missing data and controlled for pretest levels examined implementation quality's effects on individual level outcomes, statistically controlling for cluster level effects. Results show that when implemented well, students show positive outcomes compared to students receiving a poorly implemented program. Delivery significantly influenced substance use and norms, but not efficacy. Adherence marginally significantly predicted use and significantly predicted norms, but not efficacy. Findings underscore the importance of comprehensively measuring and accounting for IQ, particularly delivery, when evaluating prevention interventions.

Original languageEnglish (US)
Pages (from-to)90-99
Number of pages10
JournalPrevention science : the official journal of the Society for Prevention Research
Volume16
Issue number1
DOIs
StatePublished - Jan 1 2015
Externally publishedYes

Fingerprint

Students
Curriculum
Self Report
Statistical Factor Analysis
Regression Analysis
Pharmaceutical Preparations

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Adherence and delivery : implementation quality and program outcomes for the seventh-grade keepin' it REAL program. / Pettigrew, Jonathan; Graham, John W.; Miller-Day, Michelle; Hecht, Michael L.; Krieger, Janice L.; Shin, YoungJu.

In: Prevention science : the official journal of the Society for Prevention Research, Vol. 16, No. 1, 01.01.2015, p. 90-99.

Research output: Contribution to journalArticle

@article{508cef178aa34f428232338f919a053f,
title = "Adherence and delivery: implementation quality and program outcomes for the seventh-grade keepin' it REAL program",
abstract = "Poor implementation quality (IQ) is known to reduce program effects making it important to consider IQ for evaluation and dissemination of prevention programs. However, less is known about the ways specific implementation variables relate to outcomes. In this study, two versions of keepin' it REAL, a seventh-grade drug prevention intervention, were implemented in 78 classrooms in 25 schools in rural districts in Pennsylvania and Ohio. IQ was measured through observational coding of 276 videos. IQ variables included adherence to the curriculum, teacher engagement (attentiveness, enthusiasm, seriousness, clarity, positivity), student engagement (attention, participation), and a global rating of teacher delivery quality. Factor analysis showed that teacher engagement, student engagement, and delivery quality formed one factor, which was labeled delivery. A second factor was adherence to the curriculum. Self-report student surveys measured substance use, norms (beliefs about prevalence and acceptability of use), and efficacy (beliefs about one's ability to refuse substance offers) at two waves (pretest, immediate posttest). Mixed model regression analysis which accounted for missing data and controlled for pretest levels examined implementation quality's effects on individual level outcomes, statistically controlling for cluster level effects. Results show that when implemented well, students show positive outcomes compared to students receiving a poorly implemented program. Delivery significantly influenced substance use and norms, but not efficacy. Adherence marginally significantly predicted use and significantly predicted norms, but not efficacy. Findings underscore the importance of comprehensively measuring and accounting for IQ, particularly delivery, when evaluating prevention interventions.",
author = "Jonathan Pettigrew and Graham, {John W.} and Michelle Miller-Day and Hecht, {Michael L.} and Krieger, {Janice L.} and YoungJu Shin",
year = "2015",
month = "1",
day = "1",
doi = "10.1007/s11121-014-0459-1",
language = "English (US)",
volume = "16",
pages = "90--99",
journal = "Prevention Science",
issn = "1389-4986",
publisher = "Springer New York",
number = "1",

}

TY - JOUR

T1 - Adherence and delivery

T2 - implementation quality and program outcomes for the seventh-grade keepin' it REAL program

AU - Pettigrew, Jonathan

AU - Graham, John W.

AU - Miller-Day, Michelle

AU - Hecht, Michael L.

AU - Krieger, Janice L.

AU - Shin, YoungJu

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Poor implementation quality (IQ) is known to reduce program effects making it important to consider IQ for evaluation and dissemination of prevention programs. However, less is known about the ways specific implementation variables relate to outcomes. In this study, two versions of keepin' it REAL, a seventh-grade drug prevention intervention, were implemented in 78 classrooms in 25 schools in rural districts in Pennsylvania and Ohio. IQ was measured through observational coding of 276 videos. IQ variables included adherence to the curriculum, teacher engagement (attentiveness, enthusiasm, seriousness, clarity, positivity), student engagement (attention, participation), and a global rating of teacher delivery quality. Factor analysis showed that teacher engagement, student engagement, and delivery quality formed one factor, which was labeled delivery. A second factor was adherence to the curriculum. Self-report student surveys measured substance use, norms (beliefs about prevalence and acceptability of use), and efficacy (beliefs about one's ability to refuse substance offers) at two waves (pretest, immediate posttest). Mixed model regression analysis which accounted for missing data and controlled for pretest levels examined implementation quality's effects on individual level outcomes, statistically controlling for cluster level effects. Results show that when implemented well, students show positive outcomes compared to students receiving a poorly implemented program. Delivery significantly influenced substance use and norms, but not efficacy. Adherence marginally significantly predicted use and significantly predicted norms, but not efficacy. Findings underscore the importance of comprehensively measuring and accounting for IQ, particularly delivery, when evaluating prevention interventions.

AB - Poor implementation quality (IQ) is known to reduce program effects making it important to consider IQ for evaluation and dissemination of prevention programs. However, less is known about the ways specific implementation variables relate to outcomes. In this study, two versions of keepin' it REAL, a seventh-grade drug prevention intervention, were implemented in 78 classrooms in 25 schools in rural districts in Pennsylvania and Ohio. IQ was measured through observational coding of 276 videos. IQ variables included adherence to the curriculum, teacher engagement (attentiveness, enthusiasm, seriousness, clarity, positivity), student engagement (attention, participation), and a global rating of teacher delivery quality. Factor analysis showed that teacher engagement, student engagement, and delivery quality formed one factor, which was labeled delivery. A second factor was adherence to the curriculum. Self-report student surveys measured substance use, norms (beliefs about prevalence and acceptability of use), and efficacy (beliefs about one's ability to refuse substance offers) at two waves (pretest, immediate posttest). Mixed model regression analysis which accounted for missing data and controlled for pretest levels examined implementation quality's effects on individual level outcomes, statistically controlling for cluster level effects. Results show that when implemented well, students show positive outcomes compared to students receiving a poorly implemented program. Delivery significantly influenced substance use and norms, but not efficacy. Adherence marginally significantly predicted use and significantly predicted norms, but not efficacy. Findings underscore the importance of comprehensively measuring and accounting for IQ, particularly delivery, when evaluating prevention interventions.

UR - http://www.scopus.com/inward/record.url?scp=84979858727&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84979858727&partnerID=8YFLogxK

U2 - 10.1007/s11121-014-0459-1

DO - 10.1007/s11121-014-0459-1

M3 - Article

VL - 16

SP - 90

EP - 99

JO - Prevention Science

JF - Prevention Science

SN - 1389-4986

IS - 1

ER -