Using Indices of Fidelity to Intervention Core Components to Identify Program Active Ingredients

Tashia Abry, Chris S. Hulleman, Sara E. Rimm-Kaufman

Research output: Contribution to journalArticle

27 Citations (Scopus)

Abstract

Identifying the active ingredients of an intervention—intervention-specific components serving as key levers of change—is crucial for unpacking the intervention black box. Measures of intervention fidelity can be used to identify specific active ingredients, yet such applications are rare. We illustrate how fidelity measures can be used to identify program active ingredients in the context of a social-emotional learning intervention (Responsive Classroom). We applied one customary and two novel approaches to create indices of fidelity. In the customary approach, we averaged fidelity ratings across all core components. In the novel approaches, we computed fidelity indices for specific components by (a) averaging responses from like items and (b) deriving factor scores from a multitrait, multimethod factor analysis. We then tested indices in relation to achievement gains (N = 1,442). Indices derived using novel approaches explained more outcome variance than indices from the customary approach. Further, novel approaches revealed one component as a potential active ingredient. Discussion highlights strengths and limitations of the indices and implications for identifying program active ingredients.

Original languageEnglish (US)
Pages (from-to)320-338
Number of pages19
JournalAmerican Journal of Evaluation
Volume36
Issue number3
DOIs
StatePublished - Jan 1 2015

Fingerprint

Statistical Factor Analysis
Learning
Fidelity
factor analysis
rating
classroom
learning

Keywords

  • active ingredients
  • core components
  • implementation
  • intervention fidelity
  • Responsive Classroom
  • social and emotional learning

ASJC Scopus subject areas

  • Health(social science)
  • Social Psychology
  • Business and International Management
  • Education
  • Sociology and Political Science
  • Strategy and Management

Cite this

Using Indices of Fidelity to Intervention Core Components to Identify Program Active Ingredients. / Abry, Tashia; Hulleman, Chris S.; Rimm-Kaufman, Sara E.

In: American Journal of Evaluation, Vol. 36, No. 3, 01.01.2015, p. 320-338.

Research output: Contribution to journalArticle

Abry, Tashia ; Hulleman, Chris S. ; Rimm-Kaufman, Sara E. / Using Indices of Fidelity to Intervention Core Components to Identify Program Active Ingredients. In: American Journal of Evaluation. 2015 ; Vol. 36, No. 3. pp. 320-338.
@article{d532be17f1d946fa8fe1d4f3e4589bcd,
title = "Using Indices of Fidelity to Intervention Core Components to Identify Program Active Ingredients",
abstract = "Identifying the active ingredients of an intervention—intervention-specific components serving as key levers of change—is crucial for unpacking the intervention black box. Measures of intervention fidelity can be used to identify specific active ingredients, yet such applications are rare. We illustrate how fidelity measures can be used to identify program active ingredients in the context of a social-emotional learning intervention (Responsive Classroom). We applied one customary and two novel approaches to create indices of fidelity. In the customary approach, we averaged fidelity ratings across all core components. In the novel approaches, we computed fidelity indices for specific components by (a) averaging responses from like items and (b) deriving factor scores from a multitrait, multimethod factor analysis. We then tested indices in relation to achievement gains (N = 1,442). Indices derived using novel approaches explained more outcome variance than indices from the customary approach. Further, novel approaches revealed one component as a potential active ingredient. Discussion highlights strengths and limitations of the indices and implications for identifying program active ingredients.",
keywords = "active ingredients, core components, implementation, intervention fidelity, Responsive Classroom, social and emotional learning",
author = "Tashia Abry and Hulleman, {Chris S.} and Rimm-Kaufman, {Sara E.}",
year = "2015",
month = "1",
day = "1",
doi = "10.1177/1098214014557009",
language = "English (US)",
volume = "36",
pages = "320--338",
journal = "American Journal of Evaluation",
issn = "1098-2140",
publisher = "SAGE Publications Inc.",
number = "3",

}

TY - JOUR

T1 - Using Indices of Fidelity to Intervention Core Components to Identify Program Active Ingredients

AU - Abry, Tashia

AU - Hulleman, Chris S.

AU - Rimm-Kaufman, Sara E.

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Identifying the active ingredients of an intervention—intervention-specific components serving as key levers of change—is crucial for unpacking the intervention black box. Measures of intervention fidelity can be used to identify specific active ingredients, yet such applications are rare. We illustrate how fidelity measures can be used to identify program active ingredients in the context of a social-emotional learning intervention (Responsive Classroom). We applied one customary and two novel approaches to create indices of fidelity. In the customary approach, we averaged fidelity ratings across all core components. In the novel approaches, we computed fidelity indices for specific components by (a) averaging responses from like items and (b) deriving factor scores from a multitrait, multimethod factor analysis. We then tested indices in relation to achievement gains (N = 1,442). Indices derived using novel approaches explained more outcome variance than indices from the customary approach. Further, novel approaches revealed one component as a potential active ingredient. Discussion highlights strengths and limitations of the indices and implications for identifying program active ingredients.

AB - Identifying the active ingredients of an intervention—intervention-specific components serving as key levers of change—is crucial for unpacking the intervention black box. Measures of intervention fidelity can be used to identify specific active ingredients, yet such applications are rare. We illustrate how fidelity measures can be used to identify program active ingredients in the context of a social-emotional learning intervention (Responsive Classroom). We applied one customary and two novel approaches to create indices of fidelity. In the customary approach, we averaged fidelity ratings across all core components. In the novel approaches, we computed fidelity indices for specific components by (a) averaging responses from like items and (b) deriving factor scores from a multitrait, multimethod factor analysis. We then tested indices in relation to achievement gains (N = 1,442). Indices derived using novel approaches explained more outcome variance than indices from the customary approach. Further, novel approaches revealed one component as a potential active ingredient. Discussion highlights strengths and limitations of the indices and implications for identifying program active ingredients.

KW - active ingredients

KW - core components

KW - implementation

KW - intervention fidelity

KW - Responsive Classroom

KW - social and emotional learning

UR - http://www.scopus.com/inward/record.url?scp=84938601517&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84938601517&partnerID=8YFLogxK

U2 - 10.1177/1098214014557009

DO - 10.1177/1098214014557009

M3 - Article

AN - SCOPUS:84938601517

VL - 36

SP - 320

EP - 338

JO - American Journal of Evaluation

JF - American Journal of Evaluation

SN - 1098-2140

IS - 3

ER -