Evaluating long-term complex professional development

Using a variation of the cohort control design

Laura B Sample McMeeking, R. Brian Cobb, Carole Basile

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

This paper introduces a variation on the post-test only cohort control design and addresses questions concerning both the methodological credibility and the practical utility of employing this design variation in evaluations of large-scale complex professional development programmes in mathematics education. The original design and design variation, which adds a pre-test measure and gain scores for both pre-treatment and post-treatment conditions, are compared theoretically for their respective controls on threats to internal validity, and practically for their comparative ease of implementation in field settings. We conclude that the design variation adds important controls to selection bias that the original design cannot. From a utility perspective, however, there are tradeoffs, because this design variation requires more data collection observations, more complex data management and certain psychometric characteristics for the outcome measure.

Original languageEnglish (US)
Pages (from-to)273-285
Number of pages13
JournalEvaluation and Research in Education
Volume23
Issue number4
DOIs
StatePublished - Dec 2010
Externally publishedYes

Fingerprint

psychometrics
credibility
mathematics
threat
trend
evaluation
management
education

Keywords

  • Achievement
  • Evaluation
  • In-service education
  • Methodology
  • Quantitative
  • Students

ASJC Scopus subject areas

  • Education

Cite this

Evaluating long-term complex professional development : Using a variation of the cohort control design. / McMeeking, Laura B Sample; Cobb, R. Brian; Basile, Carole.

In: Evaluation and Research in Education, Vol. 23, No. 4, 12.2010, p. 273-285.

Research output: Contribution to journalArticle

@article{bc506d6699c642beb5f1bb889ecb44e8,
title = "Evaluating long-term complex professional development: Using a variation of the cohort control design",
abstract = "This paper introduces a variation on the post-test only cohort control design and addresses questions concerning both the methodological credibility and the practical utility of employing this design variation in evaluations of large-scale complex professional development programmes in mathematics education. The original design and design variation, which adds a pre-test measure and gain scores for both pre-treatment and post-treatment conditions, are compared theoretically for their respective controls on threats to internal validity, and practically for their comparative ease of implementation in field settings. We conclude that the design variation adds important controls to selection bias that the original design cannot. From a utility perspective, however, there are tradeoffs, because this design variation requires more data collection observations, more complex data management and certain psychometric characteristics for the outcome measure.",
keywords = "Achievement, Evaluation, In-service education, Methodology, Quantitative, Students",
author = "McMeeking, {Laura B Sample} and Cobb, {R. Brian} and Carole Basile",
year = "2010",
month = "12",
doi = "10.1080/09500790.2010.509781",
language = "English (US)",
volume = "23",
pages = "273--285",
journal = "Evaluation and Research in Education",
issn = "0950-0790",
publisher = "Routledge",
number = "4",

}

TY - JOUR

T1 - Evaluating long-term complex professional development

T2 - Using a variation of the cohort control design

AU - McMeeking, Laura B Sample

AU - Cobb, R. Brian

AU - Basile, Carole

PY - 2010/12

Y1 - 2010/12

N2 - This paper introduces a variation on the post-test only cohort control design and addresses questions concerning both the methodological credibility and the practical utility of employing this design variation in evaluations of large-scale complex professional development programmes in mathematics education. The original design and design variation, which adds a pre-test measure and gain scores for both pre-treatment and post-treatment conditions, are compared theoretically for their respective controls on threats to internal validity, and practically for their comparative ease of implementation in field settings. We conclude that the design variation adds important controls to selection bias that the original design cannot. From a utility perspective, however, there are tradeoffs, because this design variation requires more data collection observations, more complex data management and certain psychometric characteristics for the outcome measure.

AB - This paper introduces a variation on the post-test only cohort control design and addresses questions concerning both the methodological credibility and the practical utility of employing this design variation in evaluations of large-scale complex professional development programmes in mathematics education. The original design and design variation, which adds a pre-test measure and gain scores for both pre-treatment and post-treatment conditions, are compared theoretically for their respective controls on threats to internal validity, and practically for their comparative ease of implementation in field settings. We conclude that the design variation adds important controls to selection bias that the original design cannot. From a utility perspective, however, there are tradeoffs, because this design variation requires more data collection observations, more complex data management and certain psychometric characteristics for the outcome measure.

KW - Achievement

KW - Evaluation

KW - In-service education

KW - Methodology

KW - Quantitative

KW - Students

UR - http://www.scopus.com/inward/record.url?scp=79959685250&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79959685250&partnerID=8YFLogxK

U2 - 10.1080/09500790.2010.509781

DO - 10.1080/09500790.2010.509781

M3 - Article

VL - 23

SP - 273

EP - 285

JO - Evaluation and Research in Education

JF - Evaluation and Research in Education

SN - 0950-0790

IS - 4

ER -