Phys-MAPS

a programmatic physiology assessment for introductory and advanced undergraduates

Katharine Semsar, Sara Brownell, Brian A. Couch, Alison J. Crowe, Michelle K. Smith, Mindi M. Summers, Christian D. Wright, Jennifer K. Knight

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

We describe the development of a new, freely available, online, programmatic-level assessment tool, Measuring Achievement and Progress in Science in Physiology, or Phys-MAPS ( http://cperl.lassp.cornell.edu/bio-maps ). Aligned with the conceptual frameworks of Core Principles of Physiology, and Vision and Change Core Concepts, Phys-MAPS can be used to evaluate student learning of core physiology concepts at multiple time points in an undergraduate physiology program, providing a valuable longitudinal tool to gain insight into student thinking and aid in the data-driven reform of physiology curricula. Phys-MAPS questions have a modified multiple true/false design and were developed using an iterative process, including student interviews and physiology expert review to verify scientific accuracy, appropriateness for physiology majors, and clarity. The final version of Phys-MAPS was tested with 2,600 students across 13 universities, has evidence of reliability, and has no significant statement biases. Over 90% of the physiology experts surveyed agreed that each Phys-MAPS statement was scientifically accurate and relevant to a physiology major. When testing each statement for bias, differential item functioning analysis demonstrated only a small effect size (<0.008) of any tested demographic variable. Regarding student performance, Phys-MAPS can also distinguish between lower and upper division students, both across different institutions (average overall scores increase with each level of class standing; two-way ANOVA, P < 0.001) and within each of three sample institutions (each ANOVA, P ≤ 0.001). Furthermore, at the level of individual concepts, only evolution and homeostasis do not demonstrate the typical increase across class standing, suggesting these concepts likely present consistent conceptual challenges for physiology students.

Original languageEnglish (US)
Pages (from-to)15-27
Number of pages13
JournalAdvances in physiology education
Volume43
Issue number1
DOIs
StatePublished - Mar 1 2019

Fingerprint

Students
Analysis of Variance
Curriculum
Homeostasis
Demography
Learning
Interviews

Keywords

  • concept assessment
  • concept inventory
  • curriculum reform
  • major
  • program

ASJC Scopus subject areas

  • Physiology

Cite this

Phys-MAPS : a programmatic physiology assessment for introductory and advanced undergraduates. / Semsar, Katharine; Brownell, Sara; Couch, Brian A.; Crowe, Alison J.; Smith, Michelle K.; Summers, Mindi M.; Wright, Christian D.; Knight, Jennifer K.

In: Advances in physiology education, Vol. 43, No. 1, 01.03.2019, p. 15-27.

Research output: Contribution to journalArticle

Semsar, K, Brownell, S, Couch, BA, Crowe, AJ, Smith, MK, Summers, MM, Wright, CD & Knight, JK 2019, 'Phys-MAPS: a programmatic physiology assessment for introductory and advanced undergraduates', Advances in physiology education, vol. 43, no. 1, pp. 15-27. https://doi.org/10.1152/advan.00128.2018
Semsar, Katharine ; Brownell, Sara ; Couch, Brian A. ; Crowe, Alison J. ; Smith, Michelle K. ; Summers, Mindi M. ; Wright, Christian D. ; Knight, Jennifer K. / Phys-MAPS : a programmatic physiology assessment for introductory and advanced undergraduates. In: Advances in physiology education. 2019 ; Vol. 43, No. 1. pp. 15-27.
@article{fa606c10d7fb40a2b9bcaa86e5e46288,
title = "Phys-MAPS: a programmatic physiology assessment for introductory and advanced undergraduates",
abstract = "We describe the development of a new, freely available, online, programmatic-level assessment tool, Measuring Achievement and Progress in Science in Physiology, or Phys-MAPS ( http://cperl.lassp.cornell.edu/bio-maps ). Aligned with the conceptual frameworks of Core Principles of Physiology, and Vision and Change Core Concepts, Phys-MAPS can be used to evaluate student learning of core physiology concepts at multiple time points in an undergraduate physiology program, providing a valuable longitudinal tool to gain insight into student thinking and aid in the data-driven reform of physiology curricula. Phys-MAPS questions have a modified multiple true/false design and were developed using an iterative process, including student interviews and physiology expert review to verify scientific accuracy, appropriateness for physiology majors, and clarity. The final version of Phys-MAPS was tested with 2,600 students across 13 universities, has evidence of reliability, and has no significant statement biases. Over 90{\%} of the physiology experts surveyed agreed that each Phys-MAPS statement was scientifically accurate and relevant to a physiology major. When testing each statement for bias, differential item functioning analysis demonstrated only a small effect size (<0.008) of any tested demographic variable. Regarding student performance, Phys-MAPS can also distinguish between lower and upper division students, both across different institutions (average overall scores increase with each level of class standing; two-way ANOVA, P < 0.001) and within each of three sample institutions (each ANOVA, P ≤ 0.001). Furthermore, at the level of individual concepts, only evolution and homeostasis do not demonstrate the typical increase across class standing, suggesting these concepts likely present consistent conceptual challenges for physiology students.",
keywords = "concept assessment, concept inventory, curriculum reform, major, program",
author = "Katharine Semsar and Sara Brownell and Couch, {Brian A.} and Crowe, {Alison J.} and Smith, {Michelle K.} and Summers, {Mindi M.} and Wright, {Christian D.} and Knight, {Jennifer K.}",
year = "2019",
month = "3",
day = "1",
doi = "10.1152/advan.00128.2018",
language = "English (US)",
volume = "43",
pages = "15--27",
journal = "American Journal of Physiology - Advances in Physiology Education",
issn = "1043-4046",
publisher = "American Physiological Society",
number = "1",

}

TY - JOUR

T1 - Phys-MAPS

T2 - a programmatic physiology assessment for introductory and advanced undergraduates

AU - Semsar, Katharine

AU - Brownell, Sara

AU - Couch, Brian A.

AU - Crowe, Alison J.

AU - Smith, Michelle K.

AU - Summers, Mindi M.

AU - Wright, Christian D.

AU - Knight, Jennifer K.

PY - 2019/3/1

Y1 - 2019/3/1

N2 - We describe the development of a new, freely available, online, programmatic-level assessment tool, Measuring Achievement and Progress in Science in Physiology, or Phys-MAPS ( http://cperl.lassp.cornell.edu/bio-maps ). Aligned with the conceptual frameworks of Core Principles of Physiology, and Vision and Change Core Concepts, Phys-MAPS can be used to evaluate student learning of core physiology concepts at multiple time points in an undergraduate physiology program, providing a valuable longitudinal tool to gain insight into student thinking and aid in the data-driven reform of physiology curricula. Phys-MAPS questions have a modified multiple true/false design and were developed using an iterative process, including student interviews and physiology expert review to verify scientific accuracy, appropriateness for physiology majors, and clarity. The final version of Phys-MAPS was tested with 2,600 students across 13 universities, has evidence of reliability, and has no significant statement biases. Over 90% of the physiology experts surveyed agreed that each Phys-MAPS statement was scientifically accurate and relevant to a physiology major. When testing each statement for bias, differential item functioning analysis demonstrated only a small effect size (<0.008) of any tested demographic variable. Regarding student performance, Phys-MAPS can also distinguish between lower and upper division students, both across different institutions (average overall scores increase with each level of class standing; two-way ANOVA, P < 0.001) and within each of three sample institutions (each ANOVA, P ≤ 0.001). Furthermore, at the level of individual concepts, only evolution and homeostasis do not demonstrate the typical increase across class standing, suggesting these concepts likely present consistent conceptual challenges for physiology students.

AB - We describe the development of a new, freely available, online, programmatic-level assessment tool, Measuring Achievement and Progress in Science in Physiology, or Phys-MAPS ( http://cperl.lassp.cornell.edu/bio-maps ). Aligned with the conceptual frameworks of Core Principles of Physiology, and Vision and Change Core Concepts, Phys-MAPS can be used to evaluate student learning of core physiology concepts at multiple time points in an undergraduate physiology program, providing a valuable longitudinal tool to gain insight into student thinking and aid in the data-driven reform of physiology curricula. Phys-MAPS questions have a modified multiple true/false design and were developed using an iterative process, including student interviews and physiology expert review to verify scientific accuracy, appropriateness for physiology majors, and clarity. The final version of Phys-MAPS was tested with 2,600 students across 13 universities, has evidence of reliability, and has no significant statement biases. Over 90% of the physiology experts surveyed agreed that each Phys-MAPS statement was scientifically accurate and relevant to a physiology major. When testing each statement for bias, differential item functioning analysis demonstrated only a small effect size (<0.008) of any tested demographic variable. Regarding student performance, Phys-MAPS can also distinguish between lower and upper division students, both across different institutions (average overall scores increase with each level of class standing; two-way ANOVA, P < 0.001) and within each of three sample institutions (each ANOVA, P ≤ 0.001). Furthermore, at the level of individual concepts, only evolution and homeostasis do not demonstrate the typical increase across class standing, suggesting these concepts likely present consistent conceptual challenges for physiology students.

KW - concept assessment

KW - concept inventory

KW - curriculum reform

KW - major

KW - program

UR - http://www.scopus.com/inward/record.url?scp=85058612267&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058612267&partnerID=8YFLogxK

U2 - 10.1152/advan.00128.2018

DO - 10.1152/advan.00128.2018

M3 - Article

VL - 43

SP - 15

EP - 27

JO - American Journal of Physiology - Advances in Physiology Education

JF - American Journal of Physiology - Advances in Physiology Education

SN - 1043-4046

IS - 1

ER -