Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education

Flora McMartin, Ann McKenna, Ken Youssefi

Research output: Chapter in Book/Report/Conference proceedingChapter

4 Citations (Scopus)

Abstract

How to establish the trustworthiness of qualitative tools to assess students' knowledge of engineering practices, teamwork, and problem solving is a question that remains unanswered for many faculty. This paper describes a process that was conducted to ensure the dependability of scenario assignments. Scenarios are a qualitative performance assessment tool designed to assess students' knowledge of engineering practices, teamwork, and problem solving. Two scoring rubrics, a holistic and an analytic, were developed to assess students' knowledge with respect to the learning outcomes associated with the scenario tool. Careful training of the scorers, analysis of the scores, and comparison of test scores determined dependability of the tool. Students were given a scenario that describes a 'day in the life' problem faced by engineers, then they are asked to explain the process they would use to solve the problem. Initial findings suggest that faculty who score the scenarios with the analytic rubric can quickly identify students' strengths and weaknesses in these areas and adapt their course to address the areas where students need attention. At the departmental or college level, scoring the scenarios with the holistic rubric is useful to assess changes in students' learning and development over time. Careful testing of the rubric revealed that the scores can reliably differentiate among student abilities.

Original languageEnglish (US)
Title of host publicationProceedings - Frontiers in Education Conference
Place of PublicationPiscataway, NJ, United States
PublisherIEEE
Volume3
ISBN (Print)0780356438
StatePublished - 1999
Externally publishedYes
Event29th Annual Frontiers in Education Conference: 'Designing the Future of Science and Engineering Education' - San Juan, Puerto Rico
Duration: Nov 10 1999Nov 13 1999

Other

Other29th Annual Frontiers in Education Conference: 'Designing the Future of Science and Engineering Education'
CitySan Juan, Puerto Rico
Period11/10/9911/13/99

Fingerprint

Engineering education
Students
Engineers
Testing

ASJC Scopus subject areas

  • Engineering(all)
  • Industrial and Manufacturing Engineering

Cite this

McMartin, F., McKenna, A., & Youssefi, K. (1999). Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education. In Proceedings - Frontiers in Education Conference (Vol. 3). Piscataway, NJ, United States: IEEE.

Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education. / McMartin, Flora; McKenna, Ann; Youssefi, Ken.

Proceedings - Frontiers in Education Conference. Vol. 3 Piscataway, NJ, United States : IEEE, 1999.

Research output: Chapter in Book/Report/Conference proceedingChapter

McMartin, F, McKenna, A & Youssefi, K 1999, Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education. in Proceedings - Frontiers in Education Conference. vol. 3, IEEE, Piscataway, NJ, United States, 29th Annual Frontiers in Education Conference: 'Designing the Future of Science and Engineering Education', San Juan, Puerto Rico, 11/10/99.
McMartin F, McKenna A, Youssefi K. Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education. In Proceedings - Frontiers in Education Conference. Vol. 3. Piscataway, NJ, United States: IEEE. 1999
McMartin, Flora ; McKenna, Ann ; Youssefi, Ken. / Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education. Proceedings - Frontiers in Education Conference. Vol. 3 Piscataway, NJ, United States : IEEE, 1999.
@inbook{5dba85129e8142e88e97463fa04ad908,
title = "Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education",
abstract = "How to establish the trustworthiness of qualitative tools to assess students' knowledge of engineering practices, teamwork, and problem solving is a question that remains unanswered for many faculty. This paper describes a process that was conducted to ensure the dependability of scenario assignments. Scenarios are a qualitative performance assessment tool designed to assess students' knowledge of engineering practices, teamwork, and problem solving. Two scoring rubrics, a holistic and an analytic, were developed to assess students' knowledge with respect to the learning outcomes associated with the scenario tool. Careful training of the scorers, analysis of the scores, and comparison of test scores determined dependability of the tool. Students were given a scenario that describes a 'day in the life' problem faced by engineers, then they are asked to explain the process they would use to solve the problem. Initial findings suggest that faculty who score the scenarios with the analytic rubric can quickly identify students' strengths and weaknesses in these areas and adapt their course to address the areas where students need attention. At the departmental or college level, scoring the scenarios with the holistic rubric is useful to assess changes in students' learning and development over time. Careful testing of the rubric revealed that the scores can reliably differentiate among student abilities.",
author = "Flora McMartin and Ann McKenna and Ken Youssefi",
year = "1999",
language = "English (US)",
isbn = "0780356438",
volume = "3",
booktitle = "Proceedings - Frontiers in Education Conference",
publisher = "IEEE",

}

TY - CHAP

T1 - Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education

AU - McMartin, Flora

AU - McKenna, Ann

AU - Youssefi, Ken

PY - 1999

Y1 - 1999

N2 - How to establish the trustworthiness of qualitative tools to assess students' knowledge of engineering practices, teamwork, and problem solving is a question that remains unanswered for many faculty. This paper describes a process that was conducted to ensure the dependability of scenario assignments. Scenarios are a qualitative performance assessment tool designed to assess students' knowledge of engineering practices, teamwork, and problem solving. Two scoring rubrics, a holistic and an analytic, were developed to assess students' knowledge with respect to the learning outcomes associated with the scenario tool. Careful training of the scorers, analysis of the scores, and comparison of test scores determined dependability of the tool. Students were given a scenario that describes a 'day in the life' problem faced by engineers, then they are asked to explain the process they would use to solve the problem. Initial findings suggest that faculty who score the scenarios with the analytic rubric can quickly identify students' strengths and weaknesses in these areas and adapt their course to address the areas where students need attention. At the departmental or college level, scoring the scenarios with the holistic rubric is useful to assess changes in students' learning and development over time. Careful testing of the rubric revealed that the scores can reliably differentiate among student abilities.

AB - How to establish the trustworthiness of qualitative tools to assess students' knowledge of engineering practices, teamwork, and problem solving is a question that remains unanswered for many faculty. This paper describes a process that was conducted to ensure the dependability of scenario assignments. Scenarios are a qualitative performance assessment tool designed to assess students' knowledge of engineering practices, teamwork, and problem solving. Two scoring rubrics, a holistic and an analytic, were developed to assess students' knowledge with respect to the learning outcomes associated with the scenario tool. Careful training of the scorers, analysis of the scores, and comparison of test scores determined dependability of the tool. Students were given a scenario that describes a 'day in the life' problem faced by engineers, then they are asked to explain the process they would use to solve the problem. Initial findings suggest that faculty who score the scenarios with the analytic rubric can quickly identify students' strengths and weaknesses in these areas and adapt their course to address the areas where students need attention. At the departmental or college level, scoring the scenarios with the holistic rubric is useful to assess changes in students' learning and development over time. Careful testing of the rubric revealed that the scores can reliably differentiate among student abilities.

UR - http://www.scopus.com/inward/record.url?scp=0033344652&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0033344652&partnerID=8YFLogxK

M3 - Chapter

AN - SCOPUS:0033344652

SN - 0780356438

VL - 3

BT - Proceedings - Frontiers in Education Conference

PB - IEEE

CY - Piscataway, NJ, United States

ER -