Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education

Flora McMartin, Ann McKenna, Ken Youssefi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

How to establish the trustworthiness of qualitative tools to assess students' knowledge of engineering practices, teamwork, and problem solving is a question that remains unanswered for many faculty. This paper describes a process that was conducted to ensure the dependability of scenario assignments. Scenarios are a qualitative performance assessment tool designed to assess students' knowledge of engineering practices, teamwork, and problem solving. Two scoring rubrics, a holistic and an analytic, were developed to assess students' knowledge with respect to the learning outcomes associated with the scenario tool. Careful training of the scorers, analysis of the scores, and comparison of test scores determined dependability of the tool. Students were given a scenario that describes a 'day in the life' problem faced by engineers, then they are asked to explain the process they would use to solve the problem. Initial findings suggest that faculty who score the scenarios with the analytic rubric can quickly identify students' strengths and weaknesses in these areas and adapt their course to address the areas where students need attention. At the departmental or college level, scoring the scenarios with the holistic rubric is useful to assess changes in students' learning and development over time. Careful testing of the rubric revealed that the scores can reliably differentiate among student abilities.

Original languageEnglish (US)
Title of host publicationProceedings - Frontiers in Education Conference
PublisherIEEE
Pages13c1-7 - 13c1-13
ISBN (Print)0780356438
StatePublished - Dec 1 1999
Externally publishedYes
Event29th Annual Frontiers in Education Conference: 'Designing the Future of Science and Engineering Education' - San Juan, Puerto Rico
Duration: Nov 10 1999Nov 13 1999

Publication series

NameProceedings - Frontiers in Education Conference
Volume3
ISSN (Print)0190-5848

Other

Other29th Annual Frontiers in Education Conference: 'Designing the Future of Science and Engineering Education'
CitySan Juan, Puerto Rico
Period11/10/9911/13/99

ASJC Scopus subject areas

  • Software
  • Education
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education'. Together they form a unique fingerprint.

  • Cite this

    McMartin, F., McKenna, A., & Youssefi, K. (1999). Establishing the trustworthiness of scenario assignments as assessment tools for undergraduate engineering education. In Proceedings - Frontiers in Education Conference (pp. 13c1-7 - 13c1-13). (Proceedings - Frontiers in Education Conference; Vol. 3). IEEE.