Open-book problem-solving in engineering

An exploratory study

David J. Therriault, Christine S. Lee, Elliot P. Douglas, Mirka Koro-Ljungberg, Nathan McNeill

Research output: Contribution to journalArticle

Abstract

The bulk of educational research exploring open-book examination demonstrates that the open-book format reduces student anxiety and promotes higher-level learning (i.e., reduces reliance upon rote memorization and prompts students to focus on understanding concepts and principles). Previous studies examining open-book assessment provide evidence that students exhibit better performance on open-book exams compared to closed-book exams. In addition, many university faculty find it advantageous to employ exams using an open-book format, especially in engineering. However, within engineering we know little about how students approach open-book testing, particularly with regard to how they spend their time on different tasks and how this division of time may affect performance. The study in this paper examined the testing behavior of 8 senior materials science and engineering students at a large public university in the southeastern US. Students completed four engineering problems during individual laboratory sessions while engaged in a think-aloud procedure (i.e., verbally explaining their thought processes as they worked through the problems). The problems were designed to vary in terms of their closed or open-endedness and the number of decision points involved in their solution. Students' think-aloud protocols were categorized to determine the amount of time spent on each of five exam behaviors: reading from the textbook, writing, calculating, reading the test question, and talking/reflecting. Problem solutions were separately graded using a previously created rubric. The time spent on various behavior categories were then examined with respect to grades students received for their solutions. Reading from the textbook represented the bulk of students' time on the problems (35% on average). Interestingly, there was a significant negative correlation between time spent reading the textbook and students' grades. The more time that students spent with the text, the more poorly they performed. This correlation was strongest for students who had the lowest solution scores, but was still evident for students with the highest scores. Students with the lowest scores tended to search the text for information or an example problem to guide their approach to the problem, while students with the highest scores tended to use the text to confirm their knowledge. This data suggests that for our sample the textbook may have served as a distraction. The results highlight the importance of training students in the effective use resources during open-book exams so as to avoid distractive behaviors. This training would serve not only to improve exam performance, but to educate students in the effective use of resources for professional practice where open-book problem-solving is the norm.

Original languageEnglish (US)
JournalASEE Annual Conference and Exposition, Conference Proceedings
StatePublished - 2011
Externally publishedYes

Fingerprint

Students
Textbooks
Testing
Materials science

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Open-book problem-solving in engineering : An exploratory study. / Therriault, David J.; Lee, Christine S.; Douglas, Elliot P.; Koro-Ljungberg, Mirka; McNeill, Nathan.

In: ASEE Annual Conference and Exposition, Conference Proceedings, 2011.

Research output: Contribution to journalArticle

@article{e8b240222f8e4bb38f0a577756860dd5,
title = "Open-book problem-solving in engineering: An exploratory study",
abstract = "The bulk of educational research exploring open-book examination demonstrates that the open-book format reduces student anxiety and promotes higher-level learning (i.e., reduces reliance upon rote memorization and prompts students to focus on understanding concepts and principles). Previous studies examining open-book assessment provide evidence that students exhibit better performance on open-book exams compared to closed-book exams. In addition, many university faculty find it advantageous to employ exams using an open-book format, especially in engineering. However, within engineering we know little about how students approach open-book testing, particularly with regard to how they spend their time on different tasks and how this division of time may affect performance. The study in this paper examined the testing behavior of 8 senior materials science and engineering students at a large public university in the southeastern US. Students completed four engineering problems during individual laboratory sessions while engaged in a think-aloud procedure (i.e., verbally explaining their thought processes as they worked through the problems). The problems were designed to vary in terms of their closed or open-endedness and the number of decision points involved in their solution. Students' think-aloud protocols were categorized to determine the amount of time spent on each of five exam behaviors: reading from the textbook, writing, calculating, reading the test question, and talking/reflecting. Problem solutions were separately graded using a previously created rubric. The time spent on various behavior categories were then examined with respect to grades students received for their solutions. Reading from the textbook represented the bulk of students' time on the problems (35{\%} on average). Interestingly, there was a significant negative correlation between time spent reading the textbook and students' grades. The more time that students spent with the text, the more poorly they performed. This correlation was strongest for students who had the lowest solution scores, but was still evident for students with the highest scores. Students with the lowest scores tended to search the text for information or an example problem to guide their approach to the problem, while students with the highest scores tended to use the text to confirm their knowledge. This data suggests that for our sample the textbook may have served as a distraction. The results highlight the importance of training students in the effective use resources during open-book exams so as to avoid distractive behaviors. This training would serve not only to improve exam performance, but to educate students in the effective use of resources for professional practice where open-book problem-solving is the norm.",
author = "Therriault, {David J.} and Lee, {Christine S.} and Douglas, {Elliot P.} and Mirka Koro-Ljungberg and Nathan McNeill",
year = "2011",
language = "English (US)",
journal = "ASEE Annual Conference and Exposition, Conference Proceedings",
issn = "2153-5965",

}

TY - JOUR

T1 - Open-book problem-solving in engineering

T2 - An exploratory study

AU - Therriault, David J.

AU - Lee, Christine S.

AU - Douglas, Elliot P.

AU - Koro-Ljungberg, Mirka

AU - McNeill, Nathan

PY - 2011

Y1 - 2011

N2 - The bulk of educational research exploring open-book examination demonstrates that the open-book format reduces student anxiety and promotes higher-level learning (i.e., reduces reliance upon rote memorization and prompts students to focus on understanding concepts and principles). Previous studies examining open-book assessment provide evidence that students exhibit better performance on open-book exams compared to closed-book exams. In addition, many university faculty find it advantageous to employ exams using an open-book format, especially in engineering. However, within engineering we know little about how students approach open-book testing, particularly with regard to how they spend their time on different tasks and how this division of time may affect performance. The study in this paper examined the testing behavior of 8 senior materials science and engineering students at a large public university in the southeastern US. Students completed four engineering problems during individual laboratory sessions while engaged in a think-aloud procedure (i.e., verbally explaining their thought processes as they worked through the problems). The problems were designed to vary in terms of their closed or open-endedness and the number of decision points involved in their solution. Students' think-aloud protocols were categorized to determine the amount of time spent on each of five exam behaviors: reading from the textbook, writing, calculating, reading the test question, and talking/reflecting. Problem solutions were separately graded using a previously created rubric. The time spent on various behavior categories were then examined with respect to grades students received for their solutions. Reading from the textbook represented the bulk of students' time on the problems (35% on average). Interestingly, there was a significant negative correlation between time spent reading the textbook and students' grades. The more time that students spent with the text, the more poorly they performed. This correlation was strongest for students who had the lowest solution scores, but was still evident for students with the highest scores. Students with the lowest scores tended to search the text for information or an example problem to guide their approach to the problem, while students with the highest scores tended to use the text to confirm their knowledge. This data suggests that for our sample the textbook may have served as a distraction. The results highlight the importance of training students in the effective use resources during open-book exams so as to avoid distractive behaviors. This training would serve not only to improve exam performance, but to educate students in the effective use of resources for professional practice where open-book problem-solving is the norm.

AB - The bulk of educational research exploring open-book examination demonstrates that the open-book format reduces student anxiety and promotes higher-level learning (i.e., reduces reliance upon rote memorization and prompts students to focus on understanding concepts and principles). Previous studies examining open-book assessment provide evidence that students exhibit better performance on open-book exams compared to closed-book exams. In addition, many university faculty find it advantageous to employ exams using an open-book format, especially in engineering. However, within engineering we know little about how students approach open-book testing, particularly with regard to how they spend their time on different tasks and how this division of time may affect performance. The study in this paper examined the testing behavior of 8 senior materials science and engineering students at a large public university in the southeastern US. Students completed four engineering problems during individual laboratory sessions while engaged in a think-aloud procedure (i.e., verbally explaining their thought processes as they worked through the problems). The problems were designed to vary in terms of their closed or open-endedness and the number of decision points involved in their solution. Students' think-aloud protocols were categorized to determine the amount of time spent on each of five exam behaviors: reading from the textbook, writing, calculating, reading the test question, and talking/reflecting. Problem solutions were separately graded using a previously created rubric. The time spent on various behavior categories were then examined with respect to grades students received for their solutions. Reading from the textbook represented the bulk of students' time on the problems (35% on average). Interestingly, there was a significant negative correlation between time spent reading the textbook and students' grades. The more time that students spent with the text, the more poorly they performed. This correlation was strongest for students who had the lowest solution scores, but was still evident for students with the highest scores. Students with the lowest scores tended to search the text for information or an example problem to guide their approach to the problem, while students with the highest scores tended to use the text to confirm their knowledge. This data suggests that for our sample the textbook may have served as a distraction. The results highlight the importance of training students in the effective use resources during open-book exams so as to avoid distractive behaviors. This training would serve not only to improve exam performance, but to educate students in the effective use of resources for professional practice where open-book problem-solving is the norm.

UR - http://www.scopus.com/inward/record.url?scp=85029119863&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85029119863&partnerID=8YFLogxK

M3 - Article

JO - ASEE Annual Conference and Exposition, Conference Proceedings

JF - ASEE Annual Conference and Exposition, Conference Proceedings

SN - 2153-5965

ER -