Open-book problem-solving in engineering: An exploratory study

The bulk of educational research exploring open-book examination demonstrates that the open-book format reduces student anxiety and promotes higher-level learning (i.e., reduces reliance upon rote memorization and prompts students to focus on understanding concepts and principles). Previous studies examining open-book assessment provide evidence that students exhibit better performance on open-book exams compared to closed-book exams. In addition, many university faculty find it advantageous to employ exams using an open-book format, especially in engineering. However, within engineering we know little about how students approach open-book testing, particularly with regard to how they spend their time on different tasks and how this division of time may affect performance. The study in this paper examined the testing behavior of 8 senior materials science and engineering students at a large public university in the southeastern US. Students completed four engineering problems during individual laboratory sessions while engaged in a think-aloud procedure (i.e., verbally explaining their thought processes as they worked through the problems). The problems were designed to vary in terms of their closed or open-endedness and the number of decision points involved in their solution. Students' think-aloud protocols were categorized to determine the amount of time spent on each of five exam behaviors: reading from the textbook, writing, calculating, reading the test question, and talking/reflecting. Problem solutions were separately graded using a previously created rubric. The time spent on various behavior categories were then examined with respect to grades students received for their solutions. Reading from the textbook represented the bulk of students' time on the problems (35% on average). Interestingly, there was a significant negative correlation between time spent reading the textbook and students' grades. The more time that students spent with the text, the more poorly they performed. This correlation was strongest for students who had the lowest solution scores, but was still evident for students with the highest scores. Students with the lowest scores tended to search the text for information or an example problem to guide their approach to the problem, while students with the highest scores tended to use the text to confirm their knowledge. This data suggests that for our sample the textbook may have served as a distraction. The results highlight the importance of training students in the effective use resources during open-book exams so as to avoid distractive behaviors. This training would serve not only to improve exam performance, but to educate students in the effective use of resources for professional practice where open-book problem-solving is the norm.