Analyzing student generated evidence for supporting design decisions

Our work is investigating the type of knowledge students use when making decisions in the process of developing design solutions. In this paper we focus on the types of evidence students provide when presenting and choosing between various design alternatives, or when they suggest a particular design approach or solution. We are interested in seeing the extent to which students use engineering disciplinary knowledge to provide evidence for making design decisions. The major aim of our work is to investigate the role that computational and analytical abilities play in innovation in the context of engineering design and how flexible students are in applying this knowledge when developing solutions. We are using the framework of adaptive expertise to focus our work, where the framework takes into account "efficiency" and "innovation" aspects of knowledge and learning. Using the adaptive expertise framework, with a specific focus on computational/analytical knowledge, we document the type of evidence students use when selecting possible design alternatives, appropriate models or methods of analysis, and when interpreting the results to justify their decisions. In previous work we analyzed student design project reports from different academic years, and from different disciplines. Specifically, our data consisted of first-year and capstone design experiences. This data set enabled us to compare the nature of students' decision making and the type of analytical knowledge used at the "bookends" of the undergraduate experience. In this work we found that student teams in capstone courses support their decisions with evidence more often than teams in the first year course. Design teams in capstone courses supported over 55% of decisions with evidence, while the first year teams supported only an average of 31% of decisions with evidence. Of the evidence we documented, students used expert sources, literature, and their disciplinary knowledge to provide reasoning for their design decisions. In addition the amount of evidence, the type of knowledge used by the teams varied by both discipline and undergraduate year. To further study the knowledge students use in decision making, our current research efforts have focused on investigating the students' decision making during multiple stages of the design process including "Problem Definition," "Detail Design," and "Design Communication¹." To gain a better understanding of the factors that influence the teams' decisions in real time, we completed a more detailed analysis for capstone design students in Biomedical Engineering. Specifically, we collected initial, midterm, and final reports to capture decision making as it evolved throughout the course. Results from this research shed light on how students use disciplinary knowledge in the process of design, what students consider to be important technical information for design, how students make design decisions, sometimes with and without appropriate evidence to support their decisions, and finally how those decisions evolve during the period of a design course.