Toward an On-Line Knowledge Assessment Methodology: Building on the Relationship between Knowing and Doing

The researchers describe steps taken toward the development of an on-line knowledge assessment methodology in the airborne electronics troubleshooting domain. On-line assessment in this complex, ill-defined domain requires an empirical investigation of expert or ideal knowledge and a means to infer knowledge patterns from action patterns. In Study 1, actions taken while troubleshooting were examined to develop a means for identifying meaningful action patterns. Pathfinder analyses of action sequences meaningfully distinguished high and low performers and identified intervention targets. In Study 2, methods for measuring students' knowledge were examined in terms of their respective abilities to predict troubleshooting performance. Three of the evaluated measures resulted in knowledge representations that were predictive of performance. Our findings have implications for capturing conceptual models and assessing knowledge in complex domains.