Applied tests of design skills: Divergent thinking data analysis and reliability studies

A number of cognitive skills relevant to conceptual design were identified. They include Divergent Thinking, Visual Thinking, Spatial Reasoning, Qualitative Reasoning and Problem Formulation. A battery of standardized tests have been developed for these skills. We have previously reported on the contents and rationale for divergent thinking and visual thinking tests. This paper focuses on data collection and detailed statistical analysis of one test, namely the divergent thinking test. This particular test has been given to over 500 engineering students and a smaller number of practicing engineers. It is designed to evaluate four direct measures (fluency, flexibility, originality, quality) and four indirect measures (abstractability, afixability, detailability, decomplexability). The eight questions on the test overlap in some measures and the responses can be used to evaluate several measures independently (e.g., fluency and originality can be evaluated separately from the same idea set). The data on the 23 measured variables were factor analyzed using both exploratory and confirmatory procedures. Two variables were dropped from these exploratory analyses for reasons explained in the paper. For the remaining 21 variables, a four-factor solution with correlated (oblique) factors was deemed the best available solution after examining solutions with more factors. Five of the 21 variables did not load meaningfully on any of the four factors. These indirect measures did not appear to correlate strongly either among themselves, or with the other direct measures. The remaining 16 variables loaded on four factors as follows: The four factors correspond to the different measures belonging to each of the four questions. In other words, the different fluency, flexibility, or originality variables did not form factors limited to these forms of creative thinking. Instead the analyses showed factors associated with the questions themselves (with the exception of questions corresponding to indirect measures). The above four-factor structure was then taken into a confirmatory factor analytic procedure that adjusted for the missing data. After making some adjustments, the above four-factor solution was found to provide a reasonable fit to the data. Estimated correlations among the four factors (F) ranged from a high of .32 for F1 and F2 to a low of .06 for F3 and F4. All factor loadings were statistically significant.