Applied tests of design skills - Part III: Abstract reasoning

Maryam Khorshidi, Jami J. Shah, Jay Woodward

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Past studies have identified the following cognitive skills relevant to conceptual design: divergent thinking, spatial reasoning, visual thinking, abstract reasoning, and problem formulation (PF). Standardized tests are being developed to assess these skills. The tests on divergent thinking and visual thinking are fully developed and validated; this paper focuses on the development of a test of abstract reasoning in the context of engineering design. Similar to the two previous papers, this paper reports on the theoretical and empirical basis for skill identification and test development. Cognitive studies of human problem solving and design thinking revealed four indicators of abstract reasoning: qualitative deductive reasoning (DR), qualitative inductive reasoning (IR), analogical reasoning (AnR), and abductive reasoning (AbR). Each of these is characterized in terms of measurable indicators. The paper presents test construction procedures, trial runs, data collection, norming studies, and test refinement. Initial versions of the test were given to approximately 250 subjects to determine the clarity of the test problems, time allocation and to gauge the difficulty level. A protocol study was also conducted to assess test content validity. The beta version was given to approximately 100 students and the data collected was used for norming studies and test validation. Analysis of test results suggested high internal consistency; factor analysis revealed four eigenvalues above 1.0, indicating assessment of four different subskills by the test (as initially proposed by four indicators). The composite Cronbacha's alpha for all of the factors together was found to be 0.579. Future research will be conducted on criterion validity.

Original languageEnglish (US)
Article number101101
JournalJournal of Mechanical Design, Transactions Of the ASME
Volume136
Issue number10
DOIs
StatePublished - 2014

Fingerprint

Factor analysis
Conceptual design
Gages
Students
Composite materials

Keywords

  • abstract reasoning
  • design skills
  • engineering design
  • standardized tests

ASJC Scopus subject areas

  • Mechanical Engineering
  • Mechanics of Materials
  • Computer Graphics and Computer-Aided Design
  • Computer Science Applications

Cite this

Applied tests of design skills - Part III : Abstract reasoning. / Khorshidi, Maryam; Shah, Jami J.; Woodward, Jay.

In: Journal of Mechanical Design, Transactions Of the ASME, Vol. 136, No. 10, 101101, 2014.

Research output: Contribution to journalArticle

Khorshidi, Maryam ; Shah, Jami J. ; Woodward, Jay. / Applied tests of design skills - Part III : Abstract reasoning. In: Journal of Mechanical Design, Transactions Of the ASME. 2014 ; Vol. 136, No. 10.
@article{f5fe19543d7e4d49942a89d737961a11,
title = "Applied tests of design skills - Part III: Abstract reasoning",
abstract = "Past studies have identified the following cognitive skills relevant to conceptual design: divergent thinking, spatial reasoning, visual thinking, abstract reasoning, and problem formulation (PF). Standardized tests are being developed to assess these skills. The tests on divergent thinking and visual thinking are fully developed and validated; this paper focuses on the development of a test of abstract reasoning in the context of engineering design. Similar to the two previous papers, this paper reports on the theoretical and empirical basis for skill identification and test development. Cognitive studies of human problem solving and design thinking revealed four indicators of abstract reasoning: qualitative deductive reasoning (DR), qualitative inductive reasoning (IR), analogical reasoning (AnR), and abductive reasoning (AbR). Each of these is characterized in terms of measurable indicators. The paper presents test construction procedures, trial runs, data collection, norming studies, and test refinement. Initial versions of the test were given to approximately 250 subjects to determine the clarity of the test problems, time allocation and to gauge the difficulty level. A protocol study was also conducted to assess test content validity. The beta version was given to approximately 100 students and the data collected was used for norming studies and test validation. Analysis of test results suggested high internal consistency; factor analysis revealed four eigenvalues above 1.0, indicating assessment of four different subskills by the test (as initially proposed by four indicators). The composite Cronbacha's alpha for all of the factors together was found to be 0.579. Future research will be conducted on criterion validity.",
keywords = "abstract reasoning, design skills, engineering design, standardized tests",
author = "Maryam Khorshidi and Shah, {Jami J.} and Jay Woodward",
year = "2014",
doi = "10.1115/1.4027986",
language = "English (US)",
volume = "136",
journal = "Journal of Mechanical Design - Transactions of the ASME",
issn = "1050-0472",
publisher = "American Society of Mechanical Engineers(ASME)",
number = "10",

}

TY - JOUR

T1 - Applied tests of design skills - Part III

T2 - Abstract reasoning

AU - Khorshidi, Maryam

AU - Shah, Jami J.

AU - Woodward, Jay

PY - 2014

Y1 - 2014

N2 - Past studies have identified the following cognitive skills relevant to conceptual design: divergent thinking, spatial reasoning, visual thinking, abstract reasoning, and problem formulation (PF). Standardized tests are being developed to assess these skills. The tests on divergent thinking and visual thinking are fully developed and validated; this paper focuses on the development of a test of abstract reasoning in the context of engineering design. Similar to the two previous papers, this paper reports on the theoretical and empirical basis for skill identification and test development. Cognitive studies of human problem solving and design thinking revealed four indicators of abstract reasoning: qualitative deductive reasoning (DR), qualitative inductive reasoning (IR), analogical reasoning (AnR), and abductive reasoning (AbR). Each of these is characterized in terms of measurable indicators. The paper presents test construction procedures, trial runs, data collection, norming studies, and test refinement. Initial versions of the test were given to approximately 250 subjects to determine the clarity of the test problems, time allocation and to gauge the difficulty level. A protocol study was also conducted to assess test content validity. The beta version was given to approximately 100 students and the data collected was used for norming studies and test validation. Analysis of test results suggested high internal consistency; factor analysis revealed four eigenvalues above 1.0, indicating assessment of four different subskills by the test (as initially proposed by four indicators). The composite Cronbacha's alpha for all of the factors together was found to be 0.579. Future research will be conducted on criterion validity.

AB - Past studies have identified the following cognitive skills relevant to conceptual design: divergent thinking, spatial reasoning, visual thinking, abstract reasoning, and problem formulation (PF). Standardized tests are being developed to assess these skills. The tests on divergent thinking and visual thinking are fully developed and validated; this paper focuses on the development of a test of abstract reasoning in the context of engineering design. Similar to the two previous papers, this paper reports on the theoretical and empirical basis for skill identification and test development. Cognitive studies of human problem solving and design thinking revealed four indicators of abstract reasoning: qualitative deductive reasoning (DR), qualitative inductive reasoning (IR), analogical reasoning (AnR), and abductive reasoning (AbR). Each of these is characterized in terms of measurable indicators. The paper presents test construction procedures, trial runs, data collection, norming studies, and test refinement. Initial versions of the test were given to approximately 250 subjects to determine the clarity of the test problems, time allocation and to gauge the difficulty level. A protocol study was also conducted to assess test content validity. The beta version was given to approximately 100 students and the data collected was used for norming studies and test validation. Analysis of test results suggested high internal consistency; factor analysis revealed four eigenvalues above 1.0, indicating assessment of four different subskills by the test (as initially proposed by four indicators). The composite Cronbacha's alpha for all of the factors together was found to be 0.579. Future research will be conducted on criterion validity.

KW - abstract reasoning

KW - design skills

KW - engineering design

KW - standardized tests

UR - http://www.scopus.com/inward/record.url?scp=84905494274&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84905494274&partnerID=8YFLogxK

U2 - 10.1115/1.4027986

DO - 10.1115/1.4027986

M3 - Article

AN - SCOPUS:84905494274

VL - 136

JO - Journal of Mechanical Design - Transactions of the ASME

JF - Journal of Mechanical Design - Transactions of the ASME

SN - 1050-0472

IS - 10

M1 - 101101

ER -