Assessing cognitively complex strategy use in an untrained domain

George T. Jackson, Rebekah H. Guess, Danielle S. Mcnamara

Research output: Contribution to journalArticle

30 Scopus citations

Abstract

Researchers of advanced technologies are constantly seeking new ways of measuring and adapting to user performance. Appropriately adapting system feedback requires accurate assessments of user performance. Unfortunately, many assessment algorithms must be trained on and use pre-prepared data sets or corpora to provide a sufficiently accurate portrayal of user knowledge and behavior. However, if the targeted content of the tutoring system changes depending on the situation, the assessment algorithms must be sufficiently independent to apply to untrained content. Such is the case for Interactive Strategy Training for Active Reading and Thinking (iSTART), an intelligent tutoring system that assesses the cognitive complexity of strategy use while a reader self-explains a text. iSTART is designed so that teachers and researchers may add their own (new) texts into the system. The current paper explores student self-explanations from newly added texts (which iSTART had not been trained on) and focuses on evaluating the iSTART assessment algorithm by comparing it to human ratings of the students' self-explanations.

Original languageEnglish (US)
Pages (from-to)127-137
Number of pages11
JournalTopics in Cognitive Science
Volume2
Issue number1
DOIs
StatePublished - Jan 1 2010

Keywords

  • Automatic assessment
  • Empirical validation
  • Intelligent tutoring systems
  • Reading strategies

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Linguistics and Language
  • Human-Computer Interaction
  • Cognitive Neuroscience
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Assessing cognitively complex strategy use in an untrained domain'. Together they form a unique fingerprint.

  • Cite this