An evaluation of a hybrid language understanding approach for robust selection of tutoring goals

Carolyn Rosé, Kurt VanLehn

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

In this paper, we explore the problem of selecting appropriate interventions for students based on an analysis of their interactions with a tutoring system. In the context of the WHY2 conceptual physics tutoring system, we describe CarmelTC, a hybrid symbolic/statistical approach for analysing conceptual physics explanations in order to determine which Knowledge Construction Dialogues (KCDs) students need for the purpose of encouraging them to include important points that are missing. We briefly describe our tutoring approach. We then present a model that demonstrates a general problem with selecting interventions based on an analysis of student performance in circumstances where there is uncertainty with the interpretation, such as with speech or text based natural language input, complex and error prone mathematical or other formal language input, graphical input (i.e., diagrams, etc.), or gestures. In particular, when student performance completeness is high, intervention selection accuracy is more sensitive to analysis accuracy, and increasingly so as performance completeness increases. In light of this model, we have evaluated our CarmelTC approach and have demonstrated that it performs favourably in comparison with the widely used LSA approach, a Naive Bayes approach, and finally a purely symbolic approach.

Original languageEnglish (US)
Pages (from-to)325-355
Number of pages31
JournalInternational Journal of Artificial Intelligence in Education
Volume15
Issue number4
StatePublished - 2005
Externally publishedYes

Fingerprint

Students
language
evaluation
physics
Physics
student
performance
Formal languages
dialogue
uncertainty
interpretation
interaction
Uncertainty

Keywords

  • Evaluation
  • Language understanding
  • Tutorial dialogue

ASJC Scopus subject areas

  • Education
  • Computational Theory and Mathematics

Cite this

@article{d90eeaf231134382aa20d898a5e719ec,
title = "An evaluation of a hybrid language understanding approach for robust selection of tutoring goals",
abstract = "In this paper, we explore the problem of selecting appropriate interventions for students based on an analysis of their interactions with a tutoring system. In the context of the WHY2 conceptual physics tutoring system, we describe CarmelTC, a hybrid symbolic/statistical approach for analysing conceptual physics explanations in order to determine which Knowledge Construction Dialogues (KCDs) students need for the purpose of encouraging them to include important points that are missing. We briefly describe our tutoring approach. We then present a model that demonstrates a general problem with selecting interventions based on an analysis of student performance in circumstances where there is uncertainty with the interpretation, such as with speech or text based natural language input, complex and error prone mathematical or other formal language input, graphical input (i.e., diagrams, etc.), or gestures. In particular, when student performance completeness is high, intervention selection accuracy is more sensitive to analysis accuracy, and increasingly so as performance completeness increases. In light of this model, we have evaluated our CarmelTC approach and have demonstrated that it performs favourably in comparison with the widely used LSA approach, a Naive Bayes approach, and finally a purely symbolic approach.",
keywords = "Evaluation, Language understanding, Tutorial dialogue",
author = "Carolyn Ros{\'e} and Kurt VanLehn",
year = "2005",
language = "English (US)",
volume = "15",
pages = "325--355",
journal = "International Journal of Artificial Intelligence in Education",
issn = "1560-4292",
publisher = "IOS Press",
number = "4",

}

TY - JOUR

T1 - An evaluation of a hybrid language understanding approach for robust selection of tutoring goals

AU - Rosé, Carolyn

AU - VanLehn, Kurt

PY - 2005

Y1 - 2005

N2 - In this paper, we explore the problem of selecting appropriate interventions for students based on an analysis of their interactions with a tutoring system. In the context of the WHY2 conceptual physics tutoring system, we describe CarmelTC, a hybrid symbolic/statistical approach for analysing conceptual physics explanations in order to determine which Knowledge Construction Dialogues (KCDs) students need for the purpose of encouraging them to include important points that are missing. We briefly describe our tutoring approach. We then present a model that demonstrates a general problem with selecting interventions based on an analysis of student performance in circumstances where there is uncertainty with the interpretation, such as with speech or text based natural language input, complex and error prone mathematical or other formal language input, graphical input (i.e., diagrams, etc.), or gestures. In particular, when student performance completeness is high, intervention selection accuracy is more sensitive to analysis accuracy, and increasingly so as performance completeness increases. In light of this model, we have evaluated our CarmelTC approach and have demonstrated that it performs favourably in comparison with the widely used LSA approach, a Naive Bayes approach, and finally a purely symbolic approach.

AB - In this paper, we explore the problem of selecting appropriate interventions for students based on an analysis of their interactions with a tutoring system. In the context of the WHY2 conceptual physics tutoring system, we describe CarmelTC, a hybrid symbolic/statistical approach for analysing conceptual physics explanations in order to determine which Knowledge Construction Dialogues (KCDs) students need for the purpose of encouraging them to include important points that are missing. We briefly describe our tutoring approach. We then present a model that demonstrates a general problem with selecting interventions based on an analysis of student performance in circumstances where there is uncertainty with the interpretation, such as with speech or text based natural language input, complex and error prone mathematical or other formal language input, graphical input (i.e., diagrams, etc.), or gestures. In particular, when student performance completeness is high, intervention selection accuracy is more sensitive to analysis accuracy, and increasingly so as performance completeness increases. In light of this model, we have evaluated our CarmelTC approach and have demonstrated that it performs favourably in comparison with the widely used LSA approach, a Naive Bayes approach, and finally a purely symbolic approach.

KW - Evaluation

KW - Language understanding

KW - Tutorial dialogue

UR - http://www.scopus.com/inward/record.url?scp=85013590201&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85013590201&partnerID=8YFLogxK

M3 - Article

VL - 15

SP - 325

EP - 355

JO - International Journal of Artificial Intelligence in Education

JF - International Journal of Artificial Intelligence in Education

SN - 1560-4292

IS - 4

ER -