Bring it on! Challenges encountered while building a comprehensive tutoring system using ReaderBench

Marilena Panaite, Mihai Dascalu, Amy Johnson, Renu Balyan, Jianmin Dai, Danielle McNamara, Stefan Trausan-Matu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students’ practice activities. ITSs for literacy instruction commonly assess typed responses using Natural Language Processing (NLP) algorithms. One step in this direction often requires building a scoring mechanism that matches human judgments. This paper describes the challenges encountered while implementing an automated evaluation workflow and adopting solutions for increasing performance of the tutoring system. The algorithm described here comprises multiple stages, including initial pre-processing, a rule-based system for pre-classifying self-explanations, followed by classification using a Support Virtual Machine (SVM) learning algorithm. The SVM model hyper-parameters were optimized using grid search approach with 4,109 different self-explanations scored 0 to 3 (i.e., poor to great). The accuracy achieved for the model was 59% (adjacent accuracy = 97%; Kappa =.43).

Original languageEnglish (US)
Title of host publicationArtificial Intelligence in Education - 19th International Conference, AIED 2018, Proceedings
EditorsManolis Mavrikis, Carolyn Penstein Rosé, Bruce McLaren, H. Ulrich Hoppe, Rose Luckin, Kaska Porayska-Pomsta, Benedict du Boulay, Roberto Martinez-Maldonado
PublisherSpringer Verlag
Pages409-419
Number of pages11
ISBN (Print)9783319938424
DOIs
StatePublished - 2018
Event19th International Conference on Artificial Intelligence in Education, AIED 2018 - London, United Kingdom
Duration: Jun 27 2018Jun 30 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10947 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other19th International Conference on Artificial Intelligence in Education, AIED 2018
Country/TerritoryUnited Kingdom
CityLondon
Period6/27/186/30/18

Keywords

  • Intelligent tutoring systems
  • Natural language processing
  • ReaderBench
  • Self-explanations
  • Support vector machines

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Bring it on! Challenges encountered while building a comprehensive tutoring system using ReaderBench'. Together they form a unique fingerprint.

Cite this