TY - GEN
T1 - Bring it on! Challenges encountered while building a comprehensive tutoring system using ReaderBench
AU - Panaite, Marilena
AU - Dascalu, Mihai
AU - Johnson, Amy
AU - Balyan, Renu
AU - Dai, Jianmin
AU - McNamara, Danielle
AU - Trausan-Matu, Stefan
N1 - Funding Information:
Acknowledgments. This research was partially supported by the 644187 EC H2020 Realising an Applied Gaming Eco-system (RAGE) project, the FP7 2008-212578 LTfLL project, the Department of Education, Institute of Education Sciences - Grant R305A130124, as well as by the Department of Defense, Office of Naval Research - Grants N00014140343 and N000141712300. We would also like to thank Tricia Guerrero and Matthew Jacovina for their support in scoring the self-explanations.
Funding Information:
This research was partially supported by the 644187 EC H2020 Realising an Applied Gaming Eco-system (RAGE) project, the FP7 2008-212578 LTfLL project, the Department of Education, Institute of Education Sciences-Grant R305A130124, as well as by the Department of Defense, Office of Naval Research-Grants N00014140343 and N000141712300. We would also like to thank Tricia Guerrero and Matthew Jacovina for their support in scoring the self-explanations.
Publisher Copyright:
© Springer International Publishing AG, part of Springer Nature 2018.
PY - 2018
Y1 - 2018
N2 - Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students’ practice activities. ITSs for literacy instruction commonly assess typed responses using Natural Language Processing (NLP) algorithms. One step in this direction often requires building a scoring mechanism that matches human judgments. This paper describes the challenges encountered while implementing an automated evaluation workflow and adopting solutions for increasing performance of the tutoring system. The algorithm described here comprises multiple stages, including initial pre-processing, a rule-based system for pre-classifying self-explanations, followed by classification using a Support Virtual Machine (SVM) learning algorithm. The SVM model hyper-parameters were optimized using grid search approach with 4,109 different self-explanations scored 0 to 3 (i.e., poor to great). The accuracy achieved for the model was 59% (adjacent accuracy = 97%; Kappa =.43).
AB - Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students’ practice activities. ITSs for literacy instruction commonly assess typed responses using Natural Language Processing (NLP) algorithms. One step in this direction often requires building a scoring mechanism that matches human judgments. This paper describes the challenges encountered while implementing an automated evaluation workflow and adopting solutions for increasing performance of the tutoring system. The algorithm described here comprises multiple stages, including initial pre-processing, a rule-based system for pre-classifying self-explanations, followed by classification using a Support Virtual Machine (SVM) learning algorithm. The SVM model hyper-parameters were optimized using grid search approach with 4,109 different self-explanations scored 0 to 3 (i.e., poor to great). The accuracy achieved for the model was 59% (adjacent accuracy = 97%; Kappa =.43).
KW - Intelligent tutoring systems
KW - Natural language processing
KW - ReaderBench
KW - Self-explanations
KW - Support vector machines
UR - http://www.scopus.com/inward/record.url?scp=85049369683&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85049369683&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-93843-1_30
DO - 10.1007/978-3-319-93843-1_30
M3 - Conference contribution
AN - SCOPUS:85049369683
SN - 9783319938424
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 409
EP - 419
BT - Artificial Intelligence in Education - 19th International Conference, AIED 2018, Proceedings
A2 - Mavrikis, Manolis
A2 - Penstein Rosé, Carolyn
A2 - McLaren, Bruce
A2 - Hoppe, H. Ulrich
A2 - Luckin, Rose
A2 - Porayska-Pomsta, Kaska
A2 - du Boulay, Benedict
A2 - Martinez-Maldonado, Roberto
PB - Springer Verlag
T2 - 19th International Conference on Artificial Intelligence in Education, AIED 2018
Y2 - 27 June 2018 through 30 June 2018
ER -