Automated Scoring of Self-explanations Using Recurrent Neural Networks

Marilena Panaite, Stefan Ruseti, Mihai Dascalu, Renu Balyan, Danielle S. McNamara, Stefan Trausan-Matu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Intelligent Tutoring Systems (ITSs) focus on promoting knowledge acquisition, while providing relevant feedback during students’ practice. Self-explanation practice is an effective method used to help students understand complex texts by leveraging comprehension. Our aim is to introduce a deep learning neural model for automatically scoring student self-explanations that are targeted at specific sentences. The first stage of the processing pipeline performs an initial text cleaning and applies a set of predefined rules established by human experts in order to identify specific cases (e.g., students who do not understand the text, or students who simply copy and paste their self-explanations from the given input text). The second step uses a Recurrent Neural Network with pre-trained Glove word embeddings to predict self-explanation scores on a scale of 1 to 3. In contrast to previous SVM models trained on the same dataset of 4109 self-explanations, we obtain a significant increase of accuracy from 59% to 73%. Moreover, the new pipeline can be integrated in learning scenarios requiring near real-time responses from the ITS, thus addressing a major limitation in terms of processing speed exhibited by the previous approach.

Original languageEnglish (US)
Title of host publicationTransforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings
EditorsMaren Scheffel, Julien Broisin, Viktoria Pammer-Schindler, Andri Ioannou, Jan Schneider
PublisherSpringer Verlag
Pages659-663
Number of pages5
ISBN (Print)9783030297350
DOIs
StatePublished - Jan 1 2019
Externally publishedYes
Event14th European Conference on Technology Enhanced Learning, EC-TEL 2019 - Delft, Netherlands
Duration: Sep 16 2019Sep 19 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11722 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th European Conference on Technology Enhanced Learning, EC-TEL 2019
CountryNetherlands
CityDelft
Period9/16/199/19/19

Fingerprint

Recurrent neural networks
Recurrent Neural Networks
Scoring
Students
Intelligent Tutoring Systems
Intelligent systems
Pipelines
Knowledge Acquisition
Cleaning
Knowledge acquisition
Processing
Real-time
Predict
Scenarios
Text
Feedback
Model
Learning

Keywords

  • Comprehensive tutoring system
  • Natural Language Processing
  • Recurrent Neural Network
  • Self-explanations

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Panaite, M., Ruseti, S., Dascalu, M., Balyan, R., McNamara, D. S., & Trausan-Matu, S. (2019). Automated Scoring of Self-explanations Using Recurrent Neural Networks. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, & J. Schneider (Eds.), Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings (pp. 659-663). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11722 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-29736-7_61

Automated Scoring of Self-explanations Using Recurrent Neural Networks. / Panaite, Marilena; Ruseti, Stefan; Dascalu, Mihai; Balyan, Renu; McNamara, Danielle S.; Trausan-Matu, Stefan.

Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings. ed. / Maren Scheffel; Julien Broisin; Viktoria Pammer-Schindler; Andri Ioannou; Jan Schneider. Springer Verlag, 2019. p. 659-663 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11722 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Panaite, M, Ruseti, S, Dascalu, M, Balyan, R, McNamara, DS & Trausan-Matu, S 2019, Automated Scoring of Self-explanations Using Recurrent Neural Networks. in M Scheffel, J Broisin, V Pammer-Schindler, A Ioannou & J Schneider (eds), Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11722 LNCS, Springer Verlag, pp. 659-663, 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, Netherlands, 9/16/19. https://doi.org/10.1007/978-3-030-29736-7_61
Panaite M, Ruseti S, Dascalu M, Balyan R, McNamara DS, Trausan-Matu S. Automated Scoring of Self-explanations Using Recurrent Neural Networks. In Scheffel M, Broisin J, Pammer-Schindler V, Ioannou A, Schneider J, editors, Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings. Springer Verlag. 2019. p. 659-663. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-29736-7_61
Panaite, Marilena ; Ruseti, Stefan ; Dascalu, Mihai ; Balyan, Renu ; McNamara, Danielle S. ; Trausan-Matu, Stefan. / Automated Scoring of Self-explanations Using Recurrent Neural Networks. Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings. editor / Maren Scheffel ; Julien Broisin ; Viktoria Pammer-Schindler ; Andri Ioannou ; Jan Schneider. Springer Verlag, 2019. pp. 659-663 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{95aa3839d7244e0e87ae128534942712,
title = "Automated Scoring of Self-explanations Using Recurrent Neural Networks",
abstract = "Intelligent Tutoring Systems (ITSs) focus on promoting knowledge acquisition, while providing relevant feedback during students’ practice. Self-explanation practice is an effective method used to help students understand complex texts by leveraging comprehension. Our aim is to introduce a deep learning neural model for automatically scoring student self-explanations that are targeted at specific sentences. The first stage of the processing pipeline performs an initial text cleaning and applies a set of predefined rules established by human experts in order to identify specific cases (e.g., students who do not understand the text, or students who simply copy and paste their self-explanations from the given input text). The second step uses a Recurrent Neural Network with pre-trained Glove word embeddings to predict self-explanation scores on a scale of 1 to 3. In contrast to previous SVM models trained on the same dataset of 4109 self-explanations, we obtain a significant increase of accuracy from 59{\%} to 73{\%}. Moreover, the new pipeline can be integrated in learning scenarios requiring near real-time responses from the ITS, thus addressing a major limitation in terms of processing speed exhibited by the previous approach.",
keywords = "Comprehensive tutoring system, Natural Language Processing, Recurrent Neural Network, Self-explanations",
author = "Marilena Panaite and Stefan Ruseti and Mihai Dascalu and Renu Balyan and McNamara, {Danielle S.} and Stefan Trausan-Matu",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-29736-7_61",
language = "English (US)",
isbn = "9783030297350",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "659--663",
editor = "Maren Scheffel and Julien Broisin and Viktoria Pammer-Schindler and Andri Ioannou and Jan Schneider",
booktitle = "Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings",

}

TY - GEN

T1 - Automated Scoring of Self-explanations Using Recurrent Neural Networks

AU - Panaite, Marilena

AU - Ruseti, Stefan

AU - Dascalu, Mihai

AU - Balyan, Renu

AU - McNamara, Danielle S.

AU - Trausan-Matu, Stefan

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Intelligent Tutoring Systems (ITSs) focus on promoting knowledge acquisition, while providing relevant feedback during students’ practice. Self-explanation practice is an effective method used to help students understand complex texts by leveraging comprehension. Our aim is to introduce a deep learning neural model for automatically scoring student self-explanations that are targeted at specific sentences. The first stage of the processing pipeline performs an initial text cleaning and applies a set of predefined rules established by human experts in order to identify specific cases (e.g., students who do not understand the text, or students who simply copy and paste their self-explanations from the given input text). The second step uses a Recurrent Neural Network with pre-trained Glove word embeddings to predict self-explanation scores on a scale of 1 to 3. In contrast to previous SVM models trained on the same dataset of 4109 self-explanations, we obtain a significant increase of accuracy from 59% to 73%. Moreover, the new pipeline can be integrated in learning scenarios requiring near real-time responses from the ITS, thus addressing a major limitation in terms of processing speed exhibited by the previous approach.

AB - Intelligent Tutoring Systems (ITSs) focus on promoting knowledge acquisition, while providing relevant feedback during students’ practice. Self-explanation practice is an effective method used to help students understand complex texts by leveraging comprehension. Our aim is to introduce a deep learning neural model for automatically scoring student self-explanations that are targeted at specific sentences. The first stage of the processing pipeline performs an initial text cleaning and applies a set of predefined rules established by human experts in order to identify specific cases (e.g., students who do not understand the text, or students who simply copy and paste their self-explanations from the given input text). The second step uses a Recurrent Neural Network with pre-trained Glove word embeddings to predict self-explanation scores on a scale of 1 to 3. In contrast to previous SVM models trained on the same dataset of 4109 self-explanations, we obtain a significant increase of accuracy from 59% to 73%. Moreover, the new pipeline can be integrated in learning scenarios requiring near real-time responses from the ITS, thus addressing a major limitation in terms of processing speed exhibited by the previous approach.

KW - Comprehensive tutoring system

KW - Natural Language Processing

KW - Recurrent Neural Network

KW - Self-explanations

UR - http://www.scopus.com/inward/record.url?scp=85072976599&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072976599&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-29736-7_61

DO - 10.1007/978-3-030-29736-7_61

M3 - Conference contribution

AN - SCOPUS:85072976599

SN - 9783030297350

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 659

EP - 663

BT - Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings

A2 - Scheffel, Maren

A2 - Broisin, Julien

A2 - Pammer-Schindler, Viktoria

A2 - Ioannou, Andri

A2 - Schneider, Jan

PB - Springer Verlag

ER -