Predicting multi-document comprehension: Cohesion network analysis

Bogdan Nicula, Cecile A. Perret, Mihai Dascalu, Danielle McNamara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Theories of discourse comprehension assume that understanding is a process of making connections between new information (e.g., in a text) and prior knowledge, and that the quality of comprehension is a function of the coherence of the mental representation. When readers are exposed to multiple sources of information, they must make connections both within and between the texts. One challenge is how to represent this coherence and in turn how to predict readers’ levels of comprehension. In this study, we represent coherence using Cohesion Network Analysis (CNA) in which we model a global cohesion graph that semantically links reference texts to different student verbal productions. Our aim is to create an automated model of comprehension prediction based on features extracted from the CNA graph. We examine the cohesion links between the four texts read by 146 students and their (a) self-explanations generated on target sentences and (b) responses to open-ended questions. We analyze the degree to which features derived from the cohesive links from the extended CNA graph are predictive of students’ comprehension scores (on a [0 to 12] scale) using either (a) students’ self-explanations, (b) responses to comprehension questions, or (c) both. We compared the use of Linear Regression, Extra Trees Regressor, Support Vector Regression, and Multi-Layer Perceptron. Our best model used Linear Regression, obtaining a 1.29 mean absolute error when predicting comprehension scores using both sources of verbal responses (i.e., self-explanations and question answers).

Original languageEnglish (US)
Title of host publicationArtificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings
EditorsSeiji Isotani, Peter Hastings, Amy Ogan, Bruce McLaren, Eva Millán, Rose Luckin
PublisherSpringer Verlag
Pages358-369
Number of pages12
ISBN (Print)9783030232030
DOIs
StatePublished - Jan 1 2019
Event20th International Conference on Artificial Intelligence in Education, AIED 2019 - Chicago, United States
Duration: Jun 25 2019Jun 29 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11625 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference20th International Conference on Artificial Intelligence in Education, AIED 2019
CountryUnited States
CityChicago
Period6/25/196/29/19

Fingerprint

Cohesion
Network Analysis
Electric network analysis
Students
Linear regression
Graph in graph theory
Multilayer neural networks
Support Vector Regression
Linear Regression Model
Perceptron
Prior Knowledge
Multilayer
Predict
Target
Text
Prediction
Model

Keywords

  • Cohesion network analysis
  • Comprehension modeling
  • Machine learning
  • Multi-document comprehension and integration
  • Natural language processing

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Nicula, B., Perret, C. A., Dascalu, M., & McNamara, D. (2019). Predicting multi-document comprehension: Cohesion network analysis. In S. Isotani, P. Hastings, A. Ogan, B. McLaren, E. Millán, & R. Luckin (Eds.), Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings (pp. 358-369). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11625 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-030-23204-7_30

Predicting multi-document comprehension : Cohesion network analysis. / Nicula, Bogdan; Perret, Cecile A.; Dascalu, Mihai; McNamara, Danielle.

Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings. ed. / Seiji Isotani; Peter Hastings; Amy Ogan; Bruce McLaren; Eva Millán; Rose Luckin. Springer Verlag, 2019. p. 358-369 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11625 LNAI).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nicula, B, Perret, CA, Dascalu, M & McNamara, D 2019, Predicting multi-document comprehension: Cohesion network analysis. in S Isotani, P Hastings, A Ogan, B McLaren, E Millán & R Luckin (eds), Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11625 LNAI, Springer Verlag, pp. 358-369, 20th International Conference on Artificial Intelligence in Education, AIED 2019, Chicago, United States, 6/25/19. https://doi.org/10.1007/978-3-030-23204-7_30
Nicula B, Perret CA, Dascalu M, McNamara D. Predicting multi-document comprehension: Cohesion network analysis. In Isotani S, Hastings P, Ogan A, McLaren B, Millán E, Luckin R, editors, Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings. Springer Verlag. 2019. p. 358-369. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-23204-7_30
Nicula, Bogdan ; Perret, Cecile A. ; Dascalu, Mihai ; McNamara, Danielle. / Predicting multi-document comprehension : Cohesion network analysis. Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings. editor / Seiji Isotani ; Peter Hastings ; Amy Ogan ; Bruce McLaren ; Eva Millán ; Rose Luckin. Springer Verlag, 2019. pp. 358-369 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{eab63e5ff6584baeb4aa21532c9a3e93,
title = "Predicting multi-document comprehension: Cohesion network analysis",
abstract = "Theories of discourse comprehension assume that understanding is a process of making connections between new information (e.g., in a text) and prior knowledge, and that the quality of comprehension is a function of the coherence of the mental representation. When readers are exposed to multiple sources of information, they must make connections both within and between the texts. One challenge is how to represent this coherence and in turn how to predict readers’ levels of comprehension. In this study, we represent coherence using Cohesion Network Analysis (CNA) in which we model a global cohesion graph that semantically links reference texts to different student verbal productions. Our aim is to create an automated model of comprehension prediction based on features extracted from the CNA graph. We examine the cohesion links between the four texts read by 146 students and their (a) self-explanations generated on target sentences and (b) responses to open-ended questions. We analyze the degree to which features derived from the cohesive links from the extended CNA graph are predictive of students’ comprehension scores (on a [0 to 12] scale) using either (a){\^A} students’ self-explanations, (b){\^A} responses to comprehension questions, or (c){\^A} both. We compared the use of Linear Regression, Extra Trees Regressor, Support Vector Regression, and Multi-Layer Perceptron. Our best model used Linear Regression, obtaining a 1.29 mean absolute error when predicting comprehension scores using both sources of verbal responses (i.e., self-explanations and question answers).",
keywords = "Cohesion network analysis, Comprehension modeling, Machine learning, Multi-document comprehension and integration, Natural language processing",
author = "Bogdan Nicula and Perret, {Cecile A.} and Mihai Dascalu and Danielle McNamara",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-23204-7_30",
language = "English (US)",
isbn = "9783030232030",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "358--369",
editor = "Seiji Isotani and Peter Hastings and Amy Ogan and Bruce McLaren and Eva Mill{\'a}n and Rose Luckin",
booktitle = "Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings",

}

TY - GEN

T1 - Predicting multi-document comprehension

T2 - Cohesion network analysis

AU - Nicula, Bogdan

AU - Perret, Cecile A.

AU - Dascalu, Mihai

AU - McNamara, Danielle

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Theories of discourse comprehension assume that understanding is a process of making connections between new information (e.g., in a text) and prior knowledge, and that the quality of comprehension is a function of the coherence of the mental representation. When readers are exposed to multiple sources of information, they must make connections both within and between the texts. One challenge is how to represent this coherence and in turn how to predict readers’ levels of comprehension. In this study, we represent coherence using Cohesion Network Analysis (CNA) in which we model a global cohesion graph that semantically links reference texts to different student verbal productions. Our aim is to create an automated model of comprehension prediction based on features extracted from the CNA graph. We examine the cohesion links between the four texts read by 146 students and their (a) self-explanations generated on target sentences and (b) responses to open-ended questions. We analyze the degree to which features derived from the cohesive links from the extended CNA graph are predictive of students’ comprehension scores (on a [0 to 12] scale) using either (a) students’ self-explanations, (b) responses to comprehension questions, or (c) both. We compared the use of Linear Regression, Extra Trees Regressor, Support Vector Regression, and Multi-Layer Perceptron. Our best model used Linear Regression, obtaining a 1.29 mean absolute error when predicting comprehension scores using both sources of verbal responses (i.e., self-explanations and question answers).

AB - Theories of discourse comprehension assume that understanding is a process of making connections between new information (e.g., in a text) and prior knowledge, and that the quality of comprehension is a function of the coherence of the mental representation. When readers are exposed to multiple sources of information, they must make connections both within and between the texts. One challenge is how to represent this coherence and in turn how to predict readers’ levels of comprehension. In this study, we represent coherence using Cohesion Network Analysis (CNA) in which we model a global cohesion graph that semantically links reference texts to different student verbal productions. Our aim is to create an automated model of comprehension prediction based on features extracted from the CNA graph. We examine the cohesion links between the four texts read by 146 students and their (a) self-explanations generated on target sentences and (b) responses to open-ended questions. We analyze the degree to which features derived from the cohesive links from the extended CNA graph are predictive of students’ comprehension scores (on a [0 to 12] scale) using either (a) students’ self-explanations, (b) responses to comprehension questions, or (c) both. We compared the use of Linear Regression, Extra Trees Regressor, Support Vector Regression, and Multi-Layer Perceptron. Our best model used Linear Regression, obtaining a 1.29 mean absolute error when predicting comprehension scores using both sources of verbal responses (i.e., self-explanations and question answers).

KW - Cohesion network analysis

KW - Comprehension modeling

KW - Machine learning

KW - Multi-document comprehension and integration

KW - Natural language processing

UR - http://www.scopus.com/inward/record.url?scp=85068328134&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068328134&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-23204-7_30

DO - 10.1007/978-3-030-23204-7_30

M3 - Conference contribution

AN - SCOPUS:85068328134

SN - 9783030232030

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 358

EP - 369

BT - Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings

A2 - Isotani, Seiji

A2 - Hastings, Peter

A2 - Ogan, Amy

A2 - McLaren, Bruce

A2 - Millán, Eva

A2 - Luckin, Rose

PB - Springer Verlag

ER -