Multi-document Cohesion Network Analysis: Automated Prediction of Inferencing across Multiple Documents

Bogdan Nicula, Cecile A. Perret, Mihai Dascalu, Danielle S. McNamara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Open-ended comprehension questions are a common type of assessment used to evaluate how well students understand one of multiple documents. Our aim is to use natural language processing (NLP) to infer the level and type of inferencing within readers' answers to comprehension questions using the linguistic and semantic features within their responses. Our taxonomy considers three types of responses to comprehension questions from students (N=146) who read four documents: A) textbase responses (i.e., information required for the answer is present in a contiguous short sequence of text); b) single-document inference responses (i.e., requiring information from multiple text segments in a single document); and c) multi-document inference responses (i.e., information spanning multiple documents is required). The classification task was approached in two ways. First, we extracted features from students' answers to the comprehension questions using linguistic and semantic indices related to textual complexity and an extended Cohesion Network Analysis (CNA) graph to assess semantic links between the answers and the reference documents. Second, we compared different Recurrent Neural Networks (RNNs) architectures that rely on word embeddings to encode both answers and reference documents. Our best model based on RNNs predicts the answer type with an accuracy of 81%.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE 32nd International Conference on Tools with Artificial Intelligence, ICTAI 2020
EditorsMiltos Alamaniotis, Shimei Pan
PublisherIEEE Computer Society
Pages343-348
Number of pages6
ISBN (Electronic)9781728192284
DOIs
StatePublished - Nov 2020
Event32nd IEEE International Conference on Tools with Artificial Intelligence, ICTAI 2020 - Virtual, Baltimore, United States
Duration: Nov 9 2020Nov 11 2020

Publication series

NameProceedings - International Conference on Tools with Artificial Intelligence, ICTAI
Volume2020-November
ISSN (Print)1082-3409

Conference

Conference32nd IEEE International Conference on Tools with Artificial Intelligence, ICTAI 2020
Country/TerritoryUnited States
CityVirtual, Baltimore
Period11/9/2011/11/20

Keywords

  • Cohesion Network Analysis
  • Natural Language Processing
  • Predicting question type

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Multi-document Cohesion Network Analysis: Automated Prediction of Inferencing across Multiple Documents'. Together they form a unique fingerprint.

Cite this