Are you talking to me? Multi-dimensional language analysis of explanations during reading

Laura K. Allen, Cecile Perret, Caitlin Mills, Danielle McNamara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This study examines the extent to which instructions to selfexplain vs. other-explain a text lead readers to produce different forms of explanations. Natural language processing was used to examine the content and characteristics of the explanations produced as a function of instruction condition. Undergraduate students (n = 146) typed either self-explanations or other-explanations while reading a science text. The linguistic properties of these explanations were calculated using three automated text analysis tools. Machine learning classifiers in combination with the features were used to predict instruction condition (i.e., self- or other-explanation). The best machine learning model performed at rates above chance (kappa = .247; accuracy = 63%). Follow-up analyses indicated that students in the self-explanation condition generated explanations that were more cohesive and that contained words that were more related to social order (e.g., ethics). Overall, the results suggest that natural language processing techniques can be used to detect subtle differences in students' processing of complex texts.

Original languageEnglish (US)
Title of host publicationProceedings of the 9th International Conference on Learning Analytics and Knowledge
Subtitle of host publicationLearning Analytics to Promote Inclusion and Success, LAK 2019
PublisherAssociation for Computing Machinery
Pages116-120
Number of pages5
ISBN (Electronic)9781450362566
DOIs
StatePublished - Mar 4 2019
Event9th International Conference on Learning Analytics and Knowledge, LAK 2019 - Tempe, United States
Duration: Mar 4 2019Mar 8 2019

Publication series

NameACM International Conference Proceeding Series

Conference

Conference9th International Conference on Learning Analytics and Knowledge, LAK 2019
CountryUnited States
CityTempe
Period3/4/193/8/19

Fingerprint

Students
Learning systems
Processing
Linguistics
Classifiers

Keywords

  • Comprehension
  • Corpus linguistics
  • Intelligent tutoring systems
  • Natural language processing
  • Reading

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Allen, L. K., Perret, C., Mills, C., & McNamara, D. (2019). Are you talking to me? Multi-dimensional language analysis of explanations during reading. In Proceedings of the 9th International Conference on Learning Analytics and Knowledge: Learning Analytics to Promote Inclusion and Success, LAK 2019 (pp. 116-120). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3303772.3303835

Are you talking to me? Multi-dimensional language analysis of explanations during reading. / Allen, Laura K.; Perret, Cecile; Mills, Caitlin; McNamara, Danielle.

Proceedings of the 9th International Conference on Learning Analytics and Knowledge: Learning Analytics to Promote Inclusion and Success, LAK 2019. Association for Computing Machinery, 2019. p. 116-120 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Allen, LK, Perret, C, Mills, C & McNamara, D 2019, Are you talking to me? Multi-dimensional language analysis of explanations during reading. in Proceedings of the 9th International Conference on Learning Analytics and Knowledge: Learning Analytics to Promote Inclusion and Success, LAK 2019. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 116-120, 9th International Conference on Learning Analytics and Knowledge, LAK 2019, Tempe, United States, 3/4/19. https://doi.org/10.1145/3303772.3303835
Allen LK, Perret C, Mills C, McNamara D. Are you talking to me? Multi-dimensional language analysis of explanations during reading. In Proceedings of the 9th International Conference on Learning Analytics and Knowledge: Learning Analytics to Promote Inclusion and Success, LAK 2019. Association for Computing Machinery. 2019. p. 116-120. (ACM International Conference Proceeding Series). https://doi.org/10.1145/3303772.3303835
Allen, Laura K. ; Perret, Cecile ; Mills, Caitlin ; McNamara, Danielle. / Are you talking to me? Multi-dimensional language analysis of explanations during reading. Proceedings of the 9th International Conference on Learning Analytics and Knowledge: Learning Analytics to Promote Inclusion and Success, LAK 2019. Association for Computing Machinery, 2019. pp. 116-120 (ACM International Conference Proceeding Series).
@inproceedings{d6e70a2f82ce46428711f87381d08d80,
title = "Are you talking to me? Multi-dimensional language analysis of explanations during reading",
abstract = "This study examines the extent to which instructions to selfexplain vs. other-explain a text lead readers to produce different forms of explanations. Natural language processing was used to examine the content and characteristics of the explanations produced as a function of instruction condition. Undergraduate students (n = 146) typed either self-explanations or other-explanations while reading a science text. The linguistic properties of these explanations were calculated using three automated text analysis tools. Machine learning classifiers in combination with the features were used to predict instruction condition (i.e., self- or other-explanation). The best machine learning model performed at rates above chance (kappa = .247; accuracy = 63{\%}). Follow-up analyses indicated that students in the self-explanation condition generated explanations that were more cohesive and that contained words that were more related to social order (e.g., ethics). Overall, the results suggest that natural language processing techniques can be used to detect subtle differences in students' processing of complex texts.",
keywords = "Comprehension, Corpus linguistics, Intelligent tutoring systems, Natural language processing, Reading",
author = "Allen, {Laura K.} and Cecile Perret and Caitlin Mills and Danielle McNamara",
year = "2019",
month = "3",
day = "4",
doi = "10.1145/3303772.3303835",
language = "English (US)",
series = "ACM International Conference Proceeding Series",
publisher = "Association for Computing Machinery",
pages = "116--120",
booktitle = "Proceedings of the 9th International Conference on Learning Analytics and Knowledge",

}

TY - GEN

T1 - Are you talking to me? Multi-dimensional language analysis of explanations during reading

AU - Allen, Laura K.

AU - Perret, Cecile

AU - Mills, Caitlin

AU - McNamara, Danielle

PY - 2019/3/4

Y1 - 2019/3/4

N2 - This study examines the extent to which instructions to selfexplain vs. other-explain a text lead readers to produce different forms of explanations. Natural language processing was used to examine the content and characteristics of the explanations produced as a function of instruction condition. Undergraduate students (n = 146) typed either self-explanations or other-explanations while reading a science text. The linguistic properties of these explanations were calculated using three automated text analysis tools. Machine learning classifiers in combination with the features were used to predict instruction condition (i.e., self- or other-explanation). The best machine learning model performed at rates above chance (kappa = .247; accuracy = 63%). Follow-up analyses indicated that students in the self-explanation condition generated explanations that were more cohesive and that contained words that were more related to social order (e.g., ethics). Overall, the results suggest that natural language processing techniques can be used to detect subtle differences in students' processing of complex texts.

AB - This study examines the extent to which instructions to selfexplain vs. other-explain a text lead readers to produce different forms of explanations. Natural language processing was used to examine the content and characteristics of the explanations produced as a function of instruction condition. Undergraduate students (n = 146) typed either self-explanations or other-explanations while reading a science text. The linguistic properties of these explanations were calculated using three automated text analysis tools. Machine learning classifiers in combination with the features were used to predict instruction condition (i.e., self- or other-explanation). The best machine learning model performed at rates above chance (kappa = .247; accuracy = 63%). Follow-up analyses indicated that students in the self-explanation condition generated explanations that were more cohesive and that contained words that were more related to social order (e.g., ethics). Overall, the results suggest that natural language processing techniques can be used to detect subtle differences in students' processing of complex texts.

KW - Comprehension

KW - Corpus linguistics

KW - Intelligent tutoring systems

KW - Natural language processing

KW - Reading

UR - http://www.scopus.com/inward/record.url?scp=85062782008&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062782008&partnerID=8YFLogxK

U2 - 10.1145/3303772.3303835

DO - 10.1145/3303772.3303835

M3 - Conference contribution

AN - SCOPUS:85062782008

T3 - ACM International Conference Proceeding Series

SP - 116

EP - 120

BT - Proceedings of the 9th International Conference on Learning Analytics and Knowledge

PB - Association for Computing Machinery

ER -