Investigating boredom and engagement during writing using multiple sources of information: The essay, the writer, and keystrokes

Laura K. Allen, Caitlin Mills, Matthew E. Jacovina, Scott Crossley, Sidney D'mello, Danielle McNamara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

Writing training systems have been developed to provide students with instruction and deliberate practice on their writing. Although generally successful in providing accurate scores, a common criticism of these systems is their lack of personalization and adaptive instruction. In particular, these systems tend to place the strongest emphasis on delivering accurate scores, and therefore, tend to overlook additional indices that may contribute to students' success, such as their affective states during writing practice. This study takes an initial step toward addressing this gap by building a predictive model of students' affect using information that can potentially be collected by computer systems. We used individual difference measures, text indices, and keystroke analyses to predict engagement and boredom in 132 writing sessions. The results suggest that these three categories of indices were successful in modeling students' affective states during writing. Taken together, indices related to students' academic abilities, text properties, and keystroke logs were able classify high and low engagement and boredom in writing sessions with accuracies between 76.5% and 77.3%. These results suggest that information readily available in writing training systems can inform affect detectors and ultimately improve student models within intelligent tutoring systems.

Original languageEnglish (US)
Title of host publicationLAK 2016 Conference Proceedings, 6th International Learning Analytics and Knowledge Conference - Enhancing Impact: Convergence of Communities for Grounding, Implementation, and Validation
PublisherAssociation for Computing Machinery
Pages114-123
Number of pages10
Volume25-29-April-2016
ISBN (Electronic)9781450341905
DOIs
StatePublished - Apr 25 2016
Externally publishedYes
Event6th International Conference on Learning Analytics and Knowledge, LAK 2016 - Edinburgh, United Kingdom
Duration: Apr 25 2016Apr 29 2016

Other

Other6th International Conference on Learning Analytics and Knowledge, LAK 2016
CountryUnited Kingdom
CityEdinburgh
Period4/25/164/29/16

Fingerprint

Students
Intelligent systems
Computer systems
Detectors

Keywords

  • Corpus linguistics
  • Intelligent tutoring systems
  • Natural language processing
  • Stealth assessment
  • Writing

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Allen, L. K., Mills, C., Jacovina, M. E., Crossley, S., D'mello, S., & McNamara, D. (2016). Investigating boredom and engagement during writing using multiple sources of information: The essay, the writer, and keystrokes. In LAK 2016 Conference Proceedings, 6th International Learning Analytics and Knowledge Conference - Enhancing Impact: Convergence of Communities for Grounding, Implementation, and Validation (Vol. 25-29-April-2016, pp. 114-123). Association for Computing Machinery. https://doi.org/10.1145/2883851.2883939

Investigating boredom and engagement during writing using multiple sources of information : The essay, the writer, and keystrokes. / Allen, Laura K.; Mills, Caitlin; Jacovina, Matthew E.; Crossley, Scott; D'mello, Sidney; McNamara, Danielle.

LAK 2016 Conference Proceedings, 6th International Learning Analytics and Knowledge Conference - Enhancing Impact: Convergence of Communities for Grounding, Implementation, and Validation. Vol. 25-29-April-2016 Association for Computing Machinery, 2016. p. 114-123.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Allen, LK, Mills, C, Jacovina, ME, Crossley, S, D'mello, S & McNamara, D 2016, Investigating boredom and engagement during writing using multiple sources of information: The essay, the writer, and keystrokes. in LAK 2016 Conference Proceedings, 6th International Learning Analytics and Knowledge Conference - Enhancing Impact: Convergence of Communities for Grounding, Implementation, and Validation. vol. 25-29-April-2016, Association for Computing Machinery, pp. 114-123, 6th International Conference on Learning Analytics and Knowledge, LAK 2016, Edinburgh, United Kingdom, 4/25/16. https://doi.org/10.1145/2883851.2883939
Allen LK, Mills C, Jacovina ME, Crossley S, D'mello S, McNamara D. Investigating boredom and engagement during writing using multiple sources of information: The essay, the writer, and keystrokes. In LAK 2016 Conference Proceedings, 6th International Learning Analytics and Knowledge Conference - Enhancing Impact: Convergence of Communities for Grounding, Implementation, and Validation. Vol. 25-29-April-2016. Association for Computing Machinery. 2016. p. 114-123 https://doi.org/10.1145/2883851.2883939
Allen, Laura K. ; Mills, Caitlin ; Jacovina, Matthew E. ; Crossley, Scott ; D'mello, Sidney ; McNamara, Danielle. / Investigating boredom and engagement during writing using multiple sources of information : The essay, the writer, and keystrokes. LAK 2016 Conference Proceedings, 6th International Learning Analytics and Knowledge Conference - Enhancing Impact: Convergence of Communities for Grounding, Implementation, and Validation. Vol. 25-29-April-2016 Association for Computing Machinery, 2016. pp. 114-123
@inproceedings{2d28a9e063434592b6de72625d7f8c77,
title = "Investigating boredom and engagement during writing using multiple sources of information: The essay, the writer, and keystrokes",
abstract = "Writing training systems have been developed to provide students with instruction and deliberate practice on their writing. Although generally successful in providing accurate scores, a common criticism of these systems is their lack of personalization and adaptive instruction. In particular, these systems tend to place the strongest emphasis on delivering accurate scores, and therefore, tend to overlook additional indices that may contribute to students' success, such as their affective states during writing practice. This study takes an initial step toward addressing this gap by building a predictive model of students' affect using information that can potentially be collected by computer systems. We used individual difference measures, text indices, and keystroke analyses to predict engagement and boredom in 132 writing sessions. The results suggest that these three categories of indices were successful in modeling students' affective states during writing. Taken together, indices related to students' academic abilities, text properties, and keystroke logs were able classify high and low engagement and boredom in writing sessions with accuracies between 76.5{\%} and 77.3{\%}. These results suggest that information readily available in writing training systems can inform affect detectors and ultimately improve student models within intelligent tutoring systems.",
keywords = "Corpus linguistics, Intelligent tutoring systems, Natural language processing, Stealth assessment, Writing",
author = "Allen, {Laura K.} and Caitlin Mills and Jacovina, {Matthew E.} and Scott Crossley and Sidney D'mello and Danielle McNamara",
year = "2016",
month = "4",
day = "25",
doi = "10.1145/2883851.2883939",
language = "English (US)",
volume = "25-29-April-2016",
pages = "114--123",
booktitle = "LAK 2016 Conference Proceedings, 6th International Learning Analytics and Knowledge Conference - Enhancing Impact: Convergence of Communities for Grounding, Implementation, and Validation",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - Investigating boredom and engagement during writing using multiple sources of information

T2 - The essay, the writer, and keystrokes

AU - Allen, Laura K.

AU - Mills, Caitlin

AU - Jacovina, Matthew E.

AU - Crossley, Scott

AU - D'mello, Sidney

AU - McNamara, Danielle

PY - 2016/4/25

Y1 - 2016/4/25

N2 - Writing training systems have been developed to provide students with instruction and deliberate practice on their writing. Although generally successful in providing accurate scores, a common criticism of these systems is their lack of personalization and adaptive instruction. In particular, these systems tend to place the strongest emphasis on delivering accurate scores, and therefore, tend to overlook additional indices that may contribute to students' success, such as their affective states during writing practice. This study takes an initial step toward addressing this gap by building a predictive model of students' affect using information that can potentially be collected by computer systems. We used individual difference measures, text indices, and keystroke analyses to predict engagement and boredom in 132 writing sessions. The results suggest that these three categories of indices were successful in modeling students' affective states during writing. Taken together, indices related to students' academic abilities, text properties, and keystroke logs were able classify high and low engagement and boredom in writing sessions with accuracies between 76.5% and 77.3%. These results suggest that information readily available in writing training systems can inform affect detectors and ultimately improve student models within intelligent tutoring systems.

AB - Writing training systems have been developed to provide students with instruction and deliberate practice on their writing. Although generally successful in providing accurate scores, a common criticism of these systems is their lack of personalization and adaptive instruction. In particular, these systems tend to place the strongest emphasis on delivering accurate scores, and therefore, tend to overlook additional indices that may contribute to students' success, such as their affective states during writing practice. This study takes an initial step toward addressing this gap by building a predictive model of students' affect using information that can potentially be collected by computer systems. We used individual difference measures, text indices, and keystroke analyses to predict engagement and boredom in 132 writing sessions. The results suggest that these three categories of indices were successful in modeling students' affective states during writing. Taken together, indices related to students' academic abilities, text properties, and keystroke logs were able classify high and low engagement and boredom in writing sessions with accuracies between 76.5% and 77.3%. These results suggest that information readily available in writing training systems can inform affect detectors and ultimately improve student models within intelligent tutoring systems.

KW - Corpus linguistics

KW - Intelligent tutoring systems

KW - Natural language processing

KW - Stealth assessment

KW - Writing

UR - http://www.scopus.com/inward/record.url?scp=84976473432&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84976473432&partnerID=8YFLogxK

U2 - 10.1145/2883851.2883939

DO - 10.1145/2883851.2883939

M3 - Conference contribution

AN - SCOPUS:84976473432

VL - 25-29-April-2016

SP - 114

EP - 123

BT - LAK 2016 Conference Proceedings, 6th International Learning Analytics and Knowledge Conference - Enhancing Impact: Convergence of Communities for Grounding, Implementation, and Validation

PB - Association for Computing Machinery

ER -