Automatic student writing evaluation: Investigating the impact of individual differences on source-based writing

Püren Öncel, Lauren E. Flynn, Allison N. Sonia, Kennis E. Barker, Grace C. Lindsay, Caleb M. Mcclure, Danielle S. Mcnamara, Laura K. Allen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Automated Writing Evaluation systems have been developed to help students improve their writing skills through the automated delivery of both summative and formative feedback. These systems have demonstrated strong potential in a variety of educational contexts; however, they remain limited in their personalization and scope. The purpose of the current study was to begin to address this gap by examining whether individual differences could be modeled in a source-based writing context. Undergraduate students (n=106) wrote essays in response to multiple sources and then completed an assessment of their vocabulary knowledge. Natural language processing tools were used to characterize the linguistic properties of the source-based essays at four levels: descriptive, lexical, syntax, and cohesion. Finally, machine learning models were used to predict students' vocabulary scores from these linguistic features. The models accounted for approximately 29% of the variance in vocabulary scores, suggesting that the linguistic features of source-based essays are reflective of individual differences in vocabulary knowledge. Overall, this work suggests that automated text analyses can help to understand the role of individual differences in the writing process, which may ultimately help to improve personalization in computer-based learning environments.

Original languageEnglish (US)
Title of host publicationLAK 2021 Conference Proceedings - The Impact we Make
Subtitle of host publicationThe Contributions of Learning Analytics to Learning, 11th International Conference on Learning Analytics and Knowledge
PublisherAssociation for Computing Machinery
Pages620-625
Number of pages6
ISBN (Electronic)9781450389358
DOIs
StatePublished - Apr 12 2021
Event11th International Conference on Learning Analytics and Knowledge: The Impact we Make: The Contributions of Learning Analytics to Learning, LAK 2021 - Virtual, Online, United States
Duration: Apr 12 2021Apr 16 2021

Publication series

NameACM International Conference Proceeding Series

Conference

Conference11th International Conference on Learning Analytics and Knowledge: The Impact we Make: The Contributions of Learning Analytics to Learning, LAK 2021
CountryUnited States
CityVirtual, Online
Period4/12/214/16/21

Keywords

  • Individual differences
  • Machine-learning models
  • Source-based writing
  • Vocabulary knowledge

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Fingerprint Dive into the research topics of 'Automatic student writing evaluation: Investigating the impact of individual differences on source-based writing'. Together they form a unique fingerprint.

Cite this