Abstract

The assessment of writing proficiency generally includes analyses of the specific linguistic and rhetorical features contained in the singular essays produced by students. However, researchers have recently proposed that an individual’s ability to flexibly adapt the linguistic properties of their writing might more closely capture writing skill. However, the features of the task, learner, and educational context that influence this flexibility remain largely unknown. The current study extends this research by examining relations between linguistic flexibility, reading comprehension ability, and feedback in the context of an automated writing evaluation system. Students (n = 131) wrote and revised six essays in an automated writing evaluation system and were provided both summative and formative feedback on their writing. Additionally, half of the students had access to a spelling and grammar checker that provided lower-level feedback during the writing period. The results provide evidence for the fact that developing writers demonstrate linguistic flexibility across the essays that they produce. However, analyses also indicate that lower-level feedback (i.e., spelling and grammar feedback) have little to no impact on the properties of students’ essays nor on their variability across prompts or drafts. Overall, the current study provides important insights into the role of flexibility in writing skill and develops a strong foundation on which to conduct future research and educational interventions.

Original languageEnglish (US)
Title of host publicationProceedings of the 8th International Conference on Learning Analytics and Knowledge
Subtitle of host publicationTowards User-Centred Learning Analytics, LAK 2018
PublisherAssociation for Computing Machinery
Pages380-388
Number of pages9
ISBN (Electronic)9781450364003
DOIs
StatePublished - Mar 7 2018
Event8th International Conference on Learning Analytics and Knowledge, LAK 2018 - Sydney, Australia
Duration: Mar 5 2018Mar 9 2018

Other

Other8th International Conference on Learning Analytics and Knowledge, LAK 2018
CountryAustralia
CitySydney
Period3/5/183/9/18

Fingerprint

Linguistics
Feedback
Students

Keywords

  • Feedback
  • Flexibility
  • Natural language processing
  • Revision
  • Writing

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Allen, L. K., Likens, A. D., & McNamara, D. (2018). A multi-Dimensional analysis of writing flexibility in an automated writing evaluation system. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge: Towards User-Centred Learning Analytics, LAK 2018 (pp. 380-388). Association for Computing Machinery. https://doi.org/10.1145/3170358.3170404

A multi-Dimensional analysis of writing flexibility in an automated writing evaluation system. / Allen, Laura K.; Likens, Aaron D.; McNamara, Danielle.

Proceedings of the 8th International Conference on Learning Analytics and Knowledge: Towards User-Centred Learning Analytics, LAK 2018. Association for Computing Machinery, 2018. p. 380-388.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Allen, LK, Likens, AD & McNamara, D 2018, A multi-Dimensional analysis of writing flexibility in an automated writing evaluation system. in Proceedings of the 8th International Conference on Learning Analytics and Knowledge: Towards User-Centred Learning Analytics, LAK 2018. Association for Computing Machinery, pp. 380-388, 8th International Conference on Learning Analytics and Knowledge, LAK 2018, Sydney, Australia, 3/5/18. https://doi.org/10.1145/3170358.3170404
Allen LK, Likens AD, McNamara D. A multi-Dimensional analysis of writing flexibility in an automated writing evaluation system. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge: Towards User-Centred Learning Analytics, LAK 2018. Association for Computing Machinery. 2018. p. 380-388 https://doi.org/10.1145/3170358.3170404
Allen, Laura K. ; Likens, Aaron D. ; McNamara, Danielle. / A multi-Dimensional analysis of writing flexibility in an automated writing evaluation system. Proceedings of the 8th International Conference on Learning Analytics and Knowledge: Towards User-Centred Learning Analytics, LAK 2018. Association for Computing Machinery, 2018. pp. 380-388
@inproceedings{10d3230ab367498abe78c3756b992a9f,
title = "A multi-Dimensional analysis of writing flexibility in an automated writing evaluation system",
abstract = "The assessment of writing proficiency generally includes analyses of the specific linguistic and rhetorical features contained in the singular essays produced by students. However, researchers have recently proposed that an individual’s ability to flexibly adapt the linguistic properties of their writing might more closely capture writing skill. However, the features of the task, learner, and educational context that influence this flexibility remain largely unknown. The current study extends this research by examining relations between linguistic flexibility, reading comprehension ability, and feedback in the context of an automated writing evaluation system. Students (n = 131) wrote and revised six essays in an automated writing evaluation system and were provided both summative and formative feedback on their writing. Additionally, half of the students had access to a spelling and grammar checker that provided lower-level feedback during the writing period. The results provide evidence for the fact that developing writers demonstrate linguistic flexibility across the essays that they produce. However, analyses also indicate that lower-level feedback (i.e., spelling and grammar feedback) have little to no impact on the properties of students’ essays nor on their variability across prompts or drafts. Overall, the current study provides important insights into the role of flexibility in writing skill and develops a strong foundation on which to conduct future research and educational interventions.",
keywords = "Feedback, Flexibility, Natural language processing, Revision, Writing",
author = "Allen, {Laura K.} and Likens, {Aaron D.} and Danielle McNamara",
year = "2018",
month = "3",
day = "7",
doi = "10.1145/3170358.3170404",
language = "English (US)",
pages = "380--388",
booktitle = "Proceedings of the 8th International Conference on Learning Analytics and Knowledge",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - A multi-Dimensional analysis of writing flexibility in an automated writing evaluation system

AU - Allen, Laura K.

AU - Likens, Aaron D.

AU - McNamara, Danielle

PY - 2018/3/7

Y1 - 2018/3/7

N2 - The assessment of writing proficiency generally includes analyses of the specific linguistic and rhetorical features contained in the singular essays produced by students. However, researchers have recently proposed that an individual’s ability to flexibly adapt the linguistic properties of their writing might more closely capture writing skill. However, the features of the task, learner, and educational context that influence this flexibility remain largely unknown. The current study extends this research by examining relations between linguistic flexibility, reading comprehension ability, and feedback in the context of an automated writing evaluation system. Students (n = 131) wrote and revised six essays in an automated writing evaluation system and were provided both summative and formative feedback on their writing. Additionally, half of the students had access to a spelling and grammar checker that provided lower-level feedback during the writing period. The results provide evidence for the fact that developing writers demonstrate linguistic flexibility across the essays that they produce. However, analyses also indicate that lower-level feedback (i.e., spelling and grammar feedback) have little to no impact on the properties of students’ essays nor on their variability across prompts or drafts. Overall, the current study provides important insights into the role of flexibility in writing skill and develops a strong foundation on which to conduct future research and educational interventions.

AB - The assessment of writing proficiency generally includes analyses of the specific linguistic and rhetorical features contained in the singular essays produced by students. However, researchers have recently proposed that an individual’s ability to flexibly adapt the linguistic properties of their writing might more closely capture writing skill. However, the features of the task, learner, and educational context that influence this flexibility remain largely unknown. The current study extends this research by examining relations between linguistic flexibility, reading comprehension ability, and feedback in the context of an automated writing evaluation system. Students (n = 131) wrote and revised six essays in an automated writing evaluation system and were provided both summative and formative feedback on their writing. Additionally, half of the students had access to a spelling and grammar checker that provided lower-level feedback during the writing period. The results provide evidence for the fact that developing writers demonstrate linguistic flexibility across the essays that they produce. However, analyses also indicate that lower-level feedback (i.e., spelling and grammar feedback) have little to no impact on the properties of students’ essays nor on their variability across prompts or drafts. Overall, the current study provides important insights into the role of flexibility in writing skill and develops a strong foundation on which to conduct future research and educational interventions.

KW - Feedback

KW - Flexibility

KW - Natural language processing

KW - Revision

KW - Writing

UR - http://www.scopus.com/inward/record.url?scp=85045936002&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85045936002&partnerID=8YFLogxK

U2 - 10.1145/3170358.3170404

DO - 10.1145/3170358.3170404

M3 - Conference contribution

SP - 380

EP - 388

BT - Proceedings of the 8th International Conference on Learning Analytics and Knowledge

PB - Association for Computing Machinery

ER -