TY - GEN
T1 - Using automated indices of cohesion to evaluate an intelligent tutoring system and an automated writing evaluation system
AU - Crossley, Scott A.
AU - Varner, Laura K.
AU - Roscoe, Rod
AU - McNamara, Danielle
N1 - Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
PY - 2013
Y1 - 2013
N2 - We present an evaluation of the Writing Pal (W-Pal) intelligent tutoring system (ITS) and the W-Pal automated writing evaluation (AWE) system through the use ofcomputational indices related to text cohesion. Sixty-four students participated in this study. Each student was assigned to either the W-Pal ITS condition or the W-Pal AWE condition. The W-Pal ITS includes strategy instruction, game-based practice, and essay-based practice with automated feedback. In the ITS condition, students received strategy training and wrote and revised one essay in each of the 8 training sessions. In the AWE condition, students only interacted with the essay writing and feedback tools. These students wrote and revised two essays in each of the 8 sessions. Indices of local and global cohesion reported by the computational tools Coh-Metrix and the Writing Assessment Tool (WAT) were used to investigate pretest and posttest writing gains. For both the ITS and the AWE systems, training led to the increased use of global cohesion features in essay writing. This study demonstrates that automated indices of text cohesion can be used to evaluate the effects of ITSs and AWE systems and further demonstrates how text cohesion develops as a result of instruction, writing, and automated feedback.
AB - We present an evaluation of the Writing Pal (W-Pal) intelligent tutoring system (ITS) and the W-Pal automated writing evaluation (AWE) system through the use ofcomputational indices related to text cohesion. Sixty-four students participated in this study. Each student was assigned to either the W-Pal ITS condition or the W-Pal AWE condition. The W-Pal ITS includes strategy instruction, game-based practice, and essay-based practice with automated feedback. In the ITS condition, students received strategy training and wrote and revised one essay in each of the 8 training sessions. In the AWE condition, students only interacted with the essay writing and feedback tools. These students wrote and revised two essays in each of the 8 sessions. Indices of local and global cohesion reported by the computational tools Coh-Metrix and the Writing Assessment Tool (WAT) were used to investigate pretest and posttest writing gains. For both the ITS and the AWE systems, training led to the increased use of global cohesion features in essay writing. This study demonstrates that automated indices of text cohesion can be used to evaluate the effects of ITSs and AWE systems and further demonstrates how text cohesion develops as a result of instruction, writing, and automated feedback.
KW - Cohesion
KW - Computational Linguistics
KW - Corpus Linguistics
KW - Intelligent Tutoring Systems
KW - Natural Language Processing
KW - Writing Pedagogy
UR - http://www.scopus.com/inward/record.url?scp=84880012839&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84880012839&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-39112-5_28
DO - 10.1007/978-3-642-39112-5_28
M3 - Conference contribution
AN - SCOPUS:84880012839
SN - 9783642391118
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 269
EP - 278
BT - Artificial Intelligence in Education - 16th International Conference, AIED 2013, Proceedings
PB - Springer Verlag
T2 - 16th International Conference on Artificial Intelligence in Education, AIED 2013
Y2 - 9 July 2013 through 13 July 2013
ER -