Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners

Scott A. Crossley, Danielle McNamara

Research output: Contribution to journalArticle

36 Citations (Scopus)

Abstract

This study examines second language (L2) syntactic development in conjunction with the effects such development has on human judgments of writing quality (i.e., judgments of both overall writing proficiency and more fine-grained judgments of syntactic proficiency). Essays collected from 57 L2 learners in a longitudinal study were analyzed for growth and scoring patterns using syntactic complexity indices calculated by the computational tool Coh-Metrix. The analyses demonstrate that significant growth in syntactic complexity occurred in the L2 writers as a function of time spent studying English. However, only one of the syntactic features that demonstrated growth in the L2 learners was also predictive of human judgments of L2 writing quality. Interpretation of the findings suggest that over the course of a semester, L2 writers produced texts that were increasingly aligned with academic writing (i.e., texts that contain more nouns and phrasal complexity), but that human raters assessed text quality based on structures aligned with spoken discourse (i.e., clausal complexity). Thus, this study finds that the syntactic features that develop in L2 learners may not be the same syntactic features that will assist them in receiving higher evaluations of essay quality.

Original languageEnglish (US)
Pages (from-to)66-79
Number of pages14
JournalJournal of Second Language Writing
Volume26
DOIs
StatePublished - Dec 1 2014

Fingerprint

writer
semester
longitudinal study
interpretation
Computational
L2 Learners
Syntactic Complexity
Writing Development
discourse
language
evaluation
Syntactic Features
Writer
time
Proficiency
Language
Evaluation
Scoring
Writing Proficiency
Nouns

Keywords

  • Computational linguistics
  • L2 writing
  • Syntactic complexity
  • Writing development
  • Writing quality

ASJC Scopus subject areas

  • Language and Linguistics
  • Education
  • Linguistics and Language

Cite this

@article{86a1be5bb4504733b8d9a422b46e8907,
title = "Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners",
abstract = "This study examines second language (L2) syntactic development in conjunction with the effects such development has on human judgments of writing quality (i.e., judgments of both overall writing proficiency and more fine-grained judgments of syntactic proficiency). Essays collected from 57 L2 learners in a longitudinal study were analyzed for growth and scoring patterns using syntactic complexity indices calculated by the computational tool Coh-Metrix. The analyses demonstrate that significant growth in syntactic complexity occurred in the L2 writers as a function of time spent studying English. However, only one of the syntactic features that demonstrated growth in the L2 learners was also predictive of human judgments of L2 writing quality. Interpretation of the findings suggest that over the course of a semester, L2 writers produced texts that were increasingly aligned with academic writing (i.e., texts that contain more nouns and phrasal complexity), but that human raters assessed text quality based on structures aligned with spoken discourse (i.e., clausal complexity). Thus, this study finds that the syntactic features that develop in L2 learners may not be the same syntactic features that will assist them in receiving higher evaluations of essay quality.",
keywords = "Computational linguistics, L2 writing, Syntactic complexity, Writing development, Writing quality",
author = "Crossley, {Scott A.} and Danielle McNamara",
year = "2014",
month = "12",
day = "1",
doi = "10.1016/j.jslw.2014.09.006",
language = "English (US)",
volume = "26",
pages = "66--79",
journal = "Journal of Second Language Writing",
issn = "1060-3743",
publisher = "Elsevier Limited",

}

TY - JOUR

T1 - Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners

AU - Crossley, Scott A.

AU - McNamara, Danielle

PY - 2014/12/1

Y1 - 2014/12/1

N2 - This study examines second language (L2) syntactic development in conjunction with the effects such development has on human judgments of writing quality (i.e., judgments of both overall writing proficiency and more fine-grained judgments of syntactic proficiency). Essays collected from 57 L2 learners in a longitudinal study were analyzed for growth and scoring patterns using syntactic complexity indices calculated by the computational tool Coh-Metrix. The analyses demonstrate that significant growth in syntactic complexity occurred in the L2 writers as a function of time spent studying English. However, only one of the syntactic features that demonstrated growth in the L2 learners was also predictive of human judgments of L2 writing quality. Interpretation of the findings suggest that over the course of a semester, L2 writers produced texts that were increasingly aligned with academic writing (i.e., texts that contain more nouns and phrasal complexity), but that human raters assessed text quality based on structures aligned with spoken discourse (i.e., clausal complexity). Thus, this study finds that the syntactic features that develop in L2 learners may not be the same syntactic features that will assist them in receiving higher evaluations of essay quality.

AB - This study examines second language (L2) syntactic development in conjunction with the effects such development has on human judgments of writing quality (i.e., judgments of both overall writing proficiency and more fine-grained judgments of syntactic proficiency). Essays collected from 57 L2 learners in a longitudinal study were analyzed for growth and scoring patterns using syntactic complexity indices calculated by the computational tool Coh-Metrix. The analyses demonstrate that significant growth in syntactic complexity occurred in the L2 writers as a function of time spent studying English. However, only one of the syntactic features that demonstrated growth in the L2 learners was also predictive of human judgments of L2 writing quality. Interpretation of the findings suggest that over the course of a semester, L2 writers produced texts that were increasingly aligned with academic writing (i.e., texts that contain more nouns and phrasal complexity), but that human raters assessed text quality based on structures aligned with spoken discourse (i.e., clausal complexity). Thus, this study finds that the syntactic features that develop in L2 learners may not be the same syntactic features that will assist them in receiving higher evaluations of essay quality.

KW - Computational linguistics

KW - L2 writing

KW - Syntactic complexity

KW - Writing development

KW - Writing quality

UR - http://www.scopus.com/inward/record.url?scp=84914115175&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84914115175&partnerID=8YFLogxK

U2 - 10.1016/j.jslw.2014.09.006

DO - 10.1016/j.jslw.2014.09.006

M3 - Article

AN - SCOPUS:84914115175

VL - 26

SP - 66

EP - 79

JO - Journal of Second Language Writing

JF - Journal of Second Language Writing

SN - 1060-3743

ER -