Predicting second language writing proficiency in learner texts using computational tools

Yeon Joo Jung, Scott Crossley, Danielle McNamara

Research output: Contribution to journalArticle

Abstract

This study explores whether linguistic features can predict second language writing proficiency in the Michigan English Language Assessment Battery (MELAB) writing tasks. Advanced computational tools were used to automatically assess linguistic features related to lexical sophistication, syntactic complexity, cohesion, and text structure of writing samples graded by expert raters. The findings of this study show that an analysis of linguistic features can be used to significantly predict human judgments of the essays for the MELAB writing tasks. Furthermore, the findings indicate the relative contribution of a range of linguistic features in MELAB essays to overall second language (L2) writing proficiency scores. For instance, linguistic features associated with text length and lexical sophistication were found to be more predictive of writing quality in MELAB than those associated with cohesion and syntactic complexity. This study has important implications for defining writing proficiency at different levels of achievement in L2 academic writing as well as improving the current MELAB rating scale and rater training practices. Directions for future research are also discussed.

Original languageEnglish (US)
Pages (from-to)37-52
Number of pages16
JournalJournal of Asia TEFL
Volume16
Issue number1
DOIs
StatePublished - Mar 1 2019

Fingerprint

English language
language
linguistics
rating scale
expert

Keywords

  • Computational analysis
  • Linguistic features
  • Second language writing proficiency

ASJC Scopus subject areas

  • Education
  • Linguistics and Language

Cite this

Predicting second language writing proficiency in learner texts using computational tools. / Jung, Yeon Joo; Crossley, Scott; McNamara, Danielle.

In: Journal of Asia TEFL, Vol. 16, No. 1, 01.03.2019, p. 37-52.

Research output: Contribution to journalArticle

@article{09bf1eae75e74ac7afb686591f7ef5a2,
title = "Predicting second language writing proficiency in learner texts using computational tools",
abstract = "This study explores whether linguistic features can predict second language writing proficiency in the Michigan English Language Assessment Battery (MELAB) writing tasks. Advanced computational tools were used to automatically assess linguistic features related to lexical sophistication, syntactic complexity, cohesion, and text structure of writing samples graded by expert raters. The findings of this study show that an analysis of linguistic features can be used to significantly predict human judgments of the essays for the MELAB writing tasks. Furthermore, the findings indicate the relative contribution of a range of linguistic features in MELAB essays to overall second language (L2) writing proficiency scores. For instance, linguistic features associated with text length and lexical sophistication were found to be more predictive of writing quality in MELAB than those associated with cohesion and syntactic complexity. This study has important implications for defining writing proficiency at different levels of achievement in L2 academic writing as well as improving the current MELAB rating scale and rater training practices. Directions for future research are also discussed.",
keywords = "Computational analysis, Linguistic features, Second language writing proficiency",
author = "Jung, {Yeon Joo} and Scott Crossley and Danielle McNamara",
year = "2019",
month = "3",
day = "1",
doi = "10.18823/asiatefl.2019.16.1.3.37",
language = "English (US)",
volume = "16",
pages = "37--52",
journal = "Journal of Asia TEFL",
issn = "1738-3102",
publisher = "The Asian Association of Teachers of English as a Foreign Language",
number = "1",

}

TY - JOUR

T1 - Predicting second language writing proficiency in learner texts using computational tools

AU - Jung, Yeon Joo

AU - Crossley, Scott

AU - McNamara, Danielle

PY - 2019/3/1

Y1 - 2019/3/1

N2 - This study explores whether linguistic features can predict second language writing proficiency in the Michigan English Language Assessment Battery (MELAB) writing tasks. Advanced computational tools were used to automatically assess linguistic features related to lexical sophistication, syntactic complexity, cohesion, and text structure of writing samples graded by expert raters. The findings of this study show that an analysis of linguistic features can be used to significantly predict human judgments of the essays for the MELAB writing tasks. Furthermore, the findings indicate the relative contribution of a range of linguistic features in MELAB essays to overall second language (L2) writing proficiency scores. For instance, linguistic features associated with text length and lexical sophistication were found to be more predictive of writing quality in MELAB than those associated with cohesion and syntactic complexity. This study has important implications for defining writing proficiency at different levels of achievement in L2 academic writing as well as improving the current MELAB rating scale and rater training practices. Directions for future research are also discussed.

AB - This study explores whether linguistic features can predict second language writing proficiency in the Michigan English Language Assessment Battery (MELAB) writing tasks. Advanced computational tools were used to automatically assess linguistic features related to lexical sophistication, syntactic complexity, cohesion, and text structure of writing samples graded by expert raters. The findings of this study show that an analysis of linguistic features can be used to significantly predict human judgments of the essays for the MELAB writing tasks. Furthermore, the findings indicate the relative contribution of a range of linguistic features in MELAB essays to overall second language (L2) writing proficiency scores. For instance, linguistic features associated with text length and lexical sophistication were found to be more predictive of writing quality in MELAB than those associated with cohesion and syntactic complexity. This study has important implications for defining writing proficiency at different levels of achievement in L2 academic writing as well as improving the current MELAB rating scale and rater training practices. Directions for future research are also discussed.

KW - Computational analysis

KW - Linguistic features

KW - Second language writing proficiency

UR - http://www.scopus.com/inward/record.url?scp=85064190505&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064190505&partnerID=8YFLogxK

U2 - 10.18823/asiatefl.2019.16.1.3.37

DO - 10.18823/asiatefl.2019.16.1.3.37

M3 - Article

VL - 16

SP - 37

EP - 52

JO - Journal of Asia TEFL

JF - Journal of Asia TEFL

SN - 1738-3102

IS - 1

ER -