Understanding expert ratings of essay quality: Coh-Metrix analyses of first and second language writing

Scott A. Crossley, Danielle S. McNamara

Research output: Contribution to journalArticlepeer-review

63 Scopus citations

Abstract

This article reviews recent studies in which human judgements of essay quality are assessed using Coh-Metrix, an automated text analysis tool. The goal of these studies is to better understand the relationship between linguistic features of essays and human judgements of writing quality. Coh-Metrix reports on a wide range of linguistic features, affording analyses of writing at various levels of text structure, including surface, text-base, and situation model levels. Recent studies have examined linguistic features of essay quality related to co-reference, connectives, syntactic complexity, lexical diversity, spatiality, temporality, and lexical characteristics. These studies have analysed essays written by both first language and second language writers. The results support the notion that human judgements of essay quality are best predicted by linguistic indices that correlate with measures of language sophistication such as lexical diversity, word frequency, and syntactic complexity. In contrast, human judgements of essay quality are not strongly predicted by linguistic indices related to cohesion. Overall, the studies portray high quality writing as containing more complex language that may not facilitate text comprehension.

Original languageEnglish (US)
Pages (from-to)170-191
Number of pages22
JournalInternational Journal of Continuing Engineering Education and Life-Long Learning
Volume21
Issue number2-3
DOIs
StatePublished - May 2011
Externally publishedYes

Keywords

  • Cohesion
  • Computational linguistics
  • Essay quality
  • Lexical development
  • Syntactic development
  • Writing assessment

ASJC Scopus subject areas

  • Education
  • General Engineering

Fingerprint

Dive into the research topics of 'Understanding expert ratings of essay quality: Coh-Metrix analyses of first and second language writing'. Together they form a unique fingerprint.

Cite this