Classifying paragraph types using linguistic features: Is paragraph positioning important?

Scott A. Crossley, Kyle Dempsey, Danielle McNamara

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

This study examines the potential for computational tools and human raters to classify paragraphs based on positioning. In this study, a corpus of 182 paragraphs was collected from student, argumentative essays. The paragraphs selected were initial, middle, and final paragraphs and their positioning related to introductory, body, and concluding paragraphs. The paragraphs were analyzed by the computational tool Coh-Metrix on a variety of linguistic features with correlates to textual cohesion and lexical sophistication and then modeled using statistical techniques. The paragraphs were also classified by human raters based on paragraph positioning. The performance of the reported model was well above chance and reported an accuracy of classification that was similar to human judgments of paragraph type (66% accuracy for human versus 65% accuracy for our model). The model's accuracy increased when longer paragraphs that provided more linguistic coverage and paragraphs judged by human raters to be of higher quality were examined. The findings support the notions that paragraph types contain specific linguistic features that allow them to be distinguished from one another. The finding reported in this study should prove beneficial in classroom writing instruction and in automated writing assessment.

Original languageEnglish (US)
Pages (from-to)119-143
Number of pages25
JournalJournal of Writing Research
Volume3
Issue number2
StatePublished - Dec 2011

Fingerprint

linguistics
writing instruction
group cohesion
coverage
classroom
performance
Linguistic Features
Positioning
Paragraph
student
Raters

Keywords

  • Cognitive modeling
  • Computational linguistics
  • Corpus linguistics
  • Paragraph function
  • Paragraph structure

ASJC Scopus subject areas

  • Literature and Literary Theory
  • Education
  • Linguistics and Language
  • Language and Linguistics

Cite this

Classifying paragraph types using linguistic features : Is paragraph positioning important? / Crossley, Scott A.; Dempsey, Kyle; McNamara, Danielle.

In: Journal of Writing Research, Vol. 3, No. 2, 12.2011, p. 119-143.

Research output: Contribution to journalArticle

@article{d2ef921e593641f1954f135a1870c594,
title = "Classifying paragraph types using linguistic features: Is paragraph positioning important?",
abstract = "This study examines the potential for computational tools and human raters to classify paragraphs based on positioning. In this study, a corpus of 182 paragraphs was collected from student, argumentative essays. The paragraphs selected were initial, middle, and final paragraphs and their positioning related to introductory, body, and concluding paragraphs. The paragraphs were analyzed by the computational tool Coh-Metrix on a variety of linguistic features with correlates to textual cohesion and lexical sophistication and then modeled using statistical techniques. The paragraphs were also classified by human raters based on paragraph positioning. The performance of the reported model was well above chance and reported an accuracy of classification that was similar to human judgments of paragraph type (66{\%} accuracy for human versus 65{\%} accuracy for our model). The model's accuracy increased when longer paragraphs that provided more linguistic coverage and paragraphs judged by human raters to be of higher quality were examined. The findings support the notions that paragraph types contain specific linguistic features that allow them to be distinguished from one another. The finding reported in this study should prove beneficial in classroom writing instruction and in automated writing assessment.",
keywords = "Cognitive modeling, Computational linguistics, Corpus linguistics, Paragraph function, Paragraph structure",
author = "Crossley, {Scott A.} and Kyle Dempsey and Danielle McNamara",
year = "2011",
month = "12",
language = "English (US)",
volume = "3",
pages = "119--143",
journal = "Journal of Writing Research",
issn = "2030-1006",
publisher = "University of Antwerp",
number = "2",

}

TY - JOUR

T1 - Classifying paragraph types using linguistic features

T2 - Is paragraph positioning important?

AU - Crossley, Scott A.

AU - Dempsey, Kyle

AU - McNamara, Danielle

PY - 2011/12

Y1 - 2011/12

N2 - This study examines the potential for computational tools and human raters to classify paragraphs based on positioning. In this study, a corpus of 182 paragraphs was collected from student, argumentative essays. The paragraphs selected were initial, middle, and final paragraphs and their positioning related to introductory, body, and concluding paragraphs. The paragraphs were analyzed by the computational tool Coh-Metrix on a variety of linguistic features with correlates to textual cohesion and lexical sophistication and then modeled using statistical techniques. The paragraphs were also classified by human raters based on paragraph positioning. The performance of the reported model was well above chance and reported an accuracy of classification that was similar to human judgments of paragraph type (66% accuracy for human versus 65% accuracy for our model). The model's accuracy increased when longer paragraphs that provided more linguistic coverage and paragraphs judged by human raters to be of higher quality were examined. The findings support the notions that paragraph types contain specific linguistic features that allow them to be distinguished from one another. The finding reported in this study should prove beneficial in classroom writing instruction and in automated writing assessment.

AB - This study examines the potential for computational tools and human raters to classify paragraphs based on positioning. In this study, a corpus of 182 paragraphs was collected from student, argumentative essays. The paragraphs selected were initial, middle, and final paragraphs and their positioning related to introductory, body, and concluding paragraphs. The paragraphs were analyzed by the computational tool Coh-Metrix on a variety of linguistic features with correlates to textual cohesion and lexical sophistication and then modeled using statistical techniques. The paragraphs were also classified by human raters based on paragraph positioning. The performance of the reported model was well above chance and reported an accuracy of classification that was similar to human judgments of paragraph type (66% accuracy for human versus 65% accuracy for our model). The model's accuracy increased when longer paragraphs that provided more linguistic coverage and paragraphs judged by human raters to be of higher quality were examined. The findings support the notions that paragraph types contain specific linguistic features that allow them to be distinguished from one another. The finding reported in this study should prove beneficial in classroom writing instruction and in automated writing assessment.

KW - Cognitive modeling

KW - Computational linguistics

KW - Corpus linguistics

KW - Paragraph function

KW - Paragraph structure

UR - http://www.scopus.com/inward/record.url?scp=84867216493&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867216493&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:84867216493

VL - 3

SP - 119

EP - 143

JO - Journal of Writing Research

JF - Journal of Writing Research

SN - 2030-1006

IS - 2

ER -