Evaluating state-of-the-art Treebank-style parsers for Coh-Metrix and other learning technology environments

Christian F. Hempelmann, Vasile Rus, Arthur C. Graesser, Danielle McNamara

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

This paper evaluates four of the most commonly used, freely available, state-of-the-art parsers on a standard benchmark as well as with respect to a set of data relevant for measuring text cohesion, as one example of a learning technology application that requires fast and accurate syntactic parsing. We outline advantages and disadvantages of existing technologies and make recommendations. Our performance report uses traditional measures based on a gold standard as well as novel dimensions for parsing evaluation. To our knowledge, this is the first attempt to evaluate parsers across genres and grade levels for the implementation in learning technology using both gold standard and directed evaluation methods.

Original languageEnglish (US)
Pages (from-to)131-144
Number of pages14
JournalNatural Language Engineering
Volume12
Issue number2
DOIs
StatePublished - Jun 2006
Externally publishedYes

Fingerprint

gold standard
learning
Syntactics
evaluation
genre
school grade
performance
Treebank
Parsers
Gold Standard
Evaluation
Benchmark
Parsing
Syntactic Parsing
Grade Level
Cohesion

ASJC Scopus subject areas

  • Software

Cite this

Evaluating state-of-the-art Treebank-style parsers for Coh-Metrix and other learning technology environments. / Hempelmann, Christian F.; Rus, Vasile; Graesser, Arthur C.; McNamara, Danielle.

In: Natural Language Engineering, Vol. 12, No. 2, 06.2006, p. 131-144.

Research output: Contribution to journalArticle

Hempelmann, Christian F. ; Rus, Vasile ; Graesser, Arthur C. ; McNamara, Danielle. / Evaluating state-of-the-art Treebank-style parsers for Coh-Metrix and other learning technology environments. In: Natural Language Engineering. 2006 ; Vol. 12, No. 2. pp. 131-144.
@article{f05c4241d7bf4fb0bde21439be7291e5,
title = "Evaluating state-of-the-art Treebank-style parsers for Coh-Metrix and other learning technology environments",
abstract = "This paper evaluates four of the most commonly used, freely available, state-of-the-art parsers on a standard benchmark as well as with respect to a set of data relevant for measuring text cohesion, as one example of a learning technology application that requires fast and accurate syntactic parsing. We outline advantages and disadvantages of existing technologies and make recommendations. Our performance report uses traditional measures based on a gold standard as well as novel dimensions for parsing evaluation. To our knowledge, this is the first attempt to evaluate parsers across genres and grade levels for the implementation in learning technology using both gold standard and directed evaluation methods.",
author = "Hempelmann, {Christian F.} and Vasile Rus and Graesser, {Arthur C.} and Danielle McNamara",
year = "2006",
month = "6",
doi = "10.1017/S1351324906004207",
language = "English (US)",
volume = "12",
pages = "131--144",
journal = "Natural Language Engineering",
issn = "1351-3249",
publisher = "Cambridge University Press",
number = "2",

}

TY - JOUR

T1 - Evaluating state-of-the-art Treebank-style parsers for Coh-Metrix and other learning technology environments

AU - Hempelmann, Christian F.

AU - Rus, Vasile

AU - Graesser, Arthur C.

AU - McNamara, Danielle

PY - 2006/6

Y1 - 2006/6

N2 - This paper evaluates four of the most commonly used, freely available, state-of-the-art parsers on a standard benchmark as well as with respect to a set of data relevant for measuring text cohesion, as one example of a learning technology application that requires fast and accurate syntactic parsing. We outline advantages and disadvantages of existing technologies and make recommendations. Our performance report uses traditional measures based on a gold standard as well as novel dimensions for parsing evaluation. To our knowledge, this is the first attempt to evaluate parsers across genres and grade levels for the implementation in learning technology using both gold standard and directed evaluation methods.

AB - This paper evaluates four of the most commonly used, freely available, state-of-the-art parsers on a standard benchmark as well as with respect to a set of data relevant for measuring text cohesion, as one example of a learning technology application that requires fast and accurate syntactic parsing. We outline advantages and disadvantages of existing technologies and make recommendations. Our performance report uses traditional measures based on a gold standard as well as novel dimensions for parsing evaluation. To our knowledge, this is the first attempt to evaluate parsers across genres and grade levels for the implementation in learning technology using both gold standard and directed evaluation methods.

UR - http://www.scopus.com/inward/record.url?scp=33646895420&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33646895420&partnerID=8YFLogxK

U2 - 10.1017/S1351324906004207

DO - 10.1017/S1351324906004207

M3 - Article

AN - SCOPUS:33646895420

VL - 12

SP - 131

EP - 144

JO - Natural Language Engineering

JF - Natural Language Engineering

SN - 1351-3249

IS - 2

ER -