Computational replication of human paraphrase assessment

Philip M. McCarthy, Zhigiang Cai, Danielle S. McNamara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

Two sentences are paraphrases if their meanings are equivalent but their words and syntax are different. Paraphrasing can be used to aid comprehension, stimulate prior knowledge, and assist in writing skills development. While automated paraphrase assessment is both common-place and useful, research has centered solely on artificial, edited paraphrases and has used only binary dimensions (i.e., is or is-not a paraphrase). In this study, we use 1998 natural paraphrases generated by high school students that have been assessed along 10 dimensions of paraphrase (e.g., semantic completeness). This study investigates the components of paraphrase quality emerging from these dimensions, and examines whether computational approaches (e.g. LSA, MED) can simulate those human evaluations. The results suggest that semantic and syntactic evaluations are the primary components of paraphrase quality, and that computationally light systems such as LSA (semantics) and MED (syntax) present promising approaches to simulating human evaluations of paraphrases.

Original languageEnglish (US)
Title of host publicationProceedings of the 22nd International Florida Artificial Intelligence Research Society Conference, FLAIRS-22
Pages266-271
Number of pages6
StatePublished - Nov 4 2009
Externally publishedYes
Event22nd International Florida Artificial Intelligence Research Society Conference, FLAIRS-22 - Sanibel Island, FL, United States
Duration: Mar 19 2009Mar 21 2009

Publication series

NameProceedings of the 22nd International Florida Artificial Intelligence Research Society Conference, FLAIRS-22

Other

Other22nd International Florida Artificial Intelligence Research Society Conference, FLAIRS-22
CountryUnited States
CitySanibel Island, FL
Period3/19/093/21/09

    Fingerprint

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Software

Cite this

McCarthy, P. M., Cai, Z., & McNamara, D. S. (2009). Computational replication of human paraphrase assessment. In Proceedings of the 22nd International Florida Artificial Intelligence Research Society Conference, FLAIRS-22 (pp. 266-271). (Proceedings of the 22nd International Florida Artificial Intelligence Research Society Conference, FLAIRS-22).