TY - GEN
T1 - Computational considerations in correcting user-language
AU - Renner, Adam M.
AU - McCarthy, Philip M.
AU - McNamara, Danielle S.
PY - 2009
Y1 - 2009
N2 - This study evaluates the robustness of established computational indices used to assess text relatedness in user-language. The original User-Language Paraphrase Corpus (ULPC) was compared to a corrected version, in which each paraphrase was corrected for typographical and grammatical errors. Error correction significantly affected values for each of five computational indices, indicating greater similarity of the target sentence to the corrected paraphrase than to the original paraphrase. Moreover, misspelled target words accounted for a large proportion of the differences. This study also evaluated potential effects on correlations between computational indices and human ratings of paraphrases. The corrections did not yield assessments that were any more or less comparable to trained human raters than were the original paraphrases containing typographical or grammatical errors. The results suggest that although correcting for errors may optimize certain computational indices, the corrections are not necessary for comparing the indices to expert ratings.
AB - This study evaluates the robustness of established computational indices used to assess text relatedness in user-language. The original User-Language Paraphrase Corpus (ULPC) was compared to a corrected version, in which each paraphrase was corrected for typographical and grammatical errors. Error correction significantly affected values for each of five computational indices, indicating greater similarity of the target sentence to the corrected paraphrase than to the original paraphrase. Moreover, misspelled target words accounted for a large proportion of the differences. This study also evaluated potential effects on correlations between computational indices and human ratings of paraphrases. The corrections did not yield assessments that were any more or less comparable to trained human raters than were the original paraphrases containing typographical or grammatical errors. The results suggest that although correcting for errors may optimize certain computational indices, the corrections are not necessary for comparing the indices to expert ratings.
UR - http://www.scopus.com/inward/record.url?scp=68949182340&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=68949182340&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:68949182340
SN - 9781577354192
T3 - Proceedings of the 22nd International Florida Artificial Intelligence Research Society Conference, FLAIRS-22
SP - 278
EP - 283
BT - Proceedings of the 22nd International Florida Artificial Intelligence Research Society Conference, FLAIRS-22
T2 - 22nd International Florida Artificial Intelligence Research Society Conference, FLAIRS-22
Y2 - 19 March 2009 through 21 March 2009
ER -