TY - GEN
T1 - An analysis of natural language inference Benchmarks through the lens of negation
AU - Hossain, Md Mosharaf
AU - Kovatchev, Venelin
AU - Dutta, Pranoy
AU - Kao, Tiffany
AU - Wei, Elizabeth
AU - Blanco, Eduardo
N1 - Funding Information:
This material is based upon work supported by the National Science Foundation under Grant No. 1845757. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF. Funding was also provided by the Spanish Ministry of Science, Innovation, and Universities Project PGC2018-096212-B-C33. The Titan Xp used for this research was donated by the NVIDIA Corporation. Computational resources were also provided by the UNT office of High-Performance Computing. We also thank the reviewers for insightful comments.
Publisher Copyright:
© 2020 Association for Computational Linguistics.
PY - 2020
Y1 - 2020
N2 - Negation is underrepresented in existing natural language inference benchmarks. Additionally, one can often ignore the few negations in existing benchmarks and still make the right inference judgments. In this paper, we present a new benchmark for natural language inference in which negation plays an important role. We also show that state-of-the-art transformers struggle making inference judgments with the new pairs.
AB - Negation is underrepresented in existing natural language inference benchmarks. Additionally, one can often ignore the few negations in existing benchmarks and still make the right inference judgments. In this paper, we present a new benchmark for natural language inference in which negation plays an important role. We also show that state-of-the-art transformers struggle making inference judgments with the new pairs.
UR - http://www.scopus.com/inward/record.url?scp=85098430958&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85098430958&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85098430958
T3 - EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference
SP - 9106
EP - 9118
BT - EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference
PB - Association for Computational Linguistics (ACL)
T2 - 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020
Y2 - 16 November 2020 through 20 November 2020
ER -