Disentangling Indirect Answers to Yes-No Questions in Real Conversations

Krishna C. Sanagavarapu; Jathin P. Singaraju; Anusha Kakileti; Anirudh Kaza; Aaron A. Mathews; Helen Li; Nathan R. Brito; Eduardo Blanco

Disentangling Indirect Answers to Yes-No Questions in Real Conversations

Krishna C. Sanagavarapu, Jathin P. Singaraju, Anusha Kakileti, Anirudh Kaza, Aaron A. Mathews, Helen Li, Nathan R. Brito, Eduardo Blanco

Engineering, Ira A. Fulton Schools of (IAFSE)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Scopus citations

Abstract

In this paper, we explore the task of determining indirect answers to yes-no questions in real conversations. We work with transcripts of phone conversations in the Switchboard Dialog Act (SwDA) corpus and create SwDA-IndirectAnswers (SwDA-IA), a subset of SwDA consisting of all conversations containing a yes-no question with an indirect answer. We annotate the underlying direct answers to the yes-no questions (yes, probably yes, middle, probably no, or no). We show that doing so requires taking into account conversation context: the indirect answer alone is insufficient to determine the ground truth. Experimental results also show that taking into account context is beneficial. More importantly, our results demonstrate that existing corpora with synthetic indirect answers to yes-no questions are not beneficial when working with real conversations. Our best models outperform the majority baseline by a substantial margin, but the task remains a challenge (F1: 0.46).

Original language	English (US)
Title of host publication	NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics
Subtitle of host publication	Human Language Technologies, Proceedings of the Conference
Publisher	Association for Computational Linguistics (ACL)
Pages	4677-4695
Number of pages	19
ISBN (Electronic)	9781955917711
State	Published - 2022
Event	2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022 - Seattle, United States Duration: Jul 10 2022 → Jul 15 2022

Publication series

Name	NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference

Conference

Conference	2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022
Country/Territory	United States
City	Seattle
Period	7/10/22 → 7/15/22

ASJC Scopus subject areas

Computer Networks and Communications
Hardware and Architecture
Information Systems
Software

Cite this

Sanagavarapu, K. C., Singaraju, J. P., Kakileti, A., Kaza, A., Mathews, A. A., Li, H., Brito, N. R., & Blanco, E. (2022). Disentangling Indirect Answers to Yes-No Questions in Real Conversations. In NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference (pp. 4677-4695). (NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference). Association for Computational Linguistics (ACL).

Disentangling Indirect Answers to Yes-No Questions in Real Conversations. / Sanagavarapu, Krishna C.; Singaraju, Jathin P.; Kakileti, Anusha et al.
NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference. Association for Computational Linguistics (ACL), 2022. p. 4677-4695 (NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Sanagavarapu, KC, Singaraju, JP, Kakileti, A, Kaza, A, Mathews, AA, Li, H, Brito, NR & Blanco, E 2022, Disentangling Indirect Answers to Yes-No Questions in Real Conversations. in NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference. NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, Association for Computational Linguistics (ACL), pp. 4677-4695, 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, United States, 7/10/22.

Sanagavarapu KC, Singaraju JP, Kakileti A, Kaza A, Mathews AA, Li H et al. Disentangling Indirect Answers to Yes-No Questions in Real Conversations. In NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference. Association for Computational Linguistics (ACL). 2022. p. 4677-4695. (NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference).

Sanagavarapu, Krishna C. ; Singaraju, Jathin P. ; Kakileti, Anusha et al. / Disentangling Indirect Answers to Yes-No Questions in Real Conversations. NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference. Association for Computational Linguistics (ACL), 2022. pp. 4677-4695 (NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference).

@inproceedings{cc243a5d683c49a9b5d97b962e8c0098,

title = "Disentangling Indirect Answers to Yes-No Questions in Real Conversations",

abstract = "In this paper, we explore the task of determining indirect answers to yes-no questions in real conversations. We work with transcripts of phone conversations in the Switchboard Dialog Act (SwDA) corpus and create SwDA-IndirectAnswers (SwDA-IA), a subset of SwDA consisting of all conversations containing a yes-no question with an indirect answer. We annotate the underlying direct answers to the yes-no questions (yes, probably yes, middle, probably no, or no). We show that doing so requires taking into account conversation context: the indirect answer alone is insufficient to determine the ground truth. Experimental results also show that taking into account context is beneficial. More importantly, our results demonstrate that existing corpora with synthetic indirect answers to yes-no questions are not beneficial when working with real conversations. Our best models outperform the majority baseline by a substantial margin, but the task remains a challenge (F1: 0.46).",

author = "Sanagavarapu, {Krishna C.} and Singaraju, {Jathin P.} and Anusha Kakileti and Anirudh Kaza and Mathews, {Aaron A.} and Helen Li and Brito, {Nathan R.} and Eduardo Blanco",

note = "Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.; 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022 ; Conference date: 10-07-2022 Through 15-07-2022",

year = "2022",

language = "English (US)",

series = "NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference",

publisher = "Association for Computational Linguistics (ACL)",

pages = "4677--4695",

booktitle = "NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics",

}

TY - GEN

T1 - Disentangling Indirect Answers to Yes-No Questions in Real Conversations

AU - Sanagavarapu, Krishna C.

AU - Singaraju, Jathin P.

AU - Kakileti, Anusha

AU - Kaza, Anirudh

AU - Mathews, Aaron A.

AU - Li, Helen

AU - Brito, Nathan R.

AU - Blanco, Eduardo

PY - 2022

Y1 - 2022

N2 - In this paper, we explore the task of determining indirect answers to yes-no questions in real conversations. We work with transcripts of phone conversations in the Switchboard Dialog Act (SwDA) corpus and create SwDA-IndirectAnswers (SwDA-IA), a subset of SwDA consisting of all conversations containing a yes-no question with an indirect answer. We annotate the underlying direct answers to the yes-no questions (yes, probably yes, middle, probably no, or no). We show that doing so requires taking into account conversation context: the indirect answer alone is insufficient to determine the ground truth. Experimental results also show that taking into account context is beneficial. More importantly, our results demonstrate that existing corpora with synthetic indirect answers to yes-no questions are not beneficial when working with real conversations. Our best models outperform the majority baseline by a substantial margin, but the task remains a challenge (F1: 0.46).

AB - In this paper, we explore the task of determining indirect answers to yes-no questions in real conversations. We work with transcripts of phone conversations in the Switchboard Dialog Act (SwDA) corpus and create SwDA-IndirectAnswers (SwDA-IA), a subset of SwDA consisting of all conversations containing a yes-no question with an indirect answer. We annotate the underlying direct answers to the yes-no questions (yes, probably yes, middle, probably no, or no). We show that doing so requires taking into account conversation context: the indirect answer alone is insufficient to determine the ground truth. Experimental results also show that taking into account context is beneficial. More importantly, our results demonstrate that existing corpora with synthetic indirect answers to yes-no questions are not beneficial when working with real conversations. Our best models outperform the majority baseline by a substantial margin, but the task remains a challenge (F1: 0.46).

UR - http://www.scopus.com/inward/record.url?scp=85138438887&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85138438887&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85138438887

T3 - NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference

SP - 4677

EP - 4695

BT - NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics

PB - Association for Computational Linguistics (ACL)

T2 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022

Y2 - 10 July 2022 through 15 July 2022

ER -

Disentangling Indirect Answers to Yes-No Questions in Real Conversations

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Cite this