On evaluating the effects of feedback for sign language learning using explainable AI

Prajwal Paudyal, Ayan Banerjee, Sandeep Gupta

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Computer Aided Sign Language Learning (CASLL) is a recent and promising field of research which is made feasible by advances in Computer Vision and Sign Language Recognition. The importance of feedback for language learning has been established by many research works. In this work, we introduce SignGuru a chat-bot based AI tutor that can provide fine-grained feedback to learners of American Sign Language. SignGuru provides feedback directly relating to the fundamental concepts of ASL using a modular and explainable AI. The chatbot is designed not only as an interactive for learners to choose a curriculum and go through the interactive learning process, but also to perform retention and execution tests. The usability and utility of SignGuru has been validated by a userstudy using 14 ASL signs with 26 users. We demonstrate the fully functioning application with a variety of different curriculum.

Original languageEnglish (US)
Title of host publicationProceedings of the 25th International Conference on Intelligent User Interfaces Companion. IUI 2020
PublisherAssociation for Computing Machinery
Pages83-84
Number of pages2
ISBN (Electronic)9781450375139
DOIs
StatePublished - Mar 17 2020
Event25th International Conference on Intelligent User Interfaces, IUI 2020 - Cagliari, Italy
Duration: Mar 17 2020Mar 20 2020

Publication series

NameInternational Conference on Intelligent User Interfaces, Proceedings IUI

Conference

Conference25th International Conference on Intelligent User Interfaces, IUI 2020
CountryItaly
CityCagliari
Period3/17/203/20/20

Keywords

  • Datasets
  • Gaze detection
  • Neural networks
  • Text tagging

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'On evaluating the effects of feedback for sign language learning using explainable AI'. Together they form a unique fingerprint.

  • Cite this

    Paudyal, P., Banerjee, A., & Gupta, S. (2020). On evaluating the effects of feedback for sign language learning using explainable AI. In Proceedings of the 25th International Conference on Intelligent User Interfaces Companion. IUI 2020 (pp. 83-84). (International Conference on Intelligent User Interfaces, Proceedings IUI). Association for Computing Machinery. https://doi.org/10.1145/3379336.3381469