DyFAV: Dynamic Feature selection and voting for real-Time recognition of fingerspelled alphabet using wearables

Prajwal Paudyal, Junghyo Lee, Ayan Banerjee, Sandeep Gupta

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations

Abstract

Recent research has shown that reliable recognition of sign language words and phrases using user-friendly and non-invasive armbands is feasible and desirable. This work provides an analysis and implementation of including fingerspelling recognition (FR) in such systems, which is a much harder problem due to lack of distinctive hand movements. A novel algorithm called DyFAV (Dynamic Feature Selection and Voting) is proposed for this purpose that exploits the fact that fingerspelling has a finite corpus (26 letters for ASL). The system uses an independent multiple agent voting approach to identify letters with high accuracy. The independent voting of the agents ensures that the algorithm is highly parallelizable and thus recognition times can be kept low to suit real-Time mobile applications. The results are demonstrated on the entire ASL alphabet corpus for nine people with limited training and average recognition accuracy of 95.36% is achieved which is better than the state-of-Art for armband sensors. The mobile, non-invasive, and real time nature of the technology is demonstrated by evaluating performance on various types of Android phones and remote server configurations.

Original languageEnglish (US)
Title of host publicationIUI 2017 - Proceedings of the 22nd International Conference on Intelligent User Interfaces
PublisherAssociation for Computing Machinery
Pages457-467
Number of pages11
ISBN (Electronic)9781450343480
DOIs
StatePublished - Mar 7 2017
Event22nd International Conference on Intelligent User Interfaces, IUI 2017 - Limassol, Cyprus
Duration: Mar 13 2017Mar 16 2017

Publication series

NameInternational Conference on Intelligent User Interfaces, Proceedings IUI

Other

Other22nd International Conference on Intelligent User Interfaces, IUI 2017
Country/TerritoryCyprus
CityLimassol
Period3/13/173/16/17

Keywords

  • Assistive technology
  • Gesture-based interfaces
  • Sign language processing
  • Wearable and pervasive computing

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction

Fingerprint

Dive into the research topics of 'DyFAV: Dynamic Feature selection and voting for real-Time recognition of fingerspelled alphabet using wearables'. Together they form a unique fingerprint.

Cite this