SCEPTRE: A pervasive, non-invasive, and programmable gesture recognition technology

Prajwal Paudyal, Ayan Banerjee, Sandeep Gupta

Research output: Chapter in Book/Report/Conference proceedingConference contribution

24 Citations (Scopus)

Abstract

Communication and collaboration between deaf people and hearing people is hindered by lack of a common language. Although there has been a lot of research in this domain, there is room for work towards a system that is ubiquitous, non-invasive, works in real-time and can be trained interactively by the user. Such a system will be powerful enough to translate gestures performed in real-time, while also being flexible enough to be fully personalized to be used as a platform for gesture based HCI. We propose SCEPTRE which utilizes two non-invasive wrist-worn devices to decipher gesture-based communication. The system uses a multitiered template based comparison system for classification on input data from accelerometer, gyroscope and electromyography (EMG) sensors. This work demonstrates that the system is very easily trained using just one to three training instances each for twenty randomly chosen signs from the American Sign Language(ASL) dictionary and also for user-generated custom gestures. The system is able to achieve an accuracy of 97.72 % for ASL gestures.

Original languageEnglish (US)
Title of host publicationInternational Conference on Intelligent User Interfaces, Proceedings IUI
PublisherAssociation for Computing Machinery
Pages282-293
Number of pages12
Volume07-10-March-2016
ISBN (Print)9781450341370, 9781450341400
DOIs
StatePublished - Mar 7 2016
Event21st International Conference on Intelligent User Interfaces, IUI 2016 - Sonoma, United States
Duration: Mar 7 2016Mar 10 2016

Other

Other21st International Conference on Intelligent User Interfaces, IUI 2016
CountryUnited States
CitySonoma
Period3/7/163/10/16

Fingerprint

Gesture recognition
Electromyography
Communication
Gyroscopes
Audition
Human computer interaction
Glossaries
Accelerometers
Sensors

Keywords

  • Assistive technology
  • Gesture-based interfaces
  • Sign language processing
  • Wearable and pervasive computing

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction

Cite this

Paudyal, P., Banerjee, A., & Gupta, S. (2016). SCEPTRE: A pervasive, non-invasive, and programmable gesture recognition technology. In International Conference on Intelligent User Interfaces, Proceedings IUI (Vol. 07-10-March-2016, pp. 282-293). Association for Computing Machinery. https://doi.org/10.1145/2856767.2856794

SCEPTRE : A pervasive, non-invasive, and programmable gesture recognition technology. / Paudyal, Prajwal; Banerjee, Ayan; Gupta, Sandeep.

International Conference on Intelligent User Interfaces, Proceedings IUI. Vol. 07-10-March-2016 Association for Computing Machinery, 2016. p. 282-293.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Paudyal, P, Banerjee, A & Gupta, S 2016, SCEPTRE: A pervasive, non-invasive, and programmable gesture recognition technology. in International Conference on Intelligent User Interfaces, Proceedings IUI. vol. 07-10-March-2016, Association for Computing Machinery, pp. 282-293, 21st International Conference on Intelligent User Interfaces, IUI 2016, Sonoma, United States, 3/7/16. https://doi.org/10.1145/2856767.2856794
Paudyal P, Banerjee A, Gupta S. SCEPTRE: A pervasive, non-invasive, and programmable gesture recognition technology. In International Conference on Intelligent User Interfaces, Proceedings IUI. Vol. 07-10-March-2016. Association for Computing Machinery. 2016. p. 282-293 https://doi.org/10.1145/2856767.2856794
Paudyal, Prajwal ; Banerjee, Ayan ; Gupta, Sandeep. / SCEPTRE : A pervasive, non-invasive, and programmable gesture recognition technology. International Conference on Intelligent User Interfaces, Proceedings IUI. Vol. 07-10-March-2016 Association for Computing Machinery, 2016. pp. 282-293
@inproceedings{7c8eedfebe624d6c93de2664e442bbc5,
title = "SCEPTRE: A pervasive, non-invasive, and programmable gesture recognition technology",
abstract = "Communication and collaboration between deaf people and hearing people is hindered by lack of a common language. Although there has been a lot of research in this domain, there is room for work towards a system that is ubiquitous, non-invasive, works in real-time and can be trained interactively by the user. Such a system will be powerful enough to translate gestures performed in real-time, while also being flexible enough to be fully personalized to be used as a platform for gesture based HCI. We propose SCEPTRE which utilizes two non-invasive wrist-worn devices to decipher gesture-based communication. The system uses a multitiered template based comparison system for classification on input data from accelerometer, gyroscope and electromyography (EMG) sensors. This work demonstrates that the system is very easily trained using just one to three training instances each for twenty randomly chosen signs from the American Sign Language(ASL) dictionary and also for user-generated custom gestures. The system is able to achieve an accuracy of 97.72 {\%} for ASL gestures.",
keywords = "Assistive technology, Gesture-based interfaces, Sign language processing, Wearable and pervasive computing",
author = "Prajwal Paudyal and Ayan Banerjee and Sandeep Gupta",
year = "2016",
month = "3",
day = "7",
doi = "10.1145/2856767.2856794",
language = "English (US)",
isbn = "9781450341370",
volume = "07-10-March-2016",
pages = "282--293",
booktitle = "International Conference on Intelligent User Interfaces, Proceedings IUI",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - SCEPTRE

T2 - A pervasive, non-invasive, and programmable gesture recognition technology

AU - Paudyal, Prajwal

AU - Banerjee, Ayan

AU - Gupta, Sandeep

PY - 2016/3/7

Y1 - 2016/3/7

N2 - Communication and collaboration between deaf people and hearing people is hindered by lack of a common language. Although there has been a lot of research in this domain, there is room for work towards a system that is ubiquitous, non-invasive, works in real-time and can be trained interactively by the user. Such a system will be powerful enough to translate gestures performed in real-time, while also being flexible enough to be fully personalized to be used as a platform for gesture based HCI. We propose SCEPTRE which utilizes two non-invasive wrist-worn devices to decipher gesture-based communication. The system uses a multitiered template based comparison system for classification on input data from accelerometer, gyroscope and electromyography (EMG) sensors. This work demonstrates that the system is very easily trained using just one to three training instances each for twenty randomly chosen signs from the American Sign Language(ASL) dictionary and also for user-generated custom gestures. The system is able to achieve an accuracy of 97.72 % for ASL gestures.

AB - Communication and collaboration between deaf people and hearing people is hindered by lack of a common language. Although there has been a lot of research in this domain, there is room for work towards a system that is ubiquitous, non-invasive, works in real-time and can be trained interactively by the user. Such a system will be powerful enough to translate gestures performed in real-time, while also being flexible enough to be fully personalized to be used as a platform for gesture based HCI. We propose SCEPTRE which utilizes two non-invasive wrist-worn devices to decipher gesture-based communication. The system uses a multitiered template based comparison system for classification on input data from accelerometer, gyroscope and electromyography (EMG) sensors. This work demonstrates that the system is very easily trained using just one to three training instances each for twenty randomly chosen signs from the American Sign Language(ASL) dictionary and also for user-generated custom gestures. The system is able to achieve an accuracy of 97.72 % for ASL gestures.

KW - Assistive technology

KW - Gesture-based interfaces

KW - Sign language processing

KW - Wearable and pervasive computing

UR - http://www.scopus.com/inward/record.url?scp=84963731962&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84963731962&partnerID=8YFLogxK

U2 - 10.1145/2856767.2856794

DO - 10.1145/2856767.2856794

M3 - Conference contribution

AN - SCOPUS:84963731962

SN - 9781450341370

SN - 9781450341400

VL - 07-10-March-2016

SP - 282

EP - 293

BT - International Conference on Intelligent User Interfaces, Proceedings IUI

PB - Association for Computing Machinery

ER -