TY - GEN
T1 - DyFAV
T2 - 22nd International Conference on Intelligent User Interfaces, IUI 2017
AU - Paudyal, Prajwal
AU - Lee, Junghyo
AU - Banerjee, Ayan
AU - Gupta, Sandeep
PY - 2017/3/7
Y1 - 2017/3/7
N2 - Recent research has shown that reliable recognition of sign language words and phrases using user-friendly and non-invasive armbands is feasible and desirable. This work provides an analysis and implementation of including fingerspelling recognition (FR) in such systems, which is a much harder problem due to lack of distinctive hand movements. A novel algorithm called DyFAV (Dynamic Feature Selection and Voting) is proposed for this purpose that exploits the fact that fingerspelling has a finite corpus (26 letters for ASL). The system uses an independent multiple agent voting approach to identify letters with high accuracy. The independent voting of the agents ensures that the algorithm is highly parallelizable and thus recognition times can be kept low to suit real-Time mobile applications. The results are demonstrated on the entire ASL alphabet corpus for nine people with limited training and average recognition accuracy of 95.36% is achieved which is better than the state-of-Art for armband sensors. The mobile, non-invasive, and real time nature of the technology is demonstrated by evaluating performance on various types of Android phones and remote server configurations.
AB - Recent research has shown that reliable recognition of sign language words and phrases using user-friendly and non-invasive armbands is feasible and desirable. This work provides an analysis and implementation of including fingerspelling recognition (FR) in such systems, which is a much harder problem due to lack of distinctive hand movements. A novel algorithm called DyFAV (Dynamic Feature Selection and Voting) is proposed for this purpose that exploits the fact that fingerspelling has a finite corpus (26 letters for ASL). The system uses an independent multiple agent voting approach to identify letters with high accuracy. The independent voting of the agents ensures that the algorithm is highly parallelizable and thus recognition times can be kept low to suit real-Time mobile applications. The results are demonstrated on the entire ASL alphabet corpus for nine people with limited training and average recognition accuracy of 95.36% is achieved which is better than the state-of-Art for armband sensors. The mobile, non-invasive, and real time nature of the technology is demonstrated by evaluating performance on various types of Android phones and remote server configurations.
KW - Assistive technology
KW - Gesture-based interfaces
KW - Sign language processing
KW - Wearable and pervasive computing
UR - http://www.scopus.com/inward/record.url?scp=85016518552&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85016518552&partnerID=8YFLogxK
U2 - 10.1145/3025171.3025216
DO - 10.1145/3025171.3025216
M3 - Conference contribution
AN - SCOPUS:85016518552
T3 - International Conference on Intelligent User Interfaces, Proceedings IUI
SP - 457
EP - 467
BT - IUI 2017 - Proceedings of the 22nd International Conference on Intelligent User Interfaces
PB - Association for Computing Machinery
Y2 - 13 March 2017 through 16 March 2017
ER -