A comparison of techniques for sign language alphabet recognition using armband wearables

Recent research has shown that reliable recognition of sign language words and phrases using user-friendly and noninvasive armbands is feasible and desirable. This work provides an analysis and implementation of including fingerspelling recognition (FR) in such systems, which is a much harder problem due to lack of distinctive hand movements. A novel algorithm called DyFAV (Dynamic Feature Selection and Voting) is proposed for this purpose that exploits the fact that fingerspelling has a finite corpus (26 alphabets for the American Sign Language (ASL)). Detailed analysis of the algorithm used as well as comparisons with other traditional machine-learning algorithms is provided. The system uses an independent multiple-agent voting approach to identify letters with high accuracy. The independent voting of the agents ensures that the algorithm is highly parallelizable and thus recognition times can be kept low to suit real-time mobile applications. A thorough explanation and analysis is presented on results obtained on the ASL alphabet corpus for nine people with limited training. An average recognition accuracy of 95.36% is reported and compared with recognition results from other machine-learning techniques. This result is extended by including six additional validation users with data collected under similar settings as the previous dataset. Furthermore, a feature selection schema using a subset of the sensors is proposed and the results are evaluated. The mobile, noninvasive, and real-time nature of the technology is demonstrated by evaluating performance on various types of Android phones and remote server configurations. A brief discussion of the user interface is provided along with guidelines for best practices.