Augmentative and Alternative Communication (AAC) includes all forms of communication that are used to supplement speech for those with impairments in the production or comprehension of spoken language. AAC technology has mostly been used by people with severe speech or language impairments. Individuals with visual impairments also face a fundamental limitation in communicating with their sighted peers, as about 65% of the information during social interactions is conveyed using nonverbal cues. In this chapter, we present our computer vision research contributions in the design and development of a Social Interaction Assistant (SIA), which is an AAC technology that can enrich the communication experience of individuals with visual impairments.Moreover, individuals with visual impairments often have specific requirements that necessitate a personalized, adaptive approach to multimedia computing, rather than a "one-size-fits-all" approach. To address this challenge, our proposed solutions place emphasis on understanding the individual user's needs, expectations,and adaptations toward designing, developing, and deploying effective multimedia solutions. Our empirical results demonstrate the significant potential in using person-centered AAC technology to enrich the communication experience of individuals with visual impairments.

Original languageEnglish (US)
Title of host publicationComputer Vision For Assistive Healthcare
PublisherElsevier Inc.
Number of pages38
ISBN (Print)9780128134450
StatePublished - Jan 1 2018


  • Augmentative and alternative communication (AAC)
  • Batch mode active learning
  • Conformal predictions
  • Dysarthria
  • Facial expression recognition
  • Nonverbal communication
  • Person-centered technologies
  • Speech and language disorders[or impairments]

ASJC Scopus subject areas

  • Social Sciences(all)


Dive into the research topics of 'Computer Vision for Augmentative and Alternative Communication'. Together they form a unique fingerprint.

Cite this