Abstract

Augmentative and Alternative Communication (AAC) includes all forms of communication that are used to supplement speech for those with impairments in the production or comprehension of spoken language. AAC technology has mostly been used by people with severe speech or language impairments. Individuals with visual impairments also face a fundamental limitation in communicating with their sighted peers, as about 65% of the information during social interactions is conveyed using nonverbal cues. In this chapter, we present our computer vision research contributions in the design and development of a Social Interaction Assistant (SIA), which is an AAC technology that can enrich the communication experience of individuals with visual impairments.Moreover, individuals with visual impairments often have specific requirements that necessitate a personalized, adaptive approach to multimedia computing, rather than a "one-size-fits-all" approach. To address this challenge, our proposed solutions place emphasis on understanding the individual user's needs, expectations,and adaptations toward designing, developing, and deploying effective multimedia solutions. Our empirical results demonstrate the significant potential in using person-centered AAC technology to enrich the communication experience of individuals with visual impairments.

Original languageEnglish (US)
Title of host publicationComputer Vision For Assistive Healthcare
PublisherElsevier Inc.
Pages211-248
Number of pages38
ISBN (Print)9780128134450
DOIs
StatePublished - Jan 1 2018

Fingerprint

visual impairment
alternative technology
communication
communication technology
multimedia
spoken language
interaction
assistant
supplement
comprehension
experience
human being
language

Keywords

  • Augmentative and alternative communication (AAC)
  • Batch mode active learning
  • Conformal predictions
  • Dysarthria
  • Facial expression recognition
  • Nonverbal communication
  • Person-centered technologies
  • Speech and language disorders[or impairments]

ASJC Scopus subject areas

  • Social Sciences(all)

Cite this

Computer Vision for Augmentative and Alternative Communication. / Panchanathan, Sethuraman; Moore, Meredith; Demakethepalli Venkateswara, Hemanth; Chakraborty, Shayok; McDaniel, Troy.

Computer Vision For Assistive Healthcare. Elsevier Inc., 2018. p. 211-248.

Research output: Chapter in Book/Report/Conference proceedingChapter

Panchanathan, Sethuraman ; Moore, Meredith ; Demakethepalli Venkateswara, Hemanth ; Chakraborty, Shayok ; McDaniel, Troy. / Computer Vision for Augmentative and Alternative Communication. Computer Vision For Assistive Healthcare. Elsevier Inc., 2018. pp. 211-248
@inbook{5bb4f8bbc0074426b3f3438ba0a897f7,
title = "Computer Vision for Augmentative and Alternative Communication",
abstract = "Augmentative and Alternative Communication (AAC) includes all forms of communication that are used to supplement speech for those with impairments in the production or comprehension of spoken language. AAC technology has mostly been used by people with severe speech or language impairments. Individuals with visual impairments also face a fundamental limitation in communicating with their sighted peers, as about 65{\%} of the information during social interactions is conveyed using nonverbal cues. In this chapter, we present our computer vision research contributions in the design and development of a Social Interaction Assistant (SIA), which is an AAC technology that can enrich the communication experience of individuals with visual impairments.Moreover, individuals with visual impairments often have specific requirements that necessitate a personalized, adaptive approach to multimedia computing, rather than a {"}one-size-fits-all{"} approach. To address this challenge, our proposed solutions place emphasis on understanding the individual user's needs, expectations,and adaptations toward designing, developing, and deploying effective multimedia solutions. Our empirical results demonstrate the significant potential in using person-centered AAC technology to enrich the communication experience of individuals with visual impairments.",
keywords = "Augmentative and alternative communication (AAC), Batch mode active learning, Conformal predictions, Dysarthria, Facial expression recognition, Nonverbal communication, Person-centered technologies, Speech and language disorders[or impairments]",
author = "Sethuraman Panchanathan and Meredith Moore and {Demakethepalli Venkateswara}, Hemanth and Shayok Chakraborty and Troy McDaniel",
year = "2018",
month = "1",
day = "1",
doi = "10.1016/B978-0-12-813445-0.00008-3",
language = "English (US)",
isbn = "9780128134450",
pages = "211--248",
booktitle = "Computer Vision For Assistive Healthcare",
publisher = "Elsevier Inc.",

}

TY - CHAP

T1 - Computer Vision for Augmentative and Alternative Communication

AU - Panchanathan, Sethuraman

AU - Moore, Meredith

AU - Demakethepalli Venkateswara, Hemanth

AU - Chakraborty, Shayok

AU - McDaniel, Troy

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Augmentative and Alternative Communication (AAC) includes all forms of communication that are used to supplement speech for those with impairments in the production or comprehension of spoken language. AAC technology has mostly been used by people with severe speech or language impairments. Individuals with visual impairments also face a fundamental limitation in communicating with their sighted peers, as about 65% of the information during social interactions is conveyed using nonverbal cues. In this chapter, we present our computer vision research contributions in the design and development of a Social Interaction Assistant (SIA), which is an AAC technology that can enrich the communication experience of individuals with visual impairments.Moreover, individuals with visual impairments often have specific requirements that necessitate a personalized, adaptive approach to multimedia computing, rather than a "one-size-fits-all" approach. To address this challenge, our proposed solutions place emphasis on understanding the individual user's needs, expectations,and adaptations toward designing, developing, and deploying effective multimedia solutions. Our empirical results demonstrate the significant potential in using person-centered AAC technology to enrich the communication experience of individuals with visual impairments.

AB - Augmentative and Alternative Communication (AAC) includes all forms of communication that are used to supplement speech for those with impairments in the production or comprehension of spoken language. AAC technology has mostly been used by people with severe speech or language impairments. Individuals with visual impairments also face a fundamental limitation in communicating with their sighted peers, as about 65% of the information during social interactions is conveyed using nonverbal cues. In this chapter, we present our computer vision research contributions in the design and development of a Social Interaction Assistant (SIA), which is an AAC technology that can enrich the communication experience of individuals with visual impairments.Moreover, individuals with visual impairments often have specific requirements that necessitate a personalized, adaptive approach to multimedia computing, rather than a "one-size-fits-all" approach. To address this challenge, our proposed solutions place emphasis on understanding the individual user's needs, expectations,and adaptations toward designing, developing, and deploying effective multimedia solutions. Our empirical results demonstrate the significant potential in using person-centered AAC technology to enrich the communication experience of individuals with visual impairments.

KW - Augmentative and alternative communication (AAC)

KW - Batch mode active learning

KW - Conformal predictions

KW - Dysarthria

KW - Facial expression recognition

KW - Nonverbal communication

KW - Person-centered technologies

KW - Speech and language disorders[or impairments]

UR - http://www.scopus.com/inward/record.url?scp=85059725447&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85059725447&partnerID=8YFLogxK

U2 - 10.1016/B978-0-12-813445-0.00008-3

DO - 10.1016/B978-0-12-813445-0.00008-3

M3 - Chapter

SN - 9780128134450

SP - 211

EP - 248

BT - Computer Vision For Assistive Healthcare

PB - Elsevier Inc.

ER -