Abstract

Social interaction is a central component of human experience. The ability to interact with others and communicate effectively within an interactive context is a fundamental necessity for professional success as well as personal fulfillment. Individuals with visual impairment face significant challenges in social communication, which if unmitigated, may lead to lifelong needs for extensive social and economic support. Unfortunately, today's multimedia technologies largely cater to the needs of the 'able' population, resulting in solutions that mostly meet the needs of that community. Individuals with disabilities (such as visual impairment) have largely been absent in the design process, and have to adapt themselves (often unsuccessfully) to available solutions. In this paper, we propose a social interaction assistant for individuals who are blind or visually impaired, incorporating novel contributions in: 1) person recognition through batch mode active learning; 2) reliable multimodal person recognition through the conformal predictions framework; and 3) facial expression recognition through topic models. Moreover, individuals with visual impairments often have specific requirements that necessitate a personalized, adaptive approach to multimedia computing. To address this challenge, our proposed solutions place emphasis on understanding the individual user's needs, expectations and adaptations toward designing, and developing and deploying effective multimedia solutions. Our empirical results demonstrate the significant potential in using person centered multimedia solutions to enrich the lives of individuals with disabilities.

Original languageEnglish (US)
Article number7435229
Pages (from-to)942-951
Number of pages10
JournalIEEE Journal on Selected Topics in Signal Processing
Volume10
Issue number5
DOIs
StatePublished - Aug 1 2016

Fingerprint

Economics
Communication
Problem-Based Learning

Keywords

  • Computer Vision
  • Machine Learning
  • Person-centered Multimedia Computing
  • Social Interaction Assistant

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Signal Processing

Cite this

@article{1d6216042f484b13b2127066b79e5e63,
title = "Social Interaction Assistant: A Person-Centered Approach to Enrich Social Interactions for Individuals with Visual Impairments",
abstract = "Social interaction is a central component of human experience. The ability to interact with others and communicate effectively within an interactive context is a fundamental necessity for professional success as well as personal fulfillment. Individuals with visual impairment face significant challenges in social communication, which if unmitigated, may lead to lifelong needs for extensive social and economic support. Unfortunately, today's multimedia technologies largely cater to the needs of the 'able' population, resulting in solutions that mostly meet the needs of that community. Individuals with disabilities (such as visual impairment) have largely been absent in the design process, and have to adapt themselves (often unsuccessfully) to available solutions. In this paper, we propose a social interaction assistant for individuals who are blind or visually impaired, incorporating novel contributions in: 1) person recognition through batch mode active learning; 2) reliable multimodal person recognition through the conformal predictions framework; and 3) facial expression recognition through topic models. Moreover, individuals with visual impairments often have specific requirements that necessitate a personalized, adaptive approach to multimedia computing. To address this challenge, our proposed solutions place emphasis on understanding the individual user's needs, expectations and adaptations toward designing, and developing and deploying effective multimedia solutions. Our empirical results demonstrate the significant potential in using person centered multimedia solutions to enrich the lives of individuals with disabilities.",
keywords = "Computer Vision, Machine Learning, Person-centered Multimedia Computing, Social Interaction Assistant",
author = "Sethuraman Panchanathan and Shayok Chakraborty and Troy McDaniel",
year = "2016",
month = "8",
day = "1",
doi = "10.1109/JSTSP.2016.2543681",
language = "English (US)",
volume = "10",
pages = "942--951",
journal = "IEEE Journal on Selected Topics in Signal Processing",
issn = "1932-4553",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "5",

}

TY - JOUR

T1 - Social Interaction Assistant

T2 - A Person-Centered Approach to Enrich Social Interactions for Individuals with Visual Impairments

AU - Panchanathan, Sethuraman

AU - Chakraborty, Shayok

AU - McDaniel, Troy

PY - 2016/8/1

Y1 - 2016/8/1

N2 - Social interaction is a central component of human experience. The ability to interact with others and communicate effectively within an interactive context is a fundamental necessity for professional success as well as personal fulfillment. Individuals with visual impairment face significant challenges in social communication, which if unmitigated, may lead to lifelong needs for extensive social and economic support. Unfortunately, today's multimedia technologies largely cater to the needs of the 'able' population, resulting in solutions that mostly meet the needs of that community. Individuals with disabilities (such as visual impairment) have largely been absent in the design process, and have to adapt themselves (often unsuccessfully) to available solutions. In this paper, we propose a social interaction assistant for individuals who are blind or visually impaired, incorporating novel contributions in: 1) person recognition through batch mode active learning; 2) reliable multimodal person recognition through the conformal predictions framework; and 3) facial expression recognition through topic models. Moreover, individuals with visual impairments often have specific requirements that necessitate a personalized, adaptive approach to multimedia computing. To address this challenge, our proposed solutions place emphasis on understanding the individual user's needs, expectations and adaptations toward designing, and developing and deploying effective multimedia solutions. Our empirical results demonstrate the significant potential in using person centered multimedia solutions to enrich the lives of individuals with disabilities.

AB - Social interaction is a central component of human experience. The ability to interact with others and communicate effectively within an interactive context is a fundamental necessity for professional success as well as personal fulfillment. Individuals with visual impairment face significant challenges in social communication, which if unmitigated, may lead to lifelong needs for extensive social and economic support. Unfortunately, today's multimedia technologies largely cater to the needs of the 'able' population, resulting in solutions that mostly meet the needs of that community. Individuals with disabilities (such as visual impairment) have largely been absent in the design process, and have to adapt themselves (often unsuccessfully) to available solutions. In this paper, we propose a social interaction assistant for individuals who are blind or visually impaired, incorporating novel contributions in: 1) person recognition through batch mode active learning; 2) reliable multimodal person recognition through the conformal predictions framework; and 3) facial expression recognition through topic models. Moreover, individuals with visual impairments often have specific requirements that necessitate a personalized, adaptive approach to multimedia computing. To address this challenge, our proposed solutions place emphasis on understanding the individual user's needs, expectations and adaptations toward designing, and developing and deploying effective multimedia solutions. Our empirical results demonstrate the significant potential in using person centered multimedia solutions to enrich the lives of individuals with disabilities.

KW - Computer Vision

KW - Machine Learning

KW - Person-centered Multimedia Computing

KW - Social Interaction Assistant

UR - http://www.scopus.com/inward/record.url?scp=84982732354&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84982732354&partnerID=8YFLogxK

U2 - 10.1109/JSTSP.2016.2543681

DO - 10.1109/JSTSP.2016.2543681

M3 - Article

VL - 10

SP - 942

EP - 951

JO - IEEE Journal on Selected Topics in Signal Processing

JF - IEEE Journal on Selected Topics in Signal Processing

SN - 1932-4553

IS - 5

M1 - 7435229

ER -