Social interaction is a central component of human experience. The ability to interact with others and communicate effectively within an interactive context is a fundamental necessity for professional success as well as personal fulfillment. Individuals with visual impairment face significant challenges in social communication, which if unmitigated, may lead to lifelong needs for extensive social and economic support. Unfortunately, today's multimedia technologies largely cater to the needs of the 'able' population, resulting in solutions that mostly meet the needs of that community. Individuals with disabilities (such as visual impairment) have largely been absent in the design process, and have to adapt themselves (often unsuccessfully) to available solutions. In this paper, we propose a social interaction assistant for individuals who are blind or visually impaired, incorporating novel contributions in: 1) person recognition through batch mode active learning; 2) reliable multimodal person recognition through the conformal predictions framework; and 3) facial expression recognition through topic models. Moreover, individuals with visual impairments often have specific requirements that necessitate a personalized, adaptive approach to multimedia computing. To address this challenge, our proposed solutions place emphasis on understanding the individual user's needs, expectations and adaptations toward designing, and developing and deploying effective multimedia solutions. Our empirical results demonstrate the significant potential in using person centered multimedia solutions to enrich the lives of individuals with disabilities.

Original languageEnglish (US)
Article number7435229
Pages (from-to)942-951
Number of pages10
JournalIEEE Journal on Selected Topics in Signal Processing
Issue number5
StatePublished - Aug 1 2016



  • Computer Vision
  • Machine Learning
  • Person-centered Multimedia Computing
  • Social Interaction Assistant

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Signal Processing

Cite this