CS-VQA: Visual Question Answering with Compressively Sensed Images

Li Chi Huang, Kuldeep Kulkarni, Anik Jha, Suhas Lohit, Suren Jayasuriya, Pavan Turaga

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Scopus citations

Abstract

Visual Question Answering (VQA) is a complex semantic task requiring both natural language processing and visual recognition. In this paper, we explore whether VQA is solvable when images are captured in a sub-Nyquist compressive paradigm. We develop a series of deep-network architectures that exploit available compressive data to increasing degrees of accuracy, and show that VQA is indeed solvable in the compressed domain. Our results show that there is nominal degradation in VQA performance when using compressive measurements, but that accuracy can be recovered when VQA pipelines are used in conjunction with state-of-the-art deep neural networks for CS reconstruction. The results presented yield important implications for resource-constrained VQA applications.

Original languageEnglish (US)
Title of host publication2018 IEEE International Conference on Image Processing, ICIP 2018 - Proceedings
PublisherIEEE Computer Society
Pages1283-1287
Number of pages5
ISBN (Electronic)9781479970612
DOIs
StatePublished - Aug 29 2018
Event25th IEEE International Conference on Image Processing, ICIP 2018 - Athens, Greece
Duration: Oct 7 2018Oct 10 2018

Publication series

NameProceedings - International Conference on Image Processing, ICIP
ISSN (Print)1522-4880

Conference

Conference25th IEEE International Conference on Image Processing, ICIP 2018
Country/TerritoryGreece
CityAthens
Period10/7/1810/10/18

Keywords

  • Compressed sensing
  • Computer vision
  • Image reconstruction
  • Multi-layer neural network

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Signal Processing

Fingerprint

Dive into the research topics of 'CS-VQA: Visual Question Answering with Compressively Sensed Images'. Together they form a unique fingerprint.

Cite this