Active scene recognition with vision and language

Xiaodong Yu, Cornelia Fermuller, Ching Lik Teo, Yezhou Yang, Yiannis Aloimonos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Scopus citations

Abstract

This paper presents a novel approach to utilizing high level knowledge for the problem of scene recognition in an active vision framework, which we call active scene recognition. In traditional approaches, high level knowledge is used in the post-processing to combine the outputs of the object detectors to achieve better classification performance. In contrast, the proposed approach employs high level knowledge actively by implementing an interaction between a reasoning module and a sensory module (Figure 1). Following this paradigm, we implemented an active scene recognizer and evaluated it with a dataset of 20 scenes and 100+ objects. We also extended it to the analysis of dynamic scenes for activity recognition with attributes. Experiments demonstrate the effectiveness of the active paradigm in introducing attention and additional constraints into the sensing process.

Original languageEnglish (US)
Title of host publication2011 International Conference on Computer Vision, ICCV 2011
Pages810-817
Number of pages8
DOIs
StatePublished - Dec 1 2011
Event2011 IEEE International Conference on Computer Vision, ICCV 2011 - Barcelona, Spain
Duration: Nov 6 2011Nov 13 2011

Publication series

NameProceedings of the IEEE International Conference on Computer Vision

Other

Other2011 IEEE International Conference on Computer Vision, ICCV 2011
CountrySpain
CityBarcelona
Period11/6/1111/13/11

    Fingerprint

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Yu, X., Fermuller, C., Teo, C. L., Yang, Y., & Aloimonos, Y. (2011). Active scene recognition with vision and language. In 2011 International Conference on Computer Vision, ICCV 2011 (pp. 810-817). [6126320] (Proceedings of the IEEE International Conference on Computer Vision). https://doi.org/10.1109/ICCV.2011.6126320