Active scene recognition with vision and language

Xiaodong Yu, Cornelia Fermuller, Ching Lik Teo, Yezhou Yang, Yiannis Aloimonos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Citations (Scopus)

Abstract

This paper presents a novel approach to utilizing high level knowledge for the problem of scene recognition in an active vision framework, which we call active scene recognition. In traditional approaches, high level knowledge is used in the post-processing to combine the outputs of the object detectors to achieve better classification performance. In contrast, the proposed approach employs high level knowledge actively by implementing an interaction between a reasoning module and a sensory module (Figure 1). Following this paradigm, we implemented an active scene recognizer and evaluated it with a dataset of 20 scenes and 100+ objects. We also extended it to the analysis of dynamic scenes for activity recognition with attributes. Experiments demonstrate the effectiveness of the active paradigm in introducing attention and additional constraints into the sensing process.

Original languageEnglish (US)
Title of host publication2011 International Conference on Computer Vision, ICCV 2011
Pages810-817
Number of pages8
DOIs
StatePublished - 2011
Externally publishedYes
Event2011 IEEE International Conference on Computer Vision, ICCV 2011 - Barcelona, Spain
Duration: Nov 6 2011Nov 13 2011

Other

Other2011 IEEE International Conference on Computer Vision, ICCV 2011
CountrySpain
CityBarcelona
Period11/6/1111/13/11

Fingerprint

Detectors
Processing
Experiments

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Yu, X., Fermuller, C., Teo, C. L., Yang, Y., & Aloimonos, Y. (2011). Active scene recognition with vision and language. In 2011 International Conference on Computer Vision, ICCV 2011 (pp. 810-817). [6126320] https://doi.org/10.1109/ICCV.2011.6126320

Active scene recognition with vision and language. / Yu, Xiaodong; Fermuller, Cornelia; Teo, Ching Lik; Yang, Yezhou; Aloimonos, Yiannis.

2011 International Conference on Computer Vision, ICCV 2011. 2011. p. 810-817 6126320.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yu, X, Fermuller, C, Teo, CL, Yang, Y & Aloimonos, Y 2011, Active scene recognition with vision and language. in 2011 International Conference on Computer Vision, ICCV 2011., 6126320, pp. 810-817, 2011 IEEE International Conference on Computer Vision, ICCV 2011, Barcelona, Spain, 11/6/11. https://doi.org/10.1109/ICCV.2011.6126320
Yu X, Fermuller C, Teo CL, Yang Y, Aloimonos Y. Active scene recognition with vision and language. In 2011 International Conference on Computer Vision, ICCV 2011. 2011. p. 810-817. 6126320 https://doi.org/10.1109/ICCV.2011.6126320
Yu, Xiaodong ; Fermuller, Cornelia ; Teo, Ching Lik ; Yang, Yezhou ; Aloimonos, Yiannis. / Active scene recognition with vision and language. 2011 International Conference on Computer Vision, ICCV 2011. 2011. pp. 810-817
@inproceedings{6eb88223f40c457e835e1325bd4aa950,
title = "Active scene recognition with vision and language",
abstract = "This paper presents a novel approach to utilizing high level knowledge for the problem of scene recognition in an active vision framework, which we call active scene recognition. In traditional approaches, high level knowledge is used in the post-processing to combine the outputs of the object detectors to achieve better classification performance. In contrast, the proposed approach employs high level knowledge actively by implementing an interaction between a reasoning module and a sensory module (Figure 1). Following this paradigm, we implemented an active scene recognizer and evaluated it with a dataset of 20 scenes and 100+ objects. We also extended it to the analysis of dynamic scenes for activity recognition with attributes. Experiments demonstrate the effectiveness of the active paradigm in introducing attention and additional constraints into the sensing process.",
author = "Xiaodong Yu and Cornelia Fermuller and Teo, {Ching Lik} and Yezhou Yang and Yiannis Aloimonos",
year = "2011",
doi = "10.1109/ICCV.2011.6126320",
language = "English (US)",
isbn = "9781457711015",
pages = "810--817",
booktitle = "2011 International Conference on Computer Vision, ICCV 2011",

}

TY - GEN

T1 - Active scene recognition with vision and language

AU - Yu, Xiaodong

AU - Fermuller, Cornelia

AU - Teo, Ching Lik

AU - Yang, Yezhou

AU - Aloimonos, Yiannis

PY - 2011

Y1 - 2011

N2 - This paper presents a novel approach to utilizing high level knowledge for the problem of scene recognition in an active vision framework, which we call active scene recognition. In traditional approaches, high level knowledge is used in the post-processing to combine the outputs of the object detectors to achieve better classification performance. In contrast, the proposed approach employs high level knowledge actively by implementing an interaction between a reasoning module and a sensory module (Figure 1). Following this paradigm, we implemented an active scene recognizer and evaluated it with a dataset of 20 scenes and 100+ objects. We also extended it to the analysis of dynamic scenes for activity recognition with attributes. Experiments demonstrate the effectiveness of the active paradigm in introducing attention and additional constraints into the sensing process.

AB - This paper presents a novel approach to utilizing high level knowledge for the problem of scene recognition in an active vision framework, which we call active scene recognition. In traditional approaches, high level knowledge is used in the post-processing to combine the outputs of the object detectors to achieve better classification performance. In contrast, the proposed approach employs high level knowledge actively by implementing an interaction between a reasoning module and a sensory module (Figure 1). Following this paradigm, we implemented an active scene recognizer and evaluated it with a dataset of 20 scenes and 100+ objects. We also extended it to the analysis of dynamic scenes for activity recognition with attributes. Experiments demonstrate the effectiveness of the active paradigm in introducing attention and additional constraints into the sensing process.

UR - http://www.scopus.com/inward/record.url?scp=84863036307&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863036307&partnerID=8YFLogxK

U2 - 10.1109/ICCV.2011.6126320

DO - 10.1109/ICCV.2011.6126320

M3 - Conference contribution

SN - 9781457711015

SP - 810

EP - 817

BT - 2011 International Conference on Computer Vision, ICCV 2011

ER -