Integrating knowledge and reasoning in image understanding

Somak Aditya, Yezhou Yang, Chitta Baral

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep learning based data-driven approaches have been successfully applied in various image understanding applications ranging from object recognition, semantic segmentation to visual question answering. However, the lack of knowledge integration as well as higher-level reasoning capabilities with the methods still pose a hindrance. In this work, we present a brief survey of a few representative reasoning mechanisms, knowledge integration methods and their corresponding image understanding applications developed by various groups of researchers, approaching the problem from a variety of angles. Furthermore, we discuss upon key efforts on integrating external knowledge with neural networks. Taking cues from these efforts, we conclude by discussing potential pathways to improve reasoning capabilities.

Original languageEnglish (US)
Title of host publicationProceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019
EditorsSarit Kraus
PublisherInternational Joint Conferences on Artificial Intelligence
Pages6252-6259
Number of pages8
ISBN (Electronic)9780999241141
StatePublished - Jan 1 2019
Event28th International Joint Conference on Artificial Intelligence, IJCAI 2019 - Macao, China
Duration: Aug 10 2019Aug 16 2019

Publication series

NameIJCAI International Joint Conference on Artificial Intelligence
Volume2019-August
ISSN (Print)1045-0823

Conference

Conference28th International Joint Conference on Artificial Intelligence, IJCAI 2019
CountryChina
CityMacao
Period8/10/198/16/19

Fingerprint

Image understanding
Object recognition
Semantics
Neural networks
Deep learning

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Aditya, S., Yang, Y., & Baral, C. (2019). Integrating knowledge and reasoning in image understanding. In S. Kraus (Ed.), Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019 (pp. 6252-6259). (IJCAI International Joint Conference on Artificial Intelligence; Vol. 2019-August). International Joint Conferences on Artificial Intelligence.

Integrating knowledge and reasoning in image understanding. / Aditya, Somak; Yang, Yezhou; Baral, Chitta.

Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019. ed. / Sarit Kraus. International Joint Conferences on Artificial Intelligence, 2019. p. 6252-6259 (IJCAI International Joint Conference on Artificial Intelligence; Vol. 2019-August).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Aditya, S, Yang, Y & Baral, C 2019, Integrating knowledge and reasoning in image understanding. in S Kraus (ed.), Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019. IJCAI International Joint Conference on Artificial Intelligence, vol. 2019-August, International Joint Conferences on Artificial Intelligence, pp. 6252-6259, 28th International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, 8/10/19.
Aditya S, Yang Y, Baral C. Integrating knowledge and reasoning in image understanding. In Kraus S, editor, Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019. International Joint Conferences on Artificial Intelligence. 2019. p. 6252-6259. (IJCAI International Joint Conference on Artificial Intelligence).
Aditya, Somak ; Yang, Yezhou ; Baral, Chitta. / Integrating knowledge and reasoning in image understanding. Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019. editor / Sarit Kraus. International Joint Conferences on Artificial Intelligence, 2019. pp. 6252-6259 (IJCAI International Joint Conference on Artificial Intelligence).
@inproceedings{f3898690f9b54a2ba9a82293444f695b,
title = "Integrating knowledge and reasoning in image understanding",
abstract = "Deep learning based data-driven approaches have been successfully applied in various image understanding applications ranging from object recognition, semantic segmentation to visual question answering. However, the lack of knowledge integration as well as higher-level reasoning capabilities with the methods still pose a hindrance. In this work, we present a brief survey of a few representative reasoning mechanisms, knowledge integration methods and their corresponding image understanding applications developed by various groups of researchers, approaching the problem from a variety of angles. Furthermore, we discuss upon key efforts on integrating external knowledge with neural networks. Taking cues from these efforts, we conclude by discussing potential pathways to improve reasoning capabilities.",
author = "Somak Aditya and Yezhou Yang and Chitta Baral",
year = "2019",
month = "1",
day = "1",
language = "English (US)",
series = "IJCAI International Joint Conference on Artificial Intelligence",
publisher = "International Joint Conferences on Artificial Intelligence",
pages = "6252--6259",
editor = "Sarit Kraus",
booktitle = "Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019",

}

TY - GEN

T1 - Integrating knowledge and reasoning in image understanding

AU - Aditya, Somak

AU - Yang, Yezhou

AU - Baral, Chitta

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Deep learning based data-driven approaches have been successfully applied in various image understanding applications ranging from object recognition, semantic segmentation to visual question answering. However, the lack of knowledge integration as well as higher-level reasoning capabilities with the methods still pose a hindrance. In this work, we present a brief survey of a few representative reasoning mechanisms, knowledge integration methods and their corresponding image understanding applications developed by various groups of researchers, approaching the problem from a variety of angles. Furthermore, we discuss upon key efforts on integrating external knowledge with neural networks. Taking cues from these efforts, we conclude by discussing potential pathways to improve reasoning capabilities.

AB - Deep learning based data-driven approaches have been successfully applied in various image understanding applications ranging from object recognition, semantic segmentation to visual question answering. However, the lack of knowledge integration as well as higher-level reasoning capabilities with the methods still pose a hindrance. In this work, we present a brief survey of a few representative reasoning mechanisms, knowledge integration methods and their corresponding image understanding applications developed by various groups of researchers, approaching the problem from a variety of angles. Furthermore, we discuss upon key efforts on integrating external knowledge with neural networks. Taking cues from these efforts, we conclude by discussing potential pathways to improve reasoning capabilities.

UR - http://www.scopus.com/inward/record.url?scp=85074901809&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074901809&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85074901809

T3 - IJCAI International Joint Conference on Artificial Intelligence

SP - 6252

EP - 6259

BT - Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019

A2 - Kraus, Sarit

PB - International Joint Conferences on Artificial Intelligence

ER -