TY - GEN
T1 - A model-based approach to visual reasoning on CNLVR dataset
AU - Sampat, Shailaja
AU - Lee, Joohyung
N1 - Funding Information:
Acknowledgments: We are grateful to the anonymous referees for their useful comments. This work was partially supported by the National Science Foundation under Grants IIS-1526301 and IIS-1815337.
PY - 2018
Y1 - 2018
N2 - Visual Reasoning requires an understanding of complex compositional images and common-sense reasoning about sets of objects, quantities, comparisons, and spatial relationships. This paper presents a semantic parser that combines Computer Vision (CV), Natural Language Processing (NLP) and Knowledge Representation & Reasoning (KRR) to automatically solve visual reasoning problems from the Cornell Natural Language Visual Reasoning (CNLVR) dataset. Unlike the data-driven approaches applied to the same dataset, our system does not require any training but is guided by the knowledge base that is manually constructed. The system demonstrates robust overall performance which is also time and space efficient. Our system achieves 87.3% accuracy, which is 17.6% higher over the state-of-the-art method on raw image representations.
AB - Visual Reasoning requires an understanding of complex compositional images and common-sense reasoning about sets of objects, quantities, comparisons, and spatial relationships. This paper presents a semantic parser that combines Computer Vision (CV), Natural Language Processing (NLP) and Knowledge Representation & Reasoning (KRR) to automatically solve visual reasoning problems from the Cornell Natural Language Visual Reasoning (CNLVR) dataset. Unlike the data-driven approaches applied to the same dataset, our system does not require any training but is guided by the knowledge base that is manually constructed. The system demonstrates robust overall performance which is also time and space efficient. Our system achieves 87.3% accuracy, which is 17.6% higher over the state-of-the-art method on raw image representations.
UR - http://www.scopus.com/inward/record.url?scp=85071121700&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85071121700&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85071121700
T3 - Principles of Knowledge Representation and Reasoning: Proceedings of the 16th International Conference, KR 2018
SP - 62
EP - 66
BT - Principles of Knowledge Representation and Reasoning
A2 - Thielscher, Michael
A2 - Toni, Francesca
A2 - Wolter, Frank
PB - AAAI press
T2 - 16th International Conference on the Principles of Knowledge Representation and Reasoning, KR 2018
Y2 - 30 October 2018 through 2 November 2018
ER -