Conceptualization and ontology: Tools for efficient storage and retrieval of semantic visual information

Techniques for content-based image or video retrieval are not mature enough to recognize visual semantic completely. Retrieval based on color, size, texture and shape are within the state of the art. Our experiments on human factors in visual information query and retrieval show that visual information retrieval based on the semantic understanding of visual objects and content are more demanding rather than visual appearance based retrieval. Therefore, it is necessary to use captions or text annotations to photos or videos in content access of visual data. In this paper, human factors in text and image searching are carefully investigated. Based on the resulting human factors, a framework for integrated querying of visual information and textual concept is presented. The framework includes ontology-based semantic query expansion through query term rewriting and database navigation within a conceptual hierarchy within multi modal querying environments. To allow similarity based concept retrieval, a new conceptual similarity distance measure between two conceptual entities in a given conceptual space is proposed. The dissimilarity metric is a minimum weighted path length in the corresponding conceptual tree.