Using a model of the human visual system to identify features for the indexing and retrieval of images

The ability to characterize the important features of images is vital when responding to queries directed to an image database. Ideally, to produce results that are satisfactory to a human user, the system should employ methods for characterizing the image that are similar to those used by the human visual system. However, global mathematical techniques such as histograms or frequency-based transforms bear little resemblance to the early processing steps performed on the visual stream by the human visual system, as it is passed from the eye into the brain. This paper presents a model that employs a sequence of spatial convolutions and thresholding operations to mimic the neural behavior of the visual pathway, including the bipolar cells, the horizontal cells, and the retinal ganglion cells of the retina, the Lateral Geniculate Nucleus (LGN) and the simple cells of the primary visual cortex (V1). When this model is applied to an image, the result is a 2D pattern of excitation similar to that observed by neural scientists in the primary visual cortex of primates. Given the fact that the excitation pattern in the primary visual cortex is the basis for virtually all higher-level visual processing, this pattern represents the visual system's selection of the most important features in the image. By processing this 2D pattern in ways similar to the latter stages of the human visual system, an image archiving system could characterize an image in ways that are similar to the human visual system. To demonstrate one use for the model, this paper uses the patterns generated from images of several complex 3D textures to determine the direction of the light falling on each of them.