A model for visio-haptic attention for efficient resource allocation in multimodal environments

Sequences of visual and haptic exploration were obtained on surfaces of different curvature from human subjects. We then extracted regions of interest (ROI) from the data as a function of number of times a subject fixated on a certain location on object and amount of time spent on such each location. Simple models like a plane, cone, cylinder, paraboloid, hyperboloid, ellipsoid, simple-saddle and a monkey-saddle were generated. Gaussian curvature representation of each point on all the surfaces was pre-computed. The surfaces have been previously tested for haptic and visual realism and distinctness by human subjects in a separate experiment. Both visual and haptic rendering were subsequently used for exploration by human subjects to study whether there is a similarity between the visual ROI and haptic ROIs. Additionally, we wanted to see if there is a correlation between curvature values and the ROIs thus obtained. A multiple regression model was further developed to see if this data can be used to predict the visual exploration path using haptic curvature saliency measures.