The study of insect behavior plays a fundamental role in the biological sciences. One of the key factors limiting progress in all forms of insect behavior research is the rate at which data can be gathered. Insect studies are based upon either direct human observation or recorded video of behavioral interactions. However the most common practice for obtaining behavioral measurements from video is for human researchers to undertake a painstaking and time-consuming process of manually identifying behaviors through frameby- frame analysis of insect movement. Studies of ant behavior constitute a particularly relevant example of these limitations, and they are the primary motivation for the proposed work. We propose to develop an integrated solution to the problem of automating behavioral measurements from video records, consisting of theory, algorithms, software modules, and databases of behavior measurements. Our approach leverages and extends state-of-the-art methods for multiple target tracking in video. Standard multi-target video tracking algorithms trace their origins to classical work in radar systems. One problem with the application of these technologies to the analysis of social insect behavior is the need to deal with a huge degree of interaction between multiple targets, including targets which occlude each other and may be occluded by other objects for significant periods of time. We propose a novel tracking approach based on video object segmentation which utilizes the graphcut optimization method. This makes it possible to identify which portions of the video correspond to distinct targets even when these targets overlap. Accurate segmentation of the target also facilitates more accurate adaptation of the appearance model over time, as the insect appearance changes due to lighting and other environmental effects. We propose to develop novel behavior recognition methods which leverage the ability to segment video targets and analyzes the configuration and appearance of the target to assess behavior. Traditional behavior measures based on tracking systems express the behavior as a function of the state of the tracker. This has the disadvantage that tracking errors will compound recognition errors and potentially-useful information is discarded. We will validate our models, theory, and software modules by developing them for and applying them to biological experiments in ant social behavior. Two cross-cutting themes inform our proposed work. The first is a focus on algorithms and methods that support the development of modular software tools, which would allow researchers in biology to develop a customized solution to a wide range of sensing tasks. The second theme is the utilization of state-of-the-art ultra-high resolution imaging sensors to obtain more information about ant behavior and identity than is currently possible.
|Effective start/end date||9/1/10 → 8/31/14|
- NSF-BIO: Division of Biological Infrastructure (DBI): $209,014.00