A real-time video mosaicking and target tracking system

This paper describes an automatic video target tracking system that operates on the panoramic image provided by an image mosaicking preprocessing stage. In the mosaic preprocessing stage, a feature-based algorithm is applied to obtain the underlying homography between consecutive frames in a video sequence. With the first frame in the sequence chosen as the base image plane, subsequent frames are warped and merged into a panoramic scene for the video tracking stage. The tracking algorithm calculates the motion vector for each block in a warped frame by comparing it with the panoramic image, and those exceeding the dominant background motion can be considered as blocks belonging to potential moving foreground objects. Image segmentation is then used to recover the boundaries of the foreground objects. After fusing the labeled boundaries with the motion vector information, the potential targets, as well as their feature vectors, are identified. The feature vectors include information pertaining to location, size, and optical characteristics, and are input into a sub-tracker for record keeping. The input to the proposed system is a video stream from a single camera.