In this paper, a generic approach to object matching and fast tracking in video and image sequence is presented. The approach first uses Gabor filters to extract flexible and reliable features as the basis of object matching and tracking. Then, a modified Elastic Graph Matching method is proposed for accurate object matching. A novel method based on posterior probability density estimation through sequential Monte Carlo method, called as Sequential Importance Sampling (SIS) method, is also developed to track multiple objects simultaneously. Several applications of our proposed approach are given for performance evaluation, which includes moving target tracking, stereo (3D) imaging, and camera stabilization. The experimental results demonstrated the efficacy of the approach which can also be applied to many other military and civilian applications, such as moving target verification and tracking, visual surveillance of public transportation, country border control, battlefield inspection and analysis, etc.