A framework for spatiotemporal control in the tracking of visual contours

There has been a great deal of research interest in contour tracking over the last five years. This article combines themes from tracking theory—elastic models and stochastic filtering—with the notion of affine invariance to synthesize a substantially new and demonstrably effective framework for contour tracking.A mechanism is developed for incorporating a shape template into a contour tracker via an affine invariant coupling. In that way the tracker becomes selective for shape and therefore able to ignore background clutter. Affine invariance ensures that the effect of varying viewpoint is accommodated. Use of a standard statistical filtering framework allows uncertainties to be treated systematically, which accommodates object flexibility and un-modeled distortions such as the deformation of a silhouette under motion.The statistical framework also facilitates a further development. In place of heuristically determined spatial scale for feature search, both spatial scale and temporal memory are controlled automatically and in a way that is responsive to the tracking process. Typically, the tracker operates initially in a coarse scale/short memory mode while it searches for a feature. Then spatial scale diminishes to allow more precise localization while memory (temporal scale) lengths to take advantage of motion coherence. All system parameters are determined by natural assumptions and desired tracking performance, leaving none to be fixed heuristically.Versions of the tracker have been implemented at video rate, both on SUN 4 and in parallel, using a network of 11 transputers. The theoretically established properties of automatic control of spatiotemporal scale and of affine invariance are demonstrated using the implemented tracker.