Temporal Frame Sub-Sampling for Video Object Tracking

Temporal frame sub-sampling (TFS) for reducing video object tracking (VOT) computing time is investigated. With a sampling ratio N, the TFS VOT algorithm will process a shorter video by sampling 1 out of N frames of the given video. The object trajectory of the remaining frames will be interpolated linearly based on those of sampled frames. Thus, TFS can result in a significant reduction of processing time at a cost of losing tracking accuracy. More importantly, it can be applied to accelerate any VOT algorithms. However, it is observed that when the object trajectory is smooth, the tracking accuracy of a TFS VOT algorithm may be improved compared to non-TFS results. In this work, we provide an empirical analysis of this unexpected outcome of the TFS scheme. We also suggest rules of thumb to leverage this property to use TFS to enhance efficiency and accuracy of VOT.