Object tracking via collaborative multi-task learning and appearance model updating

In this paper, we propose a novel visual tracking algorithm using the collaboration of generative and discriminative trackers under the particle filter framework. Each particle denotes a single task, and we encode all the tasks simultaneously in a structured multi-task learning manner. Then, we implement generative and discriminative trackers, respectively. The discriminative tracker considers the overall information of object to represent the object appearance; while the generative tracker takes the local information of object into account for handling partial occlusions. Therefore, two models are complementary during the tracking. Furthermore, we design an effective dictionary updating mechanism. The dictionary is composed of fixed and variational parts. The variational parts are progressively updated using Metropolis–Hastings strategy. Experiments on different challenging video sequences demonstrate that the proposed tracker performs favorably against several state-of-the-art trackers.     

A parallel and robust object tracking approach synthesizing adaptive Bayesian learning and improved incremental subspace learning

This paper presents a novel tracking algorithm which integrates two complementary trackers. Firstly, an improved Bayesian tracker(B-tracker) with adaptive learning rate is presented. The classification score of B-tracker reflects tracking reliability, and a low score usually results from large appearance change. Therefore, if the score is low, we decrease the learning rate to update the classifier fast so that B-tracker can adapt to the variation and vice versa. In this way, B-tracker is more suitable than its traditional version to solve appearance change problem. Secondly, we present an improved incremental subspace learning method tracker(Stracker). We propose to calculate projected coordinates using maximum posterior probability, which results in a more accurate reconstruction error than traditional subspace learning tracker. Instead of updating at every time, we present a stopstrategy to deal with occlusion problem. Finally, we present an integrated framework(BAST), in which the pair of trackers run in parallel and return two candidate target states separately. For each candidate state, we define a tracking reliability metrics to measure whether the candidate state is reliable or not, and the reliable candidate state will be chosen as the target state at the end of each frame. Experimental results on challenging sequences show that the proposed approach is very robust and effective in comparison to the state-of-the-art trackers.     

An Adaptive Padding Correlation Filter With Group Feature Fusion for Robust Visual Tracking

In recent visual tracking research, correlation filter (CF) based trackers become popular because of their high speed and considerable accuracy. Previous methods mainly work on the extension of features and the solution of the boundary effect to learn a better correlation filter. However, the related studies are insufficient. By exploring the potential of trackers in these two aspects, a novel adaptive padding correlation filter (APCF) with feature group fusion is proposed for robust visual tracking in this paper based on the popular context-aware tracking framework. In the tracker, three feature groups are fused by use of the weighted sum of the normalized response maps, to alleviate the risk of drift caused by the extreme change of single feature. Moreover, to improve the adaptive ability of padding for the filter training of different object shapes, the best padding is selected from the preset pool according to tracking precision over the whole video, where tracking precision is predicted according to the prediction model trained by use of the sequence features of the first several frames. The sequence features include three traditional features and eight newly constructed features. Extensive experiments demonstrate that the proposed tracker is superior to most state-of-the-art correlation filter based trackers and has a stable improvement compared to the basic trackers.      

Multi-object tracking evaluated on sparse events

This article presents a visual object tracking method and applies an event-based performance evaluation metric for assessment. The proposed monocular object tracker is able to detect and track multiple object classes in non-controlled environments. The tracking framework uses Bayesian per-pixel classification to segment an image into foreground and background objects, based on observations of object appearances and motions in real-time. Furthermore, a performance evaluation method is presented and applied to different state-of-the-art trackers based on successful detections of semantically high level events. These events are extracted automatically from the different trackers an their varying types of low level tracking results. Then, a general new event metric is used to compare our tracking method with the other tracking methods against ground truth of multiple public datasets.     

Robust visual tracking using information theoretical learning

This paper presents a novel online object tracking algorithm with sparse representation for learning effective appearance models under a particle filtering framework. Compared with the state-of-the-art ? ₁ sparse tracker, which simply assumes that the image pixels are corrupted by independent Gaussian noise, our proposed method is based on information theoretical Learning and is much less sensitive to corruptions; it achieves this by assigning small weights to occluded pixels and outliers. The most appealing aspect of this approach is that it can yield robust estimations without using the trivial templates adopted by the previous sparse tracker. By using a weighted linear least squares with non-negativity constraints at each iteration, a sparse representation of the target candidate is learned; to further improve the tracking performance, target templates are dynamically updated to capture appearance changes. In our template update mechanism, the similarity between the templates and the target candidates is measured by the earth movers’ distance(EMD). Using the largest open benchmark for visual tracking, we empirically compare two ensemble methods constructed from six state-of-the-art trackers, against the individual trackers. The proposed tracking algorithm runs in real-time, and using challenging sequences performs favorably in terms of efficiency, accuracy and robustness against state-of-the-art algorithms.     

A General Framework for Combining Visual Trackers – The "Black Boxes" Approach

Over the past few years researchers have been investigating the enhancement of visual tracking performance by devising trackers that simultaneously make use of several different features. In this paper we investigate the combination of synchronous visual trackers that use different features while treating the trackers as “black boxes”. That is, instead of fusing the usage of the different types of data as has been performed in previous work, the combination here is allowed to use only the trackers' output estimates, which may be modified before their propagation to the next time step. We propose a probabilistic framework for combining multiple synchronous trackers, where each separate tracker outputs a probability density function of the tracked state, sequentially for each image. The trackers may output either an explicit probability density function, or a sample-set of it via Condensation. Unlike previous tracker combinations, the proposed framework is fairly general and allows the combination of any set of trackers of this kind, even in different state-spaces of different dimensionality, under a few reasonable assumptions. The combination may consist of different trackers that track a common object, as well as trackers that track separate, albeit related objects, thus improving the tracking performance of each object. The benefits of merely using the final estimates of the separate trackers in the combination are twofold. Firstly, the framework for the combination is fairly general and may be easily used from the software aspects. Secondly, the combination may be performed in a distributed setting, where each separate tracker runs on a different site and uses different data, while avoiding the need to share the data. The suggested framework was successfully tested using various state-spaces and datasets, demonstrating that fusing the trackers' final distribution estimates may indeed be applicable. Electronic supplementary material Electronic supplementary material is available for this article at and accessible for authorised users. First online version published in October, 2005     

Mean shift tracker combined with online learning-based detector and Kalman filtering for real-time tracking

Color-based visual object tracking is one of the most commonly used tracking methods. Among many tracking methods, the mean shift tracker is used most often because it is simple to implement and consumes less computational time. However, mean shift trackers exhibit several limitations when used for long-term tracking. In challenging conditions that include occlusions, pose variations, scale changes, and illumination changes, the mean shift tracker does not work well. In this paper, an improved tracking algorithm based on a mean shift tracker is proposed to overcome the weaknesses of existing methods based on mean shift tracker. The main contributions of this paper are to integrate mean shift tracker with an online learning-based detector and to newly define the Kalman filter-based validation region for reducing computational burden of the detector. We combine the mean shift tracker with the online learning-based detector, and integrate the Kalman filter to develop a novel tracking algorithm. The proposed algorithm can reinitialize the target when it converges to a local minima and it can cope with scale changes, occlusions and appearance changes by using the online learning-based detector. It updates the target model for the tracker in order to ensure long-term tracking. Moreover, the validation region obtained by using the Kalman filter and the Mahalanobis distance is used in order to operate detector in real-time. Through a comparison against various mean shift tracker-based methods and other state-of-the-art methods on eight challenging video sequences, we demonstrate that the proposed algorithm is efficient and superior in terms of accuracy and speed. Hence, it is expected that the proposed method can be applied to various applications which need to detect and track an object in real-time.     

Environmental Perception with Spatial Regularization Correlation Filter for Visual Tracking

With the introduction of correlation filtering (CF), the performance of visual object tracking is significantly improved. Circular shifts collecting samples is a key component of the CF tracker, and it also causes negative boundary effects. Most trackers add spatial regularization to alleviate boundary effects well. However, these trackers ignore the effect of environmental changes on tracking performance, and the filter discriminates poorly in the background interference. Here, to break these limitations, we propose a new correlation filter model, namely Environmental Perception with Spatial Regularization Correlation Filter for Visual Tracking. Specifically, we use the Average Peak to Correlation Energy (APCE) and the response value error between the two frames together to perceive environmental changes, which adjusts the learning rate to make the template more adaptable to environmental changes. To enhance the discriminatory capability of the filter, we use real background information as negative samples to train the filter model. In addition, the introduction of the regular term destroys the closed solution of CF, and this problem can be effectively solved by the use of the alternating direction method of multipliers (ADMM). Extensive experimental evaluations on three large tracking benchmarks are performed, which demonstrate the good performance of the proposed method over some of the state-of-the-art trackers.     

基于相关滤波的尺度自适应目标跟踪

针对视频目标跟踪中的尺度问题,提出了一种基于相关滤波的尺度自适应目标跟踪方法。首先利用核相关滤波获得目标的中心位置,然后将目标均分为四个子块,通过计算找出子块中心的最大响应位置,最后根据前后两帧目标子块中心位置的相对变化计算出尺度的伸缩系数,进而计算出目标尺度。在具有尺度变化的公开数据集上对本文方法进行测试,并和多种跟踪方法作对比,实验结果表明,该方法将尺度的计算问题转化为对子块中心的定位,其平均跟踪性能优于其它方法,验证了方法的有效性。     

Closed-world tracking of multiple interacting targets for indoor-sports applications

In this paper we present an efficient algorithm for tracking multiple players during indoor sports matches. A sports match can be considered as a semi-controlled environment for which a set of closed-world assumptions regarding the visual as well as the dynamical properties of the players and the court can be derived. These assumptions are then used in the context of particle filtering to arrive at a computationally fast, closed-world, multi-player tracker. The proposed tracker is based on multiple, single-player trackers, which are combined using a closed-world assumption about the interactions among players. With regard to the visual properties, the robustness of the tracker is achieved by deriving a novel sports-domain-specific likelihood function and employing a novel background-elimination scheme. The restrictions on the player’s dynamics are enforced by employing a novel form of local smoothing. This smoothing renders the tracking more robust and reduces the computational complexity of the tracker. We evaluated the proposed closed-world, multi-player tracker on a challenging data set. In comparison with several similar trackers that did not utilize all of the closed-world assumptions, the proposed tracker produced better estimates of position and prediction as well as reducing the number of failures.     

快速深度学习的鲁棒视觉跟踪

目的 基于深度学习的视觉跟踪算法具有跟踪精度高、适应性强的特点,但是,由于其模型参数多、调参复杂,使得算法的时间复杂度过高。为了提升算法的效率,通过构建新的网络结构、降低模型冗余,提出一种快速深度学习的算法。方法 鲁棒特征的提取是视觉跟踪成功的关键。基于深度学习理论,利用海量数据离线训练深度神经网络,分层提取描述图像的特征;针对网络训练时间复杂度高的问题,通过缩小网络规模得以大幅缓解,实现了在GPU驱动下的快速深度学习;在粒子滤波框架下,结合基于支持向量机的打分器的设计,完成对目标的在线跟踪。结果 该方法精简了特征提取网络的结构,降低了模型复杂度,与其他基于深度学习的算法相比,具有较高的时效性。系统的跟踪帧率总体保持在22帧/s左右。结论 实验结果表明,在目标发生平移、旋转和尺度变化,或存在光照、遮挡和复杂背景干扰时,本文算法能够实现比较稳定和相对快速的目标跟踪。但是,对目标的快速移动和运动模糊的鲁棒性不够高,容易受到相似物体的干扰。     

Robust tracking with interest points: A sparse representation approach

Visual tracking is an important task in various computer vision applications including visual surveillance, human computer interaction, event detection, video indexing and retrieval. Recent state of the art sparse representation (SR) based trackers show better robustness than many of the other existing trackers. One of the issues with these SR trackers is low execution speed. The particle filter framework is one of the major aspects responsible for slow execution, and is common to most of the existing SR trackers. In this paper,¹ we propose a robust interest point based tracker in l₁ minimization framework that runs at real-time with performance comparable to the state of the art trackers. In the proposed tracker, the target dictionary is obtained from the patches around target interest points. Next, the interest points from the candidate window of the current frame are obtained. The correspondence between target and candidate points is obtained via solving the proposed l₁ minimization problem.In order to prune the noisy matches, a robust matching criterion is proposed, where only the reliable candidate points that mutually match with target and candidate dictionary elements are considered for tracking. The object is localized by measuring the displacement of these interest points. The reliable candidate patches are used for updating the target dictionary. The performance and accuracy of the proposed tracker is benchmarked with several complex video sequences. The tracker is found to be considerably fast as compared to the reported state of the art trackers. The proposed tracker is further evaluated for various local patch sizes, number of interest points and regularization parameters. The performance of the tracker for various challenges including illumination change, occlusion, and background clutter has been quantified with a benchmark dataset containing 50 videos.     

判别稀疏表示鲁棒快速视觉跟踪

目的 L1跟踪对局部遮挡具有良好的鲁棒性,但存在易产生模型漂移和计算速度慢的问题。针对这两个问题,该文提出了一种基于判别稀疏表示的视觉跟踪方法。方法 考虑到背景和遮挡信息的干扰,提出了一种判别稀疏表示模型,并基于块坐标优化原理,采用学习迭代收缩阈值算法和软阈值操作设计出了表示模型的快速求解算法。结果 在8组图像序列中,该文方法与现有的4种经典跟踪方法分别在鲁棒性和稀疏表示的计算时间方面进行了比较。在鲁棒性的定性和定量比较实验中,该文方法不仅表现出了对跟踪过程中的多种干扰因素具有良好的适应能力,而且在位置误差阈值从0~50像素的变化过程中,其精度曲线均优于实验中的其他方法;在稀疏表示的计算时间方面,在采用大小为16×16和32×32的模板进行跟踪时,该文算法的时间消耗分别为0.152 s和0.257 s,其时效性明显优于实验中的其他方法。结论 与经典的跟踪方法相比,该文方法能够在克服遮挡、背景干扰和外观改变等诸多不良因素的同时,实现快速目标跟踪。由于该文方法不仅具有较优的稀疏表示计算速度,而且能够克服多种影响跟踪鲁棒性的干扰因素,因此可以将其应用于视频监控和体育竞技等实际场合。     

类孪生网络目标跟踪算法综述

目前,在视觉目标跟踪任务中的主流方法是基于模版匹配的跟踪器,这些方法在目标的分类和边界框的回归上具有很强的鲁棒性,主要可以分为判别相关滤波跟踪器和孪生网络跟踪器,这两种方法都有一个类孪生网络的框架。以孪生网络跟踪器为例,该方法通过模版和搜索区域之间的相关操作确定目标的位置,取得了顶尖的性能表现。近年来,Transformer在计算机视觉领域的发展十分迅速,结合了Transformer的类孪生网络跟踪器在速度和精度方面都远超传统的跟踪方法。文章简要概括了判别相关滤波跟踪器、孪生网络跟踪器的发展,以及Transformer在目标跟踪任务中的应用。     

Multi-view correlation tracking with adaptive memory-improved update model

Recently, some researchers concentrate on applying multi-view learning to the correlation filter tracking to achieve both the efficiency and accuracy. However, most of them fail to effectively collaborate multiple views to deal with more complex environment. Moreover, their methods are prone to drift in case of long-term occlusion due to the memory loss. In this paper, we propose a novel multi-view correlation filters-based tracker for robust visual tracking. First, we present an adaptive multi-view collaboration strategy to highlight different contributions of different views by jointly considering the reliability and discrimination. Second, an effective memory-improved model update rule is introduced to avoid falling into a contaminated target model. Compared with the conventional linear interpolation update rule, it can effectively deal with long-term occlusion by improving the memory of historical models. Furthermore, instead of assigning a unified learning rate for all views in each frame, we design varying learning rates for different views according to their respective evaluations on the current tracking result, which can prevent the target models of all views from being contaminated at the same time. Finally, a failure-aware scale update scheme is developed to avoid noisy scale estimation in case of temporal tracking failure. Extensive experimental results on the recent benchmark demonstrate that our tracker performs favorably against other state-of-the-art trackers with a real-time performance.

     

结合双模板融合与孪生网络的鲁棒视觉目标跟踪

目的 视觉目标跟踪算法主要包括基于相关滤波和基于孪生网络两大类。前者虽然精度较高但运行速度较慢,无法满足实时要求。后者在速度和精度方面取得了出色的跟踪性能,然而,绝大多数基于孪生网络的目标跟踪算法仍然使用单一固定的模板,导致算法难以有效处理目标遮挡、外观变化和相似干扰物等情形。针对当前孪生网络跟踪算法的不足,提出了一种高效、鲁棒的双模板融合目标跟踪方法(siamese tracker with double template fusion,Siam-DTF)。方法 使用第1帧的标注框作为初始模板,然后通过外观模板分支借助外观模板搜索模块在跟踪过程中为目标获取合适、高质量的外观模板,最后通过双模板融合模块,进行响应图融合和特征融合。融合模块结合了初始模板和外观模板各自的优点,提升了算法的鲁棒性。结果 实验在3个主流的目标跟踪公开数据集上与最新的9种方法进行比较,在OTB2015(object tracking benchmark 2015)数据集中,本文方法的AUC(area under curve)得分和精准度分别为0.701和0.918,相比于性能第2的SiamRPN++(siamese region proposal network++)算法分别提高了0.6%和1.3%;在VOT2016(visual object tracking 2016)数据集中,本文方法取得了最高的期望平均重叠(expected average overlap,EAO)和最少的失败次数,分别为0.477和0.172,而且EAO得分比基准算法SiamRPN++提高了1.6%,比性能第2的SiamMask_E算法提高了1.1%;在VOT2018数据集中,本文方法的期望平均重叠和精确度分别为0.403和0.608,在所有算法中分别排在第2位和第1位。本文方法的平均运行速度达到47帧/s,显著超出跟踪问题实时性标准要求。结论 本文提出的双模板融合目标跟踪方法有效克服了当前基于孪生网络的目标跟踪算法的不足,在保证算法速度的同时有效提高了跟踪的精确度和鲁棒性,适用于工程部署与应用。     

Online unsupervised feature learning for visual tracking

We propose a method for visual tracking-by-detection based on online feature learning. Our learning framework performs feature encoding with respect to an over-complete dictionary, followed by spatial pyramid pooling. We then learn a linear classifier based on the resulting feature encoding. Unlike previous work, we learn the dictionary online and update it to help capture the appearance of the tracked target as well as the background. In more detail, given a test image window, we extract local image patches from it and each local patch is encoded with respect to the dictionary. The encoded features are then pooled over a spatial pyramid to form an aggregated feature vector. Finally, a simple linear classifier is trained on these features.Our experiments show that the proposed powerful—albeit simple—tracker, outperforms all the state-of-the-art tracking methods that we have tested. Moreover, we evaluate the performance of different dictionary learning and feature encoding methods in the proposed tracking framework, and analyze the impact of each component in the tracking scenario. In particular, we show that a small dictionary, learned and updated online is as effective and more efficient than a huge dictionary learned offline. We further demonstrate the flexibility of feature learning by showing how it can be used within a structured learning tracking framework. The outcome is one of the best trackers reported to date, which facilitates the advantages of both feature learning and structured output prediction. We also implement a multi-object tracker, which achieves state-of-the-art performance.     

Robust visual tracking via bag of superpixels

The Bag of Words (BoW) model is one of the most popular and effective image representation methods and has been drawn increasing interest in computer vision filed. However, little attention is paid on it in visual tracking. In this paper, a visual tracking method based on Bag of Superpixels (BoS) is proposed. In BoS, the training samples are oversegmented to generate enough superpixel patches. Then K-means algorithm is performed on the collected patches to form visual words of the target and a superpixel codebook is constructed. Finally the tracking is accomplished via searching for the highest likelihood between candidates and codebooks within Bayesian inference framework. In this process, an effective updating scheme is adopted to help our tracker resist occlusions and deformations. Experimental results demonstrate that the proposed method outperforms several state-of-the-art trackers.     

利用时空特征编码的单目标跟踪网络

目的 随着深度神经网络的出现,视觉跟踪快速发展,视觉跟踪任务中的视频时空特性,尤其是时序外观一致性(temporal appearance consistency)具有巨大探索空间。本文提出一种新颖简单实用的跟踪算法——时间感知网络(temporal-aware network, TAN),从视频角度出发,对序列的时间特征和空间特征同时编码。方法 TAN内部嵌入了一个新的时间聚合模块(temporal aggregation module, TAM)用来交换和融合多个历史帧的信息,无需任何模型更新策略也能适应目标的外观变化,如形变、旋转等。为了构建简单实用的跟踪算法框架,设计了一种目标估计策略,通过检测目标的4个角点,由对角构成两组候选框,结合目标框选择策略确定最终目标位置,能够有效应对遮挡等困难。通过离线训练,在没有任何模型更新的情况下,本文提出的跟踪器TAN通过完全前向推理(fully feed-forward)实现跟踪。结果 在OTB(online object tracking:a benchmark)50、OTB100、TrackingNet、LaSOT(a high-qua...