TRBACF: Learning temporal regularized correlation filters for high performance online visual object tracking

Correlation filter-based trackers (CFTs) have recently shown remarkable performance in the field of visual object tracking. The advantage of these trackers originates from their ability to convert time-domain calculations into frequency domain calculations. However, a significant problem of these CFTs is that the model is insufficiently robust when the tracking scenarios are too complicated, meaning that the ideal tracking performance cannot be acquired. Recent work has attempted to resolve this problem by reducing the boundary effects from modeling the foreground and background of the object target effectively (e.g., CFLB, BACF, and CACF). Although these methods have demonstrated reasonable performance, they are often affected by occlusion, deformation, scale variation, and other challenging scenes. In this study, considering the relationship between the current frame and the previous frame of a moving object target in a time series, we propose a temporal regularization strategy to improve the BACF tracker (denoted as TRBACF), a typical representative of the aforementioned trackers. The TRBACF tracker can efficiently adjust the model to adapt the change of the tracking scenes, thereby enhancing its robustness and accuracy. Moreover, the objective function of our TRBACF tracker can be solved by an improved alternating direction method of multipliers, which can speed up the calculation in the Fourier domain. Extensive experimental results demonstrate that the proposed TRBACF tracker achieves competitive tracking performance compared with state-of-the-art trackers.     

Robust object tracking via deformation samples generator

In object tracking applications, it is common for trackers to experience drift problems when the object of interest becomes deformed, which compromises the ability of the tracker to track the object. It is therefore desirable to develop a learning tracker classifier that is robust to deformations. The performance of existing trackers that employ deep classification networks degrades when the amount of training data is limited and does not cover all possible scenarios. While these limitations can be mitigated in part by using larger training datasets, these datasets may still not cover all situations and the positive samples are still monotonous. To overcome this problem, we propose a novel deformation samples generator that generates samples that would normally be difficult for the tracker to classify. In the proposed framework, both the classifier and deformation samples generator learn in a joint manner. Our experiments show that the proposed approach outperforms state-of-the-art methods in both quantitative and qualitative evaluations for the visual object tracking task.     

Multi-frame co-saliency spatio-temporal regularization correlation filters for object tracking

The spatial regularization weight of the correlation filter is not related to the object content and the model degradation in the tracking process. To solve this problem, a new multi-frame co-saliency spatio-temporal regularization correlation filters (MCSRCF) is proposed for visual object tracking. To the best our knowledge, this is the first application of co-saliency regularization to CF-based tracking. In MCSRCF, grayscale features, directional gradient histogram (HOG) features and CNN features are extracted to improve the tracking precision of the tracker. Secondly, the three-dimensional spatial saliency and semantic saliency are introduced to obtain the initial weight of the spatial regularization with object content information. Then, the heterogeneous saliency fusion method is exploited to add a co-saliency spatial regularization term to the objective function to make the spatial penalty weight learn the change of the object region. In additional, the temporal saliency regularization is introduced to learn the information between adjacent frames, which reduces the overfitting effect caused by inaccurate samples. A variety of evaluations are conducted on public benchmarks, and the experimental results show that the proposed tracker achieves good robustness against many state-of-the-art trackers in various complex scenarios.     

Robust model adaption for colour-based particle filter tracking with contextual information

Color-based particle filters have emerged as an appealing method for targets tracking. As the target may undergo rapid and significant appearance changes, the template (i.e. scale of the target, color distribution histogram) also needs to be updated. Traditional updates without learning contextual information may imply a high risk of distorting the model and losing the target. In this paper, a new algorithm utilizing the environmental information to update both the scale of the tracker and the reference appearance model for the purpose of object tracking in video sequences has been put forward. The proposal makes use of the well-established color-based particle filter tracking while differentiating the foreground and background particles according to their matching score. A roaming phenomenon that yields the estimation to shrink and diverge is investigated. The proposed solution is tested using both simulated and publicly available benchmark datasets where a comparison with six state-of-the-art trackers has been carried out. The results demonstrate the feasibility of the proposal and lie down foundations for further research on tackling complex visual tracking problems.     

Re-identification framework for long term visual object tracking based on object detection and classification

In this paper, we address the problem of long-term visual object tracking and we present an efficient real-time single object tracking system suitable for integration in autonomous platforms that need to encompass intelligent capabilities. We propose a novel long-term tracking framework for classification based re-detection and tracking, that incorporates state estimation, object re-identification and automated management of tracking and detection results. Our method integrates a novel object re-identification technique which efficiently filters a number of detection candidates and systematically corrects the tracking results. Through extensive experimental validation on the UAV123, UAV20L and TLP datasets, we demonstrate the effectiveness of the proposed system and its advantage over several state-of-the art trackers. The results furthermore highlight the proposed tracker’s ability to handle challenges arising from real-world and long-term scenarios, such as variations in pose, scale, occlusions and out-of-view situations. Furthermore, we propose a variant that is suitable for deployment on autonomous robots, such as Unmanned Aerial Vehicles.     

Real-time and robust visual tracking with scene-perceptual memory

Unmanned aerial vehicle (UAV) based aerial visual tracking is one of the research hotspots in computer vision. However, the mainstream trackers for UAV still have two shortcomings: (1) the accuracy of correlation filter tracker is greatly improved with more complex model, it impedes accuracy-speed trade-off. (2) object occlusion and camera motion in the aerial tracking scene also seriously restrict the application of aerial tracking. To address these problems, and inspired by AutoTrack tracker, we propose an aerial correlation filtering tracker based on scene-perceptual memory, Fast-AutoTrack. Firstly, to perceive and judge tracking anomalies, such as object occlusion and camera motion, inspired by the peak sidelobe ratio and AutoTrack, a confidence score is designed by perceiving and remembering the changing trend of the confidence and the local historical confidence. Secondly, after tracking anomaly occurring, several search regions are predicted based on the local object motion trend and the Spatio-temporal context information for object re-detection. Finally, to accelerate the model updating, the perceptual hashing algorithm (PHA) is used to obtain the similarity of the search regions between two adjacent frames. On typical aerial tracking datasets UAVDT, UAV123@10fps, and DTB70, Fast-AutoTrack run 71.4% faster than AutoTrack with almost equal accuracy and show favorable accuracy-speed trade-off.     

Robust Struck tracker via color Haar-like feature and selective updating

Recently, Struck—a tracker based on structured support vector machine, received great attention as a consequence of its superior performance on many challenging scenes. In this work, we present an improved Struck tracker by using color Haar-like features and effective selective updating. First, we integrate color information into Haar-like features in a simple way, which models the spatial and color information simultaneously without increasing the computational complexity. Second, we make selective model updates according to the tracking status of the object. This prevents inferior patterns resulted by occlusions, abrupt appearance or illumination changes from being added to object model, which decreases the risk of model drift problem. The experimental results indicate that the proposed tracking algorithm outperforms the original Struck by a remarkable margin in precision and accuracy, and it is competitive with other state-of-the-art trackers on a tracking benchmark of 50 challenging sequences.     

基于主分量寻踪的鲁棒视觉跟踪

传统子空间跟踪易受到模型漂移的影响而导致跟踪失败.针对此问题,本文提出一种基于主分量寻踪的鲁棒视觉跟踪方法.该方法以多个模板张成的子空间作为目标表观模型,利用主分量寻踪求解候选目标的误差分量,在粒子滤波框架下利用候选目标的误差分量估计最优状态参数.为了适应目标表观变化并克服模型漂移,本文提出一种模板更新方法.当跟踪结果与目标模板相似时,该方法利用跟踪结果更新目标模板,否则利用跟踪结果的低秩分量更新目标模板.在多个具有挑战性的图像序列上的实验结果表明:与现有跟踪方法相比,文中的跟踪方法具有较优的跟踪性能.     

Deformable part-based tracking by coupled global and local correlation filters

Correlation filters have recently attracted attention in visual tracking due to their efficiency and high performance. However, their application to long-term tracking is somewhat limited since these trackers are not equipped with mechanisms to cope with challenging cases like partial occlusion, deformation or scale changes. In this paper, we propose a deformable part-based correlation filter tracking approach which depends on coupled interactions between a global filter and several part filters. Specifically, local filters provide an initial estimate, which is then used by the global filter as a reference to determine the final result. Then, the global filter provides a feedback to the part filters regarding their updates and the related deformation parameters. In this way, our proposed collaborative model handles not only partial occlusion but also scale changes. Experiments on two large public benchmark datasets demonstrate that our approach gives significantly better results compared with the state-of-the-art trackers.     

Learning discriminative update adaptive spatial-temporal regularized correlation filter for RGB-T tracking

The RGB-T trackers based on correlation filter framework have been extensively investigated for that they can track targets more accurately in most complex scenes. However, the performance of these trackers is limited when facing some specific challenging scenarios, such as occlusion and background clutter. For different tracking targets, most of these trackers utilize fixed regularization constraint to build the filter model, which is obviously unreasonable to effectively present the appearance changes and characteristics of a specific target. In addition, they adopt a simple model update mechanism based on linear interpolation, which can easily lead to model degradation in challenging scenarios, resulting in tracker drift. To solve the above problems, we propose a novel adaptive spatial-temporal regularized correlation filter model to learn an appropriate regularization for achieving robust tracking and a relative peak discriminative method for model updating to avoid the model degradation. Besides, to make better integrate the unique advantages of the two modes and adapt the changing appearance of the target, an adaptive weighting ensemble scheme and a multi-scale search mechanism are adopted, respectively. To optimize the proposed model, we designed an efficient ADMM algorithm, which greatly improved the efficiency. Extensive experiments have been carried out on two available datasets, RGBT234 and RGBT210, and the experimental results indicate that the tracker proposed by us performs favorably in both accuracy and robustness against the state-of-the-art RGB-T trackers.     

ACSiam: Asymmetric convolution structures for visual tracking with Siamese network

Object trackers based on Siamese network usually transform the tracking task into a matching problem between the candidate samples and the target template. However, with the increasing depth and width of backbone networks, researches on Siamese trackers using backbone networks are not very advanced. Therefore, it is necessary for us to further investigate the characteristics of backbone network. As a fact, the ability of backbone network to extract features can directly determine the performance of object tracker. Given this, in this paper, we first propose an asymmetric convolutional network to improve the representational capability of backbone network. And then, the strip convolution is employed to enhance the operational capability of square kernel convolution in the backbone network. Besides, we also construct a novel module named Feature Dropblock (i.e., FD) to simulate the occlusion of hidden space, which goal is to improve the performance of backbone network in the target tracking under occlusion. To demonstrate the effectiveness of the proposed tracker, extensive ablation studies are conducted. Better results are obtained on the tracking benchmarks OTB100 and VOT2018, compared to other state-of-the-art trackers.     

SiamMBFAN: Siamese tracker with multi-branch feature aggregation network

Siamese trackers have attracted considerable attention in the field of object tracking because of their high precision and speed. However, one of the main disadvantages of Siamese trackers is that their feature extraction network is relatively single. They often use AlexNet or ResNet50 as the backbone network. AlexNet is shallow and thus cannot easily extract abundant semantic information, whereas ResNet50 has many convolutional layers, reducing the real-time performance of Siamese trackers. We propose a multi-branch feature aggregation network with different designs in the shallow and deep convolutional layers. We use the residual module to build the shallow convolutional layers to extract textural and edge features. The deep convolution layers, designed with two independent branches, are built with residual and parallel modules to extract different semantic features. The proposed network has a depth of only nine modules, and thus it is a simple and effective network. We then apply the network to a Siamese tracker to form SiamMBFAN. We design multi-layer classification and regression subnetworks in the Siamese tracker by aggregating the last three modules of the two branches, improving the localization ability of the tracker. Our tracker achieves a better balance between performance and speed. Finally, SiamMBFAN is tested on four challenging benchmarks, including OTB100, VOT2016, VOT2018, and UAV123. Compared with other trackers, our tracker improves by 7% (OTB100).     

基于稀疏稠密结构表示与在线鲁棒字典学习的视觉跟踪

L1跟踪对适度的遮挡具有鲁棒性,但是存在速度慢和易产生模型漂移的不足。为了解决上述两个问题,该文首先提出一种基于稀疏稠密结构的鲁棒表示模型。该模型对目标模板系数和小模板系数分别进行L2范数和L1范数正则化增强了对离群模板的鲁棒性。为了提高目标跟踪速度,基于块坐标优化原理,用岭回归和软阈值操作建立了该模型的快速算法。其次,为降低模型漂移的发生,该文提出一种在线鲁棒的字典学习算法用于模板更新。在粒子滤波框架下,用该表示模型和字典学习算法实现了鲁棒快速的跟踪方法。在多个具有挑战性的图像序列上的实验结果表明:与现有跟踪方法相比,所提跟踪方法具有较优的跟踪性能。     

End-to-end DeepNCC framework for robust visual tracking

In this paper, we propose an NCC-based object tracking deep framework, which can be well initialized with the limited target samples in the first frame. The proposed framework contains a pretrained model, online feature fine-tuning layers and tracking processes. The pretrained model provides rich feature representations while online feature fine-tuning layers select discriminative and generic features for the tracked object. We choose normalized cross-correlation as a template tracking layer to perform the tracking process. To enable the learned features representation closely coordinated to the tracked target, we jointly train the feature representation network and tracking processes. In online tracking, an adaptive template and a fixed template are fused to find the optimal tracking results. Scale estimation and a high-confidence model update scheme are perfectly integrated into the framework to adapt to the target appearance changes. The extensive experiments demonstrate that the proposed tracker achieves superior performance compared with other state-of-the-art trackers.     

Tracking by dynamic template: Dual update mechanism

Recently, Siamese trackers have received widespread attention for visual object tracking owing to their good balance between speed and performance. Those Siamese trackers heavily depend on target template while conventional practice fixes the template to initial frame. This strategy makes it unable to cope with variation of target appearance, which often leads to tracking failures and causes the gap in performance from other tracking methods. Despite the performance gain achieved by few template update methods with target templates generated by the tracked results, these tracked templates are easy to accumulate errors and cause tracking drift. In this paper, we propose two template update mechanisms to effectively adapt the target template during the tracking process which is dubbed as DTDU (Dynamic Template with Dual Update). Unlike predecessors that directly use the tracked template, we use initial template to perform similar transformation to the tracked template. Then the similar transformed template and the tracked template are combined linearly to capture the variation of target appearance. These updated templates are stored in a memory bank and retrieved to generate the final target template. In order to enhance quick update of memory bank to accommodate the target appearance, we use the retrieved template to further update the templates in memory bank for subsequent tracking. Extensive experiments on OTB-2015, VOT2016, VOT2018 and GOT-10k datasets have proved the effectiveness of these two update mechanisms and the proposed tracker achieves a real-time speed of 44 fps.     

L2范数正则化鲁棒编码视觉跟踪

针对基于稀疏表示的视觉跟踪计算效率低和易于产生模型漂移的不足,该文提出一种基于L2范数正则化鲁棒编码的视觉跟踪方法。该方法利用L2范数正则化鲁棒编码求解候选目标的编码系数,以粒子滤波为框架,利用候选目标的加权重建误差建立似然模型跟踪目标。为了适应目标的变化并克服模型漂移问题,利用L2范数正则化鲁棒编码估计当前目标的加权矩阵用于遮挡检测,根据遮挡检测结果实现模型更新。对提出的跟踪方法进行实验的结果表明:与现有跟踪方法相比,该方法具有较优的跟踪性能。     

Robust tracking based on local structural cell graph

Structure information has been increasingly incorporated into computer vision, however most trackers have ignored the inner spatial structure of the object. In this paper, we develop a simple yet robust tracking algorithm based on local structural cell graph (LSCG). This approach exploits both partial and spatial information of the target via representing the object with local structural cells (LSCs) and constructing a graph to model the spatial structure between the inner parts of the object. The tracking is formulated as matching LSCG, whose nodes are target parts and edges are the interaction between two parts. Within the Bayesian framework, we achieve object tracking by matching graphs between the reference and candidates. Eventually, the candidate with the highest similarity is the target. In addition, an updating strategy is adopted to help our tracker adapt to the fast time-varying object appearance. Experimental results demonstrate that the proposed method outperforms several state-of-the-art trackers.     

基于特征融合的RGBT双模态孪生跟踪网络

热红外成像技术被广泛地应用于军事、遥感和安防等领域中的目标跟踪,但热红外图像对对比度较低、目标模糊等跟踪场景效果一般。因此,将热红外图像与可见光图像进行融合提高跟踪性能具有重要意义。与基于可见光或热红外图像的单模态跟踪算法相比,基于可见光/热红外(RGB/Thermal, RGBT)图像的双模态跟踪算法对光照变化、云雾遮挡具有更强的鲁棒性。提出了一种基于特征融合的RGBT双模态孪生跟踪网络架构。该网络将双模态图像中提取的深度特征进行融合,提高目标外观特征的判别力。该网络可以利用训练数据进行端到端的离线训练。公开数据集RGBT234上的实验结果表明,所提出的RGBT双模态孪生特征融合跟踪网络能够实现复杂场景下鲁棒持续的目标跟踪。     

基于相关滤波器的视觉目标跟踪方法

为了解决视觉目标跟踪中的尺度预测的难题,本 文在核相关滤波器的目标跟踪的框架下给出了一种 尺度估计策略,并对传统的核相关跟踪方法中目标模型的在线更新方法进行了修改,提出了一 种多尺度视觉 目标跟踪算法。首先,通过对正则化最小二乘分类器(RLS)学习获得位置和尺度核相关滤波 器(KCF);然后,寻找位 置和尺度KCF输出响应的最大值,完成目标位置和尺度的检测;最后,在线更新目 标模型。实验中,对 12组具有挑战性的标准视频序列进行测试。实验结果表明,相对于现 有的3种基于相关滤波器的跟 踪方法中的最优者,本文方法的平均中心位置误差(CLE)减 少了7.0pixels,平均成功率(SR)提 高了18.3%,平均距离(DP)精度提高了 5.6%;在目标发生尺度、光照、姿态变化、部 分遮挡、旋转及快速运 动等复杂情况下,本文方法均有较强的适应性,具有重要的理论和应用研究价值。