Siamese visual tracking with enriched semantics and dynamic template

Siamese tracking methods have recently drawn extensive attention due to their balanced accuracy and efficiency. However, most Siamese-based trackers use shallow backbone network, in which extracting high-level semantic features is difficult. When the appearance of distractors and targets is particularly similar, these methods may lead to tracking drift or even failure. Considering this deficiency, we propose a Siamese network with enriched semantics, named ESDT. First, a semantic enrichment module (SEM) comprising dilated convolution layers is designed to improve the classification capability of the siamese tracker. In addition, the target template is updated adaptively to cope with the target texture information changes caused by illumination and blur and further promote the tracking performance. Finally, exhaustive experimental analysis on the public datasets shows that the proposed algorithm outperforms several state-of-the-art algorithms and could track the target stably despite disturbances.     

基于增强RPN的孪生网络目标跟踪算法

目前孪生网络跟踪器已经具有比较良好的表现,但是对于卷积神经网络所提取的特征仍没有较好地利用其特点,同时孪生网络通过相似性学习进行跟踪的特性使跟踪器的准确性和鲁棒性存在不足。提出了一种金字塔式特征融合的方法,根据骨干网络特征提取层不同深度具有不同侧重的特点提高网络对目标的表征能力,然后使用注意力机制对区域推荐网络（Region Proposal Network,RPN）进行增强,最终实现更精准更鲁棒的跟踪。在OTB100数据集的实验中,新提出的SiamERPN(Siamese Enhanced RPN)算法分别得到了0.668的成功率和0.876的精度,测试结果好于基线算法和其他对比算法。     

结合全局光流的孪生区域提名网络目标跟踪算法

基于孪生网络的跟踪器受限于孪生网络跟踪框架固有的跟踪机制和搜索区域选择机制,当目标处在被遮挡、快速运动和出视野等困难场景下时,如何稳定、鲁棒地进行目标跟踪始终是孪生网络跟踪器亟需解决的问题。为此,文章提出一种结合光流的孪生区域提名网络目标跟踪算法(GOF-SiamRPN)。通过全局光流对目标的运动趋势信息进行补充,该方法可以有效地解决在这些困难场景下的跟踪问题。在VOT2019和UAV123上的实验结果表明,相比基准方法,该算法分别取得了2.0%和1.8%的性能提升。与其他先进的跟踪器相比,该算法也取得了有竞争力的跟踪效果。     

SiamMBFAN: Siamese tracker with multi-branch feature aggregation network

Siamese trackers have attracted considerable attention in the field of object tracking because of their high precision and speed. However, one of the main disadvantages of Siamese trackers is that their feature extraction network is relatively single. They often use AlexNet or ResNet50 as the backbone network. AlexNet is shallow and thus cannot easily extract abundant semantic information, whereas ResNet50 has many convolutional layers, reducing the real-time performance of Siamese trackers. We propose a multi-branch feature aggregation network with different designs in the shallow and deep convolutional layers. We use the residual module to build the shallow convolutional layers to extract textural and edge features. The deep convolution layers, designed with two independent branches, are built with residual and parallel modules to extract different semantic features. The proposed network has a depth of only nine modules, and thus it is a simple and effective network. We then apply the network to a Siamese tracker to form SiamMBFAN. We design multi-layer classification and regression subnetworks in the Siamese tracker by aggregating the last three modules of the two branches, improving the localization ability of the tracker. Our tracker achieves a better balance between performance and speed. Finally, SiamMBFAN is tested on four challenging benchmarks, including OTB100, VOT2016, VOT2018, and UAV123. Compared with other trackers, our tracker improves by 7% (OTB100).     

基于KL散度与通道选择的热红外目标跟踪算法

为了解决单一跟踪器无法有效应对复杂背景及目标外观的显著变化,对于热红外目标跟踪准确度不高的问题,基于全卷积孪生网络提出了一种多响应图集成的跟踪算法用于热红外跟踪。首先,使用预训练的卷积神经网络来提取热红外目标的多个卷积层的特征并进行通道选择,在此基础上分别构建3个对应的跟踪器,每个跟踪器独立执行跟踪并返回一个响应图。然后,利用Kullback–Leibler（KL）散度对多个响应图进行优化集成,得到一个更强的响应图。最后利用集成后的响应图来确定目标位置。为了评估所提算法的性能,在当前最全面的热红外跟踪基准LSOTB-TIR（Large-Scale Thermal Infrared Object Tracking Benchmark）上进行了实验。实验结果表明,所提算法能够适应复杂多样的红外跟踪场景,综合性能超过了现有的红外跟踪算法。     

基于非对称卷积的孪生网络视觉跟踪算法

针对孪生网络对旋转变化目标特征表达能力不足的问题,该文提出了基于非对称卷积的孪生网络跟踪算法。首先利用卷积核的可加性构建非对称卷积核组,可以将其应用于任意卷积核大小的已有网络结构。接着在孪生网络跟踪框架下,对AlexNet的卷积模块进行替换,并在训练和跟踪阶段对网络进行分别设计。最后在网络的末端并联地添加3个非对称卷积核,分别经过相关运算后得到3个响应图,进行加权融合后选取最大值即为目标的位置。实验结果表明,相比于SiamFC,在OTB2015数据集上精度提高了8.7%,成功率提高了4.5%。     

融合通道互联空间注意力的Siamese网络跟踪算法

基于Siamese网络的跟踪算法在跟踪精度和速度方面展现出巨大的潜力,然而要使离线训练的模型适应在线跟踪仍然面临着挑战。为了提升复杂场景下算法的特征提取以及判别能力,提出了一种融合通道-互联-空间注意力的Siamese网络实时跟踪算法。首先构建以深度卷积网络VGG-Net-16作为主干网络的Siamese跟踪框架,增加特征提取能力;接着设计通道-互联-空间注意力模块,增强模型的适应能力与判别能力;然后加权融合多层响应图,获取更精准的跟踪结果;最后使用大规模数据集对网络进行端到端的训练,在通用数据集OTB-2015上进行跟踪测试。实验结果表明:与当前主流算法相比,所提算法具有较强的稳健性,能更好地适应目标外观变化、相似物干扰、目标遮挡等复杂场景,在NVIDIA RTX 2060 GPU上,跟踪速度平均达到37FPS,满足实时性要求。     

Learning complementary Siamese networks for real-time high-performance visual tracking

Recently, Siamese based methods have made a breakthrough in the visual tracking field. However, the existing trackers still cannot take full advantage of the deep features. In this work, we improve the performances of Siamese trackers by complementary learning with different types of matching features. Specifically, a Matching Activation Network (MAN) is firstly designed to highlight the matching regions of the search image given a template. Since only sparse parts of feature maps contribute to the matching result, an important design choice is to emphasize the weak-matching features by erasing the strong-matching ones and learn complementary classifiers from both types of features. Then we propose a novel complementary region proposal network (CoRPN) to take complementary features as inputs and their outputs complement to each other, which are fused to improve the performance. Experiments show that our proposed tracker achieves leading performances on five tracking datasets while retaining real-time speed.     

基于关键特征信息感知和在线自适应掩模的孪生网络目标跟踪

近年来,孪生网络在视觉目标跟踪的应用给跟踪器性能带来了极大的提升,可以同时兼顾准确率和实时性。然而,孪生网络跟踪器的准确率在很大程度上受到限制。为了解决上述问题,该文基于通道注意力机制,创新地提出了关键特征信息感知模块来增强网络模型的判别能力,使网络聚焦于目标的卷积特征变化;在此基础上,该文还提出了一种在线自适应掩模策略,根据在线学习到的互相关层输出状态,自适应掩模后续帧,以此来突出前景目标。在OTB100, GOT-10k数据集上进行实验验证,所提跟踪器在不影响实时性的前提下,准确率相较于基准有了显著提升,并且在遮挡、尺度变化以及背景杂乱等复杂场景下具有鲁棒的跟踪效果。     

DSiamMFT: An RGB-T fusion tracking method via dynamic Siamese networks using multi-layer feature fusion

The task of object tracking is very important since its various applications. However, most object tracking methods are based on visible images, which may fail when visible images are unreliable, for example when the illumination conditions are poor. To address this issue, in this paper a fusion tracking method which combines information from RGB and thermal infrared images (RGB-T) is presented based on the fact that infrared images reveal thermal radiation of objects thus providing complementary features. Particularly, a fusion tracking method based on dynamic Siamese networks with multi-layer fusion, termed as DSiamMFT, is proposed. Visible and infrared images are firstly processed by two dynamic Siamese Networks, namely visible and infrared network, respectively. Then, multi-layer feature fusion is performed to adaptively integrate multi-level deep features between visible and infrared networks. Response maps produced from different fused layer features are then combined through an elementwise fusion approach to produce the final response map, based on which the target can be located. Extensive experiments on large datasets with various challenging scenarios have been conducted. The results demonstrate that the proposed method shows very competitive performance against the-state-of-art RGB-T trackers. The proposed approach also improves tracking performance significantly compared to methods based on images of single modality.     

Tracking by dynamic template: Dual update mechanism

Recently, Siamese trackers have received widespread attention for visual object tracking owing to their good balance between speed and performance. Those Siamese trackers heavily depend on target template while conventional practice fixes the template to initial frame. This strategy makes it unable to cope with variation of target appearance, which often leads to tracking failures and causes the gap in performance from other tracking methods. Despite the performance gain achieved by few template update methods with target templates generated by the tracked results, these tracked templates are easy to accumulate errors and cause tracking drift. In this paper, we propose two template update mechanisms to effectively adapt the target template during the tracking process which is dubbed as DTDU (Dynamic Template with Dual Update). Unlike predecessors that directly use the tracked template, we use initial template to perform similar transformation to the tracked template. Then the similar transformed template and the tracked template are combined linearly to capture the variation of target appearance. These updated templates are stored in a memory bank and retrieved to generate the final target template. In order to enhance quick update of memory bank to accommodate the target appearance, we use the retrieved template to further update the templates in memory bank for subsequent tracking. Extensive experiments on OTB-2015, VOT2016, VOT2018 and GOT-10k datasets have proved the effectiveness of these two update mechanisms and the proposed tracker achieves a real-time speed of 44 fps.     

基于孪生神经网络的两阶段目标跟踪方法

深度学习技术使目标跟踪的精度和鲁棒性得到了很大提高,基于孪生网络的跟踪方法通过在大规模数据集上进行训练,使模型能应对目标的各种形变,缺点是无法排除相似目标的干扰。为此,提出了一种基于孪生网络的两阶段目标跟踪方法。首先,采用修改后的残差网络提取性能更优的深度特征。区域建议网络通过相关滤波调制自适应更新模板,结合时域信息过滤掉易区分的负样本;然后,通过感兴趣池化层提取候选区域固定尺度的特征,并馈送到验证网络进行更精细的分类与回归。为了提升网络对高难度样本的区分能力,采用正负样本对联合训练的方式提高特征匹配的性能。在OTB100、VOT标准测试集和UAV123无人机航拍数据集上进行了评测,实验结果表明:所提方法能明显改进基准算法的性能。     

Visual tracking based on depth cross-correlation and feature alignment

Journal of Signal Processing Systems - Visual tracking technology based on the Siamese network have enabled excellent performance on many tracking datasets. However, these trackers cannot provide...     

Siamese visual tracking with multilayer feature fusion and corner distance IoU loss

The tracker based on the Siamese network regards tracking tasks as solving a similarity problem between the target template and search area. Using shallow networks and offline training, these trackers perform well in simple scenarios. However, due to the lack of semantic information, they have difficulty meeting the accuracy requirements of the task when faced with complex backgrounds and other challenging scenarios. In response to this problem, we propose a new model, which uses the improved ResNet-22 network to extract deep features with more semantic information. Multilayer feature fusion is used to obtain a high-quality score map to reduce the influence of interference factors in the complex background on the tracker. In addition, we propose a more powerful Corner Distance IoU (intersection over union) loss function so that the algorithm can better regression to the bounding box. In the experiments, the tracker was extensively evaluated on the object tracking benchmark data sets, OTB2013 and OTB2015, and the visual object tracking data sets, VOT2016 and VOT2017, and achieved competitive performance, proving the effectiveness of this method.     

基于双模板Siamese网络的鲁棒视觉跟踪算法

近年来,Siamese网络由于其良好的跟踪精度和较快的跟踪速度,在视觉跟踪领域引起极大关注,但大多数Siamese网络并未考虑模型更新,从而引起跟踪错误。针对这一不足,该文提出一种基于双模板Siamese网络的视觉跟踪算法。首先,保留响应图中响应值稳定的初始帧作为基准模板R,同时使用改进的APCEs模型更新策略确定动态模板T。然后,通过对候选目标区域与2个模板匹配度结果的综合分析,对结果响应图进行融合,以得到更加准确的跟踪结果。在OTB2013和OTB2015数据集上的实验结果表明,与当前5种主流跟踪算法相比,该文算法的跟踪精度和成功率具有明显优势,不仅在尺度变化、平面内旋转、平面外旋转、遮挡、光照变化情况下具有较好的跟踪效果,而且达到了46 帧/s的跟踪速度。     

Edge-aware object pixel-level representation tracking

Recently, there has been a trend in tracking to use more refined segmentation mask instead of coarse bounding box to represent the target object. Some trackers proposed segmentation branches based on the tracking framework and maintain real-time speed. However, those trackers use a simple FCNs structure and lack of the edge information modeling. This makes performance quite unsatisfactory. In this paper, we propose an edge-aware segmentation network, which uses the complementarity between target information and edge information to provide a more refined representation of the target. Firstly, We use the high-level features of the tracking backbone network and the correlation features of the classification branch of the tracking framework to fuse, and use the target edge and target segmentation mask for simultaneous supervision to obtain an optimized high-level feature with rough edge information and target information. Secondly, we use the optimized high-level features to guide the low-level features of the tracking backbone network to generate more refined edge features. Finally, we use the refined edge features to fuse with the target features of each layer to generate the final mask. Our approach has achieved leading performance on recent pixel-wise object tracking benchmark VOT2020 and segmentation datasets DAVIS2016 and DAVIS2017 while running on 47 fps. Code is available at https://github.com/TJUMMG/EATtracker.     

Attention meets involution in visual tracking

In visual tracking, both convolution and attention are widely employed for feature enhancement and fusion. However, convolution does not adequately model global dependencies of samples due to its operation on local neighbors, while attention gives too much attention to global dependencies and too little to local dependencies. It is intrinsically infeasible to combine both methods to integrate global and local information. However, a recently-proposed model called involution uses kernels differing in spatial extent but sharing across channels, making it possible to take advantage of both convolution and attention. We propose an attention-involution (Att-Inv) model that uses an attention mechanism to generate involution kernels to take both global and local dependencies of samples into account. To improve the performance of our tracker, we develop and implement strategies of backbone network modification, template updates, and regression of bounding box distributions. We evaluate our tracker using benchmarks such as GOT10k, LaSOT, TrackingNet and OxUvA. Experimental results show that it is competitive with state-of-the-art trackers.     

TRBACF: Learning temporal regularized correlation filters for high performance online visual object tracking

Correlation filter-based trackers (CFTs) have recently shown remarkable performance in the field of visual object tracking. The advantage of these trackers originates from their ability to convert time-domain calculations into frequency domain calculations. However, a significant problem of these CFTs is that the model is insufficiently robust when the tracking scenarios are too complicated, meaning that the ideal tracking performance cannot be acquired. Recent work has attempted to resolve this problem by reducing the boundary effects from modeling the foreground and background of the object target effectively (e.g., CFLB, BACF, and CACF). Although these methods have demonstrated reasonable performance, they are often affected by occlusion, deformation, scale variation, and other challenging scenes. In this study, considering the relationship between the current frame and the previous frame of a moving object target in a time series, we propose a temporal regularization strategy to improve the BACF tracker (denoted as TRBACF), a typical representative of the aforementioned trackers. The TRBACF tracker can efficiently adjust the model to adapt the change of the tracking scenes, thereby enhancing its robustness and accuracy. Moreover, the objective function of our TRBACF tracker can be solved by an improved alternating direction method of multipliers, which can speed up the calculation in the Fourier domain. Extensive experimental results demonstrate that the proposed TRBACF tracker achieves competitive tracking performance compared with state-of-the-art trackers.     

引入全局上下文特征模块的DenseNet孪生网络目标跟踪

近年来,采用孪生网络提取深度特征的方法由于其较好的跟踪精度和速度,成为目标跟踪领域的研究热点之一,但传统的孪生网络并未提取目标较深层特征来保持泛化性能,并且大多数孪生网络只提取局部领域特征,这使得模型对于外观变化是非鲁棒和局部的。针对此,该文提出一种引入全局上下文特征模块的DenseNet孪生网络目标跟踪算法。该文创新性地将DenseNet网络作为孪生网络骨干,采用一种新的密集型特征重用连接网络设计方案,在构建更深层网络的同时减少了层之间的参数量,提高了算法的性能,此外,为应对目标跟踪过程中的外观变化,该文将全局上下文特征模块(GC-Model)嵌入孪生网络分支,提升算法跟踪精度。在VOT2017和OTB50数据集上的实验结果表明,与当前较为主流的算法相比,该文算法在跟踪精度和鲁棒性上有明显优势,在尺度变化、低分辨率、遮挡等情况下具有良好的跟踪效果,且达到实时跟踪要求。     

一种基于拓扑约束的多核跟踪算法

针对视觉跟踪中常见的目标部分遮挡和尺度变化问题,提出了一种基于拓扑约束的多核跟踪算法.首先,提取满足空间分布的位于目标与背景所在边界的Harris角点作为多核跟踪器的中心,然后,采用拓扑约束对多个跟踪结果进行优化,选取跟踪性能好的核跟踪器,构造仿射变换模型,进而得到最终跟踪结果和目标尺度变化信息.实验结果表明,本文算法能对目标进行准确跟踪,并能有效地处理目标的部分遮挡和尺度变化问题.