具有尺度和旋转适应性的长时间目标跟踪

目标发生尺度和旋转变化会给长时间目标跟踪带来很大的挑战,针对该问题,本文提出了具有尺度和旋转适应性的鲁棒目标跟踪算法.首先针对跟踪过程中目标存在的尺度变化和旋转运动,提出一种基于傅里叶-梅林变换和核相关滤波的目标尺度和旋转参数估计方法.该方法能够实现连续空间的目标尺度和旋转参数估计,采用核相关滤波提高了估计的鲁棒性和准确性.然后针对长时间目标跟踪过程中,有时不可避免地会出现跟踪失败的情况（例如由于长时间半遮挡或全遮挡等）,提出一种基于直方图和方差加权的目标搜索方法.当目标丢失时,通过提出的搜索方法能够快速从图像中确定目标可能存在的区域,使得跟踪算法具有从失败中恢复的能力.本文还训练了两个核相关滤波器用于估计跟踪结果的置信度和目标平移,通过专门的核相关滤波器能够使得估计的跟踪结果置信度更加准确和鲁棒,置信度的估计结果可用于激活基于直方图和方差加权的目标搜索模块,并判断搜索窗口中是否包含目标.本文在目标跟踪标准数据集（Online object tracking benchmark,OTB）上对提出的算法和目前主流的目标跟踪算法进行对比实验,验证了本文提出算法的有效性和优越性.     

全局搜索和多实例判别特征的长时跟踪方法

长时目标跟踪相对于短时目标跟踪仍然是一个巨大的挑战. 然而现有的长时跟踪算法通常在面对目标频繁出现消失、目标外观发生剧变等挑战中表现不佳. 本文提出了一种基于局部搜索模块和全局搜索跟踪模块的全新、鲁棒且实时的长时跟踪框架. 局部搜索模块利用TransT短时跟踪器生成一系列候选框, 并通过置信度评分确定最佳候选框. 针对全局重新检测开发了一个新颖的全局搜索跟踪模块, 以Faster R-CNN为基础模型, 在RPN阶段与R-CNN阶段引入非局部操作和多级实例特征融合模块, 以充分挖掘目标实例级特征. 为了改进全局搜索跟踪模块的性能, 设计了双模板更新策略来提升跟踪器的鲁棒能力. 通过使用不同时间点上更新的模板能够更好地适应目标的变化. 根据局部或全局置信度分数判断目标是否存在, 并在下一帧中选择局部或全局搜索跟踪策略. 同时能够为局部搜索模块估计目标的位置和大小. 此外还为全局搜索跟踪器引入了排名损失函数, 隐式学习了区域提议与原始查询目标的相似度. 通过在多个跟踪数据集上进行大量实验对提出的跟踪框架进行了广泛评估. 结果一致表明, 本文提出的跟踪框架实现了令人满意的性能.     

增强二阶网络调制的目标跟踪

目的 表观模型对视觉目标跟踪的性能起着决定性的作用。基于网络调制的跟踪算法通过构建高效的子网络学习参考帧目标的表观信息,以用于测试帧目标的鲁棒匹配,在多个目标跟踪数据集上表现优异。但是,这类跟踪算法忽视了高阶信息对鲁棒建模物体表观的重要作用,致使在物体表观发生大尺度变化时易产生跟踪漂移。为此本文提出全局上下文信息增强的二阶池化调制子网络,以学习高阶特征提升跟踪器的性能。方法 首先,利用卷积神经网络（convolutional neural networks,CNN）提取参考帧和测试帧的特征;然后,对提取的特征采用不同方向的长短时记忆网络（long shot-term memory networks,LSTM）捕获每个像素的全局上下文信息,再经过二阶池化网络提取高阶信息;最后,通过调制机制引导测试帧学习最优交并比预测。同时,为提升跟踪器的稳定性,在线跟踪通过指数加权平均自适应更新物体表观特征。结果 实验结果表明,在OTB100（object tracking benchmark）数据集上,本文方法的成功率为67.9%,超越跟踪器ATOM （accurate tracking by overlap maximization）1.5%;在VOT （visual object tracking）2018数据集上平均期望重叠率（expected average overlap,EAO）为0.44,超越ATOM 4%。结论 本文通过构建全局上下文信息增强的二阶池化调制子网络来学习高效的表观模型,使跟踪器达到目前领先的性能。     

Online learning neural tracker

Object tracking is a fundamental computer vision problem and is required for many high-level tasks such as activity recognition, behavior analysis and surveillance. The main challenge in the object tracking problem is the dynamic change in object/background appearance, illumination, shape and occlusion. We present an online learning neural tracker (OLNT) to differentiate the object from the background and also adapt to changes in object/background dynamics. For target modeling and object tracking, a neural algorithm based on risk sensitive loss function is proposed to handle issues related to sample imbalance and dynamics of object. Region-based features like region-based color moments for larger mobile objects and color/texture features at pixel level for smaller mobile objects are used to discriminate the object from background. The proposed neural classifier automatically determines the number of neurons required to estimate the posterior probability map. In the online learning neural classifier, only one neuron parameter is updated per tracker to reduce the computational burden during online adaptation. The tracked object is represented using an estimated posterior probability map. The posterior probability map is used to adapt the bounding box to handle the scale change and improper initialization.For illustrating the advantage of the proposed OLNT under rapid illumination variation, change in appearance, scale/size change, and occlusion, we present results from benchmark video sequences. Finally, we also present the comparison with well-known trackers in the literature and highlight the advantage of the proposed tracker.     

Online adaptive feature weighting for spatiogram-bank tracking

In this paper, we introduce a new adaptive feature weighting framework for multi-modal tracking. Our proposed tracker can compactly and efficiently handle multiple sources of tracking data, such as colour, brightness, gradient orientation and thermal infrared, and adaptively weight the sources based on their reliability for tracking. The adaptive weight selection mechanism is inspired by the state-of-the-art Collins tracker, but instead of treating the tracked object as a bag of features, it takes advantage of the spatial information using a global object model. Additionally, our tracker incorporates scale into the weight selection process and is shown to outperform the Collins tracker in an extensive video evaluation.     

Mean shift tracker combined with online learning-based detector and Kalman filtering for real-time tracking

Color-based visual object tracking is one of the most commonly used tracking methods. Among many tracking methods, the mean shift tracker is used most often because it is simple to implement and consumes less computational time. However, mean shift trackers exhibit several limitations when used for long-term tracking. In challenging conditions that include occlusions, pose variations, scale changes, and illumination changes, the mean shift tracker does not work well. In this paper, an improved tracking algorithm based on a mean shift tracker is proposed to overcome the weaknesses of existing methods based on mean shift tracker. The main contributions of this paper are to integrate mean shift tracker with an online learning-based detector and to newly define the Kalman filter-based validation region for reducing computational burden of the detector. We combine the mean shift tracker with the online learning-based detector, and integrate the Kalman filter to develop a novel tracking algorithm. The proposed algorithm can reinitialize the target when it converges to a local minima and it can cope with scale changes, occlusions and appearance changes by using the online learning-based detector. It updates the target model for the tracker in order to ensure long-term tracking. Moreover, the validation region obtained by using the Kalman filter and the Mahalanobis distance is used in order to operate detector in real-time. Through a comparison against various mean shift tracker-based methods and other state-of-the-art methods on eight challenging video sequences, we demonstrate that the proposed algorithm is efficient and superior in terms of accuracy and speed. Hence, it is expected that the proposed method can be applied to various applications which need to detect and track an object in real-time.     

结合时空一致性的FairMOT跟踪算法优化

目的 视频多目标跟踪(multiple object tracking, MOT)是计算机视觉中的一项重要任务,现有研究分别针对目标检测和目标关联部分进行改进,均忽视了多目标跟踪中的不一致问题。不一致问题主要包括3方面,即目标检测框中心与身份特征中心不一致、帧间目标响应不一致以及训练测试过程中相似度度量方式不一致。为了解决上述不一致问题,本文提出一种基于时空一致性的多目标跟踪方法,以提升跟踪的准确度。方法 从空间、时间以及特征维度对上述不一致性进行修正。对于目标检测框中心与身份特征中心不一致,针对每个目标检测框中心到特征中心之间的空间差异,在偏移后的位置上提取目标的ReID(re-identification)特征;对帧间响应不一致,使用空间相关计算相邻帧之间的运动偏移信息,基于该偏移信息对前一帧的目标响应进行变换后得到帧间一致性响应信息,然后对目标响应进行增强;对训练和测试过程中的相似度度量不一致,提出特征正交损失函数,在训练时考虑目标两两之间的相似关系。结果 在3个数据集上与现有方法进行比较。在MOT17、MOT20和Hieve数据集中,MOTA(multiple object t...     

加权间隔结构化支持向量机目标跟踪算法

目的 目标在跟踪过程中,各种因素的干扰使得跟踪结果存在不确定性。因此,将跟踪过程中所提取样本的可靠性融入跟踪模型中,有助于克服低可靠性样本对跟踪算法的影响。为此,基于最近的结构化支持向量机（SSVM）跟踪算法,提出一种包含样本置信度的加权间隔结构化支持向量机跟踪模型（WMSSVM）,以增强SSVM跟踪算法性能。方法 首先,基于打分和位置重合率估计样本可靠性;其次,建立WMSSVM模型处理具有不同置信度的跟踪样本训练问题,并采用对偶坐标下降优化算法求解跟踪模型。结果 在包含100个视频的OTB100跟踪数据集上进行测试,提出的WMSSVM跟踪器与基准跟踪器Scale-DLSSVM相比,在精准度和成功率两个指标上分别提高了1%和2%。与最近的跟踪算法相比,提出的方法也表现出更好的性能。结论 本文首次将样本的可靠性融入结构化支持向量机跟踪模型,并提出一种基于加权间隔的结构化支持向量机跟踪模型及其优化求解方法,在包含100个视频序列的跟踪数据集上验证了提出方法的有效性,本文提出的算法能够适应复杂场景下的跟踪任务,并在背景混杂、目标形变、遮挡、运动模糊、目标出界、快速位移等类别的视频中表现出优异的性能。     

基于混合注意力的Transformer视觉目标跟踪算法

基于Transformer的视觉目标跟踪算法能够很好地捕获目标的全局信息,但是,在对目标特征的表述上还有进一步提升的空间.为了更好地提升对目标特征的表达能力,提出一种基于混合注意力的Transformer视觉目标跟踪算法.首先,引入混合注意力模块捕捉目标在空间和通道维度中的特征,实现对目标特征上下文依赖关系的建模;然后,通过多个不同空洞率的平行空洞卷积对特征图进行采样,以获得图像的多尺度特征,增强局部特征表达能力;最后,在Transformer编码器中加入所构建的卷积位置编码层,为跟踪器提供精确且长度自适应的位置编码,提升跟踪定位的精度.在OTB100、VOT2018和LaSOT等数据集上进行大量实验,实验结果表明,通过基于混合注意力的Transformer网络学习特征间的关系,能够更好地表示目标特征.与其他主流目标跟踪算法相比,所提出算法具有更好的跟踪性能,且能够达到26帧/s的实时跟踪速度.     

Robust feature learning for online discriminative tracking without large-scale pre-training

Owing to the inherent lack of training data in visual tracking, recent work in deep learning-based trackers has focused on learning a generic representation offline from large-scale training data and transferring the pre-trained feature representation to a tracking task. Offline pre-training is time-consuming, and the learned generic representation may be either less discriminative for tracking specific objects or overfitted to typical tracking datasets. In this paper, we propose an online discriminative tracking method based on robust feature learning without large-scale pre-training. Specifically, we first design a PCA filter bank-based convolutional neural network (CNN) architecture to learn robust features online with a few positive and negative samples in the high-dimensional feature space. Then, we use a simple soft-thresholding method to produce sparse features that are more robust to target appearance variations. Moreover, we increase the reliability of our tracker using edge information generated from edge box proposals during the process of visual tracking. Finally, effective visual tracking results are achieved by systematically combining the tracking information and edge box-based scores in a particle filtering framework. Extensive results on the widely used online tracking benchmark (OTB-50) with 50 videos validate the robustness and effectiveness of the proposed tracker without large-scale pre-training.     

Learning Structure Models with Context Information for Visual Tracking

Tracking objects that undergo abrupt appearance changes and heavy occlusions is a challenging problem which conventional tracking methods can barely handle.To address the problem, we propose an online structure learning algorithm that contains three layers: an object is represented by a mixture of online structure models (OSMs) which are learnt from block-based online random forest classifiers (BORFs).BORFs are able to handle occlusion problems since they model local appearances of the target.To further improve the tracking accuracy and reliability, the algorithm utilizes mixture relational models (MRMs) as multi-mode context information to integrate BORFs into OSMs.Furthermore, the mixture construction of OSMs can avoid over-fitting effectively and is more flexible to describe targets.Fusing BORFs with MRMs, OSMs capture the discriminative parts of the target, which guarantees the reliability and robustness of our tracker.In addition, OSMs incorporate with block occlusion reasoning to update our BORFs and MRMs, which can deal with appearance changes and drifting problems effectively.Experiments on challenging videos show that the proposed tracker performs better than several state-of-the-art algorithms.     

利用时空特征编码的单目标跟踪网络

目的 随着深度神经网络的出现,视觉跟踪快速发展,视觉跟踪任务中的视频时空特性,尤其是时序外观一致性(temporal appearance consistency)具有巨大探索空间。本文提出一种新颖简单实用的跟踪算法——时间感知网络(temporal-aware network, TAN),从视频角度出发,对序列的时间特征和空间特征同时编码。方法 TAN内部嵌入了一个新的时间聚合模块(temporal aggregation module, TAM)用来交换和融合多个历史帧的信息,无需任何模型更新策略也能适应目标的外观变化,如形变、旋转等。为了构建简单实用的跟踪算法框架,设计了一种目标估计策略,通过检测目标的4个角点,由对角构成两组候选框,结合目标框选择策略确定最终目标位置,能够有效应对遮挡等困难。通过离线训练,在没有任何模型更新的情况下,本文提出的跟踪器TAN通过完全前向推理(fully feed-forward)实现跟踪。结果 在OTB(online object tracking:a benchmark)50、OTB100、TrackingNet、LaSOT(a high-qua...     

A Competitive Neural Network for Multiple Object Tracking in Video Sequence Analysis

Tracking of moving objects in real situation is a challenging research issue, due to dynamic changes in objects or background appearance, illumination, shape and occlusions. In this paper, we deal with these difficulties by incorporating an adaptive feature weighting mechanism to the proposed growing competitive neural network for multiple objects tracking. The neural network takes advantage of the most relevant object features (information provided by the proposed adaptive feature weighting mechanism) in order to estimate the trajectories of the moving objects. The feature selection mechanism is based on a genetic algorithm, and the tracking algorithm is based on a growing competitive neural network where each unit is associated to each object in the scene. The proposed methods (object tracking and feature selection mechanism) are applied to detect the trajectories of moving vehicles in roads. Experimental results show the performance of the proposed system compared to the standard Kalman filter.     

Online adaptive radial basis function networks for robust object tracking

Visual tracking has been a challenging problem in computer vision over the decades. The applications of visual tracking are far-reaching, ranging from surveillance and monitoring to smart rooms. In this paper, we present a novel online adaptive object tracker based on fast learning radial basis function (RBF) networks. Pixel based color features are used for developing the target/object model. Here, two separate RBF networks are used, one of which is trained to maximize the classification accuracy of object pixels, while the other is trained for non-object pixels. The target is modeled using the posterior probability of object and non-object classes. Object localization is achieved by iteratively seeking the mode of the posterior probability of the pixels in each of the subsequent frames. An adaptive learning procedure is presented to update the object model in order to tackle object appearance and illumination changes. The superior performance of the proposed tracker is illustrated with many complex video sequences, as compared against the popular color-based mean-shift tracker. The proposed tracker is suitable for real-time object tracking due to its low computational complexity.     

基于帧内关系建模和自注意力融合的多目标跟踪方法

多目标跟踪在视频监控领域有重要的应用价值.随着卷积神经网络(convolutional neural networks,CNN),尤其是图神经网络(graph neural networks,GNN)的发展,多目标跟踪的研究现阶段取得了很大突破.其中,图神经网络由于引入目标-轨迹间的关系建模,显示出更稳定的跟踪性能.然而,已有的基于GNN的多目标跟踪方法都仅在连续两帧之间建立全局关系模型,忽视了帧内目标与周围其他目标的交互,没有考虑在帧内建立合适的局部关系模型.为了解决该问题,提出基于帧内关系建模和自注意力融合模型(INAF-GNN)的多目标跟踪方法.在帧内,INAF-GNN建立目标与邻居目标的关系图模型以获取局部跟踪特征;在帧间,INAF-GNN建立目标与轨迹关系图模型以获得全局跟踪特征,并利用注意力机制设计一个特征融合模块整合局部和全局跟踪特征.在MotChallenge行人标准数据集上进行大量的实验,与多个基于图神经网络的多目标跟踪方法相比较,结果显示,MOTA指标提高1.9%,IDF1指标提高3.6%.同时,在UA-DETRAC车辆数据集上的验证测试表明了所提出方法的有效性和泛化能力.     

一种基于背景模型的运动目标检测与跟踪算法

本文提出了一种静止摄像机条件下的运动目标检测与跟踪算法．它以一种改进的自适应混合 高斯模型为背景更新方法，用连通区检测算法分割出前景目标，以Kalman滤波为运动模型实 现对运动目标的连续跟踪．在目标跟踪时，该算法针对目标遮挡引起的各种可能情况进行了 分析，引入了对运动目标的可靠性度量，增强了目标跟踪的稳定性和可靠性．在对多个室外 视频序列的实验中，该算法显示了良好的性能，说明它对于各种外部因素的影响，如光照变 化、阴影、目标遮挡等，具有很强的适应能力．     

Visual tracking with multiple Hough detectors

We propose a visual tracking method using multiple Hough detectors to address the problem of long-term robust object tracking in unconstrained environments. The method constructs the detectors based on the feature selection by the mutual information. These detectors serve to learn the partial appearances of target and synchronously evaluate image locations via the voting based detection with the generalized Hough transform. According to the result of detections, the best detector is selected by the minimum entropy criterion and delivers the final hypotheses for target location. The feature selection allows our tracker to be able to obtain and use the most discriminative parts of target and thus more robust to its changes, e.g. occlusion and deformation. The detector selection can correct undesirable model updates and restore the tracker after tracking failure. Meanwhile, the Hough-based detection can reduce the amount of noise introduced during online self-training and thus effectively prevent the tracker from drifting. The method is evaluated on the CVPR2013 Visual Tracker Benchmark and the experimental results demonstrate our method outperforms other tracking algorithms in terms of both success rate and precision.     

多颜色直方图自适应组合Mean Shift跟踪

经典Mean Shift跟踪算法使用单一颜色直方图跟踪目标,导致其对目标外观的变化鲁棒性较差。为了解决该问题,提出一种多颜色直方图自适应组合Mean Shift跟踪算法。该算法利用多个视图的颜色核函数直方图的加权组合作为目标模型进行Mean Shift跟踪;为了适应目标外观的变化,利用目标区域对每一颜色直方图的概率图均值和方差的比值评价每一颜色直方图的可靠性,并自适应地计算其组合权值。实验结果表明,与现有Mean Shift跟踪算法相比,提出的跟踪算法对目标的外观变化具有更强的鲁棒性。     

Discriminative multi-task objects tracking with active feature selection and drift correction

In this paper, we propose a discriminative multi-task objects tracking method with active feature selection and drift correction. The developed method formulates object tracking in a particle filter framework as multi-Task discriminative tracking. As opposed to generative methods that handle particles separately, the proposed method learns the representation of all the particles jointly and the corresponding coefficients are similar. The tracking algorithm starts from the active feature selection scheme, which adaptively chooses suitable number of discriminative features from the tracked target and background in the dynamic environment. Based on the selected feature space, the discriminative dictionary is constructed and updated dynamically. Only a few of them are used to represent all the particles at each frame. In other words, all the particles share the same dictionary templates and their representations are obtained jointly by discriminative multi-task learning. The particle that has the highest similarity with the dictionary templates is selected as the next tracked target state. This jointly sparsity and discriminative learning can exploit the relationship between particles and improve tracking performance. To alleviate the visual drift problem encountered in object tracking, a two-stage particle filtering algorithm is proposed to complete drift correction and exploit both the ground truth information of the first frame and observations obtained online from the current frame. Experimental evaluations on challenging sequences demonstrate the effectiveness, accuracy and robustness of the proposed tracker in comparison with state-of-the-art algorithms.