结合连续卷积算子的自适应加权目标跟踪算法

目的 在视觉跟踪领域中,特征的高效表达是鲁棒跟踪的关键,观察到在相关滤波跟踪中,不同卷积层表达了目标的不同方面特征,提出了一种结合连续卷积算子的自适应加权目标跟踪算法。方法 针对目标定位不准确的问题,提出连续卷积算子方法,将离散的位置估计转换成连续位置估计,使得位置定位更加准确;利用不同卷积层的特征表达,提高跟踪效果。首先利用深度卷积网络结构提取多层卷积特征,通过计算相关卷积响应大小,决定在下一帧特征融合时各层特征所占的权重,凸显优势特征,然后使用从不同层训练得到的相关滤波器与提取得到的特征进行相关运算,得到最终的响应图,响应图中最大值所在的位置便是目标所在的位置和尺度。结果 与目前较流行的3种目标跟踪算法在目标跟踪基准数据库（OTB-2013）中的50组视频序列进行测试,本文算法平均跟踪成功率达到85.4%。结论 本文算法在光照变化、尺度变化、背景杂波、目标旋转、遮挡和复杂环境下的跟踪具有较高的鲁棒性。     

A grid-based Bayesian approach to robust visual tracking

Visual tracking encompasses a wide range of applications in surveillance, medicine and the military arena. There are however roadblocks that hinder exploiting the full capacity of the tracking technology. Depending on specific applications, these roadblocks may include computational complexity, accuracy and robustness of the tracking algorithms. In the paper, we present a grid-based algorithm for tracking that drastically outperforms the existing algorithms in terms of computational efficiency, accuracy and robustness. Furthermore, by judiciously incorporating feature representation, sample generation and sample weighting, the grid-based approach accommodates contrast change, jitter, target deformation and occlusion. Tracking performance of the proposed grid-based algorithm is compared with two recent algorithms, the gradient vector flow snake tracker and the Monte Carlo tracker, in the context of leukocyte (white blood cell) tracking and UAV-based tracking. This comparison indicates that the proposed tracking algorithm is approximately 100 times faster, and at the same time, is significantly more accurate and more robust, thus enabling real-time robust tracking.     

基于改进的MOSSE相关滤波的目标跟踪

视频目标跟踪存在如遮挡、形变、漂移等诸多挑战。虽然研究者提出了大量的算法来解决这一问题, 但大多数不具备普适性和实时性。为了实现目标有效稳定的实时跟踪, 本文在MOSSE相关滤波算法基础上提出了一种多模式的视觉目标跟踪算法, 该算法不仅具有相关算法的实时性, 还适应跟踪目标外观大幅度变化情况。同时, 为了适应跟踪过程中目标外形的复杂变化, 提出了一个控制模式更新率的算法, 利用具有多模式的跟踪算法可以同时处理极小的运动与外形突变。对基准视频数据的仿真实验结果表明, 与对应的单模型跟踪算法相比, 本文提出的算法可以明显改善跟踪精度和稳定性。     

面向无人机的视觉目标跟踪算法：综述与展望

近年来,无人机因其小巧灵活、智能自主等特点被广泛应用于民用和军事等领域中,特别是搜索侦察过程中首要的目标跟踪任务。无人机视觉目标跟踪场景的复杂性和运动目标的多变性,使得目标特征提取及模型建立困难,对目标跟踪性能带来巨大的挑战。本文首先介绍了无人机视觉目标跟踪的研究现状,梳理了经典和最新的目标跟踪算法,特别是基于相关滤波的跟踪算法和基于深度学习的跟踪算法,并对比了不同算法的优缺点。其次,归纳了常用的目标跟踪数据集和性能评价指标。最后,展望了无人机视觉目标跟踪算法的未来发展趋势。     

Hardware/software co-design of a real-time kernel based tracking system

The probabilistic visual tracking methods using color histograms have been proven to be robust to target model variations and background illumination changes as shown by the recent research. However, the required computational cost is high due to intensive image data processing. The embedded solution of such algorithms become challenging due to high computational power demand and algorithm complexity. This paper presents a hardware/software co-design architecture for implementation of the well-known kernel based mean shift tracking algorithm. The design uses color histogram of the target as tracking feature. The target is searched in the consecutive images by maximizing the statistical match of the color distributions. The target localization is based on gradient based iterative search instead of exhaustive search which makes the system capable of achieving frame rate up to hundreds of frames per second while tracking multiple targets. The design, which is fully standalone, is implemented on a low-cost medium-size field programmable gate array (FPGA) device. The hardware cost of the design is compared with some other tracking systems. The performance of the system in terms of speed is evaluated and compared with the software based implementation. It is expected that the proposed solution will find its utility in applications like embedded automatic video surveillance systems.     

全局搜索和多实例判别特征的长时跟踪方法

长时目标跟踪相对于短时目标跟踪仍然是一个巨大的挑战. 然而现有的长时跟踪算法通常在面对目标频繁出现消失、目标外观发生剧变等挑战中表现不佳. 本文提出了一种基于局部搜索模块和全局搜索跟踪模块的全新、鲁棒且实时的长时跟踪框架. 局部搜索模块利用TransT短时跟踪器生成一系列候选框, 并通过置信度评分确定最佳候选框. 针对全局重新检测开发了一个新颖的全局搜索跟踪模块, 以Faster R-CNN为基础模型, 在RPN阶段与R-CNN阶段引入非局部操作和多级实例特征融合模块, 以充分挖掘目标实例级特征. 为了改进全局搜索跟踪模块的性能, 设计了双模板更新策略来提升跟踪器的鲁棒能力. 通过使用不同时间点上更新的模板能够更好地适应目标的变化. 根据局部或全局置信度分数判断目标是否存在, 并在下一帧中选择局部或全局搜索跟踪策略. 同时能够为局部搜索模块估计目标的位置和大小. 此外还为全局搜索跟踪器引入了排名损失函数, 隐式学习了区域提议与原始查询目标的相似度. 通过在多个跟踪数据集上进行大量实验对提出的跟踪框架进行了广泛评估. 结果一致表明, 本文提出的跟踪框架实现了令人满意的性能.     

Collaborative signal processing for target tracking in distributed wireless sensor networks

Target tracking, especially visual target tracking, in complex situations is challenging, which is always performed in single-view system. Because of the conflict between resolution and tracking range, however, single-view tracking is not robust and accurate. This paper presents a distributed multi-view tracking system using collaborative signal processing (CSP) in distributed wireless sensor networks (DWSNs). In the proposed tracking system, target detection and classification algorithms are based on single-node processing and target tracking is performed in sink node, whereas target localization algorithm is carried out by CSP between multisensor. For conquering the disadvantages of client/server based centralized data fusion, a progressive distributed data fusion are proposed. Finally, an indoor target tracking experiment is illustrated, and then tracking performance, execution time and energy consumption of progressive distributed data fusion are compared with client/server based centralized data fusion. Experimental results demonstrate that the CSP based distributed multi-view tracking system in DWSNs can accomplish multi-target extraction, classification, localization, tracking and association quickly and accurately with little congestion, energy consumption and execution time.     

孪生网络跟踪算法并行计算结构研究

基于嵌入式平台的复杂背景目标跟踪技术在智能视频监控设备、无人机跟踪等领域有重要作用.卷积神经网络在跟踪问题上有准确率高、鲁棒性强的优点,但基于卷积特征的算法计算复杂度高,受嵌入式平台面积和功耗的限制,实时性难以满足嵌入式平台应用场景的需求.针对基于卷积特征的跟踪算法计算复杂度高、存储参数量大的难题,率先提出一种利用FPGA实现基于卷积神经网络的复杂背景目标跟踪硬件加速架构.该方法通过利用KL相对熵对目标跟踪算法Siamese-FC进行定点量化,设计了基于通道并行的卷积层加速架构.实验结果表明,定点量化后跟踪算法相比于原算法的平均精度损失不超过4.57％,FPGA部署后前向推理耗时仅为CPU的16.15％,功耗仅为CPU的13.7％.     

增强二阶网络调制的目标跟踪

目的 表观模型对视觉目标跟踪的性能起着决定性的作用。基于网络调制的跟踪算法通过构建高效的子网络学习参考帧目标的表观信息,以用于测试帧目标的鲁棒匹配,在多个目标跟踪数据集上表现优异。但是,这类跟踪算法忽视了高阶信息对鲁棒建模物体表观的重要作用,致使在物体表观发生大尺度变化时易产生跟踪漂移。为此本文提出全局上下文信息增强的二阶池化调制子网络,以学习高阶特征提升跟踪器的性能。方法 首先,利用卷积神经网络（convolutional neural networks,CNN）提取参考帧和测试帧的特征;然后,对提取的特征采用不同方向的长短时记忆网络（long shot-term memory networks,LSTM）捕获每个像素的全局上下文信息,再经过二阶池化网络提取高阶信息;最后,通过调制机制引导测试帧学习最优交并比预测。同时,为提升跟踪器的稳定性,在线跟踪通过指数加权平均自适应更新物体表观特征。结果 实验结果表明,在OTB100（object tracking benchmark）数据集上,本文方法的成功率为67.9%,超越跟踪器ATOM （accurate tracking by overlap maximization）1.5%;在VOT （visual object tracking）2018数据集上平均期望重叠率（expected average overlap,EAO）为0.44,超越ATOM 4%。结论 本文通过构建全局上下文信息增强的二阶池化调制子网络来学习高效的表观模型,使跟踪器达到目前领先的性能。     

A Pyramid Approach to Motion Tracking

This paper presents a multiresolution approach to visual motion tracking. In the approach, the foveation mechanism of the human visual system is used to model the multiresolution information perception algorithms of a Transputer-based pyramid visual tracking system. The video images of a moving target are transformed into pyramidal data structures, each of those images consists of multiple image layers with different resolutions by a Gaussian pyramid generation algorithm. The tracking of a moving target over an image sequence is accomplished by performing a foveal search that is based on an iterative intensity pattern correlation along the multiple resolution levels of the Gaussian pyramids of two successive images. Analyses are given as to the efficiency and accuracy of our tracking algorithm, showing that the algorithm is over 160 times faster than conventional mono-resolution tracking methods, with the tracking error within one pixel. To demonstrate the superiority of the multiresolution tracking algorithm in the connection to parallel computation, a scheme for mapping the tracking algorithm into a Transputer-based pyramidal parallel computing structure is proposed in the paper. Experimental results demonstrate good performance of the proposed approach.     

Robust feature learning for online discriminative tracking without large-scale pre-training

Owing to the inherent lack of training data in visual tracking, recent work in deep learning-based trackers has focused on learning a generic representation offline from large-scale training data and transferring the pre-trained feature representation to a tracking task. Offline pre-training is time-consuming, and the learned generic representation may be either less discriminative for tracking specific objects or overfitted to typical tracking datasets. In this paper, we propose an online discriminative tracking method based on robust feature learning without large-scale pre-training. Specifically, we first design a PCA filter bank-based convolutional neural network (CNN) architecture to learn robust features online with a few positive and negative samples in the high-dimensional feature space. Then, we use a simple soft-thresholding method to produce sparse features that are more robust to target appearance variations. Moreover, we increase the reliability of our tracker using edge information generated from edge box proposals during the process of visual tracking. Finally, effective visual tracking results are achieved by systematically combining the tracking information and edge box-based scores in a particle filtering framework. Extensive results on the widely used online tracking benchmark (OTB-50) with 50 videos validate the robustness and effectiveness of the proposed tracker without large-scale pre-training.     

改进协方差矩阵的智能车视觉目标跟踪方法

智能车辆视觉目标具有非线性、噪声分布非高斯性的典型特点，现有算法难以实时估计目标的状态。针对识别物体复杂且多变，很难用完全的特征来描述待识别目标及其背景的不断变化，提出了一种用于融合颜色特征及SURF（Speed-Up Robust Features）特征的协方差矩阵来改进粒子滤波算法，从而提升视觉目标跟踪的实时性，满足智能车辆的要求。首先，对采集的图像进行预处理来获取感兴趣区域。接着，通过融合颜色特征及SURF特征构造范围感兴趣区域（Region Of Interest，ROI）的目标特征协方差矩阵，建立目标状态预测模型及状态观测模型，用于改进粒子滤波算法粒子重采样过程，实现对目标的精确跟踪。最后，将该方法与Mean-shift算法和颜色属性（CN）算法进行对比。实验结果表明，在智能车视觉跟踪过程中对光环境瞬时变化、目标物体存在短时遮挡以及目标物体姿态改变时，该算法在满足智能车辆对实时性要求的前提下，有效提升算法的精确度及鲁棒性。     

多特征分层融合的相关滤波鲁棒跟踪

目的 为提高目标跟踪的鲁棒性,针对相关滤波跟踪中的多特征融合问题,提出了一种多特征分层融合的相关滤波鲁棒跟踪算法。方法 采用多通道相关滤波跟踪算法进行目标跟踪时,从目标和周围背景区域分别提取HOG（histogram of oriented gradient）、CN（color names）和颜色直方图3种特征。提出的分层融合算法首先采用自适应加权融合策略进行HOG和CN特征的特征响应图融合,通过计算特征响应图的平滑约束性和峰值旁瓣比两个指标得到融合权重。将该层融合结果与基于颜色直方图特征获得的特征响应图进行第2层融合时,采用固定系数融合策略进行特征响应图的融合。最后基于融合后的响应图估计目标的位置,并采用尺度估计算法估计得到目标更准确的包围盒。结果 采用OTB-2013（object tracking benchmark 2013）和VOT-2014（visual object tracking 2014）公开测试集验证所提跟踪算法的性能,在对多特征分层融合参数进行分析的基础上,与5种主流基于相关滤波的目标跟踪算法进行了对比分析。实验结果表明,本文算法的目标跟踪精度有所提高,其跟踪精度典型值比Staple算法提高了5.9%（0.840 vs 0.781）,同时由于有效地融合了3种特征,在多种场景下目标跟踪的鲁棒性优于其他算法。结论 提出的多特征分层融合跟踪算法在保证跟踪准确率的前提下,跟踪鲁棒性优于其他算法。当相关滤波跟踪算法采用了多个不同类型特征时,本文提出的分层融合策略具有一定的借鉴性。     

Switching particle filters for efficient visual tracking

In this article, we propose a new particle filtering scheme, called a switching particle filter, which allows robust and accurate visual tracking under typical circumstances of real-time visual tracking. This scheme switches two complementary sampling algorithms, Condensation and Auxiliary Particle Filter, in an on-line fashion based on the confidence of the filtered state of the visual target. The accuracy and robustness of the switching scheme were evaluated using real visual tracking experiments as well as computer simulations. Furthermore, we demonstrate through visual tracking experiments that our scheme not only outperforms existing particle filters but also assists on-line learning of target dynamics.     

Environmental factors and features that influence visual search in a 3D WIMP interface

The challenge of moving past the classic Window Icons Menus Pointer (WIMP) interface, i.e. by turning it ‘3D’, has resulted in much research and development. To evaluate the impact of 3D on the ‘finding a target picture in a folder’ task, we built a 3D WIMP interface that allowed the systematic manipulation of visual depth, visual aides, semantic category distribution of targets versus non-targets; and the detailed measurement of lower-level stimuli features. Across two separate experiments, one large sample web-based experiment, to understand associations, and one controlled lab environment, using eye tracking to understand user focus, we investigated how visual depth, use of visual aides, use of semantic categories, and lower-level stimuli features (i.e. contrast, colour and luminance) impact how successfully participants are able to search for, and detect, the target image. Moreover in the lab-based experiment, we captured pupillometry measurements to allow consideration of the influence of increasing cognitive load as a result of either an increasing number of items on the screen, or due to the inclusion of visual depth.Our findings showed that increasing the visible layers of depth, and inclusion of converging lines, did not impact target detection times, errors, or failure rates. Low-level features, including colour, luminance, and number of edges, did correlate with differences in target detection times, errors, and failure rates. Our results also revealed that semantic sorting algorithms significantly decreased target detection times. Increased semantic contrasts between a target and its neighbours correlated with an increase in detection errors. Finally, pupillometric data did not provide evidence of any correlation between the number of visible layers of depth and pupil size, however, using structural equation modelling, we demonstrated that cognitive load does influence detection failure rates when there is luminance contrasts between the target and its surrounding neighbours. Results suggest that WIMP interaction designers should consider stimulus-driven factors, which were shown to influence the efficiency with which a target icon can be found in a 3D WIMP interface.     

多层特征融合和并行自注意力的孪生网络目标跟踪算法

目标跟踪是计算机视觉方向上的一项重要课题,其中尺度变化、形变和旋转是目前跟踪领域较难解决的问题。针对以上跟踪中所面临的具有挑战性的问题,基于已有的孪生网络算法提出多层特征融合和并行自注意力的孪生网络目标跟踪算法(MPSiamRPN)。首先,用修改后的ResNet50对模板图片和搜索图片进行特征提取,为处理网络过深而导致目标部分特征丢失,提出多层特征融合模块(multi-layer feature fusion module, MLFF)将ResNet后三层特征进行融合;其次,引入并行自注意力模块(parallel self-attention module, PSA),该模块由通道自注意力和空间自注意力组成,通道自注意力可以选择性地强调对跟踪有益的通道特征,空间自注意力能学习目标丰富的空间信息;最后,采用区域提议网络(regional proposal network, RPN)来完成分类和回归操作,从而确定目标的位置和形状。实验显示,提出的MPSiamRPN在OTB100、VOT2018两个测试数据集上取得了具有可竞争性的结果。     

视觉目标跟踪方法研究综述

目的 随着军事侦察任务设备的发展,红外与可见光侦察技术成为军事装备中的主要侦察手段。研究视觉目标跟踪技术对提高任务设备的全天候目标侦察、目标跟踪、目标定位等战场情报获取能力具有重要意义。目前,对视觉目标跟踪技术的研究越来越深入,目标跟踪的方法和种类也越来越丰富。本文对目前应用较为广泛的4种视觉目标跟踪方法进行研究综述,为后续国内外研究者对目标跟踪相关理论及发展研究工作提供基础。方法 通过对视觉目标跟踪技术难点问题进行分析,根据目标跟踪方法建模方式的不同,将视觉目标跟踪方法分为生成式模型方法与判别式模型方法。分别对生成式模型跟踪算法中的均值漂移目标跟踪方法和粒子滤波目标跟踪方法,判别式模型跟踪算法中的相关滤波目标跟踪方法和深度学习目标跟踪方法进行研究。首先分别对4种跟踪算法的基本原理进行介绍,然后针对4种跟踪算法基本原理的不足和对应目标跟踪中的难点问题进行分析,最后针对目标跟踪的难点问题,给出对应算法的主流改进方案。结果 针对视觉目标跟踪相关技术研究进展,结合无人机侦察任务需求,对跟踪算法实际应用中存在的重点解决问题与相关目标跟踪的难点问题进行分析,给出目前的解决方案与不足,探讨研究未来无人机目标侦察跟踪技术的发展方向。结论 视觉目标跟踪技术已经取得了显著的进展,在侦察任务中的应用越来越广泛。但目标跟踪技术仍然是非常具有挑战性的问题,目标跟踪中的相关理论有待进一步完善和改进,由于实际应用中的场景复杂,目标跟踪的难点问题的挑战性更大,因此容易导致跟踪效果不佳。针对不同的应用环境,结合具体不同军事装备的特点,研究相对精确和鲁棒并且满足实时性要求的视觉目标跟踪算法,对提升装备的全天候侦察目标信息获取能力具有重要意义。     

深度长时目标跟踪算法综述

在视觉目标跟踪领域,长时跟踪因存在更为复杂的遮挡、相似物干扰和目标消失等具有现实意义的挑战场景,而越来越被研究者所重视。传统长时跟踪算法存在精度低和效率低等问题,已经无法满足如视频监控和自动驾驶等领域对跟踪器性能的应用需求。目前,大量的研究工作通过引入深度神经网络快速推动了长时跟踪技术的发展。为了深入分析深度长时跟踪算法的现状与未来发展,通过对比长短时跟踪数据集及评价指标,初步界定了长时跟踪任务范畴,归纳了长时跟踪任务的需求和难点,并介绍了长时跟踪数据集及评价体系的发展。基于深度长时目标跟踪算法的设计框架,详细描述了框架各组成部分的设计思路。以长时跟踪策略为切入点深入分析了现有研究工作,归纳了不同模型的优缺点及特性。依据对现有研究工作的整理和总结,讨论了该领域面临的挑战,并对未来的发展方向进行了展望。     

足球视频球员感知跟踪算法

目的 足球比赛视频中的球员跟踪算法为足球赛事分析提供基础的数据支持。但足球比赛中球员跟踪存在极大的挑战：球员进攻、防守和争夺球权时,目标球员可能产生快速移动、严重遮挡和周围出现若干名干扰球员的情况,目前仍没有一种能够完美解决足球比赛中球员跟踪问题的算法。因此如何解决足球场景中的困难,提升球员跟踪的准确度,成为当前研究的热点问题。方法 本文在分析足球比赛视频中球员目标特点的基础上,通过融合干扰项感知的颜色模型和目标感知的深度模型,提出并设计了一种球员感知的跟踪算法。干扰项感知的颜色模型分别提取目标、背景和干扰项的颜色直方图,利用贝叶斯公式得到搜索区域中每个像素点属于目标的似然概率。目标感知的深度模型利用孪生网络计算搜索区域与目标的相似度。针对跟踪漂移问题,使用全局跟踪器和局部跟踪器分别跟踪目标整体和目标上半身,并且在两个跟踪器的跟踪结果出现较大差异的时候分析跟踪器有效性并进行定位修正。结果 在公共的足球数据集上将本文算法与10个其他跟踪算法进行对比实验,同时对于文本算法进行了局部跟踪器的消融实验。实验结果表明,球员感知跟踪算法的平均有效重叠率达到了0.560 3,在存在同队球员和异队球员干扰的情况下,本文算法比排名第2的算法的有效重叠率分别高出3.7%和6.6%,明显优于其他算法,但是由于引入了干扰项感知的颜色模型、目标感知的深度模型以及局部跟踪器等模块增加了算法的时间复杂度,导致本文算法跟踪速度较慢。结论 本文总结了跟踪算法的整体流程并分析了实验结果,认为干扰项感知、目标感知和局部跟踪这3个策略在足球场景中的球员跟踪问题中起到了重要的作用,为未来在足球球员跟踪领域研究的继续深入提供了参考依据。