利用时空特征编码的单目标跟踪网络

目的 随着深度神经网络的出现,视觉跟踪快速发展,视觉跟踪任务中的视频时空特性,尤其是时序外观一致性(temporal appearance consistency)具有巨大探索空间。本文提出一种新颖简单实用的跟踪算法——时间感知网络(temporal-aware network, TAN),从视频角度出发,对序列的时间特征和空间特征同时编码。方法 TAN内部嵌入了一个新的时间聚合模块(temporal aggregation module, TAM)用来交换和融合多个历史帧的信息,无需任何模型更新策略也能适应目标的外观变化,如形变、旋转等。为了构建简单实用的跟踪算法框架,设计了一种目标估计策略,通过检测目标的4个角点,由对角构成两组候选框,结合目标框选择策略确定最终目标位置,能够有效应对遮挡等困难。通过离线训练,在没有任何模型更新的情况下,本文提出的跟踪器TAN通过完全前向推理(fully feed-forward)实现跟踪。结果 在OTB(online object tracking:a benchmark)50、OTB100、TrackingNet、LaSOT(a high-qua...     

Online parameter tuning for object tracking algorithms

Object tracking quality usually depends on video scene conditions (e.g. illumination, density of objects, object occlusion level). In order to overcome this limitation, this article presents a new control approach to adapt the object tracking process to the scene condition variations. More precisely, this approach learns how to tune the tracker parameters to cope with the tracking context variations. The tracking context, or context, of a video sequence is defined as a set of six features: density of mobile objects, their occlusion level, their contrast with regard to the surrounding background, their contrast variance, their 2D area and their 2D area variance. In an offline phase, training video sequences are classified by clustering their contextual features. Each context cluster is then associated to satisfactory tracking parameters. In the online control phase, once a context change is detected, the tracking parameters are tuned using the learned values. The approach has been experimented with three different tracking algorithms and on long, complex video datasets. This article brings two significant contributions: (1) a classification method of video sequences to learn offline tracking parameters and (2) a new method to tune online tracking parameters using tracking context.     

Robust object tracking via online dynamic spatial bias appearance models

This paper presents a robust object tracking method via a spatial bias appearance model learned dynamically in video. Motivated by the attention shifting among local regions of a human vision system during object tracking, we propose to partition an object into regions with different confidences and track the object using a dynamic spatial bias appearance model (DSBAM) estimated from region confidences. The confidence of a region is estimated to re ect the discriminative power of the region in a feature space, and the probability of occlusion. We propose a novel hierarchical Monte Carlo (HAMC) algorithm to learn region confidences dynamically in every frame. The algorithm consists of two levels of Monte Carlo processes implemented using two particle filtering procedures at each level and can efficiently extract high confidence regions through video frames by exploiting the temporal consistency of region confidences. A dynamic spatial bias map is then generated from the high confidence regions, and is employed to adapt the appearance model of the object and to guide a tracking algorithm in searching for correspondences in adjacent frames of video images. We demonstrate feasibility of the proposed method in video surveillance applications. The proposed method can be combined with many other existing tracking systems to enhance the robustness of these systems.     

A 2D/3D model-based object tracking framework

This paper presents a robust framework for tracking complex objects in video sequences. Multiple hypothesis tracking (MHT) algorithm reported in (IEEE Trans. Pattern Anal. Mach. Intell. 18(2) (1996)) is modified to accommodate a high level representations (2D edge map, 3D models) of objects for tracking. The framework exploits the advantages of MHT algorithm which is capable of resolving data association/uncertainty and integrates it with object matching techniques to provide a robust behavior while tracking complex objects. To track objects in 2D, a 4D feature is used to represent edge/line segments and are tracked using MHT. In many practical applications 3D models provide more information about the object's pose (i.e., rotation information in the transformation space) which cannot be recovered using 2D edge information. Hence, a 3D model-based object tracking algorithm is also presented. A probabilistic Hausdorff image matching algorithm is incorporated into the framework in order to determine the geometric transformation that best maps the model features onto their corresponding ones in the image plane. 3D model of the object is used to constrain the tracker to operate in a consistent manner. Experimental results on real and synthetic image sequences are presented to demonstrate the efficacy of the proposed framework.     

Robust object tracking under appearance change conditions

We propose a robust visual tracking framework based on particle filter to deal with the object appearance changes due to varying illumination, pose variantions, and occlusions. We mainly improve the observation model and re-sampling process in a particle filter. We use on-line updating appearance model, affine transformation, and M-estimation to construct an adaptive observation model. On-line updating appearance model can adapt to the changes of illumination partially. Affine transformation-based similarity measurement is introduced to tackle pose variantions, and M-estimation is used to handle the occluded object in computing observation likelihood. To take advantage of the most recent observation and produce a suboptimal Gaussian proposal distribution, we incorporate Kalman filter into a particle filter to enhance the performance of the resampling process. To estimate the posterior probability density properly with lower computational complexity, we only employ a single Kalman filter to propagate Gaussian distribution. Experimental results have demonstrated the effectiveness and robustness of the proposed algorithm by tracking visual objects in the recorded video sequences.     

Real-time abandoned and stolen object detection based on spatio-temporal features in crowded scenes

Abandoned and stolen object detection is a challenging task due to occlusion, changes in lighting, large perspective distortion, and the similarity in appearance of different people. This paper presents real-time detection methods of abandoned and stolen objects in a complex video. The adaptive background modeling method is applied to stable tracking and the ghost image removing. To detect abandoned and stolen objects, the methods determine spatio-temporal relationship between moving people and suspicious drops. The space first detection method measures the distance between a moving object and a non-moving object in spatial change analysis. The time first detection method conducts temporal change analysis and then spatial change analysis. The potential abandoned object is classified as a definite abandoned or stolen object by two-level detection approach. The time-to-live timer is applied by adjusting several key parameters on each camera and environment. In experiments, we show the experimental results to evaluate our proposed methods using benchmark datasets.     

Multi-object tracking via MHT with multiple information fusion in surveillance video

Tracking multiple objects is critical to automatic video content analysis and virtual reality. The major problem is how to solve data association problem when ambiguous measurements are caused by objects in close proximity. To tackle this problem, we propose a multiple information fusion-based multiple hypotheses tracking algorithm integrated with appearance feature, local motion pattern feature and repulsion–inertia model for multi-object tracking. Appearance model based on HSV–local binary patterns histogram and local motion pattern based on optical flow are adopted to describe objects. A likelihood calculation framework is proposed to incorporate the similarities of appearance, dynamic process and local motion pattern. To consider the changes in appearance and motion pattern over time, we make use of an effective template updating strategy for each object. In addition, a repulsion–inertia model is adopted to explore more useful information from ambiguous detections. Experimental results show that the proposed approach generates better trajectories with less missing objects and identity switches.     

无表观特征小目标检测与跟踪

检测跟踪模糊的小目标是计算机视觉领域中难度极大,富有挑战的任务。由于被跟踪的目标过小或过于模糊,难以提取合适的可用于检测和跟踪的表观特征,使得现有的目标检测和跟踪算法不能解决上述问题。前景运动物体区别于背景随机噪声的一个重要特征是运动物体具有一定的运动规律,基于这个假设提出一种新方法,根据物体的运动规律对其进行跟踪。首先,提出利用运动物体的时空域关联性,对视频中的运动目标进行分割和去噪;其次,提出了利用动态规划得出并优化物体的运动轨迹。各种条件下的实验结果表明了上述方法的精确性和鲁棒性。     

Fuzzy spatial constraints and ranked partitioned sampling approach for multiple object tracking

While particle filters are now widely used for object tracking in videos, the case of multiple object tracking still raises a number of issues. Among them, a first, and very important, problem concerns the exponential increase of the number of particles with the number of objects to be tracked, that can make some practical applications intractable. To achieve good tracking performances, we propose to use a Partitioned Sampling method in the estimation process with an additional feature about the ordering sequence in which the objects are processed. We call it Ranked Partitioned Sampling, where the optimal order in which objects should be processed and tracked is estimated jointly with the object state. Another essential point concerns the modeling of possible interactions between objects. As another contribution, we propose to represent these interactions within a formal framework relying on fuzzy sets theory. This allows us to easily model spatial constraints between objects, in a general and formal way. The association of these two contributions was tested on typical videos exhibiting difficult situations such as partial or total occlusions, and appearance or disappearance of objects. We show the benefit of using conjointly these two contributions, in comparison to classical approaches, through multiple object tracking and articulated object tracking experiments on real video sequences. The results show that our approach provides less tracking errors than those obtained with the classical Partitioned Sampling method, without the need for increasing the number of particles.     

Pearson-based mixture model for color object tracking

To track objects in video sequences, many studies have been done to characterize the target with respect to its color distribution. Most often, the Gaussian mixture model (GMM) is used to represent the object color density. In this paper, we propose to extend the normality assumption to more general families of distributions issued from the Pearson’s system. Precisely, we propose a method called Pearson mixture model (PMM), used in conjunction with Gaussian copula, which is dynamically updated to adapt itself to the appearance change of the object during the sequence. This model is combined with Kalman filtering to predict the position of the object in the next frame. Experimental results on gray-level and color video sequences show tracking improvements compared to classical GMM. Especially, the PMM seems robust to illumination variations, pose and scale changes, and also to partial occlusions, but its computing time is higher than the computing time of GMM.     

基于大数据分析的移动对象轨迹预测方法

对移动对象的轨迹预测将在移动目标跟踪识别中具有较好的应用价值。移动对象轨迹预测的基础是移动目标运动参量的采集和估计,移动目标的运动参量信息特征规模较大,传统的单分量时间序列分析方法难以实现准确的参量估计和轨迹预测。提出一种基于大数据多传感信息融合跟踪的移动对象轨迹预测算法。首先进行移动目标对象进行轨迹跟踪的控制对象描述和约束参量分析,对轨迹预测的大规模运动参量信息进行信息融合和自正整定性控制,通过大数据分析方法实现对移动对象运动参量的准确估计和检测,由此指导移动对象轨迹的准确预测,提高预测精度。仿真结果表明,采用该算法进行移动对象的运动参量估计和轨迹预测的精度较高,自适应性能较强,稳健性较好,相关的指标性能优于传统方法。     

Mining Appearance Models Directly From Compressed Video

In this paper, we propose an approach for learning appearance models of moving objects directly from compressed video. The appearance of a moving object changes dynamically in video due to varying object poses, lighting conditions, and partial occlusions. Efficiently mining the appearance models of objects is a crucial and challenging technology to support content-based video coding, clustering, indexing, and retrieval at the object level. The proposed approach learns the appearance models of moving objects in the spatial-temporal dimension of video data by taking advantage of the MPEG video compression format. It detects a moving object and recovers the trajectory of each macroblock covered by the object using the motion vector present in the compressed stream. The appearances are then reconstructed in the DCT domain along the object's trajectory, and modeled as a mixture of Gaussians (MoG) using DCT coefficients. We prove that, under certain assumptions, the MoG model learned from the DCT domain can achieve pixel-level accuracy when transformed back to the spatial domain, and has a better band-selectivity compared to the MoG model learned in the spatial domain. We finally cluster the MoG models to merge the appearance models of the same object together for object-level content analysis.     

A real-time object detecting and tracking system for outdoor night surveillance

Autonomous video surveillance and monitoring has a rich history. Many deployed systems are able to reliably track human motion in indoor and controlled outdoor environments. However, object detection and tracking at night remain very important problems for visual surveillance. The objects are often distant, small and their signatures have low contrast against the background. Traditional methods based on the analysis of the difference between successive frames and a background frame will do not work. In this paper, a novel real time object detection algorithm is proposed for night-time visual surveillance. The algorithm is based on contrast analysis. In the first stage, the contrast in local change over time is used to detect potential moving objects. Then motion prediction and spatial nearest neighbor data association are used to suppress false alarms. Experiments on real scenes show that the algorithm is effective for night-time object detection and tracking.     

Online multi-view subspace learning via group structure analysis for visual object tracking

In this paper, we focus on incrementally learning a robust multi-view subspace representation for visual object tracking. During the tracking process, due to the dynamic background variation and target appearance changing, it is challenging to learn an informative feature representation of tracking object, distinguished from the dynamic background. To this end, we propose a novel online multi-view subspace learning algorithm (OMEL) via group structure analysis, which consistently learns a low-dimensional representation shared across views with time changing. In particular, both group sparsity and group interval constraints are incorporated to preserve the group structure in the low-dimensional subspace, and our subspace learning model will be incrementally updated to prevent repetitive computation of previous data. We extensively evaluate our proposed OMEL on multiple benchmark video tracking sequences, by comparing with six related tracking algorithms. Experimental results show that OMEL is robust and effective to learn dynamic subspace representation for online object tracking problems. Moreover, several evaluation tests are additionally conducted to validate the efficacy of group structure assumption.     

A study on various methods used for video summarization and moving object detection for video surveillance applications

With the advancement in digital video technology, video surveillance has been playing its vital role for ensuring safety and security. The surveillance systems are deployed in wide range of applications to invigilate stuffs and to analyse the activities in the environment. From the single or multi surveillance camera, a huge amount of data is generated, stored and processed for security purpose. Due to time constraints, it is a very tedious process for an analyst to go through the full content. This limitation has been overcome by the use of video summarization. The video summarization is intended to afford comprehensible analysis of video by removing duplications and extracting key frames from the video. To make an easily interpreted outline, the various available video summarization methods will try to shot the summary of the main occurrences, scenes, or objects in a frame. Depending on the applications, it is required to summarize the happenings in the scene and detect the objects (static/dynamic) which is recorded in the video. Hence this paper provides the various methods used for video summarization and a comparative study of different techniques. It also presents different object detection, object classification and object tracking algorithms available in the literature.     

自适应紧致特征的超像素目标跟踪

目的 针对现有的超像素目标跟踪算法（RST）对同一类中分别属于目标和相似干扰物体的超像素块赋予相同特征置信度,导致难以区分目标和相似干扰物的问题,为此提出自适应紧致特征的超像素目标跟踪算法（ACFST）。方法 该方法在每帧的目标搜索区域内构建适合目标大小的自适应紧致搜索区域,并将该区域内外的特征置信度分别保持不变和降低。处于背景中的相似干扰物体会被该方法划分到紧致搜索区域外,其特征置信度被降低。当依据贝叶斯推理框架求出对应最大后验概率的目标时,紧致搜索区域外的特征置信度低,干扰物体归属目标的程度也低,不会被误判为目标。结果 在具有与目标相似干扰物体的两个视频集进行测试,本文ACFST跟踪算法与RST跟踪算法相比,平均中心误差分别缩减到5.4像素和7.5像素,成功率均提高了11%,精确率分别提高了10.6%和21.6%,使得跟踪结果更精确。结论 本文提出构建自适应紧致搜索区域,并通过设置自适应的参数控制紧致搜索区域变化,减少因干扰物体与目标之间相似而带来的误判。在具有相似物体干扰物的视频集上验证了本文算法的有效性,实验结果表明,本文算法在相似干扰物体靠近或与目标部分重叠时,能够保证算法精确地跟踪到目标,提高算法的跟踪精度,具有较强的鲁棒性,使得算法更能适应背景杂乱、目标遮挡、形变等复杂环境。     

深度学习的目标跟踪算法综述

目标跟踪是利用一个视频或图像序列的上下文信息,对目标的外观和运动信息进行建模,从而对目标运动状态进行预测并标定目标位置的一种技术,是计算机视觉的一个重要基础问题,具有重要的理论研究意义和应用价值,在智能视频监控系统、智能人机交互、智能交通和视觉导航系统等方面具有广泛应用。大数据时代的到来及深度学习方法的出现,为目标跟踪的研究提供了新的契机。本文首先阐述了目标跟踪的基本研究框架,从观测模型的角度对现有目标跟踪的历史进行回顾,指出深度学习为获得更为鲁棒的观测模型提供了可能;进而从深度判别模型、深度生成式模型等方面介绍了适用于目标跟踪的深度学习方法;从网络结构、功能划分和网络训练等几个角度对目前的深度目标跟踪方法进行分类并深入地阐述和分析了当前的深度目标跟踪方法;然后,补充介绍了其他一些深度目标跟踪方法,包括基于分类与回归融合的深度目标跟踪方法、基于强化学习的深度目标跟踪方法、基于集成学习的深度目标跟踪方法和基于元学习的深度目标跟踪方法等;之后,介绍了目前主要的适用于深度目标跟踪的数据库及其评测方法;接下来从移动端跟踪系统,基于检测与跟踪的系统等方面深入分析与总结了目标跟踪中的最新具体应用情况,最后对深度学习方法在目标跟踪中存在的训练数据不足、实时跟踪和长程跟踪等问题进行分析,并对未来的发展方向进行了展望。