基于特征描述子的视频对象分级分割和跟踪算法

不同的视频应用对视频对象的分割和跟踪的速度和精确度具有不同的要求。提出了一种视频对象的分级分割和跟踪框架,基于视频对象特征描述子算法可以实时地分割和跟踪视频对象,基于区域特征描述子算法对分割的视频对象进一步细化,提高空域准确性。该框架可以满足各种视频应用。     

Tracking of Multiple Targets Using Online Learning for Reference Model Adaptation

Recently, much work has been done in multiple object tracking on the one hand and on reference model adaptation for a single-object tracker on the other side. In this paper, we do both tracking of multiple objects (faces of people) in a meeting scenario and online learning to incrementally update the models of the tracked objects to account for appearance changes during tracking. Additionally, we automatically initialize and terminate tracking of individual objects based on low-level features, i.e., face color, face size, and object movement. Many methods unlike our approach assume that the target region has been initialized by hand in the first frame. For tracking, a particle filter is incorporated to propagate sample distributions over time. We discuss the close relationship between our implemented tracker based on particle filters and genetic algorithms. Numerous experiments on meeting data demonstrate the capabilities of our tracking approach. Additionally, we provide an empirical verification of the reference model learning during tracking of indoor and outdoor scenes which supports a more robust tracking. Therefore, we report the average of the standard deviation of the trajectories over numerous tracking runs depending on the learning rate.      

L2 vector median filters on arrays with reconfigurableoptical buses

In spite of their good filtering characteristics for vector-valued image processing, the usability of vector median filters is limited by their high computational complexity. Given an N × N image and a W × W window, the computational complexity of vector median filter is O(W⁴N²). In this paper, we design three fast and efficient parallel algorithms for vector median filtering based on the 2-norm (L₂) on the arrays with reconfigurable optical buses (AROB). For 1 ⩽ p ⩽ W ⩽ q ⩽ N, our algorithms run in O(W⁴ log W/p⁴), O(W²N²/p ⁴q² log W) and O(1) times using p⁴N² / log W, p⁴q² / log W, and W⁴N² log N processors, respectively. In the sense of the product of time and the number of processors used, the first two results are cost optimal and the last one is time optimal     

交互式分割视频运动对象的研究与实现

在 MPEG- 4视频编码标准中 ,为了实现基于视频内容的交互功能 ,视频序列的每一帧由视频对象面来表示 ,而生成视频对象面 ,需要对视频序列中运动对象进行有效分割 ,并跟踪运动对象随时间的变化 .在视频分割方法中 ,交互式分割视频对象能满足分割的效率与质量指标要求 ,因此提出了一种交互分割与自动跟踪相结合的方式来分割视频语义对象 ,即在初始分割时 ,依据用户的交互与形态学的分水线分割算法相结合提取视频对象轮廓 ,并用改进的轮廓跟踪方法有效提高视频对象轮廓的精度 ;对后续帧的跟踪 ,采用六参数仿射变换跟踪运动对象轮廓的变化 ,用平移估算的运动矢量作为初始值 ,计算六参数仿射变换的参数 .实验结果表明 ,该方法能有效地分割并跟踪视频运动对象     

An appearance-based approach for consistent labeling of humans and objects in video

In this paper, we present an approach for consistently labeling people and for detecting human–object interactions using mono-camera surveillance video. The approach is based on a robust appearance-based correlogram model combined with histogram information to model color distributions of people and objects in the scene. The models are dynamically built from non-stationary objects, which are the outputs of background subtraction, and are used to identify objects on a frame-by-frame basis. We are able to detect when people merge into groups and to segment them even during partial occlusion. We can also detect when a person deposits or removes an object. The models persist when a person or object leaves the scene and are used to identify them when they reappear. Experiments show that the models are able to accommodate perspective foreshortening that occurs with overhead camera angles, as well as partial occlusion. The results show that this is an effective approach that is able to provide important information to algorithms performing higher-level analysis, such as activity recognition, where human–object interactions play an important role.     

自动分割及跟踪视频运动对象的一种实现方法

随着MPEG-4压缩标准的制定,分割及跟踪视频运动对象的研究显得极其重要。在MPEG-4视频编码标准中,为了实现基于视频内容的交互功能,其视频序列的每一帧由视频对象面（VOP）来表示。为了生成视频对象面,需要对视频序列中的运动对象进行有效的分割;并跟踪运动对象随时间的变化,为此提出并实现了一种用于分割及跟踪视频运动对象的时空联合方法。该方法首先采用连续帧间差的4次统计量假设检验,确定运动对象的位置,自动地分离出运动区域与背景区域;在运动区域内,采用数学形态学的分水线算法来精确地提取运动对象的轮廓;最后,将提取到的运动对象作为模板,对后续的视频序列,用Hausdorff距离度量,来跟踪并提取后续帧中运动对象。实验结果表明,该方法能有效地分割和跟踪视频运动对象,且能有效减少计算复杂度,其调整参数也较少。     

部件级表观模型的目标跟踪方法

因受遮挡、运动模糊、剧烈形变等因素的影响,稳定且准确的目标跟踪是当前计算机视觉研究领域重要挑战之一.首先采用中层视觉线索的超像素描述目标/背景的部件,以部件颜色直方图作为其特征,并通过聚类部件库的特征集构建初始表观模型,部件表达的局部性和灵活性使该模型能够准确描述目标/背景;然后,利用贝叶斯滤波模型计算目标框的初始状态,并提出相似物体干扰的检测和处理算法以避免跟踪漂移,得到更健壮的结果;最后,为了减弱形变、遮挡、模糊对表观模型的影响以更好地保持目标特征,提出一种基于部件库的特征补集的在线表观模型更新算法,根据部件变化实时反映目标/背景的变化情况.在多个具有跟踪挑战的视频序列上的实验结果表明(共12个视频序列):与现有跟踪方法相比,该算法跟踪结果的中心误差更小,成功帧数更多,能够更准确并稳定、有效地跟踪目标物体.     

Fuzzy spatial constraints and ranked partitioned sampling approach for multiple object tracking

While particle filters are now widely used for object tracking in videos, the case of multiple object tracking still raises a number of issues. Among them, a first, and very important, problem concerns the exponential increase of the number of particles with the number of objects to be tracked, that can make some practical applications intractable. To achieve good tracking performances, we propose to use a Partitioned Sampling method in the estimation process with an additional feature about the ordering sequence in which the objects are processed. We call it Ranked Partitioned Sampling, where the optimal order in which objects should be processed and tracked is estimated jointly with the object state. Another essential point concerns the modeling of possible interactions between objects. As another contribution, we propose to represent these interactions within a formal framework relying on fuzzy sets theory. This allows us to easily model spatial constraints between objects, in a general and formal way. The association of these two contributions was tested on typical videos exhibiting difficult situations such as partial or total occlusions, and appearance or disappearance of objects. We show the benefit of using conjointly these two contributions, in comparison to classical approaches, through multiple object tracking and articulated object tracking experiments on real video sequences. The results show that our approach provides less tracking errors than those obtained with the classical Partitioned Sampling method, without the need for increasing the number of particles.     

Dynamic Template Tracking and Recognition

In this paper we address the problem of tracking non-rigid objects whose local appearance and motion changes as a function of time. This class of objects includes dynamic textures such as steam, fire, smoke, water, etc., as well as articulated objects such as humans performing various actions. We model the temporal evolution of the object’s appearance/motion using a linear dynamical system. We learn such models from sample videos and use them as dynamic templates for tracking objects in novel videos. We pose the problem of tracking a dynamic non-rigid object in the current frame as a maximum a-posteriori estimate of the location of the object and the latent state of the dynamical system, given the current image features and the best estimate of the state in the previous frame. The advantage of our approach is that we can specify a-priori the type of texture to be tracked in the scene by using previously trained models for the dynamics of these textures. Our framework naturally generalizes common tracking methods such as SSD and kernel-based tracking from static templates to dynamic templates. We test our algorithm on synthetic as well as real examples of dynamic textures and show that our simple dynamics-based trackers perform at par if not better than the state-of-the-art. Since our approach is general and applicable to any image feature, we also apply it to the problem of human action tracking and build action-specific optical flow trackers that perform better than the state-of-the-art when tracking a human performing a particular action. Finally, since our approach is generative, we can use a-priori trained trackers for different texture or action classes to simultaneously track and recognize the texture or action in the video.     

Mining Appearance Models Directly From Compressed Video

In this paper, we propose an approach for learning appearance models of moving objects directly from compressed video. The appearance of a moving object changes dynamically in video due to varying object poses, lighting conditions, and partial occlusions. Efficiently mining the appearance models of objects is a crucial and challenging technology to support content-based video coding, clustering, indexing, and retrieval at the object level. The proposed approach learns the appearance models of moving objects in the spatial-temporal dimension of video data by taking advantage of the MPEG video compression format. It detects a moving object and recovers the trajectory of each macroblock covered by the object using the motion vector present in the compressed stream. The appearances are then reconstructed in the DCT domain along the object's trajectory, and modeled as a mixture of Gaussians (MoG) using DCT coefficients. We prove that, under certain assumptions, the MoG model learned from the DCT domain can achieve pixel-level accuracy when transformed back to the spatial domain, and has a better band-selectivity compared to the MoG model learned in the spatial domain. We finally cluster the MoG models to merge the appearance models of the same object together for object-level content analysis.     

Real-Time Tracking of Single and Multiple Objects from Depth-Colour Imagery Using 3D Signed Distance Functions

We describe a novel probabilistic framework for real-time tracking of multiple objects from combined depth-colour imagery. Object shape is represented implicitly using 3D signed distance functions. Probabilistic generative models based on these functions are developed to account for the observed RGB-D imagery, and tracking is posed as a maximum a posteriori problem. We present first a method suited to tracking a single rigid 3D object, and then generalise this to multiple objects by combining distance functions into a shape union in the frame of the camera. This second model accounts for similarity and proximity between objects, and leads to robust real-time tracking without recourse to bolt-on or ad-hoc collision detection.     

基层层次光流的半自动时空视频分割技术

在新一代MPEG-4视频编码标准中，为了支持面向对象编码和实现基于内容的应用，视频的半自动分割成为关键技术之一，为此提出了一种基于层次光流的半自动时空视频分割算法。该算法由空域分割和时域分割组成。在空域分割中，提出的基于点的图形用户界面（PBGUI），在用户的协助下，能够精确地定义需要分割的视频对象（VO）。时域分割根据空域分割的结果采用层次光流算法对视频对象进行边界和整体跟踪。实验结果表明，利用该算法，能够精确地分割出视频对象。     

基于层次光流的半自动时空视频分割技术

在新一代 MPEG- 4视频编码标准中 ,为了支持面向对象编码和实现基于内容的应用 ,视频的半自动分割成为关键技术之一 ,为此提出了一种基于层次光流的半自动时空视频分割算法 .该算法由空域分割和时域分割组成 .在空域分割中 ,提出的基于点的图形用户界面 (PBGU I) ,在用户的协助下 ,能够精确地定义需要分割的视频对象 (VO) .时域分割根据空域分割的结果采用层次光流算法对视频对象进行边界和整体跟踪 .实验结果表明 ,利用该算法 ,能够较精确地分割出视频对象 .     

视频中遮挡情况下目标的跟踪*

提出了一种基于概率外观模型和Condensation的跟踪方法.该方法通过分析目标前景MBB重叠关系检测遮挡的产生和解除,在目标未遮挡时建立并更新目标的概率外观模型.当遮挡发生后,利用目标共面条件确定目标前后关系,通过合并目标模型计算系统观测似然度概率.算法统一在Condensation框架下进行有遮挡和无遮挡的跟踪.实验证明了该方法的有效性.     

基于多外观模型的自适应加权目标跟踪算法*

偏最小二乘(PLS)跟踪算法忽略特征间及外观模型间的差异,容易受到光照、遮挡等因素的影响,降低目标的跟踪精度.针对上述问题,文中提出基于多外观模型的自适应加权目标跟踪算法(AWMA).首先使用PLS对目标区域逐步建立多个外观模型.然后根据各外观模型中特征的重要性及目标的显著度建立自适应权重的综合模型,融合多个外观模型完成目标与样本的误差分析.最后使用粒子滤波实现目标跟踪.实验表明,文中算法能更有效地过滤噪声数据,提高目标跟踪的鲁棒性和时间性能.     

A color-based particle filter for multiple object tracking in an outdoor environment

Tracking multiple objects is more challenging than tracking a single object. Some problems arise in multiple-object tracking that do not exist in single-object tracking, such as object occlusion, the appearance of a new object and the disappearance of an existing object, updating the occluded object, etc. In this article, we present an approach to handling multiple-object tracking in the presence of occlusions, background clutter, and changing appearance. The occlusion is handled by considering the predicted trajectories of the objects based on a dynamic model and likelihood measures. We also propose target-model-update conditions, ensuring the proper tracking of multiple objects. The proposed method is implemented in a probabilistic framework such as a particle filter in conjunction with a color feature. The particle filter has proven very successful for nonlinear and non-Gaussian estimation problems. It approximates a posterior probability density of the state, such as the object’s position, by using samples or particles, where each state is denoted as the hypothetical state of the tracked object and its weight. The observation likelihood of the objects is modeled based on a color histogram. The sample weight is measured based on the Bhattacharya coefficient, which measures the similarity between each sample’s histogram and a specified target model. The algorithm can successfully track multiple objects in the presence of occlusion and noise. Experimental results show the effectiveness of our method in tracking multiple objects.     

利用时空特征编码的单目标跟踪网络

目的 随着深度神经网络的出现,视觉跟踪快速发展,视觉跟踪任务中的视频时空特性,尤其是时序外观一致性(temporal appearance consistency)具有巨大探索空间。本文提出一种新颖简单实用的跟踪算法——时间感知网络(temporal-aware network, TAN),从视频角度出发,对序列的时间特征和空间特征同时编码。方法 TAN内部嵌入了一个新的时间聚合模块(temporal aggregation module, TAM)用来交换和融合多个历史帧的信息,无需任何模型更新策略也能适应目标的外观变化,如形变、旋转等。为了构建简单实用的跟踪算法框架,设计了一种目标估计策略,通过检测目标的4个角点,由对角构成两组候选框,结合目标框选择策略确定最终目标位置,能够有效应对遮挡等困难。通过离线训练,在没有任何模型更新的情况下,本文提出的跟踪器TAN通过完全前向推理(fully feed-forward)实现跟踪。结果 在OTB(online object tracking:a benchmark)50、OTB100、TrackingNet、LaSOT(a high-qua...     

Tracking multiple people with recovery from partial and total occlusion

Robust tracking of multiple people in video sequences is a challenging task. In this paper, we present an algorithm for tracking faces of multiple people even in cases of total occlusion. Faces are detected first; then a model for each person is built. The models are handed over to the tracking module which is based on the mean shift algorithm, where each face is represented by the non-parametric distribution of the colors in the face region. The mean shift tracking algorithm is robust to partial occlusion and rotation, and is computationally efficient, but it does not deal with the problem of total occlusion. Our algorithm overcomes this problem by detecting the occlusion using an occlusion grid, and uses a non-parametric distribution of the color of the occluded person's cloth to distinguish that person after the occlusion ends. Our algorithm uses the speed and the trajectory of each occluded person to predict the locations that should be searched after occlusion ends. It integrates multiple features to handle tracking multiple people in cases of partial and total occlusion. Experiments on a large set of video clips demonstrate the robustness of the algorithm, and its capability to correctly track multiple people even when faces are temporarily occluded by other faces or by other objects in the scene.     

面向多目标跟踪系统的专用循环目标检测器

多目标跟踪技术在视频分析、信号处理等领域有着广泛的应用。在现代多目标跟踪系统通常遵循的“按检测跟踪”模式中,目标检测器的性能决定了多目标跟踪任务的跟踪精度和速度。为提高多目标跟踪系统跟踪性能,提出了面向多目标跟踪系统的专用循环目标检测器,它利用视频帧序列间高度相似性的特点,依据先前帧的目标位置信息和当前帧相对于先前帧的变化得分图来选取候选框,解决了传统二阶段目标检测器中使用候选框推荐网络带来的参数量和计算量大的问题,同时融合了目标外观特征提取分支,进一步减少了多目标跟踪系统整体运行时间。实验表明,专用循环目标检测器及其他最先进的检测器分别应用于多目标跟踪系统,采用专用循环目标检测器时能够在保证多目标跟踪系统跟踪精度的情况下提升跟踪速度。