Improving the robustness of parametric shape tracking with switched multiple models

This paper addresses the problem of tracking objects with complex motion dynamics or shape changes. It is assumed that some of the visual features detected in the image (e.g., edge strokes) are outliers i.e., they do not belong to the object boundary. A robust tracking algorithm is proposed which allows to efficiently track an object with complex shape or motion changes in clutter environments. The algorithm relies on the use of multiple models, i.e., a bank of stochastic motion models switched according to a probabilistic mechanism. Robust filtering methods are used to estimate the label of the active model as well as the state trajectory.     

Robust shape tracking in the presence of cluttered background

Many object-tracking algorithms are based on low-level features detected in the image. Typically, the object shape and position are estimated to fit the observed features. Unfortunately, image analysis methods often produce invalid features (outliers) which do not belong to the object boundary. These features have a strong influence on the shape estimates, leading to meaningless tracking results. This paper proposes a robust tracking algorithm which is able to deal with outliers, inspired in the probabilistic data association filter proposed in the context of point tracking. The algorithm is based on two key concepts. First, middle level features (strokes) are used instead of low-level ones (edge points). Second, two labels (valid/invalid) are considered for each stroke. Since the stroke labels are unknown all labeling sequences are considered and a probability (confidence degree) is assigned to each of them. In this way, all the strokes contribute to track the moving object but with different weights. This allows a robust performance of the tracker in the presence of outliers. Experimental tests are provided to assess the performance of the proposed algorithm in lip and gesture tracking and surveillance applications.     

An Occlusion-aware Edge-Based Method for Monocular 3D Object Tracking using Edge Confidence

We propose an edge-based method for 6DOF pose tracking of rigid objects using a monocular RGB camera. One of the critical problem for edge-based methods is to search the object contour points in the image corresponding to the known 3D model points. However, previous methods often produce false object contour points in case of cluttered backgrounds and partial occlusions. In this paper, we propose a novel edge-based 3D objects tracking method to tackle this problem. To search the object contour points, foreground and background clutter points are first filtered out using edge color cue, then object contour points are searched by maximizing their edge confidence which combines edge color and distance cues. Furthermore, the edge confidence is integrated into the edge-based energy function to reduce the influence of false contour points caused by cluttered backgrounds and partial occlusions. We also extend our method to multi-object tracking which can handle mutual occlusions. We compare our method with the recent state-of-art methods on challenging public datasets. Experiments demonstrate that our method improves robustness and accuracy against cluttered backgrounds and partial occlusions.     

Nonlinear Dynamical Shape Priors for Level Set Segmentation

The introduction of statistical shape knowledge into level set based segmentation methods was shown to improve the segmentation of familiar structures in the presence of noise, clutter or partial occlusions. While most work has been focused on shape priors which are constant in time, it is clear that when tracking deformable shapes certain silhouettes may become more or less likely over time. In fact, the deformations of familiar objects such as the silhouettes of a walking person are often characterized by pronounced temporal correlations. In this paper, we propose a nonlinear dynamical shape prior for level set based image segmentation. Specifically, we propose to approximate the temporal evolution of the eigenmodes of the level set function by means of a mixture of autoregressive models. We detail how such shape priors “with memory” can be integrated into a variational framework for level set segmentation. As an application, we experimentally validate that the nonlinear dynamical prior drastically improves the tracking of a person walking in different directions, despite large amounts of clutter and noise.     

基于深度特征与局部约束掩膜的相关滤波跟踪算法

为提升相关滤波算法在目标遮挡、快速运动以及背景杂乱等情况下跟踪结果的精确度和鲁棒性,提出了一种基于深度特征与局部约束掩膜(Local constrained mask, LCM)的相关滤波跟踪算法。在鉴别性相关滤波跟踪算法的基础上,利用学习得到的二值矩阵作为LCM对滤波器的能量分布进行裁剪,对模板边缘与测试图像之间产生的响应值进行抑制,实现扩大目标搜索区域的同时降低边界效应对跟踪结果的影响;将深度特征引入到特征提取过程中,通过对目标样本进行旋转、翻折和高斯模糊等处理,扩充训练样本数量,使模板学习到更为丰富的目标信息。与主流算法进行对比实验,验证了本文算法在处理目标遮挡、背景嘈杂以及光照变化等干扰时的鲁棒性。     

短程线主动轮廓跟踪算法的研究--在复杂背景和非刚性运动目标跟踪中的应用

在复杂背景下对多个非刚性目标进行跟踪是计算机视觉中的一个难点。在短程线主动轮廓模型的基础上，利用力场正则化方法，并加入运动边缘信息，提出了一种在复杂背景下多个非刚性目标进行跟踪的方法。该方法由运动检测和跟踪两部分组成：运动检测利用运动边缘信息对运动目标的运动做出检测，让轮廓曲线运动到目标轮廓附近；跟踪利用当前帧中的静态边缘信息对运动检测的结果加以修正，而跟踪这一步引入的偏差将在下一帧的运动检测中得到修正。实验表明该方法能够有效地在复杂背景中对多个非刚性运动目标进行跟踪。     

Model-driven active visual tracking

We have previously demonstrated that the performance of tracking algorithms can be improved by integrating information from multiple cues in a model-driven Bayesian reasoning framework. Here we extend our work to active vision tracking, with variable camera geometry. Many existent active tracking algorithms avoid the problem of variable camera geometry by tracking view independent features, such as corners and lines. However, the performance of algorithms based on those single features will greatly deteriorate in the presence of specularities and dense clutter. We show, by integrating multiple cues and updating the camera geometry on-line, that it is possible to track a complicated object moving arbitrarily in three-dimensional (3D) space.We use a four degree-of-freedom (4-DoF) binocular camera rig to track three focus features of an industrial object, whose complete model is known. The camera geometry is updated by using the rig control commands and kinematic model of the stereo head. The extrinsic parameters are further refined by interpolation from a previously sampled calibration of the head work space.The 2D target position estimates are obtained by a combination of blob detection, edge searching and gray-level matching, with the aid of model geometrical structure projection using current estimates of camera geometry. The information is represented in the form of a probability density distribution, and propagated in a Bayes Net. The Bayesian reasoning that is performed in the 2D image is coupled by the rigid model geometry constraint in 3D space.An αβ filter is used to smooth the tracking pursuit and to predict the position of the object in the next iteration of data acquisition. The solution of the inverse kinematic problem at the predicted position is used to control the position of the stereo head.Finally, experiments show that the target undertaking arbitrarily 3D motion can be successfully tracked in the presence of specularities and dense clutter.     

遮挡情况下基于特征相关匹配的目标跟踪算法

特征相关匹配是重要的运动目标跟踪方法.目标特征有灰度特征和边缘特征两大类,在遮挡情况下,采用哪种特征进行匹配,要根据目标本身属性来确定.本文先对目标灰度性质做出判断,然后根据灰度单一或是丰富来合理选择边缘相关匹配或者是基于多子块的灰度相关匹配来解决遮挡情况下的刚性目标跟踪问题.其中边缘匹配算法是通过当前边缘与实时更新模板的最优匹配来确定目标的运动位移量.基于多子块的灰度相关匹配算法通过目标的各个具有较明显特征的子块准确判定遮挡区域,利用剩余的未被遮挡的子块参与灰度相关匹配继续跟踪目标.实验结果表明,这种算法是十分有效的.     

基于相关匹配及自适应模板更新的目标跟踪新方法

传统的相关匹配算法在背景比较简单的情况下可以较好地跟踪到目标,但实际获得的图像存在变形、噪声、遮挡等问题,并且也很难获得比较好的跟踪效果。针对上述问题,提出了一种修正的MCD相关匹配算法和多帧累积的模板更新策略,并对实际图像进行了仿真。实验结果表明,算法在图像存在变形、噪声、遮挡时也可以达到比较理想的跟踪效果。     

Detecting objects of variable shape structure with hidden state shape models

This paper proposes a method for detecting object classes that exhibit variable shape structure in heavily cluttered images. The term "variable shape structure" is used to characterize object classes in which some shape parts can be repeated an arbitrary number of times, some parts can be optional, and some parts can have several alternative appearances. Hidden State Shape Models (HSSMs), a generalization of Hidden Markov Models (HMMs), are introduced to model object classes of variable shape structure using a probabilistic framework. A polynomial inference algorithm automatically determines object location, orientation, scale and structure by finding the globally optimal registration of model states with the image features, even in the presence of clutter. Experiments with real images demonstrate that the proposed method can localize objects of variable shape structure with high accuracy. For the task of hand shape localization and structure identification, the proposed method is significantly more accurate than previously proposed methods based on chamfer-distance matching. Furthermore, by integrating simple temporal constraints, the proposed method gains speed-ups of more than an order of magnitude, and produces highly accurate results in experiments on non-rigid hand motion tracking.     

Disambiguating Visual Motion by Form-Motion Interaction—a Computational Model

The neural mechanisms underlying motion segregation and integration still remain unclear to a large extent. Local motion estimates often are ambiguous in the lack of form features, such as corners or junctions. Furthermore, even in the presence of such features, local motion estimates may be wrong if they were generated near occlusions or from transparent objects. Here, a neural model of visual motion processing is presented that involves early stages of the cortical dorsal and ventral pathways. We investigate the computational mechanisms of V1-MT feedforward and feedback processing in the perception of coherent shape motion. In particular, we demonstrate how modulatory MT-V1 feedback helps to stabilize localized feature signals at, e.g. corners, and to disambiguate initial flow estimates that signal ambiguous movement due to the aperture problem for single shapes. In cluttered environments with multiple moving objects partial occlusions may occur which, in turn, generate erroneous motion signals at points of overlapping form. Intrinsic-extrinsic region boundaries are indicated by local T-junctions of possibly any orientation and spatial configuration. Such junctions generate strong localized feature tracking signals that inject erroneous motion directions into the integration process. We describe a simple local mechanism of excitatory form-motion interaction that modifies spurious motion cues at T-junctions. In concert with local competitive-cooperative mechanisms of the motion pathway the motion signals are subsequently segregated into coherent representations of moving shapes. Computer simulations demonstrate the competency of the proposed neural model.     

基于自适应模板更新的改进孪生卷积网络目标跟踪算法

现有的孪生网络目标跟踪算法采用边界框模板进行跟踪,在目标形变、遮挡等干扰下很容易导致跟踪漂移。在轮廓检测网络和孪生卷积网络(Siamese)跟踪网络的基础上,提出一种基于深度轮廓模板更新的改进孪生卷积网络目标跟踪算法。利用轮廓检测网络获取目标边缘轮廓,降低背景杂波干扰;利用改进的Siamese网络获得轮廓模板和搜索区域的深度特征;通过相似性匹配获得最优跟踪目标。仿真实验结果表明,所提出的改进模型能够提高目标形变、遮挡等干扰下目标跟踪性能,具有较高的工程应用价值。     

A 2D/3D model-based object tracking framework

This paper presents a robust framework for tracking complex objects in video sequences. Multiple hypothesis tracking (MHT) algorithm reported in (IEEE Trans. Pattern Anal. Mach. Intell. 18(2) (1996)) is modified to accommodate a high level representations (2D edge map, 3D models) of objects for tracking. The framework exploits the advantages of MHT algorithm which is capable of resolving data association/uncertainty and integrates it with object matching techniques to provide a robust behavior while tracking complex objects. To track objects in 2D, a 4D feature is used to represent edge/line segments and are tracked using MHT. In many practical applications 3D models provide more information about the object's pose (i.e., rotation information in the transformation space) which cannot be recovered using 2D edge information. Hence, a 3D model-based object tracking algorithm is also presented. A probabilistic Hausdorff image matching algorithm is incorporated into the framework in order to determine the geometric transformation that best maps the model features onto their corresponding ones in the image plane. 3D model of the object is used to constrain the tracker to operate in a consistent manner. Experimental results on real and synthetic image sequences are presented to demonstrate the efficacy of the proposed framework.     

融合坐标信息与模板更新的孪生网络目标跟踪

应对孪生网络单目标跟踪算法在跟踪中遇到背景杂乱、相似物影响、遮挡等复杂场景的问题导致跟踪系统精度和成功率下降的问题,提出一种融合坐标注意力机制和模板更新的跟踪算法MCUSiamRPN (MobileNet coordinate attention and updating of template SiamRPN).在SiamRPN算法基础上,采用改进的MobileNetV3为特征提取网络,多层特征信息分别送入坐标注意力模块,进行特征融合,丰富语义信息;设计了一种自适应模板更新模块,结合初始模板和当前帧的模板用于估计下一帧的最佳模板更新模板信息.在OTB100和UAV123两个数据集上进行测试,结果显示:相比于基准算法Siam RPN,精度分别提升了5.3%和3.7%;成功率分别提升了3.7%和5.2%,验证了该算法的有效性.     

A novel approach to fault diagnosis for time-delay systems

In this paper, a novel fault detection and identification (FDI) scheme for time-delay systems is presented. Different from the existing FDI design methods, the proposed approach utilizes fault tracking approximator (FTA) and iterative learning algorithm to obtain estimates of the fault functions. Performance of the FTA is rigorously analyzed by investigating its stability and fault tracking sensitivity properties in the presence of slowly developing or abrupt faults for state delayed dynamic systems. A novel feature of the FTA is that it can simultaneously detect and identify the shape and magnitude of the faults. Additionally, an extension to a class of nonlinear time-delay systems is made by using nonlinear control theories. Finally, the applicability and effectiveness of the proposed FDI scheme is illustrated by a practical industrial process.     

判别特征回归的鲁棒目标跟踪

目的 近年来,目标跟踪领域取得了很大进步,但是由于尺度变化,运动,形状畸变或者遮挡等造成的外观变化,仍然是目标跟踪中的一大挑战,因而有效的图像表达方法是提高目标跟踪鲁棒性的一个关键因素。方法 从中层视觉角度出发,首先对训练图像进行超像素分割,将得到特征向量集以及对应的置信值作为输入值,通过特征回归的方法建立目标跟踪中的判别外观模型,将跟踪图像的特征向量输入该模型,得到候选区域的置信值,从而高效地分离前景和背景,确定目标区域。结果 在公开数据集上进行跟踪实验。本文算法能较好地处理目标尺度变化、姿态变化、光照变化、形状畸变、遮挡等外观变化;和主流跟踪算法进行对比,本文算法在跟踪误差方面表现出色,在carScale、subway、tiger1视频中能取得最好结果,平均误差为12像素,3像素和21像素;和同类型的方法相比,本文算法在算法效率上表现出色,所有视频的跟踪效率均高于同类型算法,在carScale视频中的效率,是同类算法效率的32倍。结论 实验结果表明,本文目标跟踪算法具有高效性和鲁棒性,适用于目标发生外观变化时的目标跟踪问题。目前跟踪中只用了单一特征,未来考虑融合多特征来提升算法鲁棒性和准确度。     

3D SSD tracking with estimated 3D planes

We present a tracking method where full camera position and orientation is tracked from intensity differences in a video sequence. The camera pose is calculated based on 3D planes, and hence does not depend on point correspondences. The plane based formulation also allows additional constraints to be naturally added, e.g., perpendicularity between walls, floor and ceiling surfaces, co-planarity of wall surfaces etc. A particular feature of our method is that the full 3D pose change is directly computed from temporal image differences without making a commitment to a particular intermediate (e.g., 2D feature) representation. We experimentally compared our method with regular 2D SSD tracking and found it more robust and stable. This is due to 3D consistency being enforced even in the low level registration of image regions. This yields better results than first computing (and hence committing to) 2D image features and then from these compute 3D pose.     

SEDMI: Saliency based edge detection in multispectral images

Detecting edges in multispectral images is difficult because different spectral bands may contain different edges. Existing approaches calculate the edge strength of a pixel locally, based on the variation in intensity between this pixel and its neighbors. Thus, they often fail to detect the edges of objects embedded in background clutter or objects which appear in only some of the bands.We propose SEDMI, a method that aims to overcome this problem by considering the salient properties of edges in an image. Based on the observation that edges are rare events in the image, we recast the problem of edge detection into the problem of detecting events that have a small probability in a newly defined feature space. The feature space is constructed by the spatial gradient magnitude in all spectral channels. As edges are often confined to small, isolated clusters in this feature space, the edge strength of a pixel, or the confidence value that this pixel is an event with a small probability, can be calculated based on the size of the cluster to which it belongs.Experimental results on a number of multispectral data sets and a comparison with other methods demonstrate the robustness of the proposed method in detecting objects embedded in background clutter or appearing only in a few bands.     

高置信度互补学习的实时目标跟踪

目的 判别式目标跟踪算法在解决模型漂移问题时通常都是在预测结果的基础上构建更可靠的样本或采用更健壮的分类器,从而忽略了高效简洁的置信度判别环节。为此,提出高置信度互补学习的实时目标跟踪算法（HCCL-Staple）。方法 将置信度评估问题转化为子模型下独立进行的置信度计算与互补判别,对相关滤波模型计算输出的平均峰值相关能量（APCE）,结合最大响应值进行可靠性判定,当二者均以一定比例大于历史均值时,判定为可靠并进行更新,将颜色概率模型的输出通过阈值处理转化为二值图像,并基于二值图像形态学提取像素级连通分量属性（PCCP）,综合考虑连通分量数量、最大连通分量面积及矩形度进行可靠性判别,当置信度参数多数呈高置信度形态时,判定为可靠,进行更新;否则,判定为不可靠,降低该模型的融合权重并停止更新。结果 在数据集OTB-2015上的实验结果表明,HCCL-Staple算法与原算法相比,距离精度提高了3.2%,成功率提高了2.7%,跟踪速度为32.849帧/s,在颜色特征适应性较弱的场景和目标被遮挡的复杂场景中均能有效防止模型漂移,与当前各类主流的跟踪算法相比具有较好的跟踪效果。结论 两种子模型的置信度判别方法均能针对可能产生低置信度结果的敏感场景进行有效估计,且对输出形式相同的其他模型在置信度判别上具有一定的适用性。互补使用上述判别策略的HCCL-Staple算法能够有效防止模型漂移,保持高速的同时显著提升跟踪精度。     

基于OTSU分割和HOG特征的行人检测与跟踪方法

传统的HOG算法针对整幅图像进行行人特征提取,大量的非人窗口计算必然降低检测的准确率和效率。为此,提出一种基于OTSU分割和HOG特征的行人检测与跟踪方法。利用OTSU算法以最佳阈值分割图像,在分割区域的基础上进行Canny边缘检测,通过边缘的对称性计算确定行人候选区,继而采用经PCA方法降维后的HOG特征和隐马尔可夫模型对行人候选区进行检测验证。最后,以确定的行人区域为跟踪窗口,利用CamShift算法跟踪行人。多组实验结果证明,本文方法的行人检测效率和精度均有所提高,跟踪性能稳定、可靠。