Visual tracking based on adaptive interacting multiple model particle filter by fusing multiples cues

In visual tracking topic, developing a robust tracking method is very challenging, seen that there are many issues to look at, particularly, fast motion, target appearance changing, background clutter and camera motion. To override these problems, we present a new object tracking method with the fusion of interacting multiple models (IMM) and the particle filter (PF). First, the IMM is applied with a bank of parallel H∞ filter to estimate the global motion, the target motion is efficiently represented using only two parametric single models, and an adaptive strategy is preformed to adjust automatically the parameters of the two sub models at each recursive time step. Second, the particle filter is performed to estimate the local motion, we fuse the color and texture features to describe the appearance of the tracked object, we use the alpha Gaussian mixture model (α-GMM) to model the color feature distribution, the parameter α allows the probability function to possesses a flatter distribution, and the texture feature is represented by the distinctive uniform local binary pattern histogram (DULBP) based on the uniform local binary pattern (ULBP) operator; we fuse then the two features to represent the target’s appearance under the particle filter framework. We conduct quantitative and qualitative experiments on a variety of challenging public sequences; the results show that our method performs robustly and demonstrates strong accuracy.     

Disambiguating Visual Motion by Form-Motion Interaction—a Computational Model

The neural mechanisms underlying motion segregation and integration still remain unclear to a large extent. Local motion estimates often are ambiguous in the lack of form features, such as corners or junctions. Furthermore, even in the presence of such features, local motion estimates may be wrong if they were generated near occlusions or from transparent objects. Here, a neural model of visual motion processing is presented that involves early stages of the cortical dorsal and ventral pathways. We investigate the computational mechanisms of V1-MT feedforward and feedback processing in the perception of coherent shape motion. In particular, we demonstrate how modulatory MT-V1 feedback helps to stabilize localized feature signals at, e.g. corners, and to disambiguate initial flow estimates that signal ambiguous movement due to the aperture problem for single shapes. In cluttered environments with multiple moving objects partial occlusions may occur which, in turn, generate erroneous motion signals at points of overlapping form. Intrinsic-extrinsic region boundaries are indicated by local T-junctions of possibly any orientation and spatial configuration. Such junctions generate strong localized feature tracking signals that inject erroneous motion directions into the integration process. We describe a simple local mechanism of excitatory form-motion interaction that modifies spurious motion cues at T-junctions. In concert with local competitive-cooperative mechanisms of the motion pathway the motion signals are subsequently segregated into coherent representations of moving shapes. Computer simulations demonstrate the competency of the proposed neural model.     

结合时空一致性的FairMOT跟踪算法优化

目的 视频多目标跟踪(multiple object tracking, MOT)是计算机视觉中的一项重要任务,现有研究分别针对目标检测和目标关联部分进行改进,均忽视了多目标跟踪中的不一致问题。不一致问题主要包括3方面,即目标检测框中心与身份特征中心不一致、帧间目标响应不一致以及训练测试过程中相似度度量方式不一致。为了解决上述不一致问题,本文提出一种基于时空一致性的多目标跟踪方法,以提升跟踪的准确度。方法 从空间、时间以及特征维度对上述不一致性进行修正。对于目标检测框中心与身份特征中心不一致,针对每个目标检测框中心到特征中心之间的空间差异,在偏移后的位置上提取目标的ReID(re-identification)特征;对帧间响应不一致,使用空间相关计算相邻帧之间的运动偏移信息,基于该偏移信息对前一帧的目标响应进行变换后得到帧间一致性响应信息,然后对目标响应进行增强;对训练和测试过程中的相似度度量不一致,提出特征正交损失函数,在训练时考虑目标两两之间的相似关系。结果 在3个数据集上与现有方法进行比较。在MOT17、MOT20和Hieve数据集中,MOTA(multiple object t...     

多物体遮挡情况下的视觉跟踪算法

针对视频监控中多运动物体间的遮挡问题,提出一种结合全局特征匹配与局部特征匹配的目标跟踪算法.该算法采用基于直方图和基于分块的方法共同表达目标的灰度特征.遮挡发生前实时进行遮挡预判,遮挡时采用基于块分类的方法跟踪目标,遮挡结束后通过直方图匹配重新定位目标.实验结果表明了该方法的有效性和优越性.     

Real-time tracking of multiple objects in space-variant vision based on magnocellular visual pathway

In this paper, we propose a space-variant image representation model based on properties of magnocellular visual pathway, which perform motion analysis, in human retina. Then, we present an algorithm for the tracking of multiple objects in the proposed space-variant model. The proposed space-variant model has two effective image representations for object recognition and motion analysis, respectively. Each image representation is based on properties of two types of ganglion cell, which are the beginning of two basic visual pathways; one is parvocellular and the other is magnocellular. Through this model, we can get the efficient data reduction capability with no great loss of important information. And, the proposed multiple objects tracking method is restricted in space-variant image. Typically, an object-tracking algorithm consists of several processes such as detection, prediction, matching, and updating. In particular, the matching process plays an important role in multiple objects tracking. In traditional vision, the matching process is simple when the target objects are rigid. In space-variant vision, however, it is very complicated although the target is rigid, because there may be deformation of an object region in the space-variant coordinate system when the target moves to another position. Therefore, we propose a deformation formula in order to solve the matching problem in space-variant vision. By solving this problem, we can efficiently implement multiple objects tracking in space-variant vision.     

融合SPA遮挡分割的多目标跟踪方法

复杂环境下的多目标视频跟踪是计算机视觉领域的一个难点,有效处理目标间遮挡是解决多目标跟踪问题的关键。将运动分割方法引入目标跟踪领域,提出一种融合骨架点指派(SPA)遮挡分割的多目标跟踪方法。由底层光流信息得到骨架点,并估计骨架点遮挡状态;综合使用目标外观、运动、颜色信息等高级语义信息,将骨架点指派给各个目标;最后以骨架点为核,对运动前景密集分类,得到准确的目标前景像素;在粒子滤波器跟踪框架下,使用概率外观模型进行多目标跟踪。在PETS2009数据集上的实验结果表明,文中方法能够改进现有多目标跟踪方法对目标间交互适应性较差的缺点,更好地处理动态遮挡问题。     

Dynamic Template Tracking and Recognition

In this paper we address the problem of tracking non-rigid objects whose local appearance and motion changes as a function of time. This class of objects includes dynamic textures such as steam, fire, smoke, water, etc., as well as articulated objects such as humans performing various actions. We model the temporal evolution of the object’s appearance/motion using a linear dynamical system. We learn such models from sample videos and use them as dynamic templates for tracking objects in novel videos. We pose the problem of tracking a dynamic non-rigid object in the current frame as a maximum a-posteriori estimate of the location of the object and the latent state of the dynamical system, given the current image features and the best estimate of the state in the previous frame. The advantage of our approach is that we can specify a-priori the type of texture to be tracked in the scene by using previously trained models for the dynamics of these textures. Our framework naturally generalizes common tracking methods such as SSD and kernel-based tracking from static templates to dynamic templates. We test our algorithm on synthetic as well as real examples of dynamic textures and show that our simple dynamics-based trackers perform at par if not better than the state-of-the-art. Since our approach is general and applicable to any image feature, we also apply it to the problem of human action tracking and build action-specific optical flow trackers that perform better than the state-of-the-art when tracking a human performing a particular action. Finally, since our approach is generative, we can use a-priori trained trackers for different texture or action classes to simultaneously track and recognize the texture or action in the video.     

融合人群密度的自适应深度多目标跟踪算法

多目标跟踪技术不能较好地解决目标严重遮挡场景下的多目标跟踪问题,因此文中提出融合人群密度的自适应深度多目标跟踪算法.首先,融合人群密度图和目标检测结果,利用人群密度图的位置和计数信息修正检测器结果,消除漏检、误检.然后,使用自适应三元组损失改进行人重识别模型的损失函数,提高对重识别特征的辨别能力.最后,使用外观和运动信...     

A lattice-based MRF model for dynamic near-regular texture tracking

A near-regular texture (NRT) is a geometric and photometric deformation from its regular origin - a congruent wallpaper pattern formed by 2D translations of a single tile. A dynamic NRT is an NRT under motion. Although NRTs are pervasive in man-made and natural environments, effective computational algorithms for NRTs are few. This paper addresses specific computational challenges in modeling and tracking dynamic NRTs, including ambiguous correspondences, occlusions, and drastic illumination and appearance variations. We propose a lattice-based Markov-random-field (MRF) model for dynamic NRTs in a 3D spatiotemporal space. Our model consists of a global lattice structure that characterizes the topological constraint among multiple textons and an image observation model that handles local geometry and appearance variations. Based on the proposed MRF model, we develop a tracking algorithm that utilizes belief propagation and particle filtering to effectively handle the special challenges of the dynamic NRT tracking without any assumption on the motion types or lighting conditions. We provide quantitative evaluations of the proposed method against existing tracking algorithms and demonstrate its applications in video editing     

融合显著性与运动信息的相关滤波跟踪算法

主流的目标跟踪算法以矩形模板的形式建立被跟踪物体的视觉表征,无法有效区分目标与背景像素,在背景复杂、目标非刚体形变、复杂运动等挑战性因素影响下容易出现模型偏移的问题,导致跟踪失败.与此同时,像素级的显著性信息与运动先验信息作为人类视觉系统有效区分目标与背景、识别运动物体的重要信号,并没有在主流目标跟踪算法中得到有效的集...     

基于组合型表面模型的运动目标跟踪

运动目标跟踪是计算机视觉领域研究的难点课题,提出了一种基于组合型表面模型的视频运动目标跟踪算法。研究了目标的颜色特征空间和梯度特征空间,通过梯度特征和颜色特征的组合,以直方图的形式来建立目标表面模型,之后使用CamShift算法来完成一帧一帧的跟踪。实验结果表明,在图像背景复杂且目标出现遮挡的情况下,该方法仍能有效的跟踪目标。     

多物体遮挡情况下的视觉跟踪算法研究

针对视频监控中多运动物体间的遮挡问题,提出了一种新的结合全局特征和局部特征匹配的目标跟踪算法。该算法采用直方图的方法和基于分块的方法共同表达目标的灰度特征。遮挡发生前实时进行遮挡预判,遮挡时,利用基于块分类的方法跟踪目标,遮挡结束后,通过直方图匹配重新定位目标。实验结果证明了该方法的有效性和优越性。     

Robust object tracking via multi-scale patch based sparse coding histogram

There are many visual tracking algorithms that are based on sparse representation appearance model. Most of them are modeled by local patches with fixed patch scale, which make trackers less effective when objects undergone appearance changes such as illumination variation, pose change or partial occlusion. To solve the problem, a novel appearance representation model is proposed via multi-scale patch based sparse coding histogram for robust visual tracking. In this paper, the appearance of an object is modeled by different scale patches, which are represented by sparse coding histogram with different scale dictionaries. Then a similarity measure is applied to the calculation of the distance between the sparse coding histograms of target candidate and target template. Finally, the similarity score of the target candidate is passed to a particle filter to estimate the target state sequentially in the tracking process. Additionally, in order to decrease the visual drift caused by partial occlusion, an occlusion handling strategy is adopted, which takes the spatial information of multi-scale patches and occlusion into account. Based on the experimental results on some benchmarks of video sequences, our tracker outperforms state-of-the-art tracking methods.     

适用于单目视频的无标记三维人体运动跟踪

在无标记人体运动跟踪过程中,由于被跟踪目标缺乏明显的特征以及背景复杂而使得跟踪到的人体运动姿态与真实值偏差较大,不能进行长序列视频跟踪.针对这一现象,提出一种基于形变外观模板匹配进行单目视频的三维人体运动跟踪算法,其中所用的人体外观模型由三维人体骨骼模型及二维纸板模型组成.首先根据人体骨骼比例约束采用逆运动学计算出关节旋转欧拉角;然后利用正向运动学求得纸板模型中像素在三维空间中的坐标,将这些像素根据摄像机成像模型投影到二维图像中得到形变外观模板;最后采用直方图匹配得到人体运动跟踪结果.实验结果表明,该算法对于一些复杂的长序列人体运动能够得到较为理想的跟踪结果,可应用于人机交互和动画制作等领域.     

Target tracking with incomplete detection

In this paper, we address the multiple target tracking problem as a maximum a posteriori problem. We adopt a graph representation of all observations over time. To make full use of the visual observations from the image sequence, we introduce both motion and appearance likelihood. The multiple target tracking problem is formulated as finding multiple optimal paths in the graph. Due to the noisy foreground segmentation, an object may be represented by several foreground regions and similarly one foreground region may correspond to multiple objects. To deal with this problem, we propose merge, split and mean shift operations to generate new hypotheses to the measurement graph. The proposed approach uses a sliding window framework, that aggregates information across a fixed number of frames. Experimental results on both indoor and outdoor data sets are reported. Furthermore, we provide a comparison between the proposed approach with the existing methods that do not merge/split detected blobs.     

A Competitive Neural Network for Multiple Object Tracking in Video Sequence Analysis

Tracking of moving objects in real situation is a challenging research issue, due to dynamic changes in objects or background appearance, illumination, shape and occlusions. In this paper, we deal with these difficulties by incorporating an adaptive feature weighting mechanism to the proposed growing competitive neural network for multiple objects tracking. The neural network takes advantage of the most relevant object features (information provided by the proposed adaptive feature weighting mechanism) in order to estimate the trajectories of the moving objects. The feature selection mechanism is based on a genetic algorithm, and the tracking algorithm is based on a growing competitive neural network where each unit is associated to each object in the scene. The proposed methods (object tracking and feature selection mechanism) are applied to detect the trajectories of moving vehicles in roads. Experimental results show the performance of the proposed system compared to the standard Kalman filter.     

基于DenseNet的无人机在线多目标跟踪算法

多目标跟踪任务的目的,是对图像序列中不同的目标设置不同的编号(ID),最终得到不同目标的运动轨迹。本文针对跟踪过程中目标ID极易变化的现象,提出了一种新的在线多目标跟踪算法。算法主要包含三个步骤:输入预处理、特征提取和数据关联。其中预处理步骤使用NMS算法对输入的检测结果进行筛选;特征提取步骤使用密集连接的特征提取网络对目标进行外观特征的提取,输出特征向量矩阵;数据关联步骤则使用级联匹配的方式,依据目标的位置信息和外观特征信息为其分配各自的ID。此外,该文还整理了一个具有挑战性的无人机场景下的多目标跟踪测试集。实验结果表明,该方法有效地减少了错误的目标ID变化,提高了多目标跟踪算法面对复杂场景时的精度,并保持较快的运行速度。     

Tracking multiple objects through occlusion with online sampling and position estimation

To track multiple objects through occlusion, either depth information of the scene or prior models of the objects such as spatial models and smooth/predictable motion models are usually assumed before tracking. When these assumptions are unreasonable, the tracker may fail. To overcome this limitation, we propose a novel online sample based framework, inspired by the fact that the corresponding local parts of objects in sequential frames are always similar in the local color and texture features and spatial features relative to the centers of objects. Experimental results illustrate that the proposed approach works robustly under difficult and complex conditions.     

自适应在线判别外观学习的分层关联多目标跟踪

目的 复杂场景下目标频繁且长时间的遮挡、跟踪目标外观相似引起身份转换等问题给多目标跟踪带来许多挑战。针对多目标跟踪在复杂场景中因长时间遮挡引起身份转换和轨迹分段的问题,提出一种基于自适应在线判别外观学习的分层关联多目标跟踪算法。方法 利用轨迹置信度将多目标跟踪分为局部关联和全局关联两个层次。在局部关联中,置信度高的可靠轨迹利用外观、位置-大小相似度与当前帧检测点进行关联;在全局关联中,置信度低的不可靠轨迹引入运动模型和有效关联范围进一步关联分段的轨迹。在提取目标外观特征时引入增量线性可判别分析方法以解决身份转换问题,依据新增样本与目标样本均值的外观特征差异自适应地更新目标外观模型。结果 在公开数据集2D MOT2015中的PETS09-S2L1、TUD-Stadmitte、Town-Center 3个数据集中与当前10种多目标跟踪算法进行比较,该方法对各个数据集身份转换和轨迹分段都有减少,其中在Town-Center数据集中,身份转换减少了60个,轨迹分段减少了84个,跟踪准确度提高了5.2%以上。结论 本文多目标跟踪方法,能够在复杂场景中稳定有效地实现多目标跟踪,减少轨迹分段现象,其中引入的在线线性可判别外观学习对遮挡产生的身份转换具有良好的解决效果。     

A color-based particle filter for multiple object tracking in an outdoor environment

Tracking multiple objects is more challenging than tracking a single object. Some problems arise in multiple-object tracking that do not exist in single-object tracking, such as object occlusion, the appearance of a new object and the disappearance of an existing object, updating the occluded object, etc. In this article, we present an approach to handling multiple-object tracking in the presence of occlusions, background clutter, and changing appearance. The occlusion is handled by considering the predicted trajectories of the objects based on a dynamic model and likelihood measures. We also propose target-model-update conditions, ensuring the proper tracking of multiple objects. The proposed method is implemented in a probabilistic framework such as a particle filter in conjunction with a color feature. The particle filter has proven very successful for nonlinear and non-Gaussian estimation problems. It approximates a posterior probability density of the state, such as the object’s position, by using samples or particles, where each state is denoted as the hypothetical state of the tracked object and its weight. The observation likelihood of the objects is modeled based on a color histogram. The sample weight is measured based on the Bhattacharya coefficient, which measures the similarity between each sample’s histogram and a specified target model. The algorithm can successfully track multiple objects in the presence of occlusion and noise. Experimental results show the effectiveness of our method in tracking multiple objects.