Fuzzy spatial constraints and ranked partitioned sampling approach for multiple object tracking

While particle filters are now widely used for object tracking in videos, the case of multiple object tracking still raises a number of issues. Among them, a first, and very important, problem concerns the exponential increase of the number of particles with the number of objects to be tracked, that can make some practical applications intractable. To achieve good tracking performances, we propose to use a Partitioned Sampling method in the estimation process with an additional feature about the ordering sequence in which the objects are processed. We call it Ranked Partitioned Sampling, where the optimal order in which objects should be processed and tracked is estimated jointly with the object state. Another essential point concerns the modeling of possible interactions between objects. As another contribution, we propose to represent these interactions within a formal framework relying on fuzzy sets theory. This allows us to easily model spatial constraints between objects, in a general and formal way. The association of these two contributions was tested on typical videos exhibiting difficult situations such as partial or total occlusions, and appearance or disappearance of objects. We show the benefit of using conjointly these two contributions, in comparison to classical approaches, through multiple object tracking and articulated object tracking experiments on real video sequences. The results show that our approach provides less tracking errors than those obtained with the classical Partitioned Sampling method, without the need for increasing the number of particles.     

Multiframe Many–Many Point Correspondence for Vehicle Tracking in High Density Wide Area Aerial Videos

This paper presents a novel framework for tracking thousands of vehicles in high resolution, low frame rate, multiple camera aerial videos. The proposed algorithm avoids the pitfalls of global minimization of data association costs and instead maintains multiple object-centric associations for each track. Representation of object state in terms of many to many data associations per track is proposed and multiple novel constraints are introduced to make the association problem tractable while allowing sharing of detections among tracks. Weighted hypothetical measurements are introduced to better handle occlusions, mis-detections and split or merged detections. A two-frame differencing method is presented which performs simultaneous moving object detection in both. Two novel contextual constraints of vehicle following model, and discouragement of track intersection and merging are also proposed. Extensive experiments on challenging, ground truthed data sets are performed to show the feasibility and superiority of the proposed approach. Results of quantitative comparison with existing approaches are presented, and the efficacy of newly introduced constraints is experimentally established. The proposed algorithm performs better and faster than global, 1–1 data association methods.     

Recovery and Reasoning About Occlusions in 3D Using Few Cameras with Applications to 3D Tracking

In this work we propose algorithms to learn the locations of static occlusions and reason about both static and dynamic occlusion scenarios in multi-camera scenes for 3D surveillance (e.g., reconstruction, tracking). We will show that this leads to a computer system which is able to more effectively track (follow) objects in video when they are obstructed from some of the views. Because of the nature of the application area, our algorithm will be under the constraints of using few cameras (no more than 3) that are configured wide-baseline. Our algorithm consists of a learning phase, where a 3D probabilistic model of occlusions is estimated per-voxel, per-view over time via an iterative framework. In this framework, at each frame the visual hull of each foreground object (person) is computed via a Markov Random Field that integrates the occlusion model. The model is then updated at each frame using this solution, providing an iterative process that can accurately estimate the occlusion model over time and overcome the few-camera constraint. We demonstrate the application of such a model to a number of areas, including visual hull reconstruction, the reconstruction of the occluding structures themselves, and 3D tracking.     

Object tracking in image sequences using point features

This paper presents an object tracking technique based on the Bayesian multiple hypothesis tracking (MHT) approach. Two algorithms, both based on the MHT technique are combined to generate an object tracker. The first MHT algorithm is employed for contour segmentation. The segmentation of contours is based on an edge map. The segmented contours are then merged to form recognisable objects. The second MHT algorithm is used in the temporal tracking of a selected object from the initial frame. An object is represented by key feature points that are extracted from it. The key points (mostly corner points) are detected using information obtained from the edge map. These key points are then tracked through the sequence. To confirm the correctness of the tracked key points, the location of the key points on the trajectory are verified against the segmented object identified in each frame. If an acceptable number of key-points lie on or near the contour of the object in a particular frame (n-th frame), we conclude that the selected object has been tracked (identified) successfully in frame n.     

结合时空一致性的FairMOT跟踪算法优化

目的 视频多目标跟踪(multiple object tracking, MOT)是计算机视觉中的一项重要任务,现有研究分别针对目标检测和目标关联部分进行改进,均忽视了多目标跟踪中的不一致问题。不一致问题主要包括3方面,即目标检测框中心与身份特征中心不一致、帧间目标响应不一致以及训练测试过程中相似度度量方式不一致。为了解决上述不一致问题,本文提出一种基于时空一致性的多目标跟踪方法,以提升跟踪的准确度。方法 从空间、时间以及特征维度对上述不一致性进行修正。对于目标检测框中心与身份特征中心不一致,针对每个目标检测框中心到特征中心之间的空间差异,在偏移后的位置上提取目标的ReID(re-identification)特征;对帧间响应不一致,使用空间相关计算相邻帧之间的运动偏移信息,基于该偏移信息对前一帧的目标响应进行变换后得到帧间一致性响应信息,然后对目标响应进行增强;对训练和测试过程中的相似度度量不一致,提出特征正交损失函数,在训练时考虑目标两两之间的相似关系。结果 在3个数据集上与现有方法进行比较。在MOT17、MOT20和Hieve数据集中,MOTA(multiple object t...     

On-line Object Tracking for Colour Video Analysis

Real-time object tracking is recently becoming more and more important in the field of video analysis and processing. Applications like traffic control, user–computer interaction, on-line video processing and production and video surveillance need reliable and economically affordable video tracking tools. It seems, however, that most of the available solutions are computationally intensive and sometimes require expensive video hardware, quite often without guaranteeing a suitable level of reliability. In this paper, we present a new approach to real-time object tracking from colour video sequences. It relies on contours in order to track the shape, position and orientation of objects, without exploiting snakes or “traditional” active contours. A closed-loop control approach is adopted to enforce motion tracking stability, while a separate shape model is maintained, featuring a two-stage model and a median filtering technique to cope with temporary occlusions and noise. The system was tested in several different environments with different constraints, and gave very encouraging performance. Experimental results are reported and commented on.     

Structural similarity-based object tracking in multimodality surveillance videos

This paper addresses the problem of object tracking in video sequences for surveillance applications by using a recently proposed structural similarity-based image distance measure. Multimodality surveillance videos pose specific challenges to tracking algorithms, due to, for example, low or variable light conditions and the presence of spurious or camouflaged objects. These factors often cause undesired luminance and contrast variations in videos produced by infrared sensors (due to varying thermal conditions) and visible sensors (e.g., the object entering shadowy areas). Commonly used colour and edge histogram-based trackers often fail in such conditions. In contrast, the structural similarity measure reflects the distance between two video frames by jointly comparing their luminance, contrast and spatial characteristics and is sensitive to relative rather than absolute changes in the video frame. In this work, we show that the performance of a particle filter tracker is improved significantly when the structural similarity-based distance is applied instead of the conventional Bhattacharyya histogram-based distance. Extensive evaluation of the proposed algorithm is presented together with comparisons with colour, edge and mean-shift trackers using real-world surveillance video sequences from multimodal (infrared and visible) cameras.     

A real time algorithm for people tracking using contextual reasoning

In this paper we present a real-time tracking algorithm that is able to deal with complex occlusions involving a plurality of moving objects simultaneously. The rationale is grounded on a suitable representation and exploitation of the recent history of each single moving object being tracked. The object history is encoded using a state, and the transitions among the states are described through a Finite State Automata (FSA). In presence of complex situations the tracking is properly solved by making the FSA’s of the involved objects interact with each other. This is the way for basing the tracking decisions not only on the information present in the current frame, but also on conditions that have been observed more stably over a longer time span. The object history can be used to reliably discern the occurrence of the most common problems affecting object detection, making this method particularly robust in complex scenarios. An experimental evaluation of the proposed approach has been made on two publicly available datasets, the ISSIA Soccer Dataset and the PETS 2010 database.     

深度学习辅助的多行人跟踪算法

目的 目标的长距离跟踪一直是视频监控中最具挑战性的任务之一。现有的目标跟踪方法在存在遮挡、目标消失再出现等情况下往往会丢失目标,无法进行持续有效的跟踪。一方面目标消失后再次出现时,将其作为新的目标进行跟踪的做法显然不符合实际需求;另一方面,在跟踪过程中当相似的目标出现时,也很容易误导跟踪器把该相似对象当成跟踪目标,从而导致跟踪失败。为此,提出一种基于目标识别辅助的跟踪算法来解决这个问题。方法 将跟踪问题转化为寻找帧间检测到的目标之间对应关系问题,从而在目标消失再现后,采用深度学习网络实现有效的轨迹恢复,改善长距离跟踪效果,并在一定程度上避免相似目标的干扰。结果 通过在标准数据集上与同类算法进行对比实验,本文算法在目标受到遮挡、交叉运动、消失再现的情况下能够有效地恢复其跟踪轨迹,改善跟踪效果,从而可以对多个目标进行持续有效的跟踪。结论 本文创新性地提出了一种结合基于深度学习的目标识别辅助的跟踪算法,实验结果证明了该方法对遮挡重现后的目标能够有效的恢复跟踪轨迹,适用在监控视频中对多个目标进行持续跟踪。     

Adaptive pyramid mean shift for global real-time visual tracking

Tracking objects in videos using the mean shift technique has attracted considerable attention. In this work, a novel approach for global target tracking based on mean shift technique is proposed. The proposed method represents the model and the candidate in terms of background weighted histogram and color weighted histogram, respectively, which can obtain precise object size adaptively with low computational complexity. To track targets whose displacements between two successive frames are relatively large, we implement the mean shift procedure via a coarse-to-fine way for global maximum seeking. This procedure is termed as adaptive pyramid mean shift, because it uses the pyramid analysis technique and can determine the pyramid level adaptively to decrease the number of iterations required to achieve convergence. Experimental results on various tracking videos and its application to a tracking and pointing subsystem show that the proposed method can successfully cope with different situations such as camera motion, camera vibration, camera zoom and focus, high-speed moving object tracking, partial occlusions, target scale variations, etc.     

海量监控视频快速回放与检索技术

为解决海量监控视频的快速浏览和检索,介绍了一种基于目标索引的视频摘要和检索方法。该方法在光流分析的基础上,在画面的静止区域更新背景,运动的区域利用差分法分割出运动目标图像。经过优化的快速特征匹配和建立运动跟踪模型后,根据目标运动轨迹,按照时空距离进行聚类。在目标图像数据和运动参数进行XML结构化存储为索引的基础上,最后在检索时将符合条件的所有目标图像,按照其原有时间顺序逐帧贴到同一个背景图像中,形成动态的摘要视频。由于该方法剔除了背景中大量的时空冗余信息,可在较短回放时间内浏览全部有用目标,显著提高海量监控视频的查阅效率。     

Multi-object detection and tracking by stereo vision

This paper presents a new stereo vision-based model for multi-object detection and tracking in surveillance systems. Unlike most existing monocular camera-based systems, a stereo vision system is constructed in our model to overcome the problems of illumination variation, shadow interference, and object occlusion. In each frame, a sparse set of feature points are identified in the camera coordinate system, and then projected to the 2D ground plane. A kernel-based clustering algorithm is proposed to group the projected points according to their height values and locations on the plane. By producing clusters, the number, position, and orientation of objects in the surveillance scene can be determined for online multi-object detection and tracking. Experiments on both indoor and outdoor applications with complex scenes show the advantages of the proposed system.     

Detection and Tracking of Multiple,Partially Occluded Humans by Bayesian Combination of Edgelet based Part Detectors

Detection and tracking of humans in video streams is important for many applications. We present an approach to automatically detect and track multiple, possibly partially occluded humans in a walking or standing pose from a single camera, which may be stationary or moving. A human body is represented as an assembly of body parts. Part detectors are learned by boosting a number of weak classifiers which are based on edgelet features. Responses of part detectors are combined to form a joint likelihood model that includes an analysis of possible occlusions. The combined detection responses and the part detection responses provide the observations used for tracking. Trajectory initialization and termination are both automatic and rely on the confidences computed from the detection responses. An object is tracked by data association and meanshift methods. Our system can track humans with both inter-object and scene occlusions with static or non-static backgrounds. Evaluation results on a number of images and videos and comparisons with some previous methods are given. Electronic Supplementary Material Supplementary material is available in the online version of this article at     

Moving people tracking with detection by latent semantic analysis for visual surveillance applications

The latent semantic analysis (LSA) has been widely used in the fields of computer vision and pattern recognition. Most of the existing works based on LSA focus on behavior recognition and motion classification. In the applications of visual surveillance, accurate tracking of the moving people in surveillance scenes, is regarded as one of the preliminary requirement for other tasks such as object recognition or segmentation. However, accurate tracking is extremely hard under challenging surveillance scenes where similarity among multiple objects or occlusion among multiple objects occurs. Usual temporal Markov chain based tracking algorithms suffer from the ‘tracking error accumulation problem’. The accumulated errors can finally make the tracking to drift from the target. To handle the problem of tracking drift, some authors have proposed the idea of using detection along with tracking as an effective solution. However, many of the critical issues still remain unsettled in these detection based tracking algorithms. In this paper, we propose a novel moving people tracking with detection based on (probabilistic) LSA. By employing a novel ‘twin-pipeline’ training framework to find the latent semantic topics of ‘moving people’, the proposed detection can effectively detect the interest points on moving people in different indoor and outdoor environments with camera motion. Since the detected interest points on different body parts can be used to locate the position of moving people more accurately, by combining the detection with incremental subspace learning based tracking, the proposed algorithms resolves the problem of tracking drift during each target appearance update process. In addition, due to the time independent processing mechanism of detection, the proposed method is also able to handle the error accumulation problem. The detection can calibrate the tracking errors during updating of each state of the tracking algorithm. Extensive, experiments on various surveillance environments using different benchmark datasets have proved the accuracy and robustness of the proposed tracking algorithm. Further, the experimental comparison results clearly show that the proposed tracking algorithm outperforms the well known tracking algorithms such as ISL, AMS and WSL algorithms. Furthermore, the speed performance of the proposed method is also satisfactory for realistic surveillance applications.     

中值流辅助在线多示例目标跟踪

针对机器人演示学习中目标跟踪性能的严格要求,提出一种可以有效克服快速运动、遮挡和目标漂移的物体跟踪方法.首先计算中值流,并预测目标的位置偏移,以此计算高斯权重;然后修正搜索区域,并使用在线多示例分类器进行目标搜索,计算似然度;最后使用贝叶斯框架对结果进行融合,使用穷举搜索得到最优的预测位置,并更新在线分类器.实验结果表明,与现有方法相比,该方法对快速运动和目标漂移具有更强的鲁棒性,而且可以达到实时跟踪.     

基于B-样条Snake模型的人体运动跟踪方法研究

提出了一种基于B-样条曲线Snake模型的新的人体运动跟踪方法.Snake算法是通过最小能量来逼近物体的轮廓.采用改进的B-样条曲线Snake模型,每一帧图像中的目标轮廓用三次样条曲线准确地表示,使Snake模型更加稳定和具有较快的收敛速度.计算相邻帧之间的差分图像,通过利用一种基于统计关系双阈值分割方法,有效地检测出图像中运动人体,初步确定目标在每帧图像中的粗略位置.把从上一帧图像中得到的目标轮廓置于该位置,作为B-样条曲线Snake算法中轮廓提取的初始值,经运算后可得到对人体目标的准确分割与跟踪.     

自适应紧致特征的超像素目标跟踪

目的 针对现有的超像素目标跟踪算法（RST）对同一类中分别属于目标和相似干扰物体的超像素块赋予相同特征置信度,导致难以区分目标和相似干扰物的问题,为此提出自适应紧致特征的超像素目标跟踪算法（ACFST）。方法 该方法在每帧的目标搜索区域内构建适合目标大小的自适应紧致搜索区域,并将该区域内外的特征置信度分别保持不变和降低。处于背景中的相似干扰物体会被该方法划分到紧致搜索区域外,其特征置信度被降低。当依据贝叶斯推理框架求出对应最大后验概率的目标时,紧致搜索区域外的特征置信度低,干扰物体归属目标的程度也低,不会被误判为目标。结果 在具有与目标相似干扰物体的两个视频集进行测试,本文ACFST跟踪算法与RST跟踪算法相比,平均中心误差分别缩减到5.4像素和7.5像素,成功率均提高了11%,精确率分别提高了10.6%和21.6%,使得跟踪结果更精确。结论 本文提出构建自适应紧致搜索区域,并通过设置自适应的参数控制紧致搜索区域变化,减少因干扰物体与目标之间相似而带来的误判。在具有相似物体干扰物的视频集上验证了本文算法的有效性,实验结果表明,本文算法在相似干扰物体靠近或与目标部分重叠时,能够保证算法精确地跟踪到目标,提高算法的跟踪精度,具有较强的鲁棒性,使得算法更能适应背景杂乱、目标遮挡、形变等复杂环境。     

融合特征的卫星视频车辆单目标跟踪

目的 卫星视频作为新兴遥感数据,可以提供观测区域高分辨率的空间细节信息与丰富的时序变化信息,为交通监测与特定车辆目标跟踪等应用提供了不同于传统视频视角的信息。相较于传统视频数据,卫星视频中的车辆目标分辨率低、尺度小、包含的信息有限。因此,当目标边界不明、存在部分遮挡或者周边环境表观模糊时,现有的目标跟踪器往往存在严重的目标丢失问题。对此,本文提出一种基于特征融合的卫星视频车辆核相关跟踪方法。方法 对车辆目标使用原始像素和方向梯度直方图（histogram of oriented gradient,HOG）方法提取包含互补判别能力的特征,利用核相关目标跟踪器分别得到具备不变性和判别性的响应图;通过响应图融合的方式结合两种特征的互补信息,得到目标位置;使用响应分布指标（response distribution criterion,RDC）判断当前目标特征的稳定性,决定是否更新跟踪器的表征模型。本文使用的相关滤波方法具有计算量小且运算速度快的特点,具备跟踪多个车辆目标的拓展能力。结果 在8个卫星视频序列上与主流的6种相关滤波跟踪器进行比较,实验数据涵盖光照变化、快速转弯、部分遮挡、阴影干扰、道路颜色变化和相似目标临近等情况,使用准确率曲线和成功率曲线的曲线下面积（area under curve,AUC）对车辆跟踪的精度进行评价。结果表明,本文方法较好地均衡了使用不同特征的基础跟踪器（性能排名第2）的判别能力,准确率曲线AUC提高了2.9%,成功率曲线AUC下降了4.1%,成功跟踪车辆目标,不发生丢失,证明了本文方法的先进性和有效性。结论 本文提出的特征融合的卫星视频车辆核相关跟踪方法,均衡了不同特征提取器的互补信息,较好解决了卫星视频中车辆目标信息不足导致的目标丢失问题,提升了精度。     

遮挡情况下的多目标跟踪算法*

在视频监控系统中,由于背景的复杂变化,运动目标经常会出现部分或全部被遮挡的情况。为了在遮挡条件下进行多目标跟踪,针对运动目标发生遮挡情况下的Mean Shift跟踪问题进行了研究,提出一种新的抗遮挡算法。利用卡尔曼滤波器来获得每帧Mean Shift算法的起始位置,再利用Mean Shift算法得到目标跟踪位置,通过目标遮挡判定机制和目标搜索机制来解决遮挡问题。实验表明,该算法较好地解决了运动目标的遮挡问题。     

Robust and efficient foreground analysis in complex surveillance videos

Mixture of Gaussians-based background subtraction (BGS) has been widely used for detecting moving objects in surveillance videos. It is very efficient and can update the background model with slow lighting changes, however, it suffers from a number of limitations in complex surveillance conditions such as quick lighting variations, heavy occlusion, foreground fragments, slow moving or stopped object etc. To address these issues, this paper first focuses on foreground analysis within the mixture of Gaussians BGS framework in long-term scene monitoring to handle (1) quick lighting changes, (2) static objects, (3) foreground fragments, (4) abandoned and removed objects, and (5) camera view changes. Then, we propose a framework with interactive mechanisms between BGS and processing from different high levels (i.e. region, frame, and tracking) to improve the accuracy of moving object detection and tracking to handle (1) objects that stop for a significant period of time and (2) slow-moving objects. The robustness and efficiency of the proposed mechanism are tested in IBM Smart Surveillance Solution on a variety of sequences, including standard datasets. The proposed method is very efficient and handles ten video streams in real-time on a 2GB Pentium IV machine with MMX optimization.