Camera handoff and placement for automated tracking systems with multiple omnidirectional cameras

In a multi-camera surveillance system, both camera handoff and placement play an important role in generating an automated and persistent object tracking, typical of most surveillance requirements. Camera handoff should comprise three fundamental components, time to trigger handoff process, the execution of consistent labeling, and the selection of the next optimal camera. In this paper, we design an observation measure to quantitatively formulate the effectiveness of object tracking so that we can trigger camera handoff timely and select the next camera appropriately before the tracked object falls out of the field of view (FOV) of the currently observing camera. In the meantime, we present a novel solution to the consistent labeling problem in omnidirectional cameras. A spatial mapping procedure is proposed to consider both the noise inherent to the tracking algorithms used by the system and the lens distortion introduced by omnidirectional cameras. This does not only avoid the tedious process, but also increases the accuracy, to obtain the correspondence between omnidirectional cameras without human interventions. We also propose to use the Wilcoxon Signed-Rank Test to improve the accuracy of trajectory association between pairs of objects. In addition, since we need a certain amount of time to successfully carry out the camera handoff procedure, we introduce an additional constraint to optimally reserve sufficient cameras’ overlapped FOVs for the camera placement. Experiments show that our proposed observation measure can quantitatively formulate the effectiveness of tracking, so that camera handoff can smoothly transfer objects of interest. Meanwhile, our proposed consistent labeling approach can perform as accurately as the geometry-based approach without tedious calibration processes and outperform Calderara’s homography-based approach. Our proposed camera placement method exhibits a significant increase in the camera handoff success rate at the cost of slightly decreased coverage, as compared to Erdem and Sclaroff’s method without considering the requirement on overlapped FOVs.     

Can You See Me Now? Sensor Positioning for Automated and Persistent Surveillance

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera's field of view (FOV). However, visibility, which is a fundamental requirement of object tracking, is insufficient for automated persistent surveillance. In such applications, a continuous consistently labeled trajectory of the same object should be maintained across different camera views. Therefore, a sufficient uniform overlap between the cameras' FOVs should be secured so that camera handoff can successfully and automatically be executed before the object of interest becomes untraceable or unidentifiable. In this paper, we propose sensor-planning methods that improve existing algorithms by adding handoff rate analysis. Observation measures are designed for various types of cameras so that the proposed sensor-planning algorithm is general and applicable to scenarios with different types of cameras. The proposed sensor-planning algorithm preserves necessary uniform overlapped FOVs between adjacent cameras for an optimal balance between coverage and handoff success rate. In addition, special considerations such as resolution and frontal-view requirements are addressed using two approaches: 1) direct constraint and 2) adaptive weights. The resulting camera placement is compared with a reference algorithm published by Erdem and Sclaroff. Significantly improved handoff success rates and frontal-view percentages are illustrated via experiments using indoor and outdoor floor plans of various scales.      

基于卡尔曼滤波的多目标跟踪方法

针对视频多目标跟踪中由于目标间的遮挡、交错或目标漂移而导致跟踪失败的情况,提出一种基于卡尔曼滤波以及空间颜色直方图的遮挡预测跟踪算法。利用空间颜色直方图对目标进行建模,可以对不同目标进行区分进而在目标之间出现交错或目标漂移时仍能跟踪到目标。通过卡尔曼滤波算法可以 预测 目标的状态,对预测位置之间存在交错的目标进行遮挡标记,以便在下一帧中仍然可以跟踪到被遮挡的目标。采用2D MOT 2015数据集进行实验,跟踪的平均精度达到了34.1%。实验结果表明,所提方法对多目标跟踪的效果有所提高。     

An efficient continuous tracking system in real-time surveillance application

A tracking object must present a proper field of view (FOV) in a multiple active camera surveillance system; its clarity can facilitate smooth processing by the surveillance system before further processing, such as face recognition. However, when pan–tilt–zoom (PTZ) cameras are used, the tracking object can be brought into the FOV by adjusting its intrinsic parameters; consequently, selection of the best-performing camera is critical. Performance is determined by the relative positions of the camera and the tracking objects, image quality, lighting and how much of the front side of the object faces the camera. In a multi-camera surveillance system, both camera hand-off and camera assignment play an important role in automated and persistent tracking, which are typical surveillance requirements. This study investigates the use of automatic methods for tracking an object across cameras in a surveillance network using PTZ cameras. An automatic, efficient continuous tracking scheme is developed. The goal is to determine the decision criteria for hand-off using Sight Quality Indication (SQI) (which includes information on the position of the object and the proportion of the front of object faces the camera), and to perform the camera handoff task in a manner that optimizes the vision effect associated with monitoring. Experimental results reveal that the proposed algorithm can be efficiently executed, and the handoff method for feasible and continuously tracking active objects under real-time surveillance.     

Adaptive online camera coordination for multi-camera multi-target surveillance

Online camera selection is introduced as a result of the improved mobility of cameras and the increased scale of surveillance systems. Most existing camera assignment algorithms achieve an optimal observation under the assumption of the unlimited camera computational capacities. However, practical surveillance systems experience resource limitation and see a degradation in the system performance as the number of objects to be processed increases. To address this issue, we propose an adaptive camera assignment algorithm considering the limited camera computational capacities. In so doing, camera resources can be dynamically allocated to multiple objects according to their priorities and the current camera computational load. Experimental results illustrate that the proposed camera assignment algorithm is capable of maintaining a constant frame rate and achieving a substantially decreased object rejection rate in comparison with the algorithm presented by Bakhtari and Benhabib.     

夜晚车辆异常事件分析

夜晚车道模型是车辆跟踪和车辆行为分析的基础,但是当高速公路或者城市道路光线较暗时,很难通过车道检测的方法来建立车道模型,夜晚车辆快速行驶或相邻帧车辆之间重叠度较低时无法实现准确跟踪。针对此类问题提出了一种基于学习的车道模型建立方法和基于多帧的最佳匹配跟踪方法。首先利用自动多阈值分割方法提取场景中光亮的目标;其次,利用车灯的相关特征移除非车灯光亮区域;接着,利用空间信息把车灯聚类成一个车辆目标,利用多帧的最佳匹配跟踪方法进行跟踪;最后利用车辆跟踪参数与车道模型的融合对夜晚车辆异常事件进行分析。实验结果表明,该算法能够准确地检测出夜晚车辆换道、逆向行驶、交通拥挤、停车等异常事件,并且有很强的鲁棒性。     

A fast and accurate moving object tracker in active camera model

Detecting and tracking moving objects within a scene is an essential step for high-level machine vision applications such as video content analysis. In this paper, we propose a fast and accurate method for tracking an object of interest in a dynamic environment (active camera model). First, we manually select the region of the object of interest and extract three statistical features, namely the mean, the variance and the range of intensity values of the feature points lying inside the selected region. Then, using the motion information of the background’s feature points and k-means clustering algorithm, we calculate camera motion transformation matrix. Based on this matrix, the previous frame is transformed to the current frame’s coordinate system to compensate the impact of camera motion. Afterwards, we detect the regions of moving objects within the scene using our introduced frame difference algorithm. Subsequently, utilizing DBSCAN clustering algorithm, we cluster the feature points of the extracted regions in order to find the distinct moving objects. Finally, we use the same statistical features (the mean, the variance and the range of intensity values) as a template to identify and track the moving object of interest among the detected moving objects. Our approach is simple and straightforward yet robust, accurate and time efficient. Experimental results on various videos show an acceptable performance of our tracker method compared to complex competitors.     

Backpack: Detection of People Carrying Objects Using Silhouettes

We describe a video-rate surveillance algorithm for determining whether people are carrying objects or moving unencumbered from a stationary camera. The contribution of the paper is the shape analysis algorithm that both determines whether a person is carrying an object and segments the object from the person so that it can be tracked, e.g., during an exchange of objects between two people. As the object is segmented, an appearance model of the object is constructed. The method combines periodic motion estimation with static symmetry analysis of the silhouettes of a person in each frame of the sequence. Experimental results demonstrate robustness and real-time performance of the proposed algorithm.     

自适应紧致特征的超像素目标跟踪

目的 针对现有的超像素目标跟踪算法（RST）对同一类中分别属于目标和相似干扰物体的超像素块赋予相同特征置信度,导致难以区分目标和相似干扰物的问题,为此提出自适应紧致特征的超像素目标跟踪算法（ACFST）。方法 该方法在每帧的目标搜索区域内构建适合目标大小的自适应紧致搜索区域,并将该区域内外的特征置信度分别保持不变和降低。处于背景中的相似干扰物体会被该方法划分到紧致搜索区域外,其特征置信度被降低。当依据贝叶斯推理框架求出对应最大后验概率的目标时,紧致搜索区域外的特征置信度低,干扰物体归属目标的程度也低,不会被误判为目标。结果 在具有与目标相似干扰物体的两个视频集进行测试,本文ACFST跟踪算法与RST跟踪算法相比,平均中心误差分别缩减到5.4像素和7.5像素,成功率均提高了11%,精确率分别提高了10.6%和21.6%,使得跟踪结果更精确。结论 本文提出构建自适应紧致搜索区域,并通过设置自适应的参数控制紧致搜索区域变化,减少因干扰物体与目标之间相似而带来的误判。在具有相似物体干扰物的视频集上验证了本文算法的有效性,实验结果表明,本文算法在相似干扰物体靠近或与目标部分重叠时,能够保证算法精确地跟踪到目标,提高算法的跟踪精度,具有较强的鲁棒性,使得算法更能适应背景杂乱、目标遮挡、形变等复杂环境。     

Object tracking in image sequences using point features

This paper presents an object tracking technique based on the Bayesian multiple hypothesis tracking (MHT) approach. Two algorithms, both based on the MHT technique are combined to generate an object tracker. The first MHT algorithm is employed for contour segmentation. The segmentation of contours is based on an edge map. The segmented contours are then merged to form recognisable objects. The second MHT algorithm is used in the temporal tracking of a selected object from the initial frame. An object is represented by key feature points that are extracted from it. The key points (mostly corner points) are detected using information obtained from the edge map. These key points are then tracked through the sequence. To confirm the correctness of the tracked key points, the location of the key points on the trajectory are verified against the segmented object identified in each frame. If an acceptable number of key-points lie on or near the contour of the object in a particular frame (n-th frame), we conclude that the selected object has been tracked (identified) successfully in frame n.     

Computing occluding and transparent motions

Computing the motions of several moving objects in image sequences involves simultaneous motion analysis and segmentation. This task can become complicated when image motion changes significantly between frames, as with camera vibrations. Such vibrations make tracking in longer sequences harder, as temporal motion constancy cannot be assumed. The problem becomes even more difficult in the case of transparent motions.A method is presented for detecting and tracking occluding and transparent moving objects, which uses temporal integration without assuming motion constancy. Each new frame in the sequence is compared to a dynamic internal representation image of the tracked object. The internal representation image is constructed by temporally integrating frames after registration based on the motion computation. The temporal integration maintains sharpness of the tracked object, while blurring objects that have other motions. Comparing new frames to the internal representation image causes the motion analysis algorithm to continue tracking the same object in subsequent frames, and to improve the segmentation.     

基于静止分割和运动矢量的对象跟踪

运动对象的分割和跟踪是视频图象处理的一个很重要和困难的部分。提出一种非常有效的对象跟踪算法,能够检测到当前帧给定区域的对象并跟踪,不需要事先了解此对象的空间－时域的信息,也不需要进行镜头运动补偿。静止分割分用Watershed算法,而运动矢量用块匹配法。尽管所要跟踪对象会经历某些形变,依旧能获得较好的结果。     

Robot-vision architecture for real-time 6-DOF object localization

This paper presents a new robot-vision system architecture for real-time moving object localization. The 6-DOF (3 translation and 3 rotation) motion of the objects is detected and tracked accurately in clutter using a model-based approach without information of the objects’ initial positions. An object identification task and an object tracking task are combined under this architecture. The computational time-lag between the two tasks is absorbed by a large amount of frame memory. The tasks are implemented as independent software modules using stereo-vision-based methods which can deal with objects of various shapes with edges, including planar to smooth-curved objects, in cluttered environments. This architecture also leads to failure-recoverable object tracking, because the tracking processes can be automatically recovered, even if the moving objects are lost while tracking. Experimental results obtained with prototype systems demonstrate the effectiveness of the proposed architecture.     

Morphing active contours

A method for deforming curves in a given image to a desired position in a second image is introduced. The algorithm is based on deforming the first image toward the second one via a partial differential equation (PDE), while tracking the deformation of the curves of interest in the first image with an additional, coupled PDE; both the images and the curves on the frame/slices of interest are used for tracking. The technique can be applied to object tracking and sequential segmentation. The topology of the deforming curve can change without any special topology handling procedures added to the scheme. This permits, for example, the automatic tracking of scenes where, due to occlusions, the topology of the objects of interest changes from frame to frame. In addition, this work introduces the concept of projecting velocities to obtain systems of coupled PDEs for image analysis applications. We show examples for object tracking and segmentation of electronic microscopy     

Multi-object detection and tracking by stereo vision

This paper presents a new stereo vision-based model for multi-object detection and tracking in surveillance systems. Unlike most existing monocular camera-based systems, a stereo vision system is constructed in our model to overcome the problems of illumination variation, shadow interference, and object occlusion. In each frame, a sparse set of feature points are identified in the camera coordinate system, and then projected to the 2D ground plane. A kernel-based clustering algorithm is proposed to group the projected points according to their height values and locations on the plane. By producing clusters, the number, position, and orientation of objects in the surveillance scene can be determined for online multi-object detection and tracking. Experiments on both indoor and outdoor applications with complex scenes show the advantages of the proposed system.     

监控系统中的多摄像机协同

描述了一个用于室内场合对多个目标进行跟踪的分布式监控系统.该系统由多个廉价的固定镜头的摄像机构成,具有多个摄像机处理模块和一个中央模块用于协调摄像机间的跟踪任务.由于每个运动目标有可能被多个摄像机同时跟踪,因此如何选择最合适的摄像机对某一目标跟踪,特别是在系统资源紧张时,成为一个问题.提出的新算法能根据目标与摄像机之间的距离并考虑到遮挡的情况,把目标分配给相应的摄像机,因此在遮挡出现时,系统能把遮挡的目标分配给能看见目标并距离最近的那个摄像机.实验表明该系统能协调好多个摄像机进行目标跟踪,并处理好遮挡问题.     

Backtracking: Retrospective multi-target tracking

We introduce a multi-target tracking algorithm that operates on prerecorded video as typically found in post-incident surveillance camera investigation. Apart from being robust to visual challenges such as occlusion and variation in camera view, our algorithm is also robust to temporal challenges, in particular unknown variation in frame rate. The complication with variation in frame rate is that it invalidates motion estimation. As such, tracking algorithms based on motion models will show decreased performance. On the other hand, appearance based detection in individual frames suffers from a plethora of false detections. Our tracking algorithm, albeit relying on appearance based detection, deals robustly with the caveats of both approaches. The solution rests on the fact that for prerecorded video we can make fully informed choices; not only based on preceding, but also based on following frames. We start off from an appearance based object detection algorithm able to detect in each frame all target objects. From this we build a graph structure. The detections form the graph’s nodes and the vertices are formed by connecting each detection in a frame to all detections in the following frame. Thus, each path through the graph shows some particular selection of successive detections. Tracking is then reformulated as a heuristic search for optimal paths, where optimal means to find all detections belonging to a single object and excluding any other detection. We show that this approach, without an explicit motion model, is robust to both the visual and temporal challenges.     

利用部分观测跟踪被遮挡的目标

提出了一个在单个固定摄像机下进行多目标跟踪的方法.利用亮度和色度混合模型和卡尔曼滤波器来检测跟踪目标,为了利于预测和解释被遮挡的物体,建立了场景的模型.在遮挡的情况下,和传统的盲跟踪不同,本文中的目标状态是由可用的部分观测来估计的.对目标的观测取决于预测、前景观测和场景模型.这使得本文算法在定性或定量的分析下都表现出更加鲁棒的性能.     

基于视觉的增强现实运动跟踪算法

增强现实系统不仅具有虚拟现实的特点同时具有虚实结合的新特性，为实现虚拟物体与真实物体间的完善结合，必须实时地动态跟踪摄像与真实物体间的相对位置和方向，建立观测模,墼是而通过动态三维显示技术迅速地将虚拟物体添加到真实物体之上，然而目前大多数增强现实系统的注册对象均匀静物体，运动物体的注册跟踪尚很少有人涉足。该算法通过标志点的光流场估计真实环境中运动物体的运动参数，根据透视投影原理和刚体的运动特性确定摄像机与运动物体间的相对位置和方向，实现增强现实系统的运动目标跟踪注册。该算法构架简单、实时性强，易于实现，扩展了增强现实系统的应用范围。     

基于场景模型的多摄像机目标跟踪算法

在多个相机组成的视频监视系统中,当目标物移出某一相机的视野而进入下一个时,如何实现相机的交接,实现目标物的继续跟踪是监视系统中要解决的关键问题。针对该问题,提出了一种基于位置比较的多摄像机运动目标跟踪方法。为获得目标物的位置,建立多个相机与目标物世界坐标之间映射关系的场景模型,并根据目标物出现在不同相机之间的视野边界线上的瞬间时刻的位置来给出重叠视野的边界线。由此可对任意角度摆放的多个具有重叠视野的相机之间运行的目标物进行接力跟踪。该方法可以适应多个目标物同时进入场景的情况,实验结果表明,该方法具有较高的鲁棒性,能够满足视频跟踪的实时性要求。