The Multi-strand Graph for a PTZ Tracker

High-resolution images can be used to resolve matching ambiguities between trajectory fragments (tracklets), which is a key challenge in multiple-target tracking. A pan–tilt–zoom (PTZ) camera, which can pan, tilt and zoom, is a powerful and efficient tool that offers both close-up views and wide area coverage on demand. The wide area enables tracking of many targets, while the close-up view allows individuals to be identified from high-resolution images of their faces. A central component of a PTZ tracking system is a scheduling algorithm that determines which target to zoom in on, particularly when the high-resolution images are also used for tracklet matching. In this paper, we study this scheduling problem from a theoretical perspective. We propose a novel data structure, the Multi-strand Tracking Graph (MSG), which represents the set of tracklets computed by a tracker and the possible associations between them. The MSG allows efficient scheduling as well as resolving of matching ambiguities between tracklets. The main feature of the MSG is the auxiliary data saved in each vertex, which allows efficient computation while avoiding time-consuming graph traversal. Synthetic data simulations are used to evaluate our scheduling algorithm and to demonstrate its superiority over a naïve one.     

An efficient continuous tracking system in real-time surveillance application

A tracking object must present a proper field of view (FOV) in a multiple active camera surveillance system; its clarity can facilitate smooth processing by the surveillance system before further processing, such as face recognition. However, when pan–tilt–zoom (PTZ) cameras are used, the tracking object can be brought into the FOV by adjusting its intrinsic parameters; consequently, selection of the best-performing camera is critical. Performance is determined by the relative positions of the camera and the tracking objects, image quality, lighting and how much of the front side of the object faces the camera. In a multi-camera surveillance system, both camera hand-off and camera assignment play an important role in automated and persistent tracking, which are typical surveillance requirements. This study investigates the use of automatic methods for tracking an object across cameras in a surveillance network using PTZ cameras. An automatic, efficient continuous tracking scheme is developed. The goal is to determine the decision criteria for hand-off using Sight Quality Indication (SQI) (which includes information on the position of the object and the proportion of the front of object faces the camera), and to perform the camera handoff task in a manner that optimizes the vision effect associated with monitoring. Experimental results reveal that the proposed algorithm can be efficiently executed, and the handoff method for feasible and continuously tracking active objects under real-time surveillance.     

Continuous localization and mapping of a pan–tilt–zoom camera for wide area tracking

Pan–tilt–zoom (PTZ) cameras are well suited for object identification and recognition in far-field scenes. However, the effective use of PTZ cameras is complicated by the fact that a continuous online camera calibration is needed and the absolute pan, tilt and zoom values provided by the camera actuators cannot be used because they are not synchronized with the video stream. So, accurate calibration must be directly extracted from the visual content of the frames. Moreover, the large and abrupt scale changes, the scene background changes due to the camera operation and the need of camera motion compensation make target tracking with these cameras extremely challenging. In this paper, we present a solution that provides continuous online calibration of PTZ cameras which is robust to rapid camera motion, changes of the environment due to varying illumination or moving objects. The approach also scales beyond thousands of scene landmarks extracted with the SURF keypoint detector. The method directly derives the relationship between the position of a target in the ground plane and the corresponding scale and position in the image and allows real-time tracking of multiple targets with high and stable degree of accuracy even at far distances and any zoom level.     

Panoramic Gaussian Mixture Model and large-scale range background substraction method for PTZ camera-based surveillance systems

In this paper, we present a novel approach for constructing a large-scale range panoramic background model that provides fast registration of the observed frame and localizes the foreground targets with arbitrary camera direction and scale in a Pan–tilt–zoom (PTZ) camera-based surveillance system. Our method consists of three stages. (1) In the first stage, a panoramic Gaussian mixture model (PGMM) of the PTZ camera’s field of view is generated off-line for later use in on-line foreground detection. (2) In the second stage, a multi-layered correspondence ensemble is generated off-line from frames captured at different scales which is used by the correspondence propagation method to register observed frames online to the PGMM. (3) In the third stage, foreground is detected and the PGMM is updated. The proposed method has the capacity to deal with the PTZ camera’s ability to cover a wide field of view (FOV) and large-scale range. We demonstrate the advantages of the proposed PGMM background subtraction method by incorporating it with a tracking system for surveillance applications.     

A Catadioptric and Pan-Tilt-Zoom Camera Pair Object Tracking System for UAVs

Unmanned aerial vehicles (UAVs) are seeing widespread use in military, scientific, and civilian sectors in recent years. As the mission demands increase, these systems are becoming more complicated. Omnidirectional camera is a vision sensor that can captures 360° view in a single frame. In recent years omnidirectional camera usage has experienced a remarkable increase in many fields, where many innovative research has been done. Although, it is very promising, employment of omnidirectional cameras in UAVs is quite new. In this paper, an innovative sensory system is proposed, that has an omnidirectional imaging device and a pan tilt zoom (PTZ) camera. Such a system combines the advantages of both of the camera systems. The system can track any moving object within its 360° field of view and provide detailed images of it. The detection of the moving object has been accomplished by an adaptive background subtraction method implemented on the lowered resolution images of the catadioptric camera. A novel algorithm has also been developed to estimate the relative distance of the object with respect to the UAV, using tracking information of both of the cameras. The algorithms are implemented on an experimental system to validate the approach.     

An adaptive focus-of-attention model for video surveillance and monitoring

In current video surveillance systems, commercial pan/tilt/zoom (PTZ) cameras typically provide naive (or no) automatic scanning functionality to move a camera across its complete viewable field. However, the lack of scene-specific information inherently handicaps these scanning algorithms. We address this issue by automatically building an adaptive, focus-of-attention, scene-specific model using standard PTZ camera hardware. The adaptive model is constructed by first detecting local human activity (i.e., any translating object with a specific temporal signature) at discrete locations across a PTZ camera’s entire viewable field. The temporal signature of translating objects is extracted using motion history images (MHIs) and an original, efficient algorithm based on an iterative candidacy-classification-reduction process to separate the target motion from noise. The target motion at each location is then quantified and employed in the construction of a global activity map for the camera. We additionally present four new camera scanning algorithms which exploit this activity map to maximize a PTZ camera’s opportunity of observing human activity within the camera’s overall field of view. We expect that these efficient and effective algorithms are implementable within current commercial camera systems.     

Exploiting distinctive visual landmark maps in pan–tilt–zoom camera networks

Pan–tilt–zoom (PTZ) camera networks have an important role in surveillance systems. They have the ability to direct the attention to interesting events that occur in the scene. One method to achieve such behavior is to use a process known as sensor slaving: one (or more) master camera monitors a wide area and tracks moving targets so as to provide the positional information to one (or more) slave camera. The slave camera can thus point towards the targets in high resolution.In this paper we describe a novel framework exploiting a PTZ camera network to achieve high accuracy in the task of relating the feet position of a person in the image of the master camera, to his head position in the image of the slave camera. Each camera in the network can act as a master or slave camera, thus allowing the coverage of wide and geometrically complex areas with a relatively small number of sensors.The proposed framework does not require any 3D known location to be specified, and allows to take into account both zooming and target uncertainties. Quantitative results show good performance in target head localization, independently from the zooming factor in the slave camera. An example of cooperative tracking approach exploiting with the proposed framework is also presented.     

Optimizing PTZ camera calibration from two images

In this paper, we address the problem of calibrating an active pan–tilt–zoom (PTZ) camera. In this regard, we make three main contributions: first, for the general camera rotation, we provide a novel solution that yields four independent constraints from only two images, by directly decomposing the infinite homography using a series of Givens rotations. Second, for a camera varying its focal length, we present a solution for the degenerate cases of pure pan and pure tilt that occur very frequently in practical applications of PTZ cameras. Third, we derive a new optimized error function for pure rotation or pan–tilt rotation, which plays a similar role as the epipolar constraint in a freely moving camera, in terms of characterizing the reprojection error of point correspondences. Our solutions and analysis are thoroughly validated and tested on both synthetic and real data, whereby the new geometric error function is shown to outperform existing methods in terms of accuracy and noise resilience.     

Dynamic Calibration of Pan–Tilt–Zoom Cameras for Traffic Monitoring

Pan–tilt–zoom (PTZ) cameras have been widely used in recent years for monitoring and surveillance applications. These cameras provide flexible view selection as well as a wider observation range. This makes them suitable for vision-based traffic monitoring and enforcement systems. To employ PTZ cameras for image measurement applications, one first needs to calibrate the camera to obtain meaningful results. For instance, the accuracy of estimating vehicle speed depends on the accuracy of camera calibration and that of vehicle tracking results. This paper presents a novel calibration method for a PTZ camera overlooking a traffic scene. The proposed approach requires no manual operation to select the positions of special features. It automatically uses a set of parallel lane markings and the lane width to compute the camera parameters, namely, focal length, tilt angle, and pan angle. Image processing procedures have been developed for automatically finding parallel lane markings. Interesting experimental results are presented to validate the robustness and accuracy of the proposed method.     

Tracking a person with pre-recorded image database and a pan, tilt, and zoom camera

Abstract. This paper proposes a novel tracking strategy that can robustly track a person or other object within a fixed environment using a pan, tilt, and zoom camera with the help of a pre-recorded image database. We define a set of camera states which is sufficient to survey the environment for the target. Background images for these camera states are stored as an image database. During tracking, camera movements are restricted to these states. Tracking and segmentation are simplified, as each tracking image can be compared with the corresponding pre-recorded background image. Received: 26 August 1999 / Accepted: 22 February 2000     

双摄像机协同人脸鹰眼检测与定位方法

现代视频监控系统需要获取大范围场景中感兴趣目标的清晰图像,这在目标距离较远并且不断移动时单纯采用摄像机调焦方式通常有一定的困难。为了获取宽范围监控场景中远距离行人的主要面部特征,采用广角静止—窄视场运动双摄像机协同工作方式可以同时获得远距离目标的全局和细节信息。首先采用改进的Codebook背景减法从广角摄像机中检测运动目标,然后指引运动摄像机近距离跟踪观察;若行人停止运动,则利用运动摄像机对其进行放大,然后从中检测人脸,并将人脸置于视野中心放大得到清晰图像。当行人再次运动时,广角相机将初始位置再次传递给运动摄像机,由其再对行人进行跟踪。通过实验室内和室外真实场景的实验表明,广角相机的检测算法具有一定的鲁棒性,运动相机能跟踪放大行人人脸图像,算法运行速度满足实时性要求。     

一种用于视频监控的双摄像头系统实现

提出了一种新颖的用于视频监控的双摄像头系统,在此系统中全景摄像机与PTZ摄像机(云台摄像机)结合在一起,既能对大范围内的目标进行检测与跟踪又能对目标的详细图像进行捕捉。在全景摄像机获取的图像中进行运动检测,获取运动物体位置信息后利用PTZ摄像机对其进行检测分析,以实现二者的数据融合。设计了全景摄像机的反射镜面,对该双摄像头系统进行了标定,在实验室环境下的进行实验验证了系统的性能。     

3D Target Scale Estimation and Target Feature Separation for Size Preserving Tracking in PTZ Video

To achieve size preserving tracking, in addition to controlling the camera’s pan and tilt motions to keep the object of interest in the camera’s field of view (FOV), the camera’s focal length is adjusted automatically to compensate for the changes in the target’s image size caused by the relative motion between the camera and the target. The estimation accuracy of these changes determines the effectiveness of the resulting zoom control. The existing method of choice for real-time target scale estimation applies structure from motion (SFM) based on the weak perspective projection model. In this paper we propose a target scale estimation algorithm with a linear solution based on the more advanced paraperspective projection model, which improves the accuracy of scale estimation by considering center offset. Another key issue in SFM based algorithms is the separation of target and background features, especially when composite camera (pan/tilt/zoom) and target motions are involved. This paper designs a fast target feature separation/grouping algorithm, the 3D affine shape method. The resulting separation automatically adapts to the target’s 3D geometry and motion and is able to accommodate a large amount of off-plane rotation, which most existing separation/grouping algorithms find difficult to achieve. Experimental results illustrate the effectiveness of the proposed scale estimation and feature separation algorithms in tracking translating and rotating objects with a PTZ camera while preserving their sizes. In comparison with the leading size preserving tracking algorithm described by Tordoff and Murray, our algorithm is able to reduce the cumulative tracking error significantly from 17.4% to 3.3%.     

PTZ摄像机跟踪运动目标的智能控制算法的研究

针对传统的PTZ摄像机跟踪运动目标时依靠人工操作,无法连续、实时动态跟踪,甚至导致跟踪失败的缺点,提出以HSV颜色直方图作为模型特征,通过Camshift算法和卡尔曼滤波器实现运动目标的定位和预测补偿,运用闭环控制机制自动调节云台的转动和镜头的变倍,提高了系统的实时性。通过Android智能手机手动调节云台和镜头,配合自动跟踪系统,使跟踪效果更准确。结果表明:该方法是可行的,具有控制简单、定位准确的优点,能提高目标跟踪的实时性和可靠性。     

Dynamic 3D surface reconstruction and motion modeling from a pan–tilt–zoom camera

3D surface reconstruction and motion modeling has been integrated in several industrial applications. Using a pan–tilt–zoom (PTZ) camera, we present an efficient method called dynamic 3D reconstruction (D3DR) for recovering the 3D motion and structure of a freely moving target. The proposed method estimates the PTZ measurements to keep the target in the center of the field of view (FoV) of the camera with the same size. Feature extraction and tracking approach are used in the imaging framework to estimate the target's translation, position, and distance. A selection strategy is used to select keyframes that show significant changes in target movement and directly update the recovered 3D information. The proposed D3DR method is designed to work in a real-time environment, not requiring all frames captured to be used to update the recovered 3D motion and structure of the target. Using fewer frames minimizes the time and space complexity required. Experimental results conducted on real-time video streams using different targets to prove the efficiency of the proposed method. The proposed D3DR has been compared to existing offline and online 3D reconstruction methods, showing that it uses less execution time than the offline method and uses an average of 49.6% of the total number of frames captured.     

Robust tracking via monocular active vision for an intelligent teaching system

The research of this paper investigates a practical intelligent tracking teaching system, addressing the problem of teacher detection and tracking via monocular active vision in real time. The split lines and position-based visual servo rules are created to realize the robust and stable tracking, which is designed to keep the tracked teacher in the middle of image with a fixed size by automatically controlling a pan/tilt/zoom monocular camera in either rostrum region or other regions in the classroom. Face tracking in rostrum region is initiated by a face detector based on Adaboost followed by a novel long-term tracking algorithm named as informative random fern-tracking-learning-detection (IRF-TLD), which has advantages for its high accuracy and low memory requirement using real-valued feature and Gaussian random projection. Moreover, Gaussian mixture model can be automatically started to detect the teacher’s movement when face tracking fails or stand-up students are detected. Experimental results on many benchmark sequences, which include various challenges for tracking, such as occlusion, illumination and pose variations, and scaling, have demonstrated the superior performance of the proposed IRF-TLD method when compared with several state-of-the-art tracking algorithms. Extensive experiments in a series of challenging real classroom scenarios also demonstrate the effectiveness of the complete system.     

基于全景视觉的监控系统设计

设计了一个基于全景视觉的多摄像机监控网络。全景相机视野广,可以实现大范围的目标检测与跟踪。云台摄像机视角具有一定的自由度,可以捕捉目标的高分辨率图像。将全景相机与云台相机相互配合,通过多传感器的数据融合,分层次的跟踪算法及多相机调度算法,实现了大范围的多个运动目标的检测与跟踪,并能捕获目标的清晰图像。实验验证了该系统的有效性和合理性。     

A multiobject tracking framework for interactive multimedia applications

Automatic initialization and tracking of multiple people and their body parts is one of the first steps in designing interactive multimedia applications. The key problems in this context are robust detection and tracking of people and their body parts in an unconstrained environment. This paper presents an integrated framework to address detection and tracking of multiple objects in a computationally efficient manner. In particular, a neural network-based face detector was employed to detect faces and compute person specific statistical model for skin color from the face regions. A probabilistic model was proposed to fuse the color and motion information to localize the moving body parts (hands). Multiple hypothesis tracking (MHT) algorithm was adopted to track face and hands. In real world scenes extracted features (face and hands) usually contain spurious measurements that create unconvincing trajectories and needless computations. To deal with this problem a path coherence function was incorporated along with MHT to reduce the number of hypotheses, which in turn reduces the computational cost and improves the structure of trajectories. The performance of the framework was validated using experiments on synthetic and real sequence of images.     

基于DSP的视频人脸监控系统设计与实现

针对目前人脸监控系统存在不能及时决策、盲目跟踪的问题,提出综合人脸检测、识别、跟踪与控制的智能监控系统,并在TMS320DM642芯片上实现;使用Adaboost级联多层分类器检测人脸,基于Gabor小波图像相似度匹配识别人脸,对识别出的感兴趣人脸用Camshift算法实现跟踪,再通过串口驱动云台带动摄像头旋转,使感兴趣人脸保持在视场内并且在模拟显示器上显示;实验结果表明,本系统满足实时视频监控要求,对人脸旋转、遮挡、光照鲁棒性高;本系统可以推广至视频会议、视频聊天、可视电话等领域,有良好的应用前景。     

高速球形摄像机PTZ跟踪策略设计及实现

针对智能监控中基于高速球形摄像机的PTZ跟踪功能模块,设计了一种PTZ跟踪控制策略。该策略在球机机械参数未知的情况下,一方面能控制球机实时地跟踪目标使目标始终处于视野中央,另一方面可自动进行变倍动作来放大拍摄目标的局部细节。针对球机Zoom控制中跟踪窗口大小自适应调整的问题,利用SIFT算法设计了一种计算球机变倍率的方法。利用VS2005和OpenCV软件平台实现了PTZ跟踪的整体流程。实验表明,该策略能有效、稳定地进行PTZ跟踪。