Reliability-based camera handoff for cooperative tracking with multiple pan-tilt cameras

We present a new handoff method for multiple pan-tilt cameras for mobile robot tracking in an indoor environment. Camera handoff is an important step to consistently maintain the visibility of a mobile robot with maximized object tracking accuracy. First, we propose a method to estimate the position of a mobile robot using single pan-tilt camera. Then, the concept of position reliability is defined to quantitatively evaluate the accuracy of position estimation and tracking ability of individual pan-tilt cameras. Position reliability is used to decide when to trigger handoff and who to response handoff in the proposed handoff algorithm. Experimental results demonstrate that four pan-tilt cameras can systematically track a mobile robot in an indoor environment using the proposed method.     

Camera handoff with adaptive resource management for multi-camera multi-object tracking

Camera handoff is a crucial step to obtain a continuously tracked and consistently labeled trajectory of the object of interest in multi-camera surveillance systems. Most existing camera handoff algorithms concentrate on data association, namely consistent labeling, where images of the same object are identified across different cameras. However, there exist many unsolved questions in developing an efficient camera handoff algorithm. In this paper, we first design a trackability measure to quantitatively evaluate the effectiveness of object tracking so that camera handoff can be triggered timely and the camera to which the object of interest is transferred can be selected optimally. Three components are considered: resolution, distance to the edge of the camera’s field of view (FOV), and occlusion. In addition, most existing real-time object tracking systems see a decrease in the frame rate as the number of tracked objects increases. To address this issue, our handoff algorithm employs an adaptive resource management mechanism to dynamically allocate cameras’ resources to multiple objects with different priorities so that the required minimum frame rate is maintained. Experimental results illustrate that the proposed camera handoff algorithm can achieve a substantially improved overall tracking rate by 20% in comparison with the algorithm presented by Khan and Shah.     

Towards automated visual surveillance using gait for identity recognition and tracking across multiple non-intersecting cameras

Despite the fact that personal privacy has become a major concern, surveillance technology is now becoming ubiquitous in modern society. This is mainly due to the increasing number of crimes as well as the essential necessity to provide secure and safer environment. Recent research studies have confirmed now the possibility of recognizing people by the way they walk i.e. gait. The aim of this research study is to investigate the use of gait for people detection as well as identification across different cameras. We present a new approach for people tracking and identification between different non-intersecting un-calibrated stationary cameras based on gait analysis. A vision-based markerless extraction method is being deployed for the derivation of gait kinematics as well as anthropometric measurements in order to produce a gait signature. The novelty of our approach is motivated by the recent research in biometrics and forensic analysis using gait. The experimental results affirmed the robustness of our approach to successfully detect walking people as well as its potency to extract gait features for different camera viewpoints achieving an identity recognition rate of 73.6 % processed for 2270 video sequences. Furthermore, experimental results confirmed the potential of the proposed method for identity tracking in real surveillance systems to recognize walking individuals across different views with an average recognition rate of 92.5 % for cross-camera matching for two different non-overlapping views.     

Photometric visual servoing for omnidirectional cameras

2D visual servoing consists in using data provided by a vision sensor for controlling the motions of a dynamic system. Most of visual servoing approaches has relied on the geometric features that have to be tracked and matched in the image acquired by the camera. Recent works have highlighted the interest of taking into account the photometric information of the entire image. This approach was tackled with images of perspective cameras. We propose, in this paper, to extend this technique to central cameras. This generalization allows to apply this kind of method to catadioptric cameras and wide field of view cameras. Several experiments have been successfully done with a fisheye camera in order to control a 6 degrees of freedom robot and with a catadioptric camera for a mobile robot navigation task.     

Principal axis-based correspondence between multiple cameras for people tracking

Visual surveillance using multiple cameras has attracted increasing interest in recent years. Correspondence between multiple cameras is one of the most important and basic problems which visual surveillance using multiple cameras brings. In this paper, we propose a simple and robust method, based on principal axes of people, to match people across multiple cameras. The correspondence likelihood reflecting the similarity of pairs of principal axes of people is constructed according to the relationship between "ground-points" of people detected in each camera view and the intersections of principal axes detected in different camera views and transformed to the same view. Our method has the following desirable properties; 1) camera calibration is not needed; 2) accurate motion detection and segmentation are less critical due to the robustness of the principal axis-based feature to noise; 3) based on the fused data derived from correspondence results, positions of people in each camera view can be accurately located even when the people are partially occluded in all views. The experimental results on several real video sequences from outdoor environments have demonstrated the effectiveness, efficiency, and robustness of our method.     

The invariant features-based target tracking across multiple cameras

Target tracking across lenses is a popular research topic for video surveillance recently. This paper presents a method of target tracking across lenses with overlap regions. First, the target detection and tracking are completed with a single camera. Second, in order to obtain the location-invariant feature of the same target in the images with various cameras, the camera calibration is completed based on a three-dimension (3D) model. After that, for all images via multiple cameras, the coordinates of the 3D model are unified. Finally, referring to the assumption of spatial and temporal consistency of the target location across multiple cameras, the association among detected objects for the same target with different cameras is established. And a feature pool is built which contains perspective and scale features. Thus the same target is continuously tracked across multiple lenses. At last, the performance of the proposed approach is compared with KSP and PABC and demonstrated with indoor and outdoor experiments.     

An accurate and robust visual-compass algorithm for robot-mounted omnidirectional cameras

Due to their wide field of view, omnidirectional cameras are becoming ubiquitous in many mobile robotic applications.  A challenging problem consists of using these sensors, mounted on mobile robotic platforms, as visual compasses (VCs) to provide an estimate of the rotational motion of the camera/robot from the omnidirectional video stream. Existing VC algorithms suffer from some practical limitations, since they require a precise knowledge either of the camera-calibration parameters, or the 3-D geometry of the observed scene. In this paper we present a novel multiple-view geometry constraint for paracatadioptric views of lines in 3-D, that we use to design a VC algorithm that does not require either the knowledge of the camera calibration parameters, or the 3-D scene geometry. In addition, our algorithm runs in real time since it relies on a closed-form estimate of the camera/robot rotation, and can address the image-feature correspondence problem. Extensive simulations and experiments with real robots have been performed to show the accuracy and robustness of the proposed method.     

Motion analysis for event detection and tracking with a mobile omnidirectional camera

A mobile platform mounted with omnidirectional vision sensor (ODVS) can be used to monitor large areas and detect interesting events such as independently moving persons and vehicles. To avoid false alarms due to extraneous features, the image motion induced by the moving platform should be compensated. This paper describes a formulation and application of parametric egomotion compensation for an ODVS. Omni images give 360 view of surroundings but undergo considerable image distortion. To account for these distortions, the parametric planar motion model is integrated with the transformations into omni image space. Prior knowledge of approximate camera calibration and camera speed is integrated with the estimation process using a Bayesian approach. Iterative, coarse-to-fine, gradient-based estimation is used to correct the motion parameters for vibrations and other inaccuracies in prior knowledge. Experiments with a camera mounted on various types of mobile platforms demonstrate successful detection of moving persons and vehicles.Published online: 11 October 2004     

Modeling and deadlock avoidance of automated manufacturing systems with multiple automated guided vehicles.

An automated manufacturing system (AMS) contains a number of versatile machines (or workstations), buffers, an automated material handling system (MHS), and is computer-controlled. An effective and flexible alternative for implementing MHS is to use automated guided vehicle (AGV) system. The deadlock issue in AMS is very important in its operation and has extensively been studied. The deadlock problems were separately treated for parts in production and transportation and many techniques were developed for each problem. However, such treatment does not take the advantage of the flexibility offered by multiple AGVs. In general, it is intractable to obtain maximally permissive control policy for either problem. Instead, this paper investigates these two problems in an integrated way. First we model an AGV system and part processing processes by resource-oriented Petri nets, respectively. Then the two models are integrated by using macro transitions. Based on the combined model, a novel control policy for deadlock avoidance is proposed. It is shown to be maximally permissive with computational complexity of O (n2) where n is the number of machines in AMS if the complexity for controlling the part transportation by AGVs is not considered. Thus, the complexity of deadlock avoidance for the whole system is bounded by the complexity in controlling the AGV system. An illustrative example shows its application and power.     

Particle filtering with multiple and heterogeneous cameras

This work proposes a novel particle filter for tracking multiple people using multiple and heterogeneous cameras, namely monocular and stereo cameras. Our approach is to define confidence models and observation models for each type of camera. Particles are evaluated independently in each camera, and then the data are fused in accordance with the confidence. Confidence models take into account several sources of information. On the one hand, they consider occlusion information from an occlusion map calculated using a depth-ordered particle evaluation. On the other hand, the relative precision of sensors is considered so that the contribution of a sensor in the final data fusion step is proportional to its precision. We have defined confidence and observation models for monocular and stereo cameras and have designed tests to validate our proposal. The experiments show that our method is able to operate with each type individually and in combination. Two other remarkable properties of our method are that it is highly parallelizable and that it does not impose restrictions on the cameras’ positions or orientations.     

Discrete-time tracking systems incorporating Lur'e plants with multiple non-linearities

The concept of absolute tracking (Gruji? and Porter 1980) is extended so as to embrace digital Lur'e control systems subjected to multiple unmeasurable disturbances and multiple unmeasurable command inputs. It is shown that the discrete-time tracking characteristics of closed-loop digital control systems incorporating Lur'e plants with multiple non-linearities can be established on the basis of the discrete-time absolute stability characteristics of associated free systems. These general results are illustrated by synthesizing a discrete-time tracking system incorporating a Lur'e plant with two non-linearities.     

Real-time predictive zoom tracking for digital still cameras

Zoom tracking is becoming a standard feature in digital still cameras (DSCs). It involves keeping an object of interest in focus during the zooming-in or zooming-out operation. Zoom tracking is normally achieved by moving the focus motor in real-time according to the so-called trace curves in response to changes in the zoom motor position. A trace curve denotes in-focus motor positions versus zoom motor positions for a specific object distance. A zoom tracking approach is characterized by the way these trace curves are estimated and followed. In this paper, a new zoom tracking approach, named predictive zoom tracking (PZT), is introduced based on two prediction models: auto-regressive and recurrent neural network. The performance of this approach is compared with the existing zoom tracking approaches commonly used in DSCs. The real-time implementation results obtained on an actual digital camera platform indicate that the developed PZT approach not only achieves higher tracking accuracies but also effectively addresses the key challenge of zoom tracking, namely the one-to-many mapping problem.

Fast-sampling tracking systems incorporating Lur'e plants with multiple non-linearities

The class of Lur'e plants with multiple non-linearities which are amenable to fast-sampling error-actuated digital control is characterized in terms of the properties of the transfer function matrices of the linear components of such plants. It is shown that. this characterization greatly facilitates the synthesis of closed-loop digital control systems incorporating such Lur'e plants which exhibit state-bounded absolutely stable tracking. These general results are illustrated by synthesizing a fast-sampling tracking system incorporating a Lur'e plant with two non-linearities.     

Observer-based optimal output tracking for discrete-time systems with multiple state and input delays

The optimal output tracking control (OOTC) problem for a class of discrete-time systems with state and input delays is addressed. An augmented system is constructed such that the OOTC problem can be transformed into a two-point boundary value (TPBV) problem with both advance and delay terms from the necessary optimality conditions. The successive approximating method recently developed is extended to obtain an approximate solution of the TPBV problem, which is then used to obtain a feedforward and feedback tracking controller. An observer is designed for the uncertain reference input such that the feedforward controller is physically realizable. Simulations show the results are effective even with long time-delays. Recommended by Editorial Board member Poo Gyeon Park under the direction of Editor Young Il Lee. This research was supported by the National Natural Science Foundation of China (Grant No. 40776051), the Key Natural Science Foundation of Shandong Province (Grant No. Z2005G01), the Natural Science Foundation of Qingdao City (Grant No. 05-1-JC-94) and the research funds of QingDao University of Science and Technology. Hai-Hong Wang received the Ph.D. degree in Computer Science in July 2007 from Ocean University of China. She presently works in QingDao University of Science and Technology, Qingdao, P.R. China. Her current research interests include analysis and control for time-delay systems and nonlinear systems. Gong-You Tang received the Ph.D. degree in Control Theory and Applications from the South China University of Technology, P. R. China in 1991. He is a Professor at the College of Information Science and Engineering at the Ocean University of China, Qingdao, P. R. China. He is the Editor of the Journal of the Ocean University of China and Control and the Instruments in Chemical Industry. His research interests are in the areas of nonlinear systems, delay systems, large-scale systems, and networked control systems, with emphasis in optimal control, robust control, fault diagnosis and stability analysis.     

Direct pole placement adaptive tracking of non-minimum phase systems

In this paper, we propose a direct pole placement adaptive tracking scheme for non-minimum-phase, open-loop stable, linear plants with time delays. This controller utilizes the internal model principle to eliminate steady-state tracking error for signals with known distinct frequencies. The controller order depends only on the number of frequencies in the reference input, but not on the order of the plant. It is shown that with sufficiently small loop gain, the controller can guarantee stable closed loop, and asymptotic tracking.     

Radiance-based color calibration for image-based modeling with multiple cameras

Photo-consistency estimation is an important part for many image-based modeling techniques.This paper presents a novel radiance-based color calibration method to reduce the uncertainty of photo-consistency estimation across multiple cameras.The idea behind our method is to convert colors into a uniform radiometric color space in which multiple image data are corrected.Experimental results demonstrate that our method can achieve comparable color calibration effect without adjusting camera parameters and is more robust than other existing method.Additionally,we obtain an auto-determined threshold for photo-consistency check,which will lead to a better performance than existing photo-consistency based reconstruction algorithms.     

相机运动条件下的行人检测与跟踪

行人检测与跟踪在司机辅助安全系统和视频监控等领域具有重要的地位.针对目前存在的关键问题,如人体运动,相机运动,背景及形状、角度等变化对检测及跟踪带来的干扰,提出了一种将运动信息与形状信息相结合的行人检测方法,准确检测运动摄像机拍摄的直立运动人体;使用了基于小面积目标的跟踪算法进行人体跟踪;利用实际拍摄的视频序列进行算法验证.实验结果表明,混合检测算法速度快,准确率高;基于小面积的跟踪算法能够鲁棒的跟踪检测到的运动人体.