Scanning and tracking with independent cameras—a biologically motivated approach based on model predictive control

This paper presents a framework for visual scanning and target tracking with a set of independent pan-tilt cameras. The approach is systematic and based on Model Predictive Control (MPC), and was inspired by our understanding of the chameleon visual system. We make use of the most advanced results in the MPC theory in order to design the scanning and tracking controllers. The scanning algorithm combines information about the environment and a model for the motion of the target to perform optimal scanning based on stochastic MPC. The target tracking controller is a switched control combining smooth pursuit and saccades. Min-Max and minimum-time MPC theory is used for the design of the tracking control laws. We make use of the observed chameleon’s behavior to guide the scanning and the tracking controller design procedures, the way they are combined together and their tuning. Finally, simulative and experimental validation of the approach on a robotic chameleon head composed of two independent Pan-Tilt cameras is presented.

Visual-model-based,real-time 3D pose tracking for autonomous navigation: methodology and experiments

This paper presents a novel 3D-model-based computer-vision method for tracking the full six degree-of-freedom (dof) pose (position and orientation) of a rigid body, in real-time. The methodology has been targeted for autonomous navigation tasks, such as interception of or rendezvous with mobile targets. Tracking an object’s complete six-dof pose makes the proposed algorithm useful even when targets are not restricted to planar motion (e.g., flying or rough-terrain navigation). Tracking is achieved via a combination of textured model projection and optical flow. The main contribution of our work is the novel combination of optical flow with z-buffer depth information that is produced during model projection. This allows us to achieve six-dof tracking with a single camera. A localized illumination normalization filter also has been developed in order to improve robustness to shading. Real-time operation is achieved using GPU-based filters and a new data-reduction algorithm based on colour-gradient redundancy, which was developed within the framework of our project. Colour-gradient redundancy is an important property of colour images, namely, that the gradients of all colour channels are generally aligned. Exploiting this property provides a threefold increase in speed. A processing rate of approximately 80 to 100 fps has been obtained in our work when utilizing synthetic and real target-motion sequences. Sub-pixel accuracies were obtained in tests performed under different lighting conditions.

Dynamic space reconfiguration for Bayesian search and tracking with moving targets

This paper presents a technique for dynamically reconfiguring search spaces in order to enable Bayesian autonomous search and tracking missions with moving targets. In particular, marine search and rescue scenarios are considered, highlighting the need for space reconfiguration in situations where moving targets are involved. The proposed technique improves the search space configuration by maintaining the validity of the recursive Bayesian estimation. The advantage of the technique is that autonomous search and tracking can be performed indefinitely, without loss of information. Numerical results first show the effectiveness of the technique with a single search vehicle and a single moving target. The efficacy of the approach for coordinated autonomous search and tracking is shown through simulation, incorporating multiple search vehicles and multiple targets. The examples also highlight the added benefit to human mission planners resulting from the technique’s simplification of the search space allocation task.

A unified learning framework for object detection and classification using nested cascades of boosted classifiers

In this paper a unified learning framework for object detection and classification using nested cascades of boosted classifiers is proposed. The most interesting aspect of this framework is the integration of powerful learning capabilities together with effective training procedures, which allows building detection and classification systems with high accuracy, robustness, processing speed, and training speed. The proposed framework allows us to build state of the art face detection, eyes detection, and gender classification systems. The performance of these systems is validated and analyzed using standard face databases (BioID, FERET and CMU-MIT), and a new face database (UCHFACE).

Real time 3D face and facial feature tracking

Detecting and tracking human faces in video sequences is useful in a number of applications such as gesture recognition and human-machine interaction. In this paper, we show that online appearance models (holistic approaches) can be used for simultaneously tracking the head, the lips, the eyebrows, and the eyelids in monocular video sequences. Unlike previous approaches to eyelid tracking, we show that the online appearance models can be used for this purpose. Neither color information nor intensity edges are used by our proposed approach. More precisely, we show how the classical appearance-based trackers can be upgraded in order to deal with fast eyelid movements. The proposed eyelid tracking is made robust by avoiding eye feature extraction. Experiments on real videos show the usefulness of the proposed tracking schemes as well as their enhancement to our previous approach.

Real-time predictive zoom tracking for digital still cameras

Zoom tracking is becoming a standard feature in digital still cameras (DSCs). It involves keeping an object of interest in focus during the zooming-in or zooming-out operation. Zoom tracking is normally achieved by moving the focus motor in real-time according to the so-called trace curves in response to changes in the zoom motor position. A trace curve denotes in-focus motor positions versus zoom motor positions for a specific object distance. A zoom tracking approach is characterized by the way these trace curves are estimated and followed. In this paper, a new zoom tracking approach, named predictive zoom tracking (PZT), is introduced based on two prediction models: auto-regressive and recurrent neural network. The performance of this approach is compared with the existing zoom tracking approaches commonly used in DSCs. The real-time implementation results obtained on an actual digital camera platform indicate that the developed PZT approach not only achieves higher tracking accuracies but also effectively addresses the key challenge of zoom tracking, namely the one-to-many mapping problem.

A New Model for Cloud Tracking and Analysis on Satellite Images

The study of convective clouds is an important issue in weather analysis. Previous methods are based on shape matching and level set. In this paper, a method based on snake model is used for cloud tracking. Snakes are known to be more efficient than level set for contour detection however they do not handle topological changes. Therefore, geometrical criteria are introduced to characterize topological transformations. Geometrical techniques are then combined and inserted in the tracking algorithm to perform morphological operations. By applying this method, a history of the positions of the clouds is obtained. In a second stage, a data model is presented for cloud interrogation. Physical information is introduced and data are organized so that spatiotemporal queries can be performed. Results obtained with the tracking method on a real data set are presented and some query examples are given.

Online updating appearance generative mixture model for meanshift tracking

This paper proposes an appearance generative mixture model based on key frames for meanshift tracking. Meanshift tracking algorithm tracks an object by maximizing the similarity between the histogram in tracking window and a static histogram acquired at the beginning of tracking. The tracking therefore could fail if the appearance of the object varies substantially. In this paper, we assume the key appearances of the object can be acquired before tracking and the manifold of the object appearance can be approximated by piece-wise linear combination of these key appearances in histogram space. The generative process is described by a Bayesian graphical model. An Online EM algorithm is proposed to estimate the model parameters from the observed histogram in the tracking window and to update the appearance histogram. We applied this approach to track human head motion and to infer the head pose simultaneously in videos. Experiments verify that our online histogram generative model constrained by key appearance histograms alleviates the drifting problem often encountered in tracking with online updating, that the enhanced meanshift algorithm is capable of tracking object of varying appearances more robustly and accurately, and that our tracking algorithm can infer additional information such as the object poses. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.

---

Segmentation of news videos based on audio-video information

In this paper, we propose an innovative architecture to segment a news video into the so-called “stories” by both using the included video and audio information. Segmentation of news into stories is one of the key issues for achieving efficient treatment of news-based digital libraries. While the relevance of this research problem is widely recognized in the scientific community, we are in presence of a few established solutions in the field. In our approach, the segmentation is performed in two steps: first, shots are classified by combining three different anchor shot detection algorithms using video information only. Then, the shot classification is improved by using a novel anchor shot detection method based on features extracted from the audio track. Tests on a large database confirm that the proposed system outperforms each single video-based method as well as their combination.

Marker-less registration based on template tracking for augmented reality

Accurate 3D registration is a key issue in the Augmented Reality (AR) applications, particularly where are no markers placed manually. In this paper, an efficient markerless registration algorithm is presented for both outdoor and indoor AR system. This algorithm first calculates the correspondences among frames using fixed region tracking, and then estimates the motion parameters on projective transformation following the homography of the tracked region. To achieve the illumination insensitive tracking, the illumination parameters are solved jointly with motion parameters in each step. Based on the perspective motion parameters of the tracked region, the 3D registration, the camera’s pose and position, can be calculated with calibrated intrinsic parameters. A marker-less AR system is described using this algorithm, and the system architecture and working flow are also proposed. Experimental results with comparison quantitatively demonstrate the correctness of the theoretical analysis and the robustness of the registration algorithm.