3D head tracking for fall detection using a single calibrated camera

The head trajectory is an interesting source of information for behavior recognition and can be very useful for video surveillance applications, especially for fall detection. Consequently, much work has been done to track the head in the 2D image plane using a single camera or in a 3D world using multiple cameras. Tracking the head in real-time with a single camera could be very useful for fall detection. Thus, in this article, an original method to extract the 3D head trajectory of a person in a room is proposed using only one calibrated camera. The head is represented as a 3D ellipsoid, which is tracked with a hierarchical particle filter based on color histograms and shape information. Experiments demonstrated that this method can run in quasi-real-time, providing reasonable 3D errors for a monocular system. Results on fall detection using the head 3D vertical velocity or height obtained from the 3D trajectory are also presented.     

View-projection animation for 3D occlusion management

Inter-object occlusion is inherent to 3D environments and is one of the challenges of using 3D instead of 2D computer graphics for visualization. Based on an analysis of this effect, we present an interaction technique for view-projection animation that reduces inter-object occlusion in 3D environments without modifying the geometrical properties of the objects themselves. The technique allows for smooth on-demand animation between parallel and perspective projection modes as well as online manipulation of view parameters, enabling the user to quickly and easily adapt the view to reduce occlusion. A user study indicates that the technique provides many of the occlusion reduction benefits of traditional camera movement, but without the need to actually change the viewpoint. We have also implemented a prototype of the technique in the Blender 3D modeler.     

Tracking multiple people with recovery from partial and total occlusion

Robust tracking of multiple people in video sequences is a challenging task. In this paper, we present an algorithm for tracking faces of multiple people even in cases of total occlusion. Faces are detected first; then a model for each person is built. The models are handed over to the tracking module which is based on the mean shift algorithm, where each face is represented by the non-parametric distribution of the colors in the face region. The mean shift tracking algorithm is robust to partial occlusion and rotation, and is computationally efficient, but it does not deal with the problem of total occlusion. Our algorithm overcomes this problem by detecting the occlusion using an occlusion grid, and uses a non-parametric distribution of the color of the occluded person's cloth to distinguish that person after the occlusion ends. Our algorithm uses the speed and the trajectory of each occluded person to predict the locations that should be searched after occlusion ends. It integrates multiple features to handle tracking multiple people in cases of partial and total occlusion. Experiments on a large set of video clips demonstrate the robustness of the algorithm, and its capability to correctly track multiple people even when faces are temporarily occluded by other faces or by other objects in the scene.     

Robust facial feature tracking under varying face pose and facial expression

This paper presents a hierarchical multi-state pose-dependent approach for facial feature detection and tracking under varying facial expression and face pose. For effective and efficient representation of feature points, a hybrid representation that integrates Gabor wavelets and gray-level profiles is proposed. To model the spatial relations among feature points, a hierarchical statistical face shape model is proposed to characterize both the global shape of human face and the local structural details of each facial component. Furthermore, multi-state local shape models are introduced to deal with shape variations of some facial components under different facial expressions. During detection and tracking, both facial component states and feature point positions, constrained by the hierarchical face shape model, are dynamically estimated using a switching hypothesized measurements (SHM) model. Experimental results demonstrate that the proposed method accurately and robustly tracks facial features in real time under different facial expressions and face poses.     

Conics-enhanced vision approach for easy and low-cost 3D tracking

This paper presents a conics-enhanced vision approach for low-cost and easy-to-operate 3D tracking. The idea is to use paper disks as markers and recover the 3D positions of these paper disks by a property of conics: the 3D rotation and translation information of a planar circle with known radius could be recovered from its elliptic projection in one image (Int. J. Comput. Vision 10(1) (1993) 7). This property implies that tracking a paper disk in 3D could be done by tracking its elliptic projection in a sequence of 2D images. Since ellipse tracking has to consider many factors such as occlusion, background, light and so on, it is difficult to develop a general algorithm that works for all situations. In this paper, we discuss algorithms for two types of ellipse tracking: real-time tracking of single, non-occluded ellipse and off-line tracking of multiple ellipses. They are used to develop a real-time tracker to control the camera movement in a virtual environment and a multiple-tracker system that works off-line to acquire human motion data to animate a 3D human model. Implementation details and experimental results of the tracking systems are presented. The proposed 3D tracking approach is valuable for applications where accuracy and speed are not very critical but affordability and operational ease are most concerned, e.g., the educational-purpose virtual environments for kids.     

Multiple human tracking in high-density crowds

In this paper, we introduce a fully automatic algorithm to detect and track multiple humans in high-density crowds in the presence of extreme occlusion. Typical approaches such as background modeling and body part-based pedestrian detection fail when most of the scene is in motion and most body parts of most of the pedestrians are occluded. To overcome this problem, we integrate human detection and tracking into a single framework and introduce a confirmation-by-classification method for tracking that associates detections with tracks, tracks humans through occlusions, and eliminates false positive tracks. We use a Viola and Jones AdaBoost detection cascade, a particle filter for tracking, and color histograms for appearance modeling. To further reduce false detections due to dense features and shadows, we introduce a method for estimation and utilization of a 3D head plane that reduces false positives while preserving high detection rates. The algorithm learns the head plane from observations of human heads incrementally, without any a priori extrinsic camera calibration information, and only begins to utilize the head plane once confidence in the parameter estimates is sufficiently high. In an experimental evaluation, we show that confirmation-by-classification and head plane estimation together enable the construction of an excellent pedestrian tracker for dense crowds.     

Segmenting and tracking multiple objects under occlusion using multi-label graph cut

This paper proposes a new method to segment and track multiple objects through occlusion by integrating spatial-color Gaussian mixture model (SCGMM) into an energy minimization framework. When occlusion does not occur, a SCGMM is learned for each object. When the objects are subject to occlusion, energy minimization is used to segment the objects from occlusion. To make the learned SCGMMs suitable for the segmentation of the current occlusion, a displacing procedure is utilized to adapt the SCGMMs to the spatial variations. A multi-label energy function is formulated building on the displaced SCGMMs and then minimized using the multi-label graph cut algorithm, thus leading to both the segmentation and tracking results of the objects with occlusion. Experimental validation of the proposed method is performed and presented on several video sequences.     

Viewpoint invariant exemplar-based 3D human tracking

This paper proposes a clustered exemplar-based model for performing viewpoint invariant tracking of the 3D motion of a human subject from a single camera. Each exemplar is associated with multiple view visual information of a person and the corresponding 3D skeletal pose. The visual information takes the form of contours obtained from different viewpoints around the subject. The inclusion of multi-view information is important for two reasons: viewpoint invariance; and generalisation to novel motions. Visual tracking of human motion is performed using a particle filter coupled to the dynamics of human movement represented by the exemplar-based model. Dynamics are modelled by clustering 3D skeletal motions with similar movement and encoding the flow both within and between clusters. Results of single view tracking demonstrate that the exemplar-based models incorporating dynamics generalise to viewpoint invariant tracking of novel movements.     

2D human body tracking with Structural Kalman filter

Tracking moving objects is one of the most important techniques in motion analysis and understanding, and it has many difficult problems to solve. Especially, estimating and identifying moving objects, when the background and moving objects vary dynamically, are very difficult. It is possible under such a complex environment that targets may disappear totally or partially due to occlusion by other objects. The Kalman filter has been used to estimate motion information and use the information in predicting the appearance of targets in the succeeding frames. In this paper, we propose another version of the Kalman filter, to be called Structural Kalman filter, which can successfully work its role of estimating motion information under such a deteriorating condition as occlusion. Experimental results show that the suggested approach is very effective in estimating and tracking non-rigid moving objects reliably.     

Efficient generic face model fitting to images and videos

In this paper we present a robust and lightweight method for the automatic fitting of deformable 3D face models on facial images. Popular fitting techniques such as those based on statistical models of shape and appearance require a training stage based on a set of facial images and their corresponding facial landmarks, which have to be manually labeled. Therefore, new images in which to fit the model cannot differ too much in shape and appearance (including illumination variation, facial hair, wrinkles, etc.) from those used for training. By contrast, our approach can fit a generic face model in two steps: (1) the detection of facial features based on local image gradient analysis and (2) the backprojection of a deformable 3D face model through the optimization of its deformation parameters. The proposed approach can retain the advantages of both learning-free and learning-based approaches. Thus, we can estimate the position, orientation, shape and actions of faces, and initialize user-specific face tracking approaches, such as Online Appearance Models (OAMs), which have shown to be more robust than generic user tracking approaches. Experimental results show that our method outperforms other fitting alternatives under challenging illumination conditions and with a computational cost that allows its implementation in devices with low hardware specifications, such as smartphones and tablets. Our proposed approach lends itself nicely to many frameworks addressing semantic inference in face images and videos.     

信息视频图像人脸检测算法研究

本文首先采用运动信息检测算法,根据帧图像中是否包含运动信息判断图像中是否包含人脸区域,决定是否对该帧进行肤色分割,然后依据非线性的YCbCr肤色模型对需要检测的帧进行分割,进一步确定人脸区域的大致位置,并进行了仿真实验,实验结果表明该算法能够对视频图像序列中的人脸进行检测,具有良好的性能和一定的研究参考价值。     

Pose-invariant face recognition using a 3D deformable model

The paper proposes a novel, pose-invariant face recognition system based on a deformable, generic 3D face model, that is a composite of: (1) an edge model, (2) a color region model and (3) a wireframe model for jointly describing the shape and important features of the face. The first two submodels are used for image analysis and the third mainly for face synthesis. In order to match the model to face images in arbitrary poses, the 3D model can be projected onto different 2D viewplanes based on rotation, translation and scale parameters, thereby generating multiple face-image templates (in different sizes and orientations). Face shape variations among people are taken into account by the deformation parameters of the model. Given an unknown face, its pose is estimated by model matching and the system synthesizes face images of known subjects in the same pose. The face is then classified as the subject whose synthesized image is most similar. The synthesized images are generated using a 3D face representation scheme which encodes the 3D shape and texture characteristics of the faces. This face representation is automatically derived from training face images of the subject. Experimental results show that the method is capable of determining pose and recognizing faces accurately over a wide range of poses and with naturally varying lighting conditions. Recognition rates of 92.3% have been achieved by the method with 10 training face images per person.     

基于三维模型的人脸姿态估计

提出了一种基于三维模型的人脸姿态估计方法。首先根据人脸特征点重建出稀疏的三维人脸模型,然后基于三维模型采用线性回归的方法对人脸姿态进行初步估计,确定姿态范围,再对估计结果进行修正,从而对人脸姿态进行精确估计。实验表明,该方法具有较好的估计效果,提高了姿态估计精度。     

Non-rigid 3D shape tracking from multiview video

We present a fast and efficient non-rigid shape tracking method for modeling dynamic 3D objects from multiview video. Starting from an initial mesh representation, the shape of a dynamic object is tracked over time, both in geometry and topology, based on multiview silhouette and 3D scene flow information. The mesh representation of each frame is obtained by deforming the mesh representation of the previous frame towards the optimal surface defined by the time-varying multiview silhouette information with the aid of 3D scene flow vectors. The whole time-varying shape is then represented as a mesh sequence which can efficiently be encoded in terms of restructuring and topological operations, and small-scale vertex displacements along with the initial model. The proposed method has the ability to deal with dynamic objects that may undergo non-rigid transformations and topological changes. The time-varying mesh representations of such non-rigid shapes, which are not necessarily of fixed connectivity, can successfully be tracked thanks to restructuring and topological operations employed in our deformation scheme. We demonstrate the performance of the proposed method both on real and synthetic sequences.     

Real time 3D face and facial feature tracking

Detecting and tracking human faces in video sequences is useful in a number of applications such as gesture recognition and human-machine interaction. In this paper, we show that online appearance models (holistic approaches) can be used for simultaneously tracking the head, the lips, the eyebrows, and the eyelids in monocular video sequences. Unlike previous approaches to eyelid tracking, we show that the online appearance models can be used for this purpose. Neither color information nor intensity edges are used by our proposed approach. More precisely, we show how the classical appearance-based trackers can be upgraded in order to deal with fast eyelid movements. The proposed eyelid tracking is made robust by avoiding eye feature extraction. Experiments on real videos show the usefulness of the proposed tracking schemes as well as their enhancement to our previous approach.

Probabilistic people tracking with appearance models and occlusion classification: The AD-HOC system

AD-HOC (Appearance Driven Human tracking with Occlusion Classification) is a complete framework for multiple people tracking in video surveillance applications in presence of large occlusions. The appearance-based approach allows the estimation of the pixel-wise shape of each tracked person even during the occlusion. This peculiarity can be very useful for higher level processes, such as action recognition or event detection. A first step predicts the position of all the objects in the new frame while a MAP framework provides a solution for best placement. A second step associates each candidate foreground pixel to an object according to mutual object position and color similarity. A novel definition of non-visible regions accounts for the parts of the objects that are not detected in the current frame, classifying them as dynamic, scene or apparent occlusions. Results on surveillance videos are reported, using in-house produced videos and the PETS2006 test set.     

Foveated active tracking with redundant 2D motion parameters

This work presents a real-time active vision tracking system based on log-polar image motion estimation with 2D geometric deformation models. We present a very efficient parametric motion estimation method, where most computation can be done offline. We propose a redundant parameterization for the geometric deformations, which improve the convergence range of the algorithm. A foveated image representation provides extra computational savings and attenuation of background effects. A proper choice of motion models and a hierarchical organization of the iterations provide additional robustness. We present a fully integrated system with real-time performance and robustness to moderate deviations from the assumed deformation models.     

Specific person tracking using 3D LIDAR and ESPAR antenna for mobile service robots

The ability of detecting and following a specific person is indispensable for mobile service robots. Many image-based methods have been proposed for person detection and identification; however, they are sometimes vulnerable to illumination changes. This paper therefore proposes a novel approach to the problem, namely, using 3D LIDARs for person detection and identification and a directivity-controllable antenna (called ESPAR antenna) for localizing a specific person even under long-term occlusion and/or out-of-view situations. A sensor fusion framework, combined with an adaptive state-based strategy switching, has also been developed for achieving a reliable person following. Experimental results in actual outdoor environments show the effectiveness of the proposed framework.     

Real-time face tracking and recognition by sparse eigentracker with associative mapping to 3D shape

This paper proposes a novel framework of real-time face tracking and recognition by combining two eigen-based methods. The first method is a novel extension of eigenface called augmented eigenface and the second method is a sparse 3D eigentemplate tracker controlled by a particle filter. The augmented eigenface is an eigenface augmented by an associative mapping to 3D shape that is specified by a set of volumetric face models. This paper discusses how to make up the augmented eigenface and how it can be used for inference of 3D shape from partial images. The associative mapping is also generalized to subspace-to-one mappings to cover photometric image changes for a fixed shape. A novel technique, called photometric adjustment, is introduced for simple implementation of associative mapping when an image subspace should be combined to a shape. The sparse 3D eigentemplate tracker is an extension of the 3D template tracker proposed by Oka et al. In combination with the augmented eigenface, the sparse 3D eigentemplate tracker facilitates real-time 3D tracking and recognition when a monocular image sequence is provided. In the tracking, sparse 3D eigentemplate is updated by the augmented eigenface while face pose is estimated by the sparse eigentracker. Since the augmented eigenface is constructed on the conventional eigenfaces, face identification and expression recognition are also accomplished efficiently during the tracking. In the experiment, an augmented eigenface was constructed from 25 faces where 24 images were taken in different lighting conditions for each face. Experimental results show that the augmented eigenface works with the 3D eigentemplate tracker for real-time tracking and recognition.     

基于边的自适应实时三维跟踪

针对缺乏纹理特征的物体,提出了一种基于边的自适应实时三维跟踪方法。在已知物体三维模型的情况下,通过基于历史运动信息的物体边缘检测与跟踪,可以有效准确地求解出摄像机的外参。基于并扩展了现有的基于边的实时跟踪算法,其主要工作体现在以下三个方面： 1）提出自适应阈值和基于历史信息估计当前帧的运动趋势的方法,从而提高边匹配算法在快速运动时的稳定性;2）提出一种基于随机抽样一致性（RANSAC）的边匹配策略,可以有效剔除误匹配的边,从而提高复杂模型的跟踪稳定性;3）利用抽取轮廓边的算法将边跟踪算法从CAD模型扩展到一般的面片模型。实验结果证明了该方法的鲁棒高效,能够满足增强现实、虚拟装配等应用需求。