Recursive three-dimensional model reconstruction based on Kalman filtering.

A recursive two-step method to recover structure and motion from image sequences based on Kalman filtering is described in this paper. The algorithm consists of two major steps. The first step is an extended Kalman filter (EKF) for the estimation of the object's pose. The second step is a set of EKFs, one for each model point, for the refinement of the positions of the model features in the three-dimensional (3-D) space. These two steps alternate from frame to frame. The initial model converges to the final structure as the image sequence is scanned sequentially. The performance of the algorithm is demonstrated with both synthetic data and real-world objects. Analytical and empirical comparisons are made among our approach, the interleaved bundle adjustment method, and the Kalman filtering-based recursive algorithm by Azarbayejani and Pentland. Our approach outperformed the other two algorithms in terms of computation speed without loss in the quality of model reconstruction.     

A Comparison of Algorithms for Handheld Wand Tracking

We examine several algorithms for tracking a handheld wand in a 3D virtual reality system: extended Kalman filters (EKFs), interacting multiple models (IMMs), and support vector machines (SVMs). The IMMs consist of several EKF models, each of which is tuned for one particular type of user motion. For determining the types of motion, we compare hand-created rules with an automatic clustering algorithm, with mixed results. The mode-specific EKFs within the IMM are more accurate than one overall EKF. However, the IMM is comparable to a single EKF, because of the overhead of predicting the current component EKF. SVMs with a one-frame lookahead perform the best, cutting the error in half. Aside from those SVMs, different model types were best for the different dimensions of tracking (x, y, z, and rotation).     

基于强跟踪滤波器的多目标跟踪方法

在诸多的多目标跟踪算法中，相互作用多模型（IMM）算法是目前公认的最为有效的算法。但到目前为止，LMM估计方法都是建立在卡尔曼滤波器（KF）和扩展卡尔曼滤波器（EKF）基础上，因而其性能不仅依赖于所采用的模型集，而且在更大程度上依赖于所采用的滤波技术。强跟踪滤波器（STF）克服了卡尔曼和扩展卡尔曼的三大缺陷，因而设计一种基于STF的IMM目标跟踪算法显然能提高其性能。仿真实验表明，基于STF的IMM算法的跟踪性能要优于基于KF和EKF的IMM算法的跟踪性能。     

Hybrid three-dimensional camera pose estimation using particle filter sensor fusion

To properly align objects in the real and virtual worlds in an augmented reality (AR) space it is essential to keep tracking the camera's exact three-dimensional position and orientation (camera pose). State-of-the-art analysis shows that traditional vision-based or inertial sensor-based solutions are not adequate when used individually. Sensor fusion for hybrid tracking has become an active research direction during the past few years, although how to do it in a robust and principled way is still an open problem. In this paper, we develop a hybrid camera pose-tracking system that combines vision and inertial sensor technologies. We propose to use the particle filter framework for the sensor fusion system. Particle filters are sequential Monte-Carlo methods based upon a point mass (or 'particle') representation of probability densities, which can be applied to any state space model and which generalize the traditional Kalman filtering methods. We have tested our algorithm to evaluate its performance and have compared the results obtained by the particle filter with those given by a classical extended Kalman filter. Experimental results are presented     

Recursive camera-motion estimation with the trifocal tensor.

In this paper, an innovative extended Kalman filter (EKF) algorithm for pose tracking using the trifocal tensor is proposed. In the EKF, a constant-velocity motion model is used as the dynamic system, and the trifocal-tensor constraint is incorporated into the measurement model. The proposed method has the advantages of those structure- and-motion-based approaches in that the pose sequence can be computed with no prior information on the scene structure. It also has the strengths of those model-based algorithms in which no updating of the three-dimensional (3-D) structure is necessary in the computation. This results in a stable, accurate, and efficient algorithm. Experimental results show that the proposed approach outperformed other existing EKFs that tackle the same problem. An extension to the pose-tracking algorithm has been made to demonstrate the application of the trifocal constraint to fast recursive 3-D structure recovery.     

基于IMM-UKF的纯方位机动目标跟踪算法

针对在非线性机动目标跟踪中存在的滤波器易发散、跟踪误差大等问题,本文在多站纯方位跟踪的基础上,把Unscented卡尔曼滤波(Unscented Kalman filter,UKF)引进到交互多模型算法(Interacting multiple model,IMM)中,设计了交互多模型UKF滤波算法,克服了EKF中引入的较大线性化误差对机动目标跟踪算法性能的影响.最后将该算法与扩展卡尔曼滤波(EKF)、IMM-EKF算法进行了比较,仿真结果表明:IMM-UKF 算法增强了EKF滤波器的稳定性,提高了滤波收敛速度和跟踪精度.     

Hand tracking in a natural conversational environment by the interacting multiple model and probabilistic data association (IMM-PDA) algorithm

Traditional image based hand tracking algorithms use a single model Kalman filter to estimate and predict the hand state (position, velocity, and acceleration) and do not consider multiple measurements with noise and false alarms. However, these approaches may fail in the case of large maneuvers and/or a clutter measurement environment. In this paper, we apply the interacting multiple model (IMM) to catch hand maneuvers and the probabilistic data association (PDA) method to process noisy measurements and false alarms. A theoretical framework of image based hand tracking by the IMM-PDA algorithm is set up. Experiment results from several long video segments show that the IMM-PDA algorithm gives a superior performance compared to single model based Kalman filters.     

Heterogeneous multisensor fusion for mapping dynamic environments

In this paper, we propose a heterogeneous multisensor fusion algorithm for mapping in dynamic environments. The algorithm synergistically integrates the information obtained from an uncalibrated camera and sonar sensors to facilitate mapping and tracking. The sonar data is mainly used to build a weighted line-based map via the fuzzy clustering technique. The line weight, with confidence corresponding to the moving object, is determined by both sonar and vision data. The motion tracking is primarily accomplished by vision data using particle filtering and the sonar vectors originated from moving objects are used to modulate the sample weighting. A fuzzy system is implemented to fuse the two sensor data features. Additionally, in order to build a consistent global map and maintain reliable tracking of moving objects, the well-known extended Kalman filter is applied to estimate the states of robot pose and map features. Thus, more robust performance in mapping as well as tracking are achieved. The empirical results carried out on the Pioneer 2DX mobile robot demonstrate that the proposed algorithm outperforms the methods a using homogeneous sensor, in mapping as well as tracking behaviors.     

Tracking and Motion Estimation of the Articulated Object: a Hierarchical Kalman Filter Approach

Real-time motion capture plays a very important role in various applications, such as 3D interface for virtual reality systems, digital puppetry, and real-time character animation. In this paper we challenge the problem of estimating and recognizing the motion of articulated objects using theoptical motion capturetechnique. In addition, we present an effective method to control the articulated human figure in realtime.The heart of this problem is the estimation of 3D motion and posture of an articulated, volumetric object using feature points from a sequence of multiple perspective views. Under some moderate assumptions such as smooth motion and known initial posture, we develop a model-based technique for the recovery of the 3D location and motion of a rigid object using a variation of Kalman filter. The posture of the 3D volumatric model is updated by the 2D image flow of the feature points for all views. Two novel concepts – the hierarchical Kalman filter (KHF) and the adaptive hierarchical structure (AHS) incorporating the kinematic properties of the articulated object – are proposed to extend our formulation for the rigid object to the articulated one. Our formulation also allows us to avoid two classic problems in 3D tracking: the multi-view correspondence problem, and the occlusion problem. By adding more cameras and placing them appropriately, our approach can deal with the motion of the object in a very wide area. Furthermore, multiple objects can be handled by managing multiple AHSs and processing multiple HKFs.We show the validity of our approach using the synthetic data acquired simultaneously from the multiple virtual camera in a virtual environment (VE) and real data derived from a moving light display with walking motion. The results confirm that the model-based algorithm works well on the tracking of multiple rigid objects.     

Robust object tracking under appearance change conditions

We propose a robust visual tracking framework based on particle filter to deal with the object appearance changes due to varying illumination, pose variantions, and occlusions. We mainly improve the observation model and re-sampling process in a particle filter. We use on-line updating appearance model, affine transformation, and M-estimation to construct an adaptive observation model. On-line updating appearance model can adapt to the changes of illumination partially. Affine transformation-based similarity measurement is introduced to tackle pose variantions, and M-estimation is used to handle the occluded object in computing observation likelihood. To take advantage of the most recent observation and produce a suboptimal Gaussian proposal distribution, we incorporate Kalman filter into a particle filter to enhance the performance of the resampling process. To estimate the posterior probability density properly with lower computational complexity, we only employ a single Kalman filter to propagate Gaussian distribution. Experimental results have demonstrated the effectiveness and robustness of the proposed algorithm by tracking visual objects in the recorded video sequences.     

基于高斯和SCKF 的姿态角辅助三维目标跟踪

首先, 根据目标运动与姿态角的关系, 分析目标在偏航角和俯仰角下的速度变化, 进而推导出姿态角辅助三维目标跟踪模型; 然后, 针对姿态角量测非高斯情况, 在分析均方根容积卡尔曼滤波的基础上, 提出新的高斯和均方根容积卡尔曼滤波算法, 以提高非线性非高斯的处理能力; 最后, 结合不同运动模式下姿态角分量的特点, 建立姿态角分量不同的跟踪模型, 通过模型切换实现对姿态角机动的跟踪. 仿真结果验证了所提出跟踪模型和滤波算法的正确性和有效性.     

Multi-sensor based real-time 6-DoF pose tracking for wearable augmented reality

Wearable augmented reality (WAR) combines a live view of a real scene with computer-generated graphic on resource-limited platforms. One of the crucial technologies for WAR is a real-time 6-DoF pose tracking, facilitating registration of virtual components within in a real scene. Generally, artificial markers are typically applied to provide pose tracking for WAR applications. However, these marker-based methods suffer from marker occlusions or large viewpoint changes. Thus, a multi-sensor based tracking approach is applied in this paper, and it can perform real-time 6-DoF pose tracking with real-time scale estimation for WAR on a consumer smartphone. By combining a wide-angle monocular camera and an inertial sensor, a more robust 6-DoF motion tracking is demonstrated with the mutual compensations of the heterogeneous sensors. Moreover, with the help of the depth sensor, the scale initialization of the monocular tracking is addressed, where the initial scale is propagated within the subsequent sensor-fusion process, alleviating the scale drift in traditional monocular tracking approaches. In addition, a sliding-window based Kalman filter framework is used to provide a low jitter pose tracking for WAR. Finally, experiments are carried out to demonstrate the feasibility and robustness of the proposed tracking method for WAR applications.     

利用部分观测跟踪被遮挡的目标

提出了一个在单个固定摄像机下进行多目标跟踪的方法.利用亮度和色度混合模型和卡尔曼滤波器来检测跟踪目标,为了利于预测和解释被遮挡的物体,建立了场景的模型.在遮挡的情况下,和传统的盲跟踪不同,本文中的目标状态是由可用的部分观测来估计的.对目标的观测取决于预测、前景观测和场景模型.这使得本文算法在定性或定量的分析下都表现出更加鲁棒的性能.     

Rigid body segmentation and shape description from dense opticalflow under weak perspective

We present an algorithm for identifying and tracking independently moving rigid objects from optical flow. Some previous attempts at segmentation via optical flow have focused on finding discontinuities in the flow field. While discontinuities do indicate a change in scene depth, they do not in general signal a boundary between two separate objects. The proposed method uses the fact that each independently moving object has a unique epipolar constraint associated with its motion. Thus motion discontinuities based on self-occlusion can be distinguished from those due to separate objects. The use of epipolar geometry allows for the determination of individual motion parameters for each object as well as the recovery of relative depth for each point on the object. The algorithm assumes an affine camera where perspective effects are limited to changes in overall scale. No camera calibration parameters are required. A Kalman filter based approach is used for tracking motion parameters with time     

Efficient three-dimensional metric object modeling from uncalibrated image sequences

This paper presents a scheme that addresses the practical issues associated with producing a geometric model of a scene using a passive sensing technique. The proposed image-based scheme comprises a recursive structure recovery method and a recursive surface reconstruction technique. The former method employs a robust corner-tracking algorithm that copes with the appearance and disappearance of features and a corner-based structure and motion estimation algorithm that handles the inclusion and expiration of features. The novel formulation and dual extended Kalman filter computational framework of the estimation algorithm provide an efficient approach to metric structure recovery that does not require any prior knowledge about the camera or scene. The newly developed surface reconstruction technique employs a visibility constraint to iteratively refine and ultimately yield a triangulated surface that envelops the recovered scene structure and can produce views consistent with those of the original image sequence. Results on simulated data and synthetic and real imagery illustrate that the proposed scheme is robust, accurate, and has good numerical stability, even when features are repeatedly absent or their image locations are affected by extreme levels of noise.     

AR预测模型的IMM跟踪算法

针对LOS/NLOS混合条件下对机动目标的鲁棒跟踪问题,提出一种基于AR预测模型的交互式多模型（Interacting Multiple Model,IMM）跟踪算法（ARIMM）。该算法利用AR预测模型对运动状态建模,针对LOS与NLOS条件下观测噪声的分布不同分别使用无迹卡尔曼滤波器（Unscented Kalman Filter,UKF）和改进的无迹卡尔曼滤波器（Robust Unscented Kalman Filter,RUKF）,通过IMM方法估计出移动台的位置,利用该位置更新AR模型的参数,使AR模型与真实运动状态更加匹配,实现精确跟踪。仿真结果表明,在LOS/NLOS混合条件下,与传统的UKF和RUKF算法相比,该算法对机动目标跟踪的鲁棒性更好。     

交互式多模型算法在机器人目标跟踪中的应用

将交互式多模型(IMM)算法应用于视觉伺服机器人对机动目标的跟踪。使用匀速运动(CV)和匀加速运动(CA)模型表示目标的两种运动状态,利用马尔可夫链进行模型切换,根据目标前一时刻的状态和当前的观测值,预测目标当前的状态。在Matlab上对IMM滤波算法和Kalman滤波算法进行了仿真实验研究,结果表明,不管目标处于何种运动状态,IMM算法估计量的误差均值都比Kalman滤波算法的误差均值小,尤以目标作机动运动时更为突出,证明了应用IMM算法可以提高跟踪机动目标的精度。     

A framework for heading-guided recognition of human activity

A combined 2D, 3D approach is presented that allows for robust tracking of moving people and recognition of actions. It is assumed that the system observes multiple moving objects via a single, uncalibrated video camera. Low-level features are often insufficient for detection, segmentation, and tracking of non-rigid moving objects. Therefore, an improved mechanism is proposed that integrates low-level (image processing), mid-level (recursive 3D trajectory estimation), and high-level (action recognition) processes. A novel extended Kalman filter formulation is used in estimating the relative 3D motion trajectories up to a scale factor. The recursive estimation process provides a prediction and error measure that is exploited in higher-level stages of action recognition. Conversely, higher-level mechanisms provide feedback that allows the system to reliably segment and maintain the tracking of moving objects before, during, and after occlusion. Heading-guided recognition (HGR) is proposed as an efficient method for adaptive classification of activity. The HGR approach is demonstrated using “motion history images” that are then recognized via a mixture-of-Gaussians classifier. The system is tested in recognizing various dynamic human outdoor activities: running, walking, roller blading, and cycling. In addition, experiments with real and synthetic data sets are used to evaluate stability of the trajectory estimator with respect to noise.     

Pose robust face tracking by combining view-based AAMs and temporal filters

Active appearance models (AAMs) are useful for face tracking for the advantages of detailed face interpretation, accurate alignment and high efficiency. However, they are sensitive to initial parameters and may easily be stuck in local minima due to the gradient-descent optimization, which makes the AAM based face tracker unstable in the presence of large pose deviation and fast motion. In this paper, we propose to combine the view-based AAMs with two novel temporal filters to overcome the limitations. First, we build a new view space based on the shape parameters of AAMs, instead of the model parameters controlling both the shape and appearance, for the purpose of pose estimation. Then the Kalman filter is used to simultaneously update the pose and shape parameters for a better fitting of each frame. Second, we propose a temporal matching filter which is twofold. The inter-frame local appearance constraint is incorporated into AAM fitting, where the mechanism of the active shape model (ASM) is also implemented in a unified framework to find more accurate matching points. Moreover, we propose to initialize the shape with correspondences found by a random forest based local feature matching. By introducing the local information and temporal correspondences, the twofold temporal matching filter improves the tracking stability when confronted with fast appearance changes. Experimental results show that our algorithm is more pose robust than basic AAMs and some state-of-art AAM based methods, and that it can also handle large expressions and non-extreme illumination changes in test video sequences.     

Robust camera pose tracking for augmented reality using particle filtering framework

In this paper, we present new solutions for the problem of estimating the camera pose using particle filtering framework. The proposed approach is suitable for real-time augmented reality (AR) applications in which the camera is held by the user. This work demonstrates that particle filtering improve the robustness of the tracking comparing to existing approaches, such as those based on the Kalman filter. We propose a tracking framework for both points and lines features, the particle filter is used to compute the posterior density for the camera 3D motion parameters. We also analyze the sensitivity of our technique when outliers are present in the match data. Outliers arise frequently due to incorrect correspondences which occur because of either image noise or occlusion. Results from real data in an augmented reality setup are then presented, demonstrating the efficiency and robustness of the proposed method.