Nonrigid motion analysis based on dynamic refinement of finiteelement models

We propose new algorithms for accurate nonrigid motion tracking. Given an initial model representing general knowledge of the object, a set of sparse correspondences, and incomplete or missing information about geometry or material properties, we can recover dense motion vectors using finite element models. The method is based on the iterative analysis of the differences between the actual and predicted behaviors. Unknown parameters are recovered using an iterative descent search for the best nonlinear finite element model that approximates nonrigid motion of the given object. During this search process, we not only estimate material properties, but also infer dense point correspondences from our initial set of sparse correspondences. Thus, during tracking, the model is refined which, in turn, improves tracking quality. Experimental results demonstrate the success of the proposed algorithm. Our work demonstrates the possibility of accurate quantitative analysis of nonrigid motion in range image sequences with objects consisting of multiple materials and 3D volumes     

Matching point features under small nonrigid motion

This paper describes a method for matching point features between images of objects that have undergone small nonrigid motion. Feature points are assumed to be available and, given a properly extracted set of feature points, a robust matching is established under the condition that the local nonrigid motion of each point is restricted to a circle of radius δ, where δ is not too large. This is in contrast to other techniques for point matching which assume either rigid motion or nonrigid motion of a known kind. The point matching problem is viewed in terms of weighted bipartite graph matching. In order to account for the possibility that the feature selector can be imprecise, we incorporate a greedy matching strategy with the weighted graph matching algorithm. Our algorithm is robust and insensitive to noise and missing features. The resulting matching can be used with image warping or other techniques for nonrigid motion analysis, image subtraction, etc. We present our experimental results on sequences of mammograms, images of a deformable clay object and satellite cloud images. In the first two cases we provide quantitative comparison with known ground truth.     

Recovery of nonrigid motion and structure

The authors introduce a physically correct model of elastic nonrigid motion. This model is based on the finite element method, but decouples the degrees of freedom by breaking down object motion into rigid and nonrigid vibration or deformation modes. The result is an accurate representation for both rigid and nonrigid motion that has greatly reduced dimensionality, capturing the intuition that nonrigid motion is normally coherent and not chaotic. Because of the small number of parameters involved, this representation is used to obtain accurate overstrained estimates of both rigid and nonrigid global motion. It is also shown that these estimates can be integrated over time by use of an extended Kalman filter, resulting in stable and accurate estimates of both three-dimensional shape and three-dimensional velocity. The formulation is then extended to include constrained nonrigid motion. Examples of tracking single nonrigid objects and multiple constrained objects are presented     

Tracking nonrigid motion and structure from 2D satellite cloudimages without correspondences

Tracking both structure and motion of nonrigid objects from monocular images is an important problem in vision. In this paper, a hierarchical method which integrates local analysis (that recovers small details) and global analysis (that appropriately limits possible nonrigid behaviors) is developed to recover dense depth values and nonrigid motion from a sequence of 2D satellite cloud images without any prior knowledge of point correspondences. This problem is challenging not only due to the absence of correspondence information but also due to the lack of depth cues in the 2D cloud images (scaled orthographic projection). In our method, the cloud images are segmented into several small regions and local analysis is performed for each region. A recursive algorithm is proposed to integrate local analysis with appropriate global fluid model constraints, based on which a structure and motion analysis system, SMAS, is developed. We believe that this is the first reported system in estimating dense structure and nonrigid motion under scaled orthographic views using fluid model constraints. Experiments on cloud image sequences captured by meteorological satellites (GOES-8 and GOES-9) have been performed using our system, along with their validation and analyses. Both structure and 3D motion correspondences are estimated to subpixel accuracy. Our results are very encouraging and have many potential applications in earth and space sciences, especially in cloud models for weather prediction     

Shape and nonrigid motion estimation through physics-basedsynthesis

A physics-based framework for 3-D shape and nonrigid motion estimation for real-time computer vision systems is presented. The framework features dynamic models that incorporate the mechanical principles of rigid and nonrigid bodies into conventional geometric primitives. Through the efficient numerical simulation of Lagrange equations of motion, the models can synthesize physically correct behaviors in response to applied forces and imposed constraints. Applying continuous Kalman filtering theory, a recursive shape and motion estimator that employs the Lagrange equations as a system model is developed. The system model continually synthesizes nonrigid motion in response to generalized forces that arise from the inconsistency between the incoming observations and the estimated model state. The observation forces also account formally for instantaneous uncertainties and incomplete information. A Riccati procedure updates a covariance matrix that transforms the forces in accordance with the system dynamics and prior observation history. Experiments involving model fitting and tracking of articulated and flexible objects from noisy 3-D data are described     

短程线主动轮廓跟踪算法的研究--在复杂背景和非刚性运动目标跟踪中的应用

在复杂背景下对多个非刚性目标进行跟踪是计算机视觉中的一个难点。在短程线主动轮廓模型的基础上，利用力场正则化方法，并加入运动边缘信息，提出了一种在复杂背景下多个非刚性目标进行跟踪的方法。该方法由运动检测和跟踪两部分组成：运动检测利用运动边缘信息对运动目标的运动做出检测，让轮廓曲线运动到目标轮廓附近；跟踪利用当前帧中的静态边缘信息对运动检测的结果加以修正，而跟踪这一步引入的偏差将在下一帧的运动检测中得到修正。实验表明该方法能够有效地在复杂背景中对多个非刚性运动目标进行跟踪。     

Chain Vibration and Dynamic Stress in Three-Dimensional Multibody Tracked Vehicles

A three-dimensional computational finite element procedure for the vibration and dynamic stress analysis of the track link chains of off-road vehicles is presented in this paper. The numerical procedure developed in this investigation integrates classical constrained multibody dynamics methods with finite element capabilities. The nonlinear equations of motion of the three-dimensional tracked vehicle model in which the track link s are considered flexible bodies, are obtained using the floating frame of reference formulation. Three-dimensional contact force models are used to describe the interaction of the track chain links with the vehicle components and the ground. The dynamic equations of motion are first presented in terms of a coupled set of reference and elastic coordinates of the track links. Assuming that the structural flexibility of the track links does not have a significant effect on their overall rigid body motion as well as the vehicle dynamics, a partially linearized set of differential equations of motion of the track links is obtained. The equations associated with the rigid body motion are used to predict the generalized contact, inertia, and constraint forces associated with the deformation degrees of freedom of the track links. These forces are introduced to the track link flexibility equations which are used to calculate the deformations of the links resulting from the vehicle motion. A detailed three-dimensional finite element model of the track link is developed and utilized to predict the natural frequencies and mode shapes. The terms that represent the rigid body inertia, centrifugal and Coriolis forces in the equations of motion associated with the elastic coordinates of the track link are described in detail. A computational procedure for determining the generalized constraint forces associated with the elastic coordinates of the deformable chain links is presented. The finite element model is then used to determine the deformations of the track links resulting from the contact, inertia, and constraint forces. The results of the dynamic stress analysis of the track links are presented and the differences between these results and the results obtained by using the static stress analysis are demonstrated.     

A factorization-based approach for articulated nonrigid shape, motion and kinematic chain recovery from video

Recovering articulated shape and motion, especially human body motion, from video is a challenging problem with a wide range of applications in medical study, sport analysis and animation, etc. Previous work on articulated motion recovery generally requires prior knowledge of the kinematic chain and usually does not concern the recovery of the articulated shape. The non-rigidity of some articulated part, e.g. human body motion with nonrigid facial motion, is completely ignored. We propose a factorization-based approach to recover the shape, motion and kinematic chain of an articulated object with nonrigid parts altogether directly from video sequences under a unified framework. The proposed approach is based on our modeling of the articulated non-rigid motion as a set of intersecting motion subspaces. A motion subspace is the linear subspace of the trajectories of an object. It can model a rigid or non-rigid motion. The intersection of two motion subspaces of linked parts models the motion of an articulated joint or axis. Our approach consists of algorithms for motion segmentation, kinematic chain building, and shape recovery. It handles outliers and can be automated. We test our approach through synthetic and real experiments and demonstrate how to recover articulated structure with non-rigid parts via a single-view camera without prior knowledge of its kinematic chain.     

Adaptive-size meshes for rigid and nonrigid shape analysis andsynthesis

A physically based modeling method that uses adaptive-size meshes to model surfaces of rigid and nonrigid objects is presented. The initial model uses an a priori determined mesh size. However, the mesh size increases or decreases dynamically during surface reconstruction to locate nodes near surface areas of interest (like high curvature points) and to optimize the fitting error. Further, presented with multiple 3-D data frames, the mesh size varies as the data surface undergoes nonrigid motion. This model is used to reconstruct 3-D surfaces, analyze the nonrigid motion, track the corresponding points in nonrigid motion, and create graphic animation and visualization. The method was tested on real range data, on simulated nonrigid motion, and on real data for the left ventricular motion     

Outward-looking circular motion analysis of large image sequences

This work presents a novel and simple method of analyzing the motion of a large image sequence captured by a calibrated outward-looking video camera moving on a circular trajectory for large-scale environment applications. Previous circular motion algorithms mainly focus on inward-looking turntable-like setups. They are not suitable for outward-looking motion where the conic trajectory of corresponding points degenerates to straight lines. The circular motion of a calibrated camera essentially has only one unknown rotation angle for each frame. The motion recovery for the entire sequence computes only one fundamental matrix of a pair of frames to extract the angular motion of the pair using Laguerre's formula and then propagates the computation of the unknown rotation angles to the other frames by tracking one point over at least three frames. Finally, a maximum-likelihood estimation is developed for the optimization of the whole sequence. Extensive experiments demonstrate the validity of the method and the feasibility of the application in image-based rendering.     

Direct estimation of range flow on deformable shape from a videorate range camera

A method of estimating range flow (space displacement vector field) on nonrigid as well as rigid objects from a sequence of range images is described. This method can directly estimate the deformable motion parameters by solving a system of linear equations that are obtained from substituting a linear transformation of nonrigid objects expressed by the Jacobian matrix into motion constraints based on an extension of the conventional scheme used in intensity image sequences. The range flow is directly computed by substituting these estimated motion parameters into the linear transformation. The algorithm is supported by experimental estimations of range flow on a sheet of paper, a piece of cloth, human skin, and a rubber balloon being inflated, using real range image sequences acquired by a video rate range camera     

Rigid body motion from range image sequences

An algorithm is described for recovering the six degrees of freedom of motion of a vehicle from a sequence of range images of a static environment taken by a range camera rigidly attached to the vehicle. The technique utilizes a least-squares minimization of the difference between the measured rate of change of elevation at a point and the rate predicted by the so-called elevation rate constraint equation. It is assumed that most of the surface is smooth enough so that local tangent planes can be constructed, and that the motion between frames is smaller than the size of most features in the range image. This method does not depend on the determination of correspondences between isolated high-level features in the range images. The algorithm has been successfully applied to data obtained from the range imager on the Autonomous Land Vehicle (ALV). Other sensors on the ALV provide an initial approximation to the motion between frames. It was found that the outputs of the vehicle sensors themselves are not suitable for accurate motion recovery because of errors in dead reckoning resulting from such problems as wheel slippage. The sensor measurements are used only to approximately register range data. The algorithm described here then recovers the difference between the true motion and that estimated from the sensor outputs.     

The Problem of Degeneracy in Structure and Motion Recovery from Uncalibrated Image Sequences

The aim of this work is the recovery of 3D structure and camera projection matrices for each frame of an uncalibrated image sequence. In order to achieve this, correspondences are required throughout the sequence. A significant and successful mechanism for automatically establishing these correspondences is by the use of geometric constraints arising from scene rigidity. However, problems arise with such geometry guided matching if general viewpoint and general structure are assumed whilst frames in the sequence and/or scene structure do not conform to these assumptions. Such cases are termed degenerate.In this paper we describe two important cases of degeneracy and their effects on geometry guided matching. The cases are a motion degeneracy where the camera does not translate between frames, and a structure degeneracy where the viewed scene structure is planar. The effects include the loss of correspondences due to under or over fitting of geometric models estimated from image data, leading to the failure of the tracking method. These degeneracies are not a theoretical curiosity, but commonly occur in real sequences where models are statistically estimated from image points with measurement error.We investigate two strategies for tackling such degeneracies: the first uses a statistical model selection test to identify when degeneracies occur: the second uses multiple motion models to overcome the degeneracies. The strategies are evaluated on real sequences varying in motion, scene type, and length from 13 to 120 frames.     

基于广义形态内插的非刚体运动描述方法

提出一种基于形态变换的非刚体运动的广义内插方法.通过对非刚体的凸剖分及凸子集全局优化匹配,与传统的线性内插方法相比,解决了任意非同拓扑结构(包括有孔及凹多边形)的内插问题.理论证明,此种内插方法是一种全局优化的内插方法,并证明了此方法的正确性,讨论了不同凸剖分对内插的影响,把非刚体的运动分解为非刚体的变形与刚体的旋转.实验证明,此内插方法效果自然,质量好,速度快,可用于非刚体运动描述及二维动画关键帧的内插.     

On 3-D scene flow and structure recovery from multiview image sequences

Two novel systems computing dense three-dimensional (3-D) scene flow and structure from multiview image sequences are described in this paper. We do not assume rigidity of the scene motion, thus allowing for nonrigid motion in the scene. The first system, integrated model-based system (IMS), assumes that each small local image region is undergoing 3-D affine motion. Non-linear motion model fitting based on both optical flow constraints and stereo constraints is then carried out on each local region in order to simultaneously estimate 3-D motion correspondences and structure. The second system is based on extended gradient-based system (EGS), a natural extension of two-dimensional (2-D) optical flow computation. In this method, a new hierarchical rule-based stereo matching algorithm is first developed to estimate the initial disparity map. Different available constraints under a multiview camera setup are further investigated and utilized in the proposed motion estimation. We use image segmentation information to adopt and maintain the motion and depth discontinuities. Within the framework for EGS, we present two different formulations for 3-D scene flow and structure computation. One formulation assumes that initial disparity map is accurate, while the other does not. Experimental results on both synthetic and real imagery demonstrate the effectiveness of our 3-D motion and structure recovery schemes. Empirical comparison between IMS and EGS is also reported.     

Modeling and Animating Realistic Faces from Images

We present a new set of techniques for modeling and animating realistic faces from photographs and videos. Given a set of face photographs taken simultaneously, our modeling technique allows the interactive recovery of a textured 3D face model. By repeating this process for several facial expressions, we acquire a set of face models that can be linearly combined to express a wide range of expressions. Given a video sequence, this linear face model can be used to estimate the face position, orientation, and facial expression at each frame. We illustrate these techniques on several datasets and demonstrate robust estimations of detailed face geometry and motion.     

Spatially varying image based lighting by light probe sequences

We present a novel technique for capturing spatially or temporally resolved light probe sequences, and using them for image based lighting. For this purpose we have designed and built a real-time light probe, a catadioptric imaging system that can capture the full dynamic range of the lighting incident at each point in space at video frame rates, while being moved through a scene. The real-time light probe uses a digital imaging system which we have programmed to capture high quality, photometrically accurate color images of 512×512 pixels with a dynamic range of 10000000:1 at 25 frames per second. By tracking the position and orientation of the light probe, it is possible to transform each light probe into a common frame of reference in world coordinates, and map each point and direction in space along the path of motion to a particular frame and pixel in the light probe sequence. We demonstrate our technique by rendering synthetic objects illuminated by complex real world lighting, first by using traditional image based lighting methods and temporally varying light probe illumination, and second an extension to handle spatially varying lighting conditions across large objects and object motion along an extended path.     

基于运动平滑性的动态图象序列的特征点对应

建立帧间的运动特征点对应是图象序列分析中的一项非常重要且非常困难的工作。根据图象序列的特点,该文提出了一种运动平滑的代价方程,并将运动平滑性与帧间特征点的欧氏距离相结合,以完成图象特征点运动轨迹的确立。由于此文的方法没有建立在运动的刚性约束之上,所以该方法同样适用于各种非刚体运动的特征点对应。