Design and Use of Linear Models for Image Motion Analysis

Linear parameterized models of optical flow, particularly affine models, have become widespread in image motion analysis. The linear model coefficients are straightforward to estimate, and they provide reliable estimates of the optical flow of smooth surfaces. Here we explore the use of parameterized motion models that represent much more varied and complex motions. Our goals are threefold: to construct linear bases for complex motion phenomena; to estimate the coefficients of these linear models; and to recognize or classify image motions from the estimated coefficients. We consider two broad classes of motions: i) generic motion features such as motion discontinuities and moving bars; and ii) non-rigid, object-specific, motions such as the motion of human mouths. For motion features we construct a basis of steerable flow fields that approximate the motion features. For object-specific motions we construct basis flow fields from example motions using principal component analysis. In both cases, the model coefficients can be estimated directly from spatiotemporal image derivatives with a robust, multi-resolution scheme. Finally, we show how these model coefficients can be use to detect and recognize specific motions such as occlusion boundaries and facial expressions.     

Facial expression recognition system based on rigid and non-rigid motion separation and 3D pose estimation

This study presents a facial expression recognition system which separates the non-rigid facial expression from the rigid head rotation and estimates the 3D rigid head rotation angle in real time. The extracted trajectories of the feature points contain both rigid head motion components and non-rigid facial expression motion components. A 3D virtual face model is used to obtain accurate estimation of the head rotation angle such that the non-rigid motion components can be precisely separated to enhance the facial expression recognition performance. The separation performance of the proposed system is further improved through the use of a restoration mechanism designed to recover feature points lost during large pan rotations. Having separated the rigid and non-rigid motions, hidden Markov models (HMMs) are employed to recognize a prescribed set of facial expressions defined in terms of facial action coding system (FACS) action units (AUs).     

基于模型的头部运动估计和面部图像合成

文中讨论一种基于模型的头部运动估计和面部图像合成方法。首先建立了一个基于人脸几何模型的可变形三维面部模型,此模型可根据不同人脸图像特征修正特定人脸模型。为了使特定人脸模型与特定人脸图像相匹配,需根据变形模型修正人脸模型。文中采用自动调整与人机交互相结合的方法实现特定人脸模型匹配。在调整完模型形状之后,应用３个方向的面部图像进行纹理映射生成不同视点方向的面部图像。应用合成面部图像与输入面部图像最佳匹     

基于分层弹性运动分析的非刚体跟踪方法

采用时--空分层的弹性运动跟踪策略, 提出了一种分析长时运动稳定结构与短时运动局部变化的非刚体运动跟踪方法. 首先, 基于序贯形状聚类的分段弹性运动跟踪模型, 将整段图像序列分割成若干子段, 并利用弹性运动分析方法得到子段内各帧边缘点的对应关系和各类的平均形状, 获取短时局部运动变化细节. 然后, 通过基于贝叶斯网的整体搜索算法寻找时序上相邻聚类平均形状之间的对应关系, 进而得到整段运动的公共形状, 用于表示长时运动稳定结构. 通过计算公共形状与各类平均形状之间的变形关系, 可以建立各聚类平均形状之间的对应关系, 实现分段运动的连接. 本方法的特点是不依赖先验模型、 通用性好、 目标的描述能力强. 实验表明, 本方法与现有不依赖模型的方法相比,具有更好的长时稳定性和更高的跟踪精确度.     

单目人体图像序列的运动及结构参数估计

利用人体行走的单目动态图像序列,估计手臂及腿部的运动及结构参数.在物体中心坐标的运动表示方式下,提出了基于弹性连接概念的非刚体运动模型,通过加入弹性系数的方法,使非刚体运动模型和刚体运动模型得到了良好的统一.在此模型的基础上,用Levenberg-Marquardt方法实现了运动及结构参数的估计.实验证明了该方法的有效性,运动模型中的弹性系数也反映了一定程度的运动模式.     

面向纹理特征的真实感三维人脸动画方法

纹理变化是人脸表情的重要组成部分，传统的人脸动画方法通常只是对纹理图像做简单的拉伸变换，没有考虑人脸细微纹理特征的变化，比如皱纹、酒窝等，该文提出了一种面向纹理特征变化的真实感三维人脸动画方法．给出局部表情比率图(Partial Expression Ratio Image，PERI)的概念及其获取方法，在此基础上，进一步给出了面向MPEG-4的PERI参数化与面向三维人脸动画的多方向PERI方法，前者通过有机结合MPEG-4的人脸动画参数(Facial Anlmation Parameter，FAP)，实现人脸动画中细微表情特征的参数化表示；后者通过多方向PERI纹理特征调整方法，使得三维人脸模型在不同角度都具有较好的细微表情特征,该文提出的方法克服了传统人脸动画只考虑人脸曲面形变控制而忽略纹理变化的缺陷，实现面向纹理变化的具有细微表情特征的真实感三维人脸动画，实验表明，该文提出的方法能有效捕捉纹理变化细节，提高人脸动画的真实感。     

High Resolution Tracking of Non-Rigid Motion of Densely Sampled 3D Data Using Harmonic Maps

We present a novel automatic method for high resolution, non-rigid dense 3D point tracking. High quality dense point clouds of non-rigid geometry moving at video speeds are acquired using a phase-shifting structured light ranging technique. To use such data for the temporal study of subtle motions such as those seen in facial expressions, an efficient non-rigid 3D motion tracking algorithm is needed to establish inter-frame correspondences. The novelty of this paper is the development of an algorithmic framework for 3D tracking that unifies tracking of intensity and geometric features, using harmonic maps with added feature correspondence constraints. While the previous uses of harmonic maps provided only global alignment, the proposed introduction of interior feature constraints allows to track non-rigid deformations accurately as well. The harmonic map between two topological disks is a diffeomorphism with minimal stretching energy and bounded angle distortion. The map is stable, insensitive to resolution changes and is robust to noise. Due to the strong implicit and explicit smoothness constraints imposed by the algorithm and the high-resolution data, the resulting registration/deformation field is smooth, continuous and gives dense one-to-one inter-frame correspondences. Our method is validated through a series of experiments demonstrating its accuracy and efficiency.     

Forward non-rigid motion tracking for facial MoCap

For the existing motion capture (MoCap) data processing methods, manual interventions are always inevitable, most of which are derived from the data tracking process. This paper addresses the problem of tracking non-rigid 3D facial motions from sequences of raw MoCap data in the presence of noise, outliers and long time missing. We present a novel dynamic spatiotemporal framework to automatically solve the problem. First, based on a 3D facial topological structure, a sophisticated non-rigid motion interpreter (SNRMI) is put forward; together with a dynamic searching scheme, it cannot only track the non-missing data to the maximum extent but recover missing data (it can accurately recover more than five adjacent markers under long time (about 5 seconds) missing) accurately. To rule out wrong tracks of the markers labeled in open structures (such as mouth, eyes), a semantic-based heuristic checking method was raised. Second, since the existing methods have not taken the noise propagation problem into account, a forward processing framework is presented to solve the problem. Another contribution is the proposed method could track facial non-rigid motions automatically and forward, and is believed to greatly reduce even eliminate the requirements of human interventions during the facial MoCap data processing. Experimental results proved the effectiveness, robustness and accuracy of our system.     

具有真实感的三维人脸动画

具有真实感的三维人脸模型的构造和动画是计算机图形学领域中一个重要的研究课题.如何在三维人脸模型上实时地模拟人脸的运动,产生具有真实感的人脸表情和动作,是其中的一个难点.提出一种实时的三维人脸动画方法,该方法将人脸模型划分成若干个运动相对独立的功能区,然后使用提出的基于加权狄里克利自由变形DFFD(Dirichlet free-form deformation)和刚体运动模拟的混合技术模拟功能区的运动.同时,通过交叉的运动控制点模拟功能区之间运动的相互影响.在该方法中,人脸模型的运动通过移动控制点来驱动.为了简化人脸模型的驱动,提出了基于MPEG-4中脸部动画参数FAP(facial animation parameters)流和基于肌肉模型的两种高层驱动方法.这两种方法不但具有较高的真实感,而且具有良好的计算性能,能实时模拟真实人脸的表情和动作.     

Reconstruction of non-rigid 3D shapes from stereo-motion

Several non-rigid structure from motion methods have been proposed so far in order to recover both the motion and the non-rigid structure of an object. However, these monocular algorithms fail to give reliable 3D shape estimates when the overall rigid motion of the sequence is small. Aiming to overcome this limitation, in this paper we propose a novel approach for the 3D Euclidean reconstruction of deformable objects observed by an uncalibrated stereo rig. Using a stereo setup drastically improves the 3D model estimation when the observed 3D shape is mostly deforming without undergoing strong rigid motion. Our approach is based on the following steps. Firstly, the stereo system is automatically calibrated and used to compute metric rigid structures from pairs of views. Afterwards, these 3D shapes are aligned to a reference view using a RANSAC method in order to compute the mean shape of the object and to select the subset of points which have remained rigid throughout the sequence. The selected rigid points are then used to compute frame-wise shape registration and to robustly extract the motion parameters from frame to frame. Finally, all this information is used as initial estimates of a non-linear optimization which allows us to refine the initial solution and also to recover the non-rigid 3D model. Exhaustive results on synthetic and real data prove the performance of our proposal estimating motion, non-rigid models and stereo camera parameters even when there is no rigid motion in the original sequence.