单目图像序列中基于正则化的局部三维非刚体运动估计

通过局部的三维非刚体运动估计进而达到全局估计结果是三维非刚体运动估计中的重要方法。本文提出了在单目图像序列中利用正则化的手段解决局部的三维非刚体运动估计。首先，在帧间特征点匹配已确立的前提下，利用仿射运动模型并结合中心投影方式提出了运动估计的最小二乘模型；然后针对三维运动估计的不适定性提出了正则化的运动估计方法，以正则化的形式融入运动的先验知识，使运动估计的结果更具鲁棒性；最后利用Levenberg-Marquart方法实现运动参数的求解。仿真图像序列的实验反映了本文方法的有效性。     

Tracking a dynamic set of feature points

We address the problems of tracking a set of feature points over a long sequence of monocular images as well as how to include and track new feature points detected in successive frames. Due to the 3-D movement of the camera, different parts of the images exhibit different image motion. Tracking discrete features can therefore be decomposed into several independent and local problems. Accordingly, we propose a localized feature tracking algorithm. The trajectory of each feature point is described by a 2-D kinematic model. Then to track a feature point, an interframe motion estimation scheme is designed to obtain the estimates of interframe motion parameters. Subsequently, using the estimates of motion parameters, corresponding points are identified to subpixel accuracy. Afterwards, the temporal information is processed to facilitate the tracking scheme. Since different feature points are tracked independently, the algorithm is able to handle the image motion arising from general 3-D camera movements. On the other hand, in addition to tracking feature points detected at the beginning, an efficient way to dynamically include new points extracted in subsequent frames is devised so that the information in a sequence is preserved. Experimental results for several image sequences are also reported.     

Automatic scalable face model design for 2D model-based video coding

Scalable low bit-rate video coding is vital for the transmission of video signals over wireless channels. A scalable model-based video coding scheme is proposed in this paper to achieve this. This paper mainly addresses automatic scalable face model design. Firstly, a robust and adaptive face segmentation method is proposed, which is based on piecewise skin-colour distributions. 43 million skin pixels from 900 images are used to train the skin-colour model, which can identify skin-colour pixels reliably under different lighting conditions. Next, reliable algorithms are proposed for detecting the eyes, mouth and chin that are used to verify the face candidatures. Then, based on the detected facial features and human face muscular distributions, a heuristic scalable face model is designed to represent the rigid and non-rigid motion of head and facial features. A novel motion estimation algorithm is proposed to estimate the object model motion hierarchically. Experimental results are provided to illustrate the performance of the proposed algorithms for facial feature detection and the accuracy of the designed scalable face model for representing face motion.     

Affine-structure-based facial image encoding

A real-time algorithm for affine-structure-based video compression for facial images is presented. The face undergoing motion is segmented and triangulated to yield a set of control points. The set of control points generated by triangulation are tracked across a few frames using an intensity-based correlation technique. For accurate motion and structure estimation a Kalman-filter-based algorithm is used to track features on the facial image. The structure information of the control points is transmitted only during the bootstrapping stage. After that only the motion information is transmitted to the decoder. This reduces the number of motion parameters associated with control points in each frame. The local motion of the eyes and lips is captured using local 2-D affine transformations. For real time implementation a quad-tree based search technique is adopted to solve local correlation. Any remaining reconstruction error is accounted for using predictive encoding. Results on real image sequences demonstrate the applicability of the method     

模型基编码的运动参数估计及误差准则

在人脸序列的图像编码中 ,模型基编码方法可以获得高的主观图像质量和低的码率 ,而受到广泛重视。但是 ,其运动参数的可靠估计还是一个难点 ,而且也没有一个较好的适合视觉特性的误差准则。本文提出了基于特征点的运动参数估计算法 ,并根据边沿 ,亮度和端点特性来自动提取特征点及自适应调整点的数目。提出用重建的图像的质量来估价运动参数误差 ,并给出了误差面积和轮廓转折率误差二个函数。这二个函数较好地反映了运动参数误差引入的图像几何失真。     

基于LED与肌肉模型的特定人脸建模与动画

特定三维人脸的建模与动画是计算机图形学中一个非常令人感兴趣的领域.本文提出了一种新的从两幅正交照片建立特定人脸的模型以及动画方法,首先以主动轮廓跟踪技术snake自动获取人脸特征点的准确位置,然后以文中的局部弹性变形(local elastic deformation)方法进行通用人脸模型到特定人脸的定制,并辅以采用图像镶嵌技术生成的大分辨率纹理图像施行纹理绘制,该方法以特征点的位移和非特征点与特征点的相对位置为基础计算局部人脸面部的变形,同时还能够实现人脸剧烈的面部变化和动作,与肌肉模型相结合,可很好地实时完成人脸的动画,具有快速高效的特点.最后,给出了所得到的实验结果.     

An efficient search strategy for block motion estimation usingimage features

Block motion estimation using the exhaustive full search is computationally intensive. Fast search algorithms offered in the past tend to reduce the amount of computation by limiting the number of locations to be searched. Nearly all of these algorithms rely on this assumption: the mean absolute difference (MAD) distortion function increases monotonically as the search location moves away from the global minimum. Essentially, this assumption requires that the MAD error surface be unimodal over the search window. Unfortunately, this is usually not true in real-world video signals. However, we can reasonably assume that it is monotonic in a small neighborhood around the global minimum. Consequently, one simple strategy, but perhaps the most efficient and reliable, is to place the checking point as close as possible to the global minimum. In this paper, some image features are suggested to locate the initial search points. Such a guided scheme is based on the location of certain feature points. After applying a feature detecting process to each frame to extract a set of feature points as matching primitives, we have extensively studied the statistical behavior of these matching primitives, and found that they are highly correlated with the MAD error surface of real-world motion vectors. These correlation characteristics are extremely useful for fast search algorithms. The results are robust and the implementation could be very efficient. A beautiful point of our approach is that the proposed search algorithm can work together with other block motion estimation algorithms. Results of our experiment on applying the present approach to the block-based gradient descent search algorithm (BBGDS), the diamond search algorithm (DS) and our previously proposed edge-oriented block motion estimation show that the proposed search strategy is able to strengthen these searching algorithms. As compared to the conventional approach, the new algorithm, through the extraction of image features, is more robust, produces smaller motion compensation errors, and has a simple computational complexity.     

Super-resolution without dense flow

Super-resolution is a widely applied technique that improves the resolution of input images by software methods. Most conventional reconstruction-based super-resolution algorithms assume accurate dense optical flow fields between the input frames, and their performance degrades rapidly when the motion estimation result is not accurate enough. However, optical flow estimation is usually difficult, particularly when complicated motion is presented in real-world videos. In this paper, we explore a new way to solve this problem by using sparse feature point correspondences between the input images. The feature point correspondences, which are obtained by matching a set of feature points, are usually precise and much more robust than dense optical flow fields. This is because the feature points represent well-selected significant locations in the image, and performing matching on the feature point set is usually very accurate. In order to utilize the sparse correspondences in conventional super-resolution, we extract an adaptive support region with a reliable local flow field from each corresponding feature point pair. The normalized prior is also proposed to increase the visual consistency of the reconstructed result. Extensive experiments on real data were carried out, and results show that the proposed algorithm produces high-resolution images with better quality, particularly in the presence of large-scale or complicated motion fields.     

一种改进的窄带算法在彩色序列图像人脸跟踪中的应用

该文提出了一种基于几何主动轮廓模型的人脸跟踪方法.通过直方图反向投影,使人脸区域表现为一个一致性区域与背景相区别.研究了一种改进的窄带算法实现曲线演化:以等间隔分布的节点表示运动曲线,只在这些节点上计算Level set函数的变化值,窄带区内其余点的Level set值的更新通过插值和查表的方法实现;根据节点的局部图像信息决定节点的运动方向和时间步长值.实验表明该算法能在满足一定精度的前提下,快速地对运动人脸进行跟踪.     

Automatic facial feature extraction by genetic algorithms

An automatic facial feature extraction algorithm is presented. The algorithm is composed of two main stages: the face region estimation stage and the feature extraction stage. In the face region estimation stage, a second-chance region growing method is adopted to estimate the face region of a target image. In the feature extraction stage, genetic search algorithms are applied to extract the facial feature points within the face region. It is shown by simulation results that the proposed algorithm can automatically and exactly extract facial features with limited computational complexity.     

Optical Flow Estimation for a Periodic Image Sequence

We propose a temporal modeling approach for determining image motion from a sequence of images wherein the inherent motion is periodic over time. To exploit the periodic nature of the motion, we use a Fourier harmonic representation to model the temporal evolution of the motion field for the entire sequence. We then determine the motion field simultaneously for the different image frames by estimating the parameters of this representation model, where the model order in the Fourier representation serves as a regularization parameter on the temporal coherence of the motion field. This approach can take advantage of the statistics of all the available data in the image sequence. In our experiments, we tested the proposed approach on several motion types at different noise levels, including translational motion, convergent/divergent motion, and cardiac motion. Our results demonstrate that this approach could lead to more robust estimation of the motion field in the presence of strong imaging noise compared to a frame-by-frame estimation approach.     

基于动态时间规整和主动外观模型的动态表情识别

针对静态表情特征缺乏时间信息,不能充分体现表情的细微变化,该文提出一种针对非特定人的动态表情识别方法:基于动态时间规整(Dynamic Time Warping, DTW)和主动外观模型(Active Appearance Model, AAM)的动态表情识别。首先采用基于局部梯度DT-CWT(Dual-Tree Complex Wavelet Transform)主方向模式(Dominant Direction Pattern, DDP)特征的DTW对表情序列进行规整。然后采用AAM定位出表情图像的66个特征点并进行跟踪,利用中性脸的特征点构建人脸几何模型,通过人脸几何模型的匹配克服不同人呈现表情的差异,并通过计算表情序列中相邻两帧图像对应特征点的位移获得表情的变化特征。最后采用最近邻分类器进行分类识别。在CK+库和实验室自建库HFUT-FE(HeFei University of Technology-Face Emotion)上的实验结果表明,所提算法具有较高的准确性。     

动态前景干扰下的电子稳像算法

针对存在动态前景目标的抖动视频序列,提出了一种以块匹配为基础的块差分平方前景目标去除的电子稳像方法。首先,将图像分成大小为16×16的特征块,通过设定阈值筛选出具有足够梯度信息来估计运动矢量的特征块,从而减少参与计算的特征块数量以提高处理速度;然后,采用归一化互相关(Normalized Cross Correlation,NCC)匹配算法和迭代搜索策略来完成特征块的匹配;最后,采用牛顿迭代法计算仿射变换模型下的帧间运动参数,并根据这些参数完成运动补偿,实现图像序列的稳定。实验表明,算法成功地去除了视频序列中动态前景目标的干扰,对存在平移及旋转运动的视频序列,能够快速有效地去除抖动,获得稳定的视频序列。     

基于边界约束的样品视频非对称运动估计方法

针对常规运动估计方法应用到样品视频编码时存在无效搜索点冗余搜索、有效搜索点遗漏的问题,提出基于边界约束的非对称运动估计方法。首先,在原有预定搜索范围的基础上,采用定点统计运动参量的方法对科学仪器的样品视频运动性能进行测试统计,根据统计结果,对视频运动范围设定边界,减少运动搜索点数;然后提出了基于边界约束的非对称搜索模型,依据样品视频的运动特征,优化搜索算法。来自电子探针和电子显微镜的不同样品视频编码实验表明,与多方向搜索算法(MDS)比较,所提方法的运动估计时间缩短了约33%,编码性能保持甚至超过了多方向搜索算法。     

Motion estimation of skeletonized angiographic images using elastic registration

An approach for estimating the motion of arteries in digital angiographic image sequences is proposed. Binary skeleton images are registered using an elastic registration algorithm in order to estimate the motion of the corresponding arteries. This algorithm operates recursively on the skeleton images by considering an autoregressive (AR) model of the deformation in conjunction with a dynamic programming (DP) algorithm. The AR model is used at the pixel level and provides a suitable cost function to DP through the innovation process. In addition, a moving average (MA) model for the motion of the entire skeleton is used in combination with the local AR model for improved registration results. The performance of this motion estimation method is demonstrated on simulated and real digital angiographic image sequences. It is shown that motion estimation using elastic registration of skeletons is very successful especially with low contrast and noisy angiographic images.     

Three-dimensional motion estimation of objects for video coding

Three-dimensional (3-D) motion estimation is applied to the problem of motion compensation for video coding. We suppose that the video sequence consists of the perspective projections of a collection of rigid bodies which undergo a rototranslational motion. Motion compensation can be performed on the sequence once the shape of the objects and the motion parameters are determined. We show that the motion equations of a rigid body can be formulated as a nonlinear dynamic system whose state is represented by the motion parameters and by the scaled depths of the object feature points. An extended Kalman filter is used to estimate both the motion and the object shape parameters simultaneously. The inclusion of the shape parameters in the estimation procedure adds a set of constraints to the filter equations that appear to be essential for reliable motion estimation. Our experiments show that the proposed approach gives two advantages. First, the filter can give more reliable estimates in the presence of measurement noise in comparison with other motion estimators that separately compute motion and structure. Second, the filter can efficiently track abrupt motion changes. Moreover, the structure imposed by the model implies that the reconstructed motion is very natural as opposed to more common block-based schemes. Also, the parameterization of the model allows for a very efficient coding of the motion information     

Reliable tracking of facial features in semantic-based video coding

A new method of tracking the position of important facial features for semantic-based moving image coding is presented. Reliable and fast tracking of the facial features in head-and-shoulders scenes is of paramount importance for reconstruction of the speakers motion in videophone systems. The proposed method is based on eigenvalue decomposition of the sub-images extracted from subsequent frames of the video sequence. The motion of each facial feature (the left eye, the right eye, the nose and the lips) is tracked separately; this means that the algorithm can be easily adapted for a parallel machine. No restrictions, other than the presence of the speaker's face, were imposed on the actual contents of the scene. The algorithm was tested on numerous widely used head-and-shoulders video sequences containing moderate head pan, rotation and zoom, with remarkably good results. Tracking was maintained even when the facial features were occluded. The algorithm can also be used in other semantic-based systems     

A media conversion from speech to facial image for intelligentman-machine interface

An automatic field motion image synthesis scheme (driven by speech) and a real-time image synthesis design are presented. The purpose of this research is to realize an intelligent human-machine interface or intelligent communication system with talking head images. A human face is reconstructed on the display of a terminal using a 3-D surface model and texture mapping technique. Facial motion images are synthesized naturally by transformation of the lattice points on 3-D wire frames. Two driving motion methods, a text-to-image conversion scheme and a voice-to-image conversion scheme, are proposed. In the first method, the synthesized head image can appear to speak some given words and phrases naturally. In the second case, some mouth and jaw motions can be synthesized in synchronization with voice signals from a speaker. Facial expressions other than mouth shape and jaw position can be added at any moment, so it is easy to make the facial model appear angry, to smile, to appear sad, etc., by special modification rules. These schemes were implemented on a parallel image computer system. A real-time image synthesizer was able to generate facial motion images on the display at a TV image video rate     

平面旋转人脸检测与特征定位方法研究

提出了一种基于角点检测、AdaBoost算法和C-V方法的平面旋转人脸检测及特征定位方法.方法首先根据AdaBoost算法训练样本得到脸、眼、鼻、嘴4个检测器;然后以角点作为眼睛的候选点,枚举任意两个角点构造可能的人脸区域,并在区域内运用人脸检测器进行检测;接着利用眼、鼻、嘴检测器检测出人脸特征所在的矩形区域;最后利用C-V方法从各个特征区域中分割出人脸特征的轮廓,进而得到人脸关键特征点的位置.在CMU平面旋转测试集上的检测率为94.6%,误报24个,提取出的特征点位置准确.实验结果表明方法是有效的.