Free form deformation based face cartoon stylization

A novel free form based face cartoon stylization method is presented in this paper. First, a face cartoon library with marked feature points is constructed. And then select the input image as the target image and an appropriate cartoon image from cartoon library as the reference image, apply the deformation between the corresponding feature points of the images to the target image. Finally, we apply an image stylization process to the result image. As an alternative method, we also choose an appropriate cartoon image as the target image and the input image as the reference image to apply the free form deformation. The experimental results show that our method is straightforward and quick with diversified styles, delivering more infection.     

利用图像变形生成个性化人脸卡通

提出了一种利用图像变形技术生成个性化人脸卡通的方法,该方法以真实人脸图像、卡通人脸图像和中性标准人脸网格为输入。根据提取的特征点,在人脸上分区域构建多个径向基神经网络;将标准人脸网格变形,分别和人脸以及卡通脸配准,得到人脸和卡通脸的个性化网格;将卡通脸图像作为纹理映射到个性化人脸网格,并做色调分离提取原始人脸的光照信息,得到个性化卡通人脸图像。     

Semi-automatic cartoon generation by motion planning

To reduce tedious work in cartoon animation, some computer-assisted systems including automatic Inbetweening and cartoon reusing systems have been proposed. In existing automatic Inbetweening systems, accurate correspondence construction, which is a prerequisite for Inbetweening, cannot be achieved. For cartoon reusing systems, the lack of efficient similarity estimation method and reusing mechanism makes it impractical for the users. The semi-supervised graph-based cartoon reusing approach proposed in this paper aims at generating smooth cartoons from the existing data. In this approach, the similarity between cartoon frames can be accurately evaluated by calculating the distance based on local shape context, which is expected to be rotation and scaling invariant. By the semi-supervised algorithm, given an initial frame, the most similar cartoon frames in the cartoon library are selected as candidates of the next frame. The smooth cartoons can be generated by carrying out the algorithm repeatedly to select new cartoon frames after the cartoonists specifying the motion path in a background image. Experimental results of the candidate frame selection in our cartoon dataset suggest the effectiveness of the proposed local shape context for similarity evaluation. The other experiments show the excellent performance on cartoon generation of our approach.     

An MLS-based cartoon deformation

We present an image deformation method driven by skeleton; it is based on MLS deformation algorithm (Schaefer et al. in SIGGRAPH, vol. 25, pp. 533–540, 2006). We improve the MLS deformation by defining a new weight function based on skeleton. Being different from the weight function based on control points, our weight function has benefited from the shape information of undeformed object and keeps deformation local, therefore our method can achieve a realistic effect. In cartoon video, we propose a new method to track the skeleton in the video, to build new origin skeleton and new target skeleton on each frame, and to apply our image deformation method to each frame and maintain spatiotemporal consistency. Results demonstrate that our method is able to decrease the effect of squeeze and use less control points.     

骨架驱动的三维模型变形方法

为发展三维网格模型的变形技术,研究了多种三维模型变形算法,通过对骨架驱动变形算法的深入研究,针对现行算法多是以单一骨架驱动变形的不足,提出了一种新的基于多骨架点驱动的交互式局部变形方法.有效结合模型的骨架图结构,确定各骨架点对应的局部区域.并将骨架点拟合为二次Bézier曲线,通过交互式拖动任意骨架点计算与之相连的多骨架点的动态变化,实现模型局部区域的自然形变.实验结果表明了该算法的有效性和直观性.     

Real-time face tracking and recognition by sparse eigentracker with associative mapping to 3D shape

This paper proposes a novel framework of real-time face tracking and recognition by combining two eigen-based methods. The first method is a novel extension of eigenface called augmented eigenface and the second method is a sparse 3D eigentemplate tracker controlled by a particle filter. The augmented eigenface is an eigenface augmented by an associative mapping to 3D shape that is specified by a set of volumetric face models. This paper discusses how to make up the augmented eigenface and how it can be used for inference of 3D shape from partial images. The associative mapping is also generalized to subspace-to-one mappings to cover photometric image changes for a fixed shape. A novel technique, called photometric adjustment, is introduced for simple implementation of associative mapping when an image subspace should be combined to a shape. The sparse 3D eigentemplate tracker is an extension of the 3D template tracker proposed by Oka et al. In combination with the augmented eigenface, the sparse 3D eigentemplate tracker facilitates real-time 3D tracking and recognition when a monocular image sequence is provided. In the tracking, sparse 3D eigentemplate is updated by the augmented eigenface while face pose is estimated by the sparse eigentracker. Since the augmented eigenface is constructed on the conventional eigenfaces, face identification and expression recognition are also accomplished efficiently during the tracking. In the experiment, an augmented eigenface was constructed from 25 faces where 24 images were taken in different lighting conditions for each face. Experimental results show that the augmented eigenface works with the 3D eigentemplate tracker for real-time tracking and recognition.     

An efficient 3D face recognition approach using local geometrical signatures

This paper presents a computationally efficient 3D face recognition system based on a novel facial signature called Angular Radial Signature (ARS) which is extracted from the semi-rigid region of the face. Kernel Principal Component Analysis (KPCA) is then used to extract the mid-level features from the extracted ARSs to improve the discriminative power. The mid-level features are then concatenated into a single feature vector and fed into a Support Vector Machine (SVM) to perform face recognition. The proposed approach addresses the expression variation problem by using facial scans with various expressions of different individuals for training. We conducted a number of experiments on the Face Recognition Grand Challenge (FRGC v2.0) and the 3D track of Shape Retrieval Contest (SHREC 2008) datasets, and a superior recognition performance has been achieved. Our experimental results show that the proposed system achieves very high Verification Rates (VRs) of 97.8% and 88.5% at a 0.1% False Acceptance Rate (FAR) for the “neutral vs. nonneutral” experiments on the FRGC v2.0 and the SHREC 2008 datasets respectively, and 96.7% for the ROC III experiment of the FRGC v2.0 dataset. Our experiments also demonstrate the computational efficiency of the proposed approach.     

基于轮廓线局部描述符的三维人脸识别

提出了一种基于等测地轮廓线的局部描述符来识别三维人脸。首先对三维人脸数据进行预处理, 得到统一的人脸区域并进行姿态归一化; 然后根据测地距离提取到鼻尖点相同距离的点组成等测地轮廓线, 对轮廓线进行重采样, 并对轮廓线上每个采样点的邻域提取局部描述符; 最后在建立测试人脸和库集人脸的点对应关系后进行局部描述符的加权融合和比较, 给出最终识别结果。算法在FRGC(face recognition grand challenge)v2. 0数据库上进行测试, 实验结果表明该方法具有较好的识别性能。     

2D face fitting-assisted 3D face reconstruction for pose-robust face recognition

Recent face recognition algorithm can achieve high accuracy when the tested face samples are frontal. However, when the face pose changes largely, the performance of existing methods drop drastically. Efforts on pose-robust face recognition are highly desirable, especially when each face class has only one frontal training sample. In this study, we propose a 2D face fitting-assisted 3D face reconstruction algorithm that aims at recognizing faces of different poses when each face class has only one frontal training sample. For each frontal training sample, a 3D face is reconstructed by optimizing the parameters of 3D morphable model (3DMM). By rotating the reconstructed 3D face to different views, pose virtual face images are generated to enlarge the training set of face recognition. Different from the conventional 3D face reconstruction methods, the proposed algorithm utilizes automatic 2D face fitting to assist 3D face reconstruction. We automatically locate 88 sparse points of the frontal face by 2D face-fitting algorithm. Such 2D face-fitting algorithm is so-called Random Forest Embedded Active Shape Model, which embeds random forest learning into the framework of Active Shape Model. Results of 2D face fitting are added to the 3D face reconstruction objective function as shape constraints. The optimization objective energy function takes not only image intensity, but also 2D fitting results into account. Shape and texture parameters of 3DMM are thus estimated by fitting the 3DMM to the 2D frontal face sample, which is a non-linear optimization problem. We experiment the proposed method on the publicly available CMUPIE database, which includes faces viewed from 11 different poses, and the results show that the proposed method is effective and the face recognition results toward pose variants are promising.     

A deformation model to reduce the effect of expressions in 3D face recognition

In 3D face recognition, most work utilizes the rigid parts of face surfaces for matching to exclude the distortion caused by expressions. However, across a broad range of expressions, the rigid parts may not always be uniform and cover large parts of faces. On the other hand, the non-rigid regions of face surfaces also contain useful information for recognition. In this paper, we include the non-rigid regions besides the rigid parts for 3D face recognition. A deformation model is proposed to deform the non-rigid regions to the shapes that are more similar between intra-personal samples but less similar between inter-personal samples. Together with the rigid regions, the deformed parts make samples more discriminable so that the effect of expressions is reduced. The first part of our model uses the target gradient fields from enrolled samples to depress the distortion of the non-rigid regions. The gradient field works in the differential domain. According to the Poisson equation, a smooth deformed shape can be computed by a linear system. The second part of the model is the definition of a surface property that determines the deformation ability of different face regions. Unlike the target gradient fields that improve the similarity of intra-personal samples, the original topology and surface property can keep inter-personal samples sufficiently dissimilar. Our deformation model can be used to improve existing 3D face recognition methods. Experiments are carried out on FRGC and BU-3DFE databases. There are about 8–10% improvements obtained after applying this deformation model to the baseline ICP method. Compared with other deformation models, the experimental results show that our model has advantages on both recognition performance and computational efficiency.     

三维模型任意剖面的程序式凹凸映射生成算法

为解决传统二维凹凸映射无法适用于三维模型任意剖面的问题,提出一种三维模型任意剖面的程序式凹凸映射生成算法。该算法使用球、圆柱等作为模型内部的微观颗粒,使用三维Perlin噪声对颗粒添加扰动,提升模拟真实感。根据颗粒与剖面的位置关系,将剖面分为撕裂面、空心切割面、实心切割面。分别应用不同的程序式凹凸映射算法,计算法线扰动信息,得到剖面的凹凸映射纹理。仿真结果表明,该算法可以在三维模型任意剖面上产生凹凸映射效果,渲染一帧大约只需0.5ms,满足实时切割渲染的需求。     

基于改进三维形变模型的三维人脸重建和密集人脸对齐方法

针对现在广泛使用的三维形变模型表达能力不够,导致重建出的三维人脸模型泛化性能不佳的问题,提出了一种在姿态、表情和光照未知的条件下的基于单张人脸图片的三维人脸重建和密集人脸对齐的新方法。首先,通过卷积神经网络对现有的三维形变模型进行改进,以提高三维人脸模型的表达能力;然后,基于人脸光滑性和图像相似性,在特征点和像素层面提出新的损失函数,并使用弱监督学习训练卷积神经网络模型;最后,通过训练出的网络模型进行三维人脸重建和密集人脸对齐。实验结果表明,对于三维人脸重建任务,所提模型在AFLW2000-3D上实现了2.25的归一化平均误差;对于密集人脸对齐任务,所提模型在AFLW2000-3D和AFLW-LFPA上分别实现了3.80和3.34的归一化平均误差。与原始使用三维形变模型的方法相比,所提模型在三维人脸重建和密集人脸对齐上的归一化平均误差分别降低了7.4%和7.8%。针对不同光照环境以及角度的人脸图片,该网络模型的重建准确,鲁棒性好,且具有较高的三维人脸重建和密集人脸对齐质量。     

基于改进三维形变模型的三维人脸重建和密集人脸对齐方法

针对现在广泛使用的三维形变模型表达能力不够，导致重建出的三维人脸模型泛化性能不佳的问题，提出了一种在姿态、表情和光照未知的条件下的基于单张人脸图片的三维人脸重建和密集人脸对齐的新方法。首先，通过卷积神经网络对现有的三维形变模型进行改进，以提高三维人脸模型的表达能力；然后，基于人脸光滑性和图像相似性，在特征点和像素层面提出新的损失函数，并使用弱监督学习训练卷积神经网络模型；最后，通过训练出的网络模型进行三维人脸重建和密集人脸对齐。实验结果表明，对于三维人脸重建任务，所提模型在AFLW2000-3D上实现了2.25的归一化平均误差；对于密集人脸对齐任务，所提模型在AFLW2000-3D和AFLW-LFPA上分别实现了3.80和3.34的归一化平均误差。与原始使用三维形变模型的方法相比，所提模型在三维人脸重建和密集人脸对齐上的归一化平均误差分别降低了7.4%和7.8%。针对不同光照环境以及角度的人脸图片，该网络模型的重建准确，鲁棒性好，且具有较高的三维人脸重建和密集人脸对齐质量。     

改进的三维人脸识别方法

针对目前三维人脸几何特征识别算法中计算量大和设备昂贵,尤其是在特征融合时加权值确定的不精确性问题,提出了根据双目立体视觉原理,通过对普通二维图像确定脸部关键部位特征点的三维几何特征信息,并且依照类内距离越小越好,类间距离越大越好的准则设定适应度函数,使用人脸样本数据根据遗传算法进行训练,得到使适应度函数最小时的最优解,从而获得三维人脸几何特征融合时的最佳加权值。实验结果表明了该算法的可行性和有效性。     

Robust3D: a robust 3D face reconstruction application

In the process of reconstructing a historical event such as a rock concert only from video, the reconstruction of faces and expressions of the musicians is obviously important. However, in the process of rebuilding appearance, because of the low quality of the video of the recorded concert, the result of the reconstruction may be far from the real appearance. In this paper, a robust 3D face reconstruction application is described that can be applied to a video recording. The application first uses DeblurGAN program to run anti-ambiguity calculation and removes the ambiguity in the concert video. Then, the super-resolution program is used to enlarge every frame of the concert video by four times, thus making every frame of the video clearer. Finally, the 3D faces are obtained after 3D reconstruction of the processed video frames via the 3DMM_CNN program.

     

三维人脸建模及在跨姿态人脸匹配中的有效性验证

针对现有三维人脸采集技术对采集场景存在诸多限制,提出了自由场景下基于多张图像的三维人脸建模技术,并对其进行了有效性验证。首先,提出一个姿态及深度值迭代计算模型,实现了特征点深度值的准确估计;然后,进行了基于多张图像的深度值融合及整体形状建模;最后,将深度迭代优化算法（IPDO）与目前最优的非线性最小二乘法（NLS1_SR）在Bosphorus Database数据集上进行了对比,建模精度提高了9%,所重建的三维人脸模型投影图像与二维图像具有较高的相似度。实验结果表明,在大姿态变化条件下,该识别算法借助三维信息相较于未借助的情况下,其识别率可以提高50%以上。