基于特征发现的卡通人脸肖像生成

通过对成年男女各100幅真实照片进行特征提取和特征统计,获得平均人脸特征分布信息,对新输入的人脸照片进行特征比对,发现其相对突出的特征,采用主动形状模型特征提取和特征线对相结合的方法,对突出的特征实现自动变形,生成人物的卡通肖像．实验结果表明,该方法具有人脸数据量大、特征提取和发现的自动化、变形效果好等优点．     

Semi‐Supervised Learning in Reconstructed Manifold Space for 3D Caricature Generation

Recently, automatic 3D caricature generation has attracted much attention from both the research community and the game industry. Machine learning has been proven effective in the automatic generation of caricatures. However, the lack of 3D caricature samples makes it challenging to train a good model. This paper addresses this problem by two steps. First, the training set is enlarged by reconstructing 3D caricatures. We reconstruct 3D caricatures based on some 2D caricature samples with a Principal Component Analysis (PCA)‐based method. Secondly, between the 2D real faces and the enlarged 3D caricatures, a regressive model is learnt by the semi‐supervised manifold regularization (MR) method. We then predict 3D caricatures for 2D real faces with the learnt model. The experiments show that our novel approach synthesizes the 3D caricature more effectively than traditional methods. Moreover, our system has been applied successfully in a massive multi‐user educational game to provide human‐like avatars.     

Example-based caricature generation with exaggeration control

Caricature is a popular artistic media widely used for effective communications. The fascination of caricature lies in its expressive depiction of a person’s prominent features, which is usually realized through the so-called exaggeration technique. This paper proposes a new example-based automatic caricature generation system supporting the exaggeration of both the shape of facial components and the spatial relationships among the components. Given the photograph of a face, the system automatically computes the feature vectors representing the shape of facial components as well as the spatial relationship among the components. Those features are exaggerated and then used to search the learning database for the corresponding caricature components and for arranging the retrieved components to create the caricature. Experimental results show that our system can generate the caricatures of the example style capturing the prominent features of the subjects.     

刚性区域特征点的3维人脸识别

目的 针对3维人脸识别中存在表情变化的问题,提出了一种基于刚性区域特征点的3维人脸识别方法。方法 该方法首先在人脸纹理图像上提取人脸图像的特征点,并删除非刚性区域内的特征点,然后根据采样点的序号,在人脸空间几何信息上得到人脸图像特征点的3维几何信息,并建立以特征点为中心的刚性区域内的子区域,最后以子区域为局部特征进行人脸识别测试,得到不同子区域对人脸识别的贡献,并以此作为依据对人脸识别的结果进行加权统计。结果 在FRGC v2.0的3维人脸数据库上进行实验测试,该方法的识别准确率为98.5%,当错误接受率（FAR）为0.001时的验证率为99.2%,结果表明,该方法对非中性表情下的3维人脸识别具有很好的准确性。结论 该方法可以有效克服表情变化对3维人脸识别的影响,同时对3维数据中存在的空洞和尖锐噪声等因素具有较好的鲁棒性,对提高3维人脸识别性能具有重要意义。     

Manifold Constrained Transfer of Facial Geometric Knowledge for 3D Caricature Reconstruction

3D caricatures are important attractive elements of the interface in virtual environment such as online game. However, very limited 3D caricatures exist in the real world. Meanwhile, creating 3D caricatures manually is rather costly, and even professional skills are needed. This paper proposes a novel and effective manifold transfer algorithm to reconstruct 3D caricatures according to their original 2D caricatures. We first manually create a small dataset with only 100 3D caricature models and use them to initialize the whole 3D dataset. After that, manifold transfer algorithm is carried out to refine the dataset. The algorithm comprises of two steps. The first is to perform manifold alignment between 2D and 3D caricatures to get a "standard" manifold map; the second is to reconstruct all the 3D caricatures based on the manifold map. The proposed approach utilizes and transfers knowledge of 2D caricatures to the target 3D caricatures well. Comparative experiments show that the approach reconstructs 3D caricatures more effectively and the results conform more to the styles of the original 2D caricatures than the Principal Components Analysis (PCA) based method.     

Manifold‐Based 3D Face Caricature Generation with Individualized Facial Feature Extraction

Caricature is an interesting art to express exaggerated views of different persons and things through drawing. The face caricature is popular and widely used for different applications. To do this, we have to properly extract unique/specialized features of a person's face. A person's facial feature not only depends on his/her natural appearance, but also the associated expression style. Therefore, we would like to extract the neutural facial features and personal expression style for different applicaions. In this paper, we represent the 3D neutral face models in BU–3DFE database by sparse signal decomposition in the training phase. With this decomposition, the sparse training data can be used for robust linear subspace modeling of public faces. For an input 3D face model, we fit the model and decompose the 3D model geometry into a neutral face and the expression deformation separately. The neutral geomertry can be further decomposed into public face and individualized facial feature. We exaggerate the facial features and the expressions by estimating the probability on the corresponding manifold. The public face, the exaggerated facial features and the exaggerated expression are combined to synthesize a 3D caricature for a 3D face model. The proposed algorithm is automatic and can effectively extract the individualized facial features from an input 3D face model to create 3D face caricature.     

一种人脸图像局部变形技术

图像局部变形是指仅在图像的一小部分区域内做变形,改变某些局部范围的特征,同时保持其它区域不发生变化。目前常用的图像变形方法基本上考虑的是图像的整体变形,在直接运用于局部变形时效果不佳。文章提出了一种人脸图像局部变形方法,它首先在局部变形区域内确定特征点和影响半径,然后计算出区域像素点几何位置的变化,最后使用双线性插值原理进行灰度赋值。该方法可以获得很好的人脸局部变形效果,具有连续性好、算法简洁、实时互动等优点,在人脸识别领域有可观的应用前景。     

Caricaturistic visualization

Caricatures are pieces of art depicting persons or sociological conditions in a non-veridical way. In both cases caricatures are referring to a reference model. The deviations from the reference model are the characteristic features of the depicted subject. Good caricatures exaggerate the characteristics of a subject in order to accent them. The concept of caricaturistic visualization is based on the caricature metaphor. The aim of caricaturistic visualization is an illustrative depiction of characteristics of a given dataset by exaggerating deviations from the reference model. We present the general concept of caricaturistic visualization as well as a variety of examples. We investigate different visual representations for the depiction of caricatures. Further, we present the caricature matrix, a technique to make differences between datasets easily identifiable.     

Aligned texture map creation for pose invariant face recognition

In last years, Face recognition based on 3D techniques is an emergent technology which has demonstrated better results than conventional 2D approaches. Using texture (180° multi-view image) and depth maps is supposed to increase the robustness towards the two main challenges in Face Recognition: Pose and illumination. Nevertheless, 3D data should be acquired under highly controlled conditions and in most cases depends on the collaboration of the subject to be recognized. Thus, in applications such as surveillance or control access points, this kind of 3D data may not be available during the recognition process. This leads to a new paradigm using some mixed 2D-3D face recognition systems where 3D data is used in the training but either 2D or 3D information can be used in the recognition depending on the scenario. Following this concept, where only part of the information (partial concept) is used in the recognition, a novel method is presented in this work. This has been called Partial Principal Component Analysis (P²CA) since they fuse the Partial concept with the fundamentals of the well known PCA algorithm. This strategy has been proven to be very robust in pose variation scenarios showing that the 3D training process retains all the spatial information of the face while the 2D picture effectively recovers the face information from the available data. Furthermore, in this work, a novel approach for the automatic creation of 180° aligned cylindrical projected face images using nine different views is presented. These face images are created by using a cylindrical approximation for the real object surface. The alignment is done by applying first a global 2D affine transformation of the image, and afterward a local transformation of the desired face features using a triangle mesh. This local alignment allows a closer look to the feature properties and not the differences. Finally, these aligned face images are used for training a pose invariant face recognition approach (P²CA).     

基于二维图像的三维人脸建模技术

建立三维人脸模型和表情动画是计算机图形学领域的一个研究热点。文章提出了一种基于二维图像的三维人脸建模方法,首先在给定的人脸的正侧面照片上提取事先定义好的反映人脸特征的特征点信息,与一个一般人脸模型上对应点的信息进行比较和修改,得到反映给定人脸特征的特定人脸模型。最后使用纹理映射技术给特定人脸模型添加纹理信息,形成真实感的虚拟三维人脸模型。     

Building Three Dimensional Head Models

This paper proposes a camera-based real-time system for building a three dimensional (3D) human head model. The proposed system is first trained in a semi-automatic way to locate the user's facial area and is then used to build a 3D model based on the front and profile views of the user's face. This is achieved by directing the user to position his or her face and profile in a highlighted area, which is used to train a neural network to distinguish the background from the face. With a blink from the user, the system is then capable of accurately locating a set of characteristic feature points on the front and profile views of the face, which are used for the adaptation of a generic 3D face model. This adaptation procedure is initialized with a rigid transformation of the model aiming to minimize the distances of the 3D model feature nodes from the calculated 3D coordinates of the 2D feature points. Then, a nonrigid transformation ensures that the feature nodes are displaced optimally close to their exact calculated positions, dragging their neighbors in a way that deforms the facial model in a natural looking manner. A male hair model is created using a 3D ellipsoid, which is truncated and merged with the adapted face model. A cylindrical texture map is finally built from the two image views covering the whole area of the head by exploiting the inherent face symmetry. The final result is a complete, textured model of a specific person's head.     

基于二维图像三维重建的人脸特征提取技术研究

采用基于二维图像的三维重建对人脸特征进行提取.首先应用平行双目视觉原理获取人脸的二维图像,然后对图像进行预处理,消除图像上的噪音点,增强图像,以便提取特征点,对这些二维图像上的特征点进行优化计算,最后得到整体人脸的三维特征点信息.     

Application of the Self-Organizing Feature Map Algorithm in Facial Image Morphing

A new facial image morphing algorithm based on the Kohonen self-organizing feature map (SOM) algorithm is proposed to generate a smooth 2D transformation that reflects anchor point correspondences. Using only a 2D face image and a small number of anchor points, we show that the proposed morphing algorithm provides a powerful mechanism for processing facial expressions.     

基于双流循环映射网络的肖像漫画化

肖像风格迁移旨在将参考艺术肖像画中迁移到人物照片上,同时保留人物面部的基本语义结构。然而,由于人类视觉对肖像面部语义结构的敏感性,使得肖像风格迁移任务比一般图像的风格迁移更具挑战性,现有的风格迁移方法未考虑漫画风格的抽象性以及肖像面部语义结构的保持,所以应用到肖像漫画化任务时会出现严重的结构坍塌及特征信息混乱等问题。为此,提出了一个双流循环映射网DSCM。首先,引入了一个结构一致性损失来保持肖像整体语义结构的完整性;其次,设计了一个结合U2-Net的特征编码器在不同尺度下帮助网络捕获输入图像更多有用的特征信息;最后,引入了风格鉴别器来对编码后的风格特征进行鉴别从而辅助网络学习到更接近目标图像的抽象漫画风格特征。实验与五种先进方法进行了定性及定量的比较,该方法均优于其他方法,其不仅能够完整地保持肖像的整体结构和面部的基本语义结构,而且能够充分学习到风格类型。     

智能配镜三维特征参数提取方法研究

摘 要：针对智能配镜中三维面部特征点提取算法复杂度较高的问题,提出一种将三维点 云转换为映射图像定位特征点的方法。采用 Voronoi 方法计算面部三角网格各顶点处的高斯曲 率、平均曲率。选取鼻尖、眼角等曲率特征明显的区域估计面部点云姿态。根据曲率旋转不变 性,使用初选的点云方向向量简化旋转矩阵的计算,使面部点云正面朝向视点。将点云映射转 换为图像,三维网格模型中三角面片一对一映射到图像中的三角形。搭建卷积神经网络,使用 Texas 3DFRD 数据集进行模型训练。进行人脸对齐,预测所得各面部特征点分别限制在图像某 三角形中。根据图像中三角形映射查找三维网格模型中对应三角面片,通过三角面片顶点坐标 计算配镜所需的面部特征点位置坐标,实现配镜特征参数的提取。     

Realistic wrinkle generation for 3D face modeling based on automatically extracted curves and improved shape control functions

Realistic wrinkles are extremely important for enhancing the realism of three-dimensional (3D) virtual face models. This paper proposes an approach for generating realistic wrinkle on a 3D face model based on a face image. It includes image preprocessing, automatically extracting wrinkle curves and generating wrinkles on a 3D surface. For image preprocessing, we use a linear transform method to conduct a grayscale conversion. We then use a transfinite-pixel neighborhood averaging method to reduce the noise, and a high pass filter to sharpen the image. For the automatic extraction of wrinkle curves, an improved Canny edge detector is employed. For wrinkle generation on a 3D surface, a number of novel techniques are employed. Some feature points are firstly defined both on the face image and on the 3D face model. By aligning these feature points, the extracted wrinkle curves are then projected onto the 3D face model. Finally, three shape control functions are used to produce more realistic properties of the 3D wrinkles. They are the proposed cross-section shape control function (CSCF) to determine the cross-section shape and size, the depth attenuation function (DAF) and the width attenuation function (WAF) to control the depth and width amplitude variations of the wrinkles, respectively. For better results, an adaptive subdivision is applied to the predefined influence region to adjust the resolution around the wrinkle mesh. The experiment results of applying this method to individualized 3D human models demonstrate the ability of our method to generate more natural wrinkles.     

Bayesian shape model for facial feature extraction and recognition

A facial feature extraction algorithm using the Bayesian shape model (BSM) is proposed in this paper. A full-face model consisting of the contour points and the control points is designed to describe the face patch, using which the warping/normalization of the extracted face patch can be performed efficiently. First, the BSM is utilized to match and extract the contour points of a face. In BSM, the prototype of the face contour can be adjusted adaptively according to its prior distribution. Moreover, an affine invariant internal energy term is introduced to describe the local shape deformations between the prototype contour in the shape domain and the deformable contour in the image domain. Thus, both global and local shape deformations can be tolerated. Then, the control points are estimated from the matching result of the contour points based on the statistics of the full-face model. Finally, the face patch is extracted and normalized using the piece-wise affine triangle warping algorithm. Experimental results based on real facial feature extraction demonstrate that the proposed BSM facial feature extraction algorithm is more accurate and effective as compared to that of the active shape model (ASM).     

匹配图像与3维模型特征点的真实感3维头重建

提出并实现一种基于两张正交图像和一个标准3维头模型,并利用2D图像特征点和3D模型特征点的匹配进行3维头模型重建的算法。首先,进行面部区域和头发区域的分割,利用色彩传递对输入图像进行颜色处理。对正面图像利用改进后的ASM（主动形状模型）模型进行特征点定位。改进局部最大曲率跟踪(LMCT)方法,更为鲁棒的定位了侧面特征点。在匹配图像特征点与标准3维头上预先定义的特征点的基础上,利用径向基函数进行标准头形变,获得特定人的3维头部形状模型。采用重建好的3维头作为桥梁,自动匹配输入图像,进行无缝纹理融合。最后,将所得纹理映射到形状模型上,获得对应输入图像的特定真实感3维头模型。     

基于特征点的特定人脸三维网格的生成

提出了一种在通用人脸模型的基础上,依据特定人脸正面图象和侧面图象得到特定人脸的三维网格模型的方法。该方法首先从特定人脸正面和侧面图象上提取特征点,然后依据这些特征点信息对相应人脸网格模型进行特征点变换,在此基础上对其余非特征点使用插值变换,从而将特定人的特征信息溶入到通用人脸模型中去,由此得到特定人脸的三维网格模型。通过试验取得了一定的效果。     

面向移动设备的快速人脸纹理映射

提出一种面向未来掌上移动设备（如高端手机,PDA等）的灵活实用的纹理映射方法.该方法仅需要一张正面人脸照片作为输入,不要求模型和纹理的精确匹配,通过简单的交互实现在较低资源配置下人脸纹理的提取.采用了一种交互调整映射的方案,通过用户对模型中特征点及其影响区域的编辑,实现对局部纹理坐标的定义,得到满意的映射效果.实验结果表明,文中方法具有较高的效率和真实感,可以用于产生真实感三维人脸表情动画.