基于弱监督学习的三维人脸形状与纹理重建

三维人脸相较于二维人脸包含了更多特征信息, 可应用于如人脸识别、影视娱乐、医疗美容等更多实际应用场景, 因此三维人脸重建技术一直是计算机视觉领域的研究热点. 由于真实三维人脸数据较难获取, 很多基于深度学习的重建算法首先利用传统重建方法为大量二维人脸图像构建三维标签, 作为训练数据, 这些数据可能并不精准, 从而导致算法的重建精度受到影响. 为此, 本文提出一种基于multi-level损失函数的弱监督学习模型, 结合传统三维人脸形变模型3DMM与深度学习方法, 直接从大量无三维标签的二维人脸图像中学习三维人脸特征信息, 从而实现基于单张二维人脸图像的三维人脸重建算法. 此外, 为解决二维人脸图像中常存在遮挡或大姿态情况而影响人脸纹理重建的问题, 本文使用基于CelebAMask-HQ数据集的人脸解析分割算法对图像进行预处理去除遮挡区域. 实验结果表明, 基于本文方法的三维人脸重建质量与重建精度均实现了一定的提升.

3D Face Shape and Texture Reconstruction Based on Weakly Supervised Learning

Compared with two-dimensional faces, three-dimensional faces contain more feature information and can be applied to more practical application scenarios, such as face recognition, film and television entertainment, medical beauty, etc. Therefore, 3D face reconstruction technology has become a research focus in the field of computer vision. Due to real 3D face data is difficult to obtain, many deep learning-based reconstruction algorithms first use traditional reconstruction methods to construct 3D labels for a large number of 2D face images. These training data may not be accurate which will affect the reconstructive accuracy of these algorithm. To this end, this study proposes a weakly supervised learning model based on a multi-level loss function, which combines traditional 3D morphable model 3DMM and deep learning methods to directly learn 3D face feature from a large number of 2D face images without 3D labels to implement a3D face reconstruction algorithm based on a single 2D face image. In addition, in order to solve the problem that occlusion or large poses in 2D face images often affect the reconstruction of face texture, this paper uses a face parse segmentation algorithm based on the CelebAMask-HQ dataset to preprocess the images to remove the occlusion areas. The experimental results show that the quality and accuracy of 3D face reconstruction based on the proposed method have been improved greatly.