Real-time recovery of moving 3D faces for emerging applications

Existing face imaging systems are not suitable to meet the face representation and recognition demands for emerging applications in areas such as interactive gaming, enhanced learning environments and directed advertising. This is mainly due to the poor capture and characterisation of facial data that compromises their spatial and temporal precision. For emerging applications it is not only necessary to have a high level of precision for the representation of facial data, but also to characterise dynamic faces as naturally as possible and in a timely manner. This study proposes a new framework for capturing and recovering dynamic facial information in real-time at significantly high order of spatial and temporal accuracy to capture and model subtle facial changes for enhanced realism in 3D face visualisation and higher precision for face recognition applications. We also present a novel, fast, and robust correspondence mapping approach for 3D registration of moving 3D faces.     

从视频中恢复三维人脸的实时方法

三维人脸恢复是视觉交互的一个难点问题,提出了一种从视频中实时恢复三维人脸的新方法.该方法利用主动形状模型进行人脸特征点提取和跟踪,确保了三维形状恢复和特征跟踪的有效性和一致性;采用非刚体形状和运动估计方法构建三维形变基,有效地适应人脸形状变化的多样性;采用非线性优化算法估算人脸姿态和三维形变基参数,实现了三维人脸形状和姿态的实时恢复.实验结果表明,该方法不仅能从视频中实时恢复三维人脸模型,而且可有效跟踪人脸各种姿态的变化.     

Reshaping 3D facial scans for facial appearance modeling and 3D facial expression analysis

3D face scans have been widely used for face modeling and analysis. Due to the fact that face scans provide variable point clouds across frames, they may not capture complete facial data or miss point-to-point correspondences across various facial scans, thus causing difficulties to use such data for analysis. This paper presents an efficient approach to representing facial shapes from face scans through the reconstruction of face models based on regional information and a generic model. A new approach for 3D feature detection and a hybrid approach using two vertex mapping algorithms, displacement mapping and point-to-surface mapping, and a regional blending algorithm are proposed to reconstruct the facial surface detail. The resulting models can represent individual facial shapes consistently and adaptively, establishing facial point correspondences across individual models. The accuracy of the generated models is evaluated quantitatively. The applicability of the models is validated through the application of 3D facial expression recognition using the static 3DFE and dynamic 4DFE databases. A comparison with the state of the art has also been reported.     

BJUT-3D三维人脸数据库及其处理技术

BJUT-3D是目前国际上最大的中国人的三维人脸数据库,其中包括经过顸处理的1200名中国人的三维人脸数据,这一数据资源对于三维人脸识别与建模方面的研究有重要意义.首先介绍了BJUT-3D数据库的数据获取条件、数据形式,并针对数据库建立过程中数据预处理技术进行了讨论.最后作为数据库的直接应用,进行了多姿态人脸识别和人脸姿态估计算法的研究.实验结果证实,该算法具有良好的性能.     

3D颅骨和相貌匹配技术

以三棱柱为同一参照物,采用超平面拟和方法拟和三棱柱的4个平面,然后提取顶面的三角形,通过重合技巧使两个三角形重合,完成3D颅骨和相貌之间匹配。     

A template of ease allowance for garments based on a 3D reverse methodology

Nowadays, with the very competitive environment and the emergence of the mass customization concept, garment design is become a crucial step for clothing companies. Indeed, garments should be designed quickly under controlled costs and perfectly fitted to consumers. No traditional pattern making in 2D matches these new constraints since it requires the production of expensive and time consuming physical prototypes. The study presented in this paper relies on 3D virtual design that lets achieve customized garments. An original image processing method based on ellipses and a reverse methodology using scans of a reference body and garment enables evaluation of the overall 3D ease of the garment. The garment is then designed virtually and the result is checked with a 3D simulation on the reference body.     

3D printed deformable product handle material for improved ergonomics

In product handle ergonomic design optimisation researchers focused mainly on the size and shape of the handles; however, interface handle materials have been neglected despite showing potential to improve ergonomics. Deformable elastic cellular meta-materials with pre-engineered mechanical response based on the biomechanical evaluation of human hand soft tissue during grasping were designed and manufactured using commercial 3D printing technology. A sawing task has been utilized for the evaluation of subjective comfort rating. Based on distinct mechanical behaviour of cellular solids, 3D printed handle interface material stays stiff at the low grasping forces and deforms only when certain amount of contact pressure is reached. Cellular density can be easily adjusted to meet the desired biomechanical response. Hereby stability of the handle in hands is maximised while providing more uniform contact pressure distribution on the soft tissue at higher grasping forces. By this means comfort rating is also increased compared to stiff handle interface materials such as plastic. Results also suggest the handle material has greater influence on the comfort rating than the handle size and shape.Relevance to industryApplication of this research includes the utilization of this methodology and design techniques in development of handles for powered and non-powered tools and handheld products for improved comfort and also ergonomics.     

三维人脸建模与应用

三维建模是计算机视觉和计算机图形学领域中一个基本的问题，人脸借助其特有的普遍性和易用性成为众多先进三维建模算法的实验平台。但是由于人脸的复杂性、易变性，建立逼真的三维人脸模型成为众多研究者挑战的课题。本文分析了三维人脸建模的研究内容和研究难点，总结了1990年以来出现的最新方法和进展，从三维数据的获取、标准三维人脸建模、特定三维人脸建模及应用等三维建模的几个主要环节进行了分类综述，重点介绍了一些主要算法的基本思路和最新进展，并对该领域的研究热点和方向作了较为详细的分析。     

Realistic wrinkle generation for 3D face modeling based on automatically extracted curves and improved shape control functions

Realistic wrinkles are extremely important for enhancing the realism of three-dimensional (3D) virtual face models. This paper proposes an approach for generating realistic wrinkle on a 3D face model based on a face image. It includes image preprocessing, automatically extracting wrinkle curves and generating wrinkles on a 3D surface. For image preprocessing, we use a linear transform method to conduct a grayscale conversion. We then use a transfinite-pixel neighborhood averaging method to reduce the noise, and a high pass filter to sharpen the image. For the automatic extraction of wrinkle curves, an improved Canny edge detector is employed. For wrinkle generation on a 3D surface, a number of novel techniques are employed. Some feature points are firstly defined both on the face image and on the 3D face model. By aligning these feature points, the extracted wrinkle curves are then projected onto the 3D face model. Finally, three shape control functions are used to produce more realistic properties of the 3D wrinkles. They are the proposed cross-section shape control function (CSCF) to determine the cross-section shape and size, the depth attenuation function (DAF) and the width attenuation function (WAF) to control the depth and width amplitude variations of the wrinkles, respectively. For better results, an adaptive subdivision is applied to the predefined influence region to adjust the resolution around the wrinkle mesh. The experiment results of applying this method to individualized 3D human models demonstrate the ability of our method to generate more natural wrinkles.     

基于改进三维形变模型的三维人脸重建和密集人脸对齐方法

针对现在广泛使用的三维形变模型表达能力不够,导致重建出的三维人脸模型泛化性能不佳的问题,提出了一种在姿态、表情和光照未知的条件下的基于单张人脸图片的三维人脸重建和密集人脸对齐的新方法。首先,通过卷积神经网络对现有的三维形变模型进行改进,以提高三维人脸模型的表达能力;然后,基于人脸光滑性和图像相似性,在特征点和像素层面提出新的损失函数,并使用弱监督学习训练卷积神经网络模型;最后,通过训练出的网络模型进行三维人脸重建和密集人脸对齐。实验结果表明,对于三维人脸重建任务,所提模型在AFLW2000-3D上实现了2.25的归一化平均误差;对于密集人脸对齐任务,所提模型在AFLW2000-3D和AFLW-LFPA上分别实现了3.80和3.34的归一化平均误差。与原始使用三维形变模型的方法相比,所提模型在三维人脸重建和密集人脸对齐上的归一化平均误差分别降低了7.4%和7.8%。针对不同光照环境以及角度的人脸图片,该网络模型的重建准确,鲁棒性好,且具有较高的三维人脸重建和密集人脸对齐质量。     

基于改进三维形变模型的三维人脸重建和密集人脸对齐方法

针对现在广泛使用的三维形变模型表达能力不够，导致重建出的三维人脸模型泛化性能不佳的问题，提出了一种在姿态、表情和光照未知的条件下的基于单张人脸图片的三维人脸重建和密集人脸对齐的新方法。首先，通过卷积神经网络对现有的三维形变模型进行改进，以提高三维人脸模型的表达能力；然后，基于人脸光滑性和图像相似性，在特征点和像素层面提出新的损失函数，并使用弱监督学习训练卷积神经网络模型；最后，通过训练出的网络模型进行三维人脸重建和密集人脸对齐。实验结果表明，对于三维人脸重建任务，所提模型在AFLW2000-3D上实现了2.25的归一化平均误差；对于密集人脸对齐任务，所提模型在AFLW2000-3D和AFLW-LFPA上分别实现了3.80和3.34的归一化平均误差。与原始使用三维形变模型的方法相比，所提模型在三维人脸重建和密集人脸对齐上的归一化平均误差分别降低了7.4%和7.8%。针对不同光照环境以及角度的人脸图片，该网络模型的重建准确，鲁棒性好，且具有较高的三维人脸重建和密集人脸对齐质量。     

多视角2.5维人脸数据的自动配准与三维融合*

提出了一种有效的从三视角2.5维人脸图像到完整三维人脸模型的融合方法。首先用ICP(迭代最近点)方法对三视角人脸图像上手动选取的特征区域进行粗配准,然后用一种调整能量最优方法进行精确配准,最终合成完整的三维人脸模型。通过对融合结果进行相似度测量,实验说明了方法的有效性和优越性。     

Representation of a human head with bi-cubic B-splines technique based on the laser scanning technique in 3D surface anthropometry

Three-dimensional (3D) anthropometry based on the laser scanning technique not only provides one-dimensional measurements calculated in accordance with the landmarks which are pre-located on the human body surface manually, but also the 3D shape information between the landmarks. This new technique used in recent ergonomic research has brought new challenges to resolving the application problem that was generally avoided by anthropometric experts in their researches. The current research problem is concentrating on how to shift and develop one-dimensional measurements (1D landmarks) into three-dimensional measurements (3D land-surfaces). The main purpose of this paper is to test whether the function of B-splines can be used to fit 3D scanned human heads, and to for further study to develop a computer aided ergonomic design tool (CAED). The result shows that B-splines surfaces can effectively reconstruct 3D human heads based on the laser scanning technique.     

三维人脸建模及在跨姿态人脸匹配中的有效性验证

针对现有三维人脸采集技术对采集场景存在诸多限制,提出了自由场景下基于多张图像的三维人脸建模技术,并对其进行了有效性验证。首先,提出一个姿态及深度值迭代计算模型,实现了特征点深度值的准确估计;然后,进行了基于多张图像的深度值融合及整体形状建模;最后,将深度迭代优化算法（IPDO）与目前最优的非线性最小二乘法（NLS1_SR）在Bosphorus Database数据集上进行了对比,建模精度提高了9%,所重建的三维人脸模型投影图像与二维图像具有较高的相似度。实验结果表明,在大姿态变化条件下,该识别算法借助三维信息相较于未借助的情况下,其识别率可以提高50%以上。     

三维人脸建模研究的进展与展望

介绍三维人脸建模的研究目的及内容,分析该领域的研究热点和方向,以及未来的技术应用与发展.三维人脸建模及动画技术是计算机图形学研究的一个热点,人脸以其普遍性和易用性成为众多三维建模算法的实践平台,同时又由于人脸的复杂性和易变性,三维人脸建模也是一个极具挑战性的研究课题.     

基于D3D立体镜显示器图像生成算法研究

介绍了自动立体镜显示器的结构及其三维显示的原理,研究了基于D3D技术的自动立体镜三维图像生成原理,对点蒙板方法做了深入的研究,给出了九片图中观察角度、位置以及视点观察空间的变换运算公式,在分析其优缺点的基础上提出了改进算法。并对最终图像的合成算法作了改进。在自动立体镜显示器上分别用原始方法、点蒙板方法、以及改进的点蒙板方法对图像进行显示,并在显示前准备时间、一帧图像渲染时间、三维深度感觉三方面对三者进行了对比实验。     

Relationship between eye fixation patterns and Kansei evaluation of 3D chair forms

Understanding how to induce Kansei (emotion or affect) in consumers through form is critical in product design and development. Conventional Kansei evaluations, which involve subjectively evaluating the overall form of a product, do not clarify the effects of the individual parts of a product on people’s Kansei evaluation. A microscale analysis of eye movement of people looking at product form may redeem this flaw in subjective evaluation. However, simultaneously recording eye movement when people making Kansei evaluation is challenging, previous studies have typically investigated either the relationship between form and eye movement or the relationship between form and Kansei separately. The eye movement of people while performing Kansei evaluations on product forms still has not been clarified. To address this issue, the present study used an eye tracking system to analyze the changes in the fixation points of people performing various Kansei evaluations. Twenty participants were recruited for 8 Kansei evaluations on the form of 16 chairs by using the semantic differential (SD) rating, while their eye movements on these evaluations were tracked simultaneously. Through factor analysis on the data of Kansei evaluations, two principal factors, valence (pleasure) and arousal, were extracted from the 8 Kansei scales to constitute a Kansei plane which is compatible to Russell’s circumplex model (plane) of affect By adopting the factor scores of the 16 chairs as coordinates, the 16 chairs were mapped into the Kansei plane. Further analysis on the eye fixation on the chairs located in this plane concluded the following results: (a) Pleasure had a more significant effect on the participants’ visual attention compared to arousal; the participants required more fixation points when evaluating the chair form that induced displeasure. (b) The participants typically fixated on two parts of the chairs during their Kansei evaluations, namely the seat and the backrest, indicating that seats and backrests are the two primary features people consider when evaluating chairs. The results clarify the effect of various Kansei on eye movements; thereby enable predicting people’s Kansei evaluations of product forms through analyzing their eye movement.     

三维全身扫描仪的建模仿真

随着计算机全面进入3D时代,三维全身扫描系统对社会的价值越来越显著,获取人体表面的三维几何信息,是建立3D人体模型的先决条件。该文提出了一种基于结构光方法的三维全身扫描仪的系统仿真方案,使用3D Studio MAX软件仿真得到虚拟扫描头运动形成的带激光条纹的图像序列,最后用仿真数据进行三维重建。关键步骤包括用自适应阈值法提取激光带中心,深度图像的获取,以及多扫描头数据的融合等。实验取得了满意的效果,获得了人体的完整轮廓线,验证了系统仿真方案的合理性与高效性。