A Comprehensive Survey to Face Hallucination

This paper comprehensively surveys the development of face hallucination (FH), including both face super-resolution and face sketch-photo synthesis techniques. Indeed, these two techniques share the same objective of inferring a target face image (e.g. high-resolution face image, face sketch and face photo) from a corresponding source input (e.g. low-resolution face image, face photo and face sketch). Considering the critical role of image interpretation in modern intelligent systems for authentication, surveillance, law enforcement, security control, and entertainment, FH has attracted growing attention in recent years. Existing FH methods can be grouped into four categories: Bayesian inference approaches, subspace learning approaches, a combination of Bayesian inference and subspace learning approaches, and sparse representation-based approaches. In spite of achieving a certain level of development, FH is limited in its success by complex application conditions such as variant illuminations, poses, or views. This paper provides a holistic understanding and deep insight into FH, and presents a comparative analysis of representative methods and promising future directions.     

人脸画像合成研究的综述与对比分析

人脸画像合成通常是在给定一些训练画像-照片的前提下,将一张输入的人脸照片转换为画像的过程.目前并没有一个系统性的实验对比分析揭示当前此过程面临的挑战以及可能的解决思路.文中对具有代表性的各类方法进行综合深入对比与分析.人脸画像合成方法归纳为2类:数据驱动类方法(即基于样本的方法)和模型驱动类方法.数据驱动方法由3类方法组成:基于子空间学习的方法、基于稀疏表示的方法和基于贝叶斯推断的方法.模型驱动方法直接学习照片到画像的映射关系.文中给出一些之前文献中并未发现的有意义的结论和展望.     

Deep Appearance Models: A Deep Boltzmann Machine Approach for Face Modeling

International Journal of Computer Vision - The “interpretation through synthesis” approach to analyze face images, particularly Active Appearance Models (AAMs) method, has become one of...     

结合pHash和稀疏编码的素描人脸合成方法

为解决已有素描人脸合成方法存在的细节模糊和清晰度低的问题,提出一种感知哈希算法（Perceptual Hash,pHash）与稀疏编码（Sparse Coding,SC）相结合的素描人脸合成方法。首先根据图像的信息熵对人脸照片-素描对进行自适应分块处理,利用感知哈希算法计算出大图像块的哈希指纹,并对小图像块进行稀疏编码;然后选取与测试照片块最相似的[K]个初始候选照片块,得到与之对应的素描块;最后引入二次稀疏编码方法,合成最终的素描块,进而合成整幅素描人脸图像。利用现有的人脸数据库验证了算法的有效性,该算法经优化后可用于素描人脸合成。     

多判别器循环生成对抗网络的素描人脸合成

素描人脸合成在娱乐和刑侦领域具有重要应用价值。为了解决传统素描人脸合成方法生成图像面部细节模糊,缺失真实感等问题,改进了CycleGAN网络结构,提出一种基于多判别器循环生成对抗网络的素描人脸合成方法。该方法选取残差网络作为生成网络模型,在生成器隐藏层中增加多个判别器,提高网络对生成图像细节特征的提取能力;并建立了重构误差约束映射关系,最小化生成图像与目标图像之间的距离。通过在CUHK和AR人脸数据库中的对比实验,证明了相比于原始CycleGAN框架该方法性能有明显提升;相比于目前领先的方法,所提方法生成的素描图像细节特征更清晰,真实感更强。     

Face Photo-Sketch Synthesis and Recognition

In this paper, we propose a novel face photo-sketch synthesis and recognition method using a multiscale Markov Random Fields (MRF) model. Our system has three components: 1) given a face photo, synthesizing a sketch drawing; 2) given a face sketch drawing, synthesizing a photo; and 3) searching for face photos in the database based on a query sketch drawn by an artist. It has useful applications for both digital entertainment and law enforcement. We assume that faces to be studied are in a frontal pose, with normal lighting and neutral expression, and have no occlusions. To synthesize sketch/photo images, the face region is divided into overlapping patches for learning. The size of the patches decides the scale of local face structures to be learned. From a training set which contains photo-sketch pairs, the joint photo-sketch model is learned at multiple scales using a multiscale MRF model. By transforming a face photo to a sketch (or transforming a sketch to a photo), the difference between photos and sketches is significantly reduced, thus allowing effective matching between the two in face sketch recognition. After the photo-sketch transformation, in principle, most of the proposed face photo recognition approaches can be applied to face sketch recognition in a straightforward way. Extensive experiments are conducted on a face sketch database including 606 faces, which can be downloaded from our Web site (http://mmlab.ie.cuhk.edu.hk/facesketch.html).     

基于联合模型的人脸识别定位算法仿真研究

研究人脸定位和识别精度问题。由于人脸在拍摄过程中可能出现的形变,位置变化,光照变换等因素影响,造成人脸模糊不清,为了提高人脸识别定位的精确度,提出了一种新的ASM和AAM联合模型迭代的人脸定位和识别算法。首先利用ASM提取人脸轮廓上关键点特征,并对人脸定位。在初始定位的基础上,利用AAM对人脸进行投影,产生训练集合中没有出现的合成人脸数据。以上两步交替进行,产生足够的、稳定的人脸形变图像。识别过程中,将变换矩阵与原始合成数据进行比对。仿真结果显示,改进的方法能稳定地提取人脸轮廓,并准确定位,具有很高的识别效率。     

Pose robust face tracking by combining view-based AAMs and temporal filters

Active appearance models (AAMs) are useful for face tracking for the advantages of detailed face interpretation, accurate alignment and high efficiency. However, they are sensitive to initial parameters and may easily be stuck in local minima due to the gradient-descent optimization, which makes the AAM based face tracker unstable in the presence of large pose deviation and fast motion. In this paper, we propose to combine the view-based AAMs with two novel temporal filters to overcome the limitations. First, we build a new view space based on the shape parameters of AAMs, instead of the model parameters controlling both the shape and appearance, for the purpose of pose estimation. Then the Kalman filter is used to simultaneously update the pose and shape parameters for a better fitting of each frame. Second, we propose a temporal matching filter which is twofold. The inter-frame local appearance constraint is incorporated into AAM fitting, where the mechanism of the active shape model (ASM) is also implemented in a unified framework to find more accurate matching points. Moreover, we propose to initialize the shape with correspondences found by a random forest based local feature matching. By introducing the local information and temporal correspondences, the twofold temporal matching filter improves the tracking stability when confronted with fast appearance changes. Experimental results show that our algorithm is more pose robust than basic AAMs and some state-of-art AAM based methods, and that it can also handle large expressions and non-extreme illumination changes in test video sequences.     

Unsupervised joint face alignment with gradient correlation coefficient

This work proposes an unsupervised joint alignment framework, referred to as “Gradient Correlation Congealing,” which aligns an image ensemble by maximizing a sum of gradient correlation coefficient function defined over all images. We, respectively, develop two different formulations to optimize the objective function regarding the role of “template.” While most existing face alignment methods suffer from outliers, e.g., occlusions, the proposed algorithms are able to align faces undergoing partial occlusions. Moreover, our algorithms can cope with nonuniform illumination changes (even extremely difficult ones), and also, they do not require any predefined templates. We test the novel approaches against four typical joint alignment methods including Least-Squares Congealing, Learned-Miller Congealing, Lucas–Kanade entropy Congealing, and RASL using three challenging face databases: AR, Yale B, and LFW. Experimental results prove the efficiency of our approaches under different conditions, especially when faces are partially occluded, and the proposed algorithms perform much better than all considered methods.     

基于聚类优化主动模型人脸识别算法研究

研究人脸识别和跟踪准确度问题。针对在使用大数据样本进行训练前提下,以往AAM算法人脸识别与定位不准确的缺陷,提出了一种新的利用聚类算法对样本空间进行划分,并在此基础上训练多个AAM的分层人脸识别和跟踪算法。首先利用所有训练样本训练得到初始AAM,然后利用一个全新的相似度计算公式,将所有训练样本划分成若干个子类别,在此基础上,针对每个子类别,训练一个相对稳定的AAM。在识别与跟踪过程中,先使用初始AAM进行定位,然后根据子类AAM进行精细化定位,从而得到比以往算法更为精确的定位效果。仿真结果显示改进的算法能准确定位出人脸所在位置,并且具有很高的运算效率,可以方便的实现实时监控系统的人脸跟踪定位及识别等目标。     

基于改进主动表观模型算法的人脸特征定位

人脸特征点的精确定位一直是人脸图像处理的重要研究内容,特征点定位精确与否直接影响后续工作结果的好坏。在基于反向组合AAM（Active Appearance Models）人脸特征点定位算法的基础上,提出结合特征点局部纹理模型来对AAM初始形状参数做最优化以及对AAM匹配模板升级的改进。改进的算法采用特征点局部纹理模型和AAM全局纹理模型结合的方法来最优化AAM初始形状参数,并在此前提下对AAM匹配模板进行升级,使其更接近待匹配图像的信息。在精确的匹配模板和优化的初始形状参数下,匹配的最终精度会得到提升。实验和理论证明,改进后的算法比传统反向组合AAM算法以及现有改进的PAAM（Progressive AAM）算法以及简单的结合ASM和AAM的改进算法都有更好的特征点定位精度。     

Recent Advances in Facial Appearance Capture

Facial appearance capture is now firmly established within academic research and used extensively across various application domains, perhaps most prominently in the entertainment industry through the design of virtual characters in video games and films. While significant progress has occurred over the last two decades, no single survey currently exists that discusses the similarities, differences, and practical considerations of the available appearance capture techniques as applied to human faces. A central difficulty of facial appearance capture is the way light interacts with skin—which has a complex multi‐layered structure—and the interactions that occur below the skin surface can, by definition, only be observed indirectly. In this report, we distinguish between two broad strategies for dealing with this complexity. “Image‐based methods” try to exhaustively capture the exact face appearance under different lighting and viewing conditions, and then render the face through weighted image combinations. “Parametric methods” instead fit the captured reflectance data to some parametric appearance model used during rendering, allowing for a more lightweight and flexible representation but at the cost of potentially increased rendering complexity or inexact reproduction. The goal of this report is to provide an overview that can guide practitioners and researchers in assessing the tradeoffs between current approaches and identifying directions for future advances in facial appearance capture.     

基于深度概率图模型的鲁棒人脸画像合成

针对基于数据驱动的人脸画像合成算法像素特征缺乏对光照变化和复杂背景的鲁棒性,常合成低质量的画像的问题,文中提出基于深度概率图模型的鲁棒人脸画像合成算法.采用预处理方法调整测试照片的光照亮度和人脸姿态,使之与训练照片一致.采用深度特征代替像素特征进行近邻匹配,采用深度概率图模型对画像重建权重和深度特征权重联合建模,得到合成画像的最佳重构表示.为了提高画像合成速度,提出快速近邻搜索方法.实验验证文中算法的鲁棒性和快速性.     

基于AAM和T型结构的人脸3D姿态估计

在分析已有的人脸姿态估计方法基础上,提出了一种基于主动表观模型(AAM)和T型结构的人脸3D姿态估计方法。对多姿态的人脸样本进行训练,得到多姿态的AAM模板集;利用训练得到的多姿态的AAM模板集进行最佳模板匹配,并对人脸的特征点进行精确定位;用人脸的双眼和嘴部构建T型模型,进行人脸3D姿态的参数估计。实验结果表明,该方法能适应较大的姿态旋转角度,并具有良好的姿态估计精度。     

A Comprehensive Performance Evaluation of Deformable Face Tracking “In-the-Wild”

Recently, technologies such as face detection, facial landmark localisation and face recognition and verification have matured enough to provide effective and efficient solutions for imagery captured under arbitrary conditions (referred to as “in-the-wild”). This is partially attributed to the fact that comprehensive “in-the-wild” benchmarks have been developed for face detection, landmark localisation and recognition/verification. A very important technology that has not been thoroughly evaluated yet is deformable face tracking “in-the-wild”. Until now, the performance has mainly been assessed qualitatively by visually assessing the result of a deformable face tracking technology on short videos. In this paper, we perform the first, to the best of our knowledge, thorough evaluation of state-of-the-art deformable face tracking pipelines using the recently introduced 300 VW benchmark. We evaluate many different architectures focusing mainly on the task of on-line deformable face tracking. In particular, we compare the following general strategies: (a) generic face detection plus generic facial landmark localisation, (b) generic model free tracking plus generic facial landmark localisation, as well as (c) hybrid approaches using state-of-the-art face detection, model free tracking and facial landmark localisation technologies. Our evaluation reveals future avenues for further research on the topic.     

基于主动表观模型姿态矫正和局部加权匹配人脸识别

非约束环境下,光照、姿态、表情、遮挡等复杂背景因素给人脸识别带来严重影响。提出一种基于AAM（active appearance model）的图像对齐和局部匹配人脸识别算法,使之能够增强人脸识别算法对姿态、表情变化的鲁棒性。AAM能够快速准确地定位人脸的特征点,进而将图像扭转到一个标准正面人脸模型中。接着,提出一种新的基于信息熵的Gabor jet加权方法用于提高人脸识别率;并且对Borda count分类器组合方法进行了改进,认为在投票过程中为其设置阈值来排除“噪声”的干扰可以提高识别率。通过与多种人脸识别方法的实验结果比较表明,使用AAM矫正图像后,联合熵加权Gabor方法和加阈值Borda能够取得比单独使用更好的成绩。     

基于跨批次存储预训练的素描人脸识别方法

素描人脸识别技术在刑侦领域应用广泛,有助于缩小嫌疑人的搜寻范围。由于素描人脸样本数量不足,导致经典的深度学习模型无法达到理想的识别精度。针对此问题,提出一种基于跨批次预训练的素描人脸识别方法,通过在有限素描人脸数据集进行跨批次预训练的方式缓解训练样本稀缺问题,从而提高人脸识别模型的泛化能力。该方法通过跨批次存储机制缓解GPU存储限制扩大单批次预训练样本数量,从而获得更优的模型初始参数,并在其基础上根据三元组损失进一步优化模型,以提升网络性能。提出的方法在UoM-SGFS素描人脸数据集上的Rank-1识别精度为72.53%,在PRIP-VSGC数据集上Rank-10识别精度为62.47%。相比CDAN、DANN、SSD等方法识别率有显著提高。     

From sketch to solid: an algebraic cross-section criterion for the realizability of a wireframe sketch

An intermediate step in the construction of a polyhedron from a partial-view sketch is the derivation of a realizable wireframe sketch, i.e., a complete sketch which is guaranteed to be the projection of a polyhedron. This paper presents a robust realizability-test based on the classical “cross-section criterion” that was developed in a geometric “ruler-and-compass” framework.     

高维数据聚类可视分析方法综述

数据聚类的可视分析方法利用可视化与交互技术帮助用户对聚类过程与结果进行 多角度分析,从而发现数据内部隐藏的结构和关系。但由于高维数据自身的“维度诅咒”问题 使得聚类分析面临着许多挑战,例如模型参数设定、数据特征捕捉、结果解释以及可视化展现 等。本文从高维数据聚类过程中遇到的问题出发,首先总结了高维数据聚类过程中常用的数据 处理方法并对其性能进行了比较,这些方法能够较好地解决“维度诅咒”问题,帮助用户挖掘 数据中存在的聚类模式。在分析和理解不同聚类结果中包含的数据内部结构和规律时,由于前 期采取的数据处理方法不同,因此需要采取不同的探索分析策略,所以本文将近10 年来高维数 据聚类的可视分析方法分为2 大类进行总结,即基于降维的聚类可视分析方法和基于子空间聚 类的可视分析方法。最后对该领域目前存在的机遇与挑战进行了讨论。     

结合多尺度HOG特征和语义属性的合成素描人脸识别

合成素描的人脸识别问题属于异质人脸识别研究领域,在刑侦领域具有重要的实际应用.由于合成素描与人脸照片属于不同模态,对不同模态人脸进行鲁棒的表征是识别的关键.针对合成素描人脸在某些区域缺乏纹理细节,单纯依赖局部细节特征识别率较低的问题,文中提出一种融合多尺度HOG特征并加以语义属性约束的合成素描人脸识别的算法.首先提取出合成素描人脸的全局HOG特征以及五官等关键部位的局部HOG特征来表征人脸的整体结构特征和细节特征,之后将得到的整体结构特征和各个部位的细节特征进行分数层融合,最后用语义属性特征对匹配结果进行重排序.在PRIP-VSGC和UoM-SGFS数据集上进行验证,文中算法rank10的识别率分别达到88.6%和96.7%,与现有算法相比有明显的提高.