Face Hallucination: Theory and Practice

In this paper, we study face hallucination, or synthesizing a high-resolution face image from an input low-resolution image, with the help of a large collection of other high-resolution face images. Our theoretical contribution is a two-step statistical modeling approach that integrates both a global parametric model and a local nonparametric model. At the first step, we derive a global linear model to learn the relationship between the high-resolution face images and their smoothed and down-sampled lower resolution ones. At the second step, we model the residue between an original high-resolution image and the reconstructed high-resolution image after applying the learned linear model by a patch-based non-parametric Markov network to capture the high-frequency content. By integrating both global and local models, we can generate photorealistic face images. A practical contribution is a robust warping algorithm to align the low-resolution face images to obtain good hallucination results. The effectiveness of our approach is demonstrated by extensive experiments generating high-quality hallucinated face images from low-resolution input with no manual alignment.     

Face hallucination based on sparse local-pixel structure

In this paper, we propose a face-hallucination method, namely face hallucination based on sparse local-pixel structure. In our framework, a high resolution (HR) face is estimated from a single frame low resolution (LR) face with the help of the facial dataset. Unlike many existing face-hallucination methods such as the from local-pixel structure to global image super-resolution method (LPS-GIS) and the super-resolution through neighbor embedding, where the prior models are learned by employing the least-square methods, our framework aims to shape the prior model using sparse representation. Then this learned prior model is employed to guide the reconstruction process. Experiments show that our framework is very flexible, and achieves a competitive or even superior performance in terms of both reconstruction error and visual quality. Our method still exhibits an impressive ability to generate plausible HR facial images based on their sparse local structures.     

Parts-based face super-resolution via non-negative matrix factorization

Face super-resolution refers to inferring the high-resolution face image from its low-resolution one. In this paper, we propose a parts-based face hallucination framework which consists of global face reconstruction and residue compensation. In the first phase, correlation-constrained non-negative matrix factorization (CCNMF) algorithm combines non-negative matrix factorization and canonical correlation analysis to hallucinate the global high-resolution face. In the second phase, the High-dimensional Coupled NMF (HCNMF) algorithm is used to compensate the error residue in hallucinated images. The proposed CCNMF algorithm can generate global face more similar to the ground truth face by learning a parts-based local representation of facial images; while the HCNMF can learn the relation between high-resolution residue and low-resolution residue to better preserve high frequency details. The experimental results validate the effectiveness of our method.     

基于类别相关近邻子空间的最大似然稀疏表示鲁棒图像识别算法

为了构建一个快速鲁棒的图像识别算法, 提出基于类别相关近邻子空间的最大似然稀疏表示图像识别算法. 考虑到每个测试样本的不同分布特性及训练样本选择的类别代表性原则, 不再将所有训练样本作为稀疏表示的字典, 而是基于距离相近准则选择合适子空间, 从每个类别中选取自适应数量的局部近邻构成新的字典, 在减少训练样本的同时保留了稀疏表示原有的子空间结构. 然后基于最大似然稀疏表示识别方法, 将稀疏表示的保真度表示为余项的最大似然函数, 并将识别问题转化为加权的稀疏优化问题. 在公用人脸与数字识别数据库上的实验证明该算法的合理性, 提高识别速度的同时保证了识别精度和算法的鲁棒性, 特别是对于遮挡与干扰图像具有较好的适应性.     

Robust Video Face Recognition Under Pose Variation

With the abundance of video data, the interest in more effective methods for recognizing faces from surveillance videos has grown. However, most algorithms proposed in this field have an assumption that each image set lies in a single linear subspace, or a mixture of linear subspaces. As a result, 3-dimensional shape information, which leads to the nonlinear transformation of face images, is ignored. This paper proposes a robust video face recognition across pose variation in video (RVPose) based on sparse representation. The key idea is performing alignment and recognition based on sparse representation simultaneously. Moreover, by considering that multi-pose faces of the same subject possess the same texture and 3-dimensional shape, RVPose aligns a sequence of faces with pose variations simultaneously, which is reduced to a 3-dimensional shape-constrained video alignment problem. Finally, aligned video sequence is recognized based on sparse represent. Experiments conducted on public video datasets demonstrate the effectiveness of the proposed algorithm.     

基于非负稀疏表示的人脸识别

对稀疏表示在人脸识别中的应用进行了研究,提出了人脸识别的非负稀疏表示方法和采样方法.提出了非负稀疏表示的乘性迭代算法,分析了该方法与非负矩阵分解的联系,设计了基于非负稀疏表示的分类算法.在仿射传播算法的基础上,提出了人脸数据集的采样方法,并在人脸图像集上进行了实验.与稀疏表示相比,非负稀疏表示在计算复杂度和鲁棒性上具有优越性;与随机采样方法相比,该采样方法具有较高的识别精度.     

结合位置先验与稀疏表示的单帧人脸图像超分辨率算法

提出了一种结合位置先验与稀疏表示的人脸图像超分辨率算法,可对单帧输入的低分辨率人脸图像基于训练集进行超分辨率重建。利用压缩感知理论中的信号分解方法,〖BP(〗明确哪些方法更好〖BP)〗,将稀疏表示与人脸位置先验信息相结合,使用经过分类的超完备冗余字典,来分别稀疏逼近输入信号的块向量结构。利用最佳的K项原子,线性组合重建出高分辨率图像块。最后按照图像块最初在人脸的位置,将它们拼接为整体人脸。在CAS-PEAL-R1人脸图库上的实验结果表明,该算法使用相对较少的原子,就可以重建出质量较好的高分辨率人脸图像。     

基于分类稀疏表示的人脸表情识别

为挖掘不同人脸表情图像的统计特性差异,提出一种基于分类稀疏表示的表情识别算法。首先通过对不同类别表情图像的字典学习,构建满足各类表情图像统计特性的基函数子集,进而采用Lasso算法获得表情图像在由基函数集所张成特征子空间中的稀疏表示,最后通过比较表情图像在各基函数子集上的重构误差实现不同表情的分类识别。基于JAFFE人脸表情数据库的实验结果表明,该算法可以有效克服人脸身份对表情识别的影响,具有较高的表情识别率和鲁棒性。     

利用CNN和PCA约束优化模型实现稀疏表示分类

目的 传统的稀疏表示分类方法运用高维数据提升算法的稀疏分类能力,早已引起了广泛关注,但其忽视了测试样本与训练样本间的信息冗余,导致了不确定性的决策分类问题。为此,本文提出一种基于卷积神经网络和PCA约束优化模型的稀疏表示分类方法（EPCNN-SRC）。方法 首先通过深度卷积神经网络计算,在输出层提取对应的特征图像,用以表征原始样本的鲁棒人脸特征。然后在此特征基础上,构建一个PCA（principal component analysis）约束优化模型来线性表示测试样本,计算对应的PCA系数。最后使用稀疏表示分类算法重构测试样本与每类训练样本的PCA系数来完成分类。结果 本文设计的分类模型与一些典型的稀疏分类方法相比,取得了更好的分类性能,在AR、FERET、FRGC和LFW人脸数据库上的实验结果显示,当每类仅有一个训练样本时,EPCNN-SRC算法的识别率分别达到96.92%、96.15%、86.94%和42.44%,均高于传统的表示分类方法,充分验证了本文算法的有效性。同时,本文方法不仅提升了对测试样本稀疏表示的鲁棒性,而且在保证识别率的基础上,有效降低了算法的时间复杂度,在FERET数据库上的运行时间为4.92 s,均低于一些传统方法的运行时间。结论 基于卷积神经网络和PCA约束优化模型的稀疏表示分类方法,将深度学习特征与PCA方法相结合,不仅具有较好的识别准确度,而且对稀疏分类也具有很好的鲁棒性,尤其在小样本问题上优势显著。     

基于核稀疏分类与多尺度分块旋转扩展的鲁棒图像识别

针对在图像旋转或局部扭曲变形等复杂情况下的图像识别问题,提出一种基于核稀疏分类与多尺度分块旋转扩展的鲁棒图像识别算法。该算法首先对图像进行多尺度分块与旋转扩展,使得字典能近似测试图像局部的旋转扭曲与各种排列组合。为了增加字典类间稀疏度,改善系统效率,提出一种字典降维策略。通过核随机坐标下降方法高效求解核稀疏分类的凸优化问题,进而通过对比不同类对测试图像的重构误差完成图像识别。实验表明,与经典方法相比,文中方法具有更好的识别效果,对图像旋转或局部扭曲变形等复杂情况具有较好的鲁棒性。     

融合LBP和局部稀疏表示的人脸表情识别

为更好获取人脸局部表情特征,提出了一种融合局部二值模式（Local Binary Pattern,LBP）和局部稀疏表示的人脸表情特征与识别方法。为深入分析表情对人脸子区域的影响,根据五官特征对人脸进行非均匀分区,并提取局部LBP特征;为精细刻画人脸局部纹理,整合人脸局部特征,设计了人脸局部稀疏重构表示方法,并根据表情对各局部子区域的影响因子,加权融合局部重构残差进行人脸表情识别。在JAFFE2表情人脸库上的对比实验,验证了该方法的可行性和鲁棒性。     

Gabor特征判别分析人脸识别方法的误配准鲁棒性分析

人脸识别领域中,Gabor特征人脸表示方法因其在应用中获得的高首选识别率而被认为是一种理想的人脸特征表示方法。文章用一种全新的量化评价方法,结合配准精度和识别率,从误配准鲁棒性角度评价Gabor特征在人脸识别中的优越性。实验表明,和图像灰度信息特征相比,Gabor特征不仅在精确配准时具有高识别率,而且对由于人脸特征定位不精确而导致的图像变化的鲁棒性也更强。     

多视角级联回归模型人脸特征点定位

针对人脸姿态偏转较大导致人脸特征点定位精度低的问题，提出了多视角人脸特征点定位算法，采用随机森林局部学习与全局线性回归相结合的级联姿态回归（Cascaded Pose Regression，CPR）人脸特征点定位模型，在不同的人脸姿态视角下建立不同的模型，以多模型代替单一模型来提高人脸特征点定位的精度。首先采用CPR模型对不同视角下的人脸建立不同的模型；然后采用多视角生成模型（Multi-View Generative Model，MVGM）来评估输入人脸图片的姿态；最后根据评估的姿态选择相对应的模型，进而实现特征点的精确定位。仿真实验结果表明，相比于现有的几种人脸特征点定位算法，所提算法实现了更精确的定位效果。     

Multiplicative noise removal via adaptive learned dictionaries and TV regularization

Multiplicative noise removal is a key issue in image processing problem. While a large amount of literature on this subject are total variation (TV)-based and wavelet-based methods, recently sparse representation of images has shown to be efficient approach for image restoration. TV regularization is efficient to restore cartoon images while dictionaries are well adapted to textures and some tricky structures. Following this idea, in this paper, we propose an approach that combines the advantages of sparse representation over dictionary learning and TV regularization method. The method is proposed to solve multiplicative noise removal problem by minimizing the energy functional, which is composed of the data-fidelity term, a sparse representation prior over adaptive learned dictionaries, and TV regularization term. The optimization problem can be efficiently solved by the split Bregman algorithm. Experimental results validate that the proposed model has a superior performance than many recent methods, in terms of peak signal-to-noise ratio, mean absolute-deviation error, mean structure similarity, and subjective visual quality.     

基于组稀疏表示的在线单帧图像超分辨率算法

基于稀疏表示的图像超分辨率重建算法以近似随机抽取的方式选取字典中的原子来拟合图像片,而实际中的字典原子的选择体现出了很强的结构稀疏性,从而导致算法计算复杂且引入了大量的误差,影响重建图像的质量。针对该问题,提出了一种基于组稀疏表示的在线图像超分辨率重建算法。该方法引入组稀疏理论,仅利用输入的低分辨率图像作为样本来构建组稀疏字典,通过结合组稀疏性和几何对偶性来构建超分辨率图像算法的成本函数,并使用提出的一种迭代的方法进行求解。实验表明,该算法在视觉观察和参数比较上都优于当前主流的超分辨率算法。     

Improving Medical CT Image Blind Restoration Algorithm Based on Dictionary Learning by Alternating Direction Method of Multipliers

In this paper, the medical CT image blind restoration is translated into two sub problems, namely, image estimation based on dictionary learning and point spread function estimation. A blind restoration algorithm optimized by the alternating direction method of multipliers for medical CT images was proposed. At present, the existing methods of blind image restoration based on dictionary learning have the problem of low efficiency and precision. This paper aims to improve the effectiveness and accuracy of the algorithm and to improve the robustness of the algorithm. The local CT images are selected as training samples, and the K-SVD algorithm is used to construct the dictionary by iterative optimization, which is beneficial to improve the efficiency of the algorithm. Then, the orthogonal matching pursuit algorithm is employed to implement the dictionary update. Dictionary learning is accomplished by sparse representation of medical CT images. The alternating direction method of multipliers (ADMM) is used to solve the objective function and realize the local image restoration, so as to eliminate the influence of point spread function. Secondly, the local restoration image is used to estimate the point spread function, and the convex quadratic optimization method is used to solve the point spread function sub problems. Finally, the optimal estimation of point spread function is obtained by iterative method, and the global sharp image is obtained by the alternating direction method of multipliers. Experimental results show that, compared with the traditional adaptive dictionary restoration algorithm, the new algorithm improves the objective image quality metrics, such as peak signal to noise ratio, structural similarity, and universal image quality index. The new algorithm optimizes the restoration effect, improves the robustness of noise immunity and improves the computing efficiency.     

Robust face alignment and tracking by combining local search and global fitting

When a face in an image is considerably occluded, existing local search and global fitting methods often cannot find the facial features due to failures in the local facial feature detectors or the fitting limitations of appearance modeling. To solve these problems, we propose a new face alignment method that combines the local search and global fitting methods, where local misalignments in the local search method are restricted by holistic appearance fitting in the global fitting method and the divergent or shrinking alignments in the global fitting method are avoided by the restricting local movements in the local search method. The proposed alignment method consists of two stages: the initialization stage detects the face, estimates the facial pose and obtains the initial facial features by locating a pose-specific mean shape on the detected face; the optimization stage then obtains the facial features by updating the parameter set from the combined Hessian matrix and the combined gradient vector. We also extend the proposed face alignment to face tracking by adding a template image that is warped from the facial features obtained in the previous frame. In the experiments, the proposed method yields more accurate and stable face alignment or tracking under heavy occlusion and pose variation than the existing methods.     

基于旋转扩展和稀疏表示的鲁棒遥感图像目标识别

针对含有残缺图像的遥感图像目标识别问题,提出一种基于旋转扩展和稀疏表示的目标识别方法.首先对训练集进行旋转扩展,使得测试图像能近似用训练集稀疏表示,然后通过求解一个l1范数最小化问题得到测试图像相对于训练集的一个稀疏表示,进而根据不同类对应的稀疏表示对测试图像的近似程度进行识别.与代表性的方法进行比较,实验结果与分析表明,该方法识别率优于已有方法,对残缺图像的识别有较好的鲁棒性,且在小样本、低采样率情况下也能保持较好的识别性能.     

自适应超完备字典学习的SAR图像降噪

提出一种基于自适应超完备字典学习的SAR图像降噪。该算法建立在超完备字典稀疏表示基础上,具有较强的数据稀疏性和稳健的建模假设。算法依据相干斑噪声统计特性,通过分步优化字典原子和变换系数自适应构造超完备字典,利用获得的超完备字典将图像局部信息投影到高维空间中,实现图像的稀疏表示,运用正则化方法建立多目标优化模型。最后通过对优化问题的求解重建SAR图像场景分辨单元的平均强度,实现SAR图像的降噪。实验结果表明,该算法对相干斑噪声有很好的抑制效果,并且具有保持图像细节信息的优点。     

HSR: <Emphasis Type="Italic">L</Emphasis><Subscript>1/2</Subscript>-regularized sparse representation for fast face recognition using hierarchical feature selection

In this paper, we propose a novel method for fast face recognition called L _1/2-regularized sparse representation using hierarchical feature selection. By employing hierarchical feature selection, we can compress the scale and dimension of global dictionary, which directly contributes to the decrease of computational cost in sparse representation that our approach is strongly rooted in. It consists of Gabor wavelets and extreme learning machine auto-encoder (ELM-AE) hierarchically. For Gabor wavelets’ part, local features can be extracted at multiple scales and orientations to form Gabor-feature-based image, which in turn improves the recognition rate. Besides, in the presence of occluded face image, the scale of Gabor-feature-based global dictionary can be compressed accordingly because redundancies exist in Gabor-feature-based occlusion dictionary. For ELM-AE part, the dimension of Gabor-feature-based global dictionary can be compressed because high-dimensional face images can be rapidly represented by low-dimensional feature. By introducing L _1/2 regularization, our approach can produce sparser and more robust representation compared to L ₁-regularized sparse representation-based classification (SRC), which also contributes to the decrease of the computational cost in sparse representation. In comparison with related work such as SRC and Gabor-feature-based SRC, experimental results on a variety of face databases demonstrate the great advantage of our method for computational cost. Moreover, we also achieve approximate or even better recognition rate.