一种基于增强现实的虚拟眼镜试戴的方法

选用单目的 CCD(Charge-coupled device)摄像头作为输入传感器,提出了一种基于标识物特征与人脸特征共同作用的增强现实技术设计并实现了一个虚拟眼镜试戴系统。通过单目摄像头采集现实场景的信息,对采集的图像进行分析,识别,并跟踪标识物与人脸特征,使虚拟眼镜模型可以准确地叠加在人脸之上,从而实现虚拟眼镜的试戴。实验证明,该方法可以准确地识别标识物,快速完成虚拟眼镜的试戴,为眼镜试戴技术提供了新的思路。     

Integrating face and gait for human recognition at a distance in video.

This paper introduces a new video-based recognition method to recognize noncooperating individuals at a distance in video who expose side views to the camera. Information from two biometrics sources, side face and gait, is utilized and integrated for recognition. For side face, an enhanced side-face image (ESFI), a higher resolution image compared with the image directly obtained from a single video frame, is constructed, which integrates face information from multiple video frames. For gait, the gait energy image (GEI), a spatio-temporal compact representation of gait in video, is used to characterize human-walking properties. The features of face and gait are obtained separately using the principal component analysis and multiple discriminant analysis combined method from ESFI and GEI, respectively. They are then integrated at the match score level by using different fusion strategies. The approach is tested on a database of video sequences, corresponding to 45 people, which are collected over seven months. The different fusion methods are compared and analyzed. The experimental results show that: 1) the idea of constructing ESFI from multiple frames is promising for human recognition in video, and better face features are extracted from ESFI compared to those from the original side-face images (OSFIs); 2) the synchronization of face and gait is not necessary for face template ESFI and gait template GEI; the synthetic match scores combine information from them; and 3) an integrated information from side face and gait is effective for human recognition in video.     

Open-set face recognition across look-alike faces in real-world scenarios

The open-set problem is among the problems that have significantly changed the performance of face recognition algorithms in real-world scenarios. Open-set operates under the supposition that not all the probes have a pair in the gallery. Most face recognition systems in real-world scenarios focus on handling pose, expression and illumination problems on face recognition. In addition to these challenges, when the number of subjects is increased for face recognition, these problems are intensified by look-alike faces for which there are two subjects with lower intra-class variations. In such challenges, the inter-class similarity is higher than the intra-class variation for these two subjects. In fact, these look-alike faces can be created as intrinsic, situation-based and also by facial plastic surgery. This work introduces three real-world open-set face recognition methods across facial plastic surgery changes and a look-alike face by 3D face reconstruction and sparse representation. Since some real-world databases for face recognition do not have multiple images per person in the gallery, with just one image per subject in the gallery, this paper proposes a novel idea to overcome this challenge by 3D modeling from gallery images and synthesizing them for generating several images. Accordingly, a 3D model is initially reconstructed from frontal face images in a real-world gallery. Then, each 3D reconstructed face in the gallery is synthesized to several possible views and a sparse dictionary is generated based on the synthesized face image for each person. Also, a likeness dictionary is defined and its optimization problem is solved by the proposed method. Finally, the face recognition is performed for open-set face recognition using three proposed representation classifications. Promising results are achieved for face recognition across plastic surgery and look-alike faces on three databases including the plastic surgery face, look-alike face and LFW databases compared to several state-of-the-art methods. Also, several real-world and open-set scenarios are performed to evaluate the proposed method on these databases in real-world scenarios.     

基于内容的多模态可逆密写方法

为了提高生物认证信息在网络传输过程中的安全性,提出一种基于内容相关性分析的多模态双重可逆密写 方法。与现存大多数方法不同,为了充分利用载体图像丰富的内容和提高方法的隐藏性能,首先采用最小二乘回归方 法分析掌纹图像与人脸图像之间的内容相关性,即用人脸图像表示掌纹图像,未被表示的部分掌纹图像被嵌入到相应 的人脸图像中,另外,重构系数作为密钥存储;然后,为了不引起攻击者的注意,将得到的含密人脸图像嵌入到随机选 取的自然载体图像中;最后,将得到含有掌纹信息和人脸信息的含密图像进行传输。提出的方法实现了生物认证信息 的双重可逆信息隐藏,而且哈希函数和密钥的使用提高了该方法的安全性。大量实验结果表明该,方法具有很好的安 全性、不可见性和很高的嵌入容量。特别地,采用双重隐藏机制进一步增强了生物认证信息的安全性,确保了多模态 生物认证的有效性。     

On robust optical flow estimation on image sequences with differently exposed frames using primal–dual optimization

Optical flow methods are used to estimate pixelwise motion information based on consecutive frames in image sequences. The image sequences traditionally contain frames that are similarly exposed. However, many real-world scenes contain high dynamic range content that cannot be captured well with a single exposure setting. Such scenes result in certain image regions being over- or underexposed, which can negatively impact the quality of motion estimates in those regions. Motivated by this, we propose to capture high dynamic range scenes using different exposure settings every other frame. A framework for OF estimation on such image sequences is presented, that can straightforwardly integrate techniques from the state-of-the-art in conventional OF methods. Different aspects of robustness of OF methods are discussed, including estimation of large displacements and robustness to natural illumination changes that occur between the frames, and we demonstrate experimentally how to handle such challenging flow estimation scenarios. The flow estimation is formulated as an optimization problem whose solution is obtained using an efficient primal–dual method.     

基于动态贝叶斯网络的音视频联合说话人跟踪

将多传感器信息融合技术用于说话人跟踪问题, 提出了一种基于动态贝叶斯网络的音视频联合说话人跟踪方法. 在动态贝叶斯网络中, 该方法分别采用麦克风阵列声源定位、人脸肤色检测以及音视频互信息最大化三种感知方式获取与说话人位置相关的量测信息; 然后采用粒子滤波对这些信息进行融合, 通过贝叶斯推理实现说话人的有效跟踪; 并运用信息熵理论对三种感知方式进行动态管理, 以提高跟踪系统的整体性能. 实验结果验证了本文方法的有效性.     

Simultaneous Learning of Nonlinear Manifolds Based on the Bottleneck Neural Network

Manifold learning methods are important techniques for nonlinear extraction of high-dimensional data structures. These methods usually extract a global manifold for data. However, in many real-world problems, there is not only one global manifold, but also additional information about the objects is shared by a large number of manifolds. These manifolds can share information for data reconstruction. To simultaneously extract these data manifolds, this paper proposes a nonlinear method based on the deep neural network (NN) named nonlinear manifold separator NN (NMSNN). Unlike unsupervised learning of bottleneck NN, data labels were used for simultaneous manifold learning. This paper makes use of NMSNN for extracting both expression and identity manifolds for facial images of the CK+ database. These manifolds have been evaluated by different metrics. The identity manifold is used for changing image identity. The result of identity recognition by K-nearest neighbor classifier shows that virtual identities are exactly sanitized. The virtual images for different expressions of test subjects are generated by expression manifold. The facial expression recognition rate of 92.86 % is achieved for virtual expressions of test persons. It is shown that NMSNN can be used to enrich datasets by sanitizing virtual images. As a result, 8 and 19 % improvements are gained in the face recognition task by a single image of each person on CK+ and Bosphorus databases, respectively.     

Discriminative sparse representation for face recognition

Recently Sparse Representation (or coding) based Classification (SRC) has gained great success in face recognition. In SRC, the testing image is expected to be best represented as a sparse linear combination of training images from the same class, and the representation fidelity is measured by the ?₂-norm or ?₁-norm of the coding residual. However, SRC emphasizes the sparsity too much and overlooks the spatial information during local feature encoding process which has been demonstrated to be critical in real-world face recognition problems. Besides, some work considers the spatial information but overlooks the different discriminative ability in different face regions. In this paper, we propose to weight spatial locations based on their discriminative abilities in sparse coding for robust face recognition. Specifically, we learn the weights at face locations according to the information entropy in each face region, so as to highlight locations in face images that are important for classification. Furthermore, in order to construct a robust weights to fully exploit structure information of each face region, we employed external data to learn the weights, which can cover all possible face image variants of different persons, so the robustness of obtained weights can be guaranteed. Finally, we consider the group structure of training images (i.e. those from the same subject) and added an ?_2,1-norm (group Lasso) constraint upon the formulation, which enforcing the sparsity at the group level. Extensive experiments on three benchmark face datasets demonstrate that our proposed method is much more robust and effective than baseline methods in dealing with face occlusion, corruption, lighting and expression changes, etc.     

Landmark selection for spectral clustering based on Weighted PageRank

Spectral clustering methods have various real-world applications, such as face recognition, community detection, protein sequences clustering etc. Although spectral clustering methods can detect arbitrary shaped clusters, resulting thus in high clustering accuracy, the heavy computational cost limits their scalability. In this paper, we propose an accelerated spectral clustering method based on landmark selection. According to the Weighted PageRank algorithm, the most important nodes of the data affinity graph are selected as landmarks. Furthermore, the selected landmarks are provided to a landmark spectral clustering technique to achieve scalable and accurate clustering. In our experiments, by using two benchmark face and shape image data sets, we examine several landmark selection strategies for scalable spectral clustering that either ignore or consider the topological properties of the data in the affinity graph. Also, we show that the proposed method outperforms baseline and accelerated spectral clustering methods, in terms of computational cost and clustering accuracy, respectively. Finally, we provide future directions in spectral clustering.     

Two-dimensional locality adaptive discriminant analysis

Two-dimensional Linear Discriminant Analysis (2DLDA), which is supervised and extracts the most discriminating features, has been widely used in face image representation and recognition. However, 2DLDA is inapplicable to many real-world situations because it assumes that the input data obeys the Gaussian distribution and emphasizes the global relationship of data merely. To handle this problem, we present a Two-dimensional Locality Adaptive Discriminant Analysis (2DLADA). Compared to 2DLDA, our method has two salient advantages: (1) it does not depend on any assumptions on the data distribution and is more suitable in real world applications; (2) it adaptively exploits the intrinsic local structure of data manifold. Performance on artificial dataset and real-world datasets demonstrate the superiority of our proposed method.

     

一种快速的彩色序列图像人脸检测算法

针对视频中的彩色序列图像，提出了一种人脸检测算法。该算法是一个由粗到精的检测过程。首先采用运动检测分析方法，根据多帧差分图像中运动物体边缘点的水平投影确定目标的水平位置，并结合肤色检测算法进一步确定人脸位置，然后用训练好的支持向量机进行人脸验证。实验结果表明，针对一般的彩色序列图像任意姿态人脸检测问题，该算法快速有效。     

应用深度光学应变特征图的人脸活体检测

目的 随着人脸识别系统应用的日益广泛,提高身份认证的安全性,提升人脸活体检测的有效性已经成为迫切需要解决的问题。针对活体检测中真实用户的照片存在的人脸欺骗问题,提出一种新的解决照片攻击的人脸活体检测算法。方法 利用局部二值模式LBP（local binary pattern）、TV-L1（total variation regularization and the robust L1 norm）光流法、光学应变和深度网络实现的人脸活体检测方法。对原始数据进行预处理得到LBP特征图;对LBP特征图提取光流信息,提高对噪声适应的鲁棒性;计算光流的导数得到图像的光学应变图,以表征相邻两帧之间的微纹理性质的微小移动量;通过卷积神经网络模型（CNN）将每个应变图编码成特征向量,最终将特征向量传递给长短期记忆LSTM（long short term memory）模型进行分类,实现真假人脸的判别。结果 实验在两个公开的人脸活体检测数据库上进行,并将本文算法与具有代表性的活体检测算法进行对比。在南京航空航天大学（NUAA）人脸活体检测数据库中,算法精度达到99.79%;在Replay-attack数据库中,算法精度达到98.2%,对比实验的结果证明本文算法对照片攻击的识别更加准确。结论 本文提出的针对照片攻击的人脸活体检测算法,融合光学应变图像和深度学习模型的优点,使得人脸活体检测更加准确。     

Illumination robust single sample face recognition using multi-directional orthogonal gradient phase faces

Due to the limitation of the storage space in the real-world face recognition application systems, only one sample image per person is often stored in the system, which is the so-called single sample problem. Moreover, real-world illumination has impact on recognition performance. This paper presents an illumination robust single sample face recognition approach, which utilizes multi-directional orthogonal gradient phase faces to solve the above limitations. In the proposed approach, an illumination insensitive orthogonal gradient phase face is obtained by using two vertical directional gradient values of the original image. Multi-directional orthogonal gradient phase faces can be used to extend samples for single sample face recognition. Simulated experiments and comparisons on a subset of Yale B database, Yale database, a subset of PIE database and VALID face database show that the proposed approach is not only an outstanding method for single sample face recognition under illumination but also more effective when addressing illumination, expression, decoration, etc.     

基于深度学习的人脸图像加密算法研究

图像作为视觉传达的重要信息载体,以一种直观、形象的方式向受众传递信息。但是,图像会在不知不觉中带来个人隐私信息泄露等安全隐患。本文从保护图像中隐私安全角度出发,深度融合人脸检测、人脸对齐方法以及混合混沌序列的图像加解密算法,提出了一种基于深度学习算法的人脸图像信息加密算法,即FIIE（Face Image Information Encryption ）算法,用于保护图片中的面部核心部位隐私信息。FIIE算法的具体描述如下：首先,采用WLDER FACE数据集中的人脸图像对MTCNN模型展开训练,并利用训练好的模型根据人脸特征点获取图像中人脸所在的矩形框坐标;然后,通过上述人脸区域坐标生成掩膜,运用生成的掩膜使原图与Logistic混沌序列做位运算,最后,对图像中人脸特定区域的加密。通过实验表明,本算法可以准确识别图像中人脸信息特定区域,实现对图像中面部信息的有效加密,保障用户的隐私安全。     

单幅人脸图像的全景纹理图生成方法

目的针对从单幅人脸图像中恢复面部纹理图时获得的信息不完整、纹理细节不够真实等问题,提出一种基于生成对抗网络的人脸全景纹理图生成方法。方法将2维人脸图像与3维人脸模型之间的特征关系转换为编码器中的条件参数,从图像数据与人脸条件参数的多元高斯分布中得到隐层数据的概率分布,用于在生成器中学习人物的头面部纹理特征。在新创建的人脸纹理图数据集上训练一个全景纹理图生成模型,利用不同属性的鉴别器对输出结果进行评估反馈,提升生成纹理图的完整性和真实性。结果实验与当前最新方法进行了比较,在Celeb A-HQ和LFW(labled faces in the wild)数据集中随机选取单幅正面人脸测试图像,经生成结果的可视化对比及3维映射显示效果对比,纹理图的完整度和显示效果均优于其他方法。通过全局和面部区域的像素量化指标进行数据比较,相比于UVGAN,全局峰值信噪比(peak signal to noise ratio,PSNR)和全局结构相似性(structural similarity index,SSIM)分别提高了7.9 d B和0.088,局部PSNR和局部SSIM分别提高了2.8 d B和0...     

Occlusion-Aware 3D Morphable Models and an Illumination Prior for Face Image Analysis

Faces in natural images are often occluded by a variety of objects. We propose a fully automated, probabilistic and occlusion-aware 3D morphable face model adaptation framework following an analysis-by-synthesis setup. The key idea is to segment the image into regions explained by separate models. Our framework includes a 3D morphable face model, a prototype-based beard model and a simple model for occlusions and background regions. The segmentation and all the model parameters have to be inferred from the single target image. Face model adaptation and segmentation are solved jointly using an expectation–maximization-like procedure. During the E-step, we update the segmentation and in the M-step the face model parameters are updated. For face model adaptation we apply a stochastic sampling strategy based on the Metropolis–Hastings algorithm. For segmentation, we apply loopy belief propagation for inference in a Markov random field. Illumination estimation is critical for occlusion handling. Our combined segmentation and model adaptation needs a proper initialization of the illumination parameters. We propose a RANSAC-based robust illumination estimation technique. By applying this method to a large face image database we obtain a first empirical distribution of real-world illumination conditions. The obtained empirical distribution is made publicly available and can be used as prior in probabilistic frameworks, for regularization or to synthesize data for deep learning methods.     

M3 CSR: Multi-view,multi-scale and multi-component cascade shape regression

Automatic face alignment is a fundamental step in facial image analysis. However, this problem continues to be challenging due to the large variability of expression, illumination, occlusion, pose, and detection drift in the real-world face images. In this paper, we present a multi-view, multi-scale and multi-component cascade shape regression (M³CSR) model for robust face alignment. Firstly, face view is estimated according to the deformable facial parts for learning view specified CSR, which can decrease the shape variance, alleviate the drift of face detection and accelerate shape convergence. Secondly, multi-scale HoG features are used as the shape-index features to incorporate local structure information implicitly, and a multi-scale optimization strategy is adopted to avoid trapping in local optimum. Finally, a component-based shape refinement process is developed to further improve the performance of face alignment. Extensive experiments on the IBUG dataset and the 300-W challenge dataset demonstrate the superiority of the proposed method over the state-of-the-art methods.     

刚性区域特征点的3维人脸识别

目的 针对3维人脸识别中存在表情变化的问题,提出了一种基于刚性区域特征点的3维人脸识别方法。方法 该方法首先在人脸纹理图像上提取人脸图像的特征点,并删除非刚性区域内的特征点,然后根据采样点的序号,在人脸空间几何信息上得到人脸图像特征点的3维几何信息,并建立以特征点为中心的刚性区域内的子区域,最后以子区域为局部特征进行人脸识别测试,得到不同子区域对人脸识别的贡献,并以此作为依据对人脸识别的结果进行加权统计。结果 在FRGC v2.0的3维人脸数据库上进行实验测试,该方法的识别准确率为98.5%,当错误接受率（FAR）为0.001时的验证率为99.2%,结果表明,该方法对非中性表情下的3维人脸识别具有很好的准确性。结论 该方法可以有效克服表情变化对3维人脸识别的影响,同时对3维数据中存在的空洞和尖锐噪声等因素具有较好的鲁棒性,对提高3维人脸识别性能具有重要意义。     

Deep peak-neutral difference feature for facial expression recognition

Facial expression recognition (FER) is important in vision-related applications. Deep neural networks demonstrate impressive performance for face recognition; however, it should be noted that this method relies heavily on a great deal of manually labeled training data, which is not available for facial expressions in real-world applications. Hence, we propose a powerful facial feature called deep peak–neutral difference (DPND) for FER. DPND is defined as the difference between two deep representations of the fully expressive (peak) and neutral facial expression frames. The difference tends to emphasize the facial parts that are changed in the transition from the neutral to the expressive face and to eliminate the face identity information retained in the fine-tuned deep neural network for facial expression, the network has been trained on large-scale face recognition dataset. Furthermore, unsupervised clustering and semi-supervised classification methods are presented to automatically acquire the neutral and peak frames from the expression sequence. The proposed facial expression feature achieved encouraging results on public databases, which suggests that it has strong potential to recognize facial expressions in real-world applications.

     

复杂光照环境下视频人脸序列的自动检测方法

基于视频序列人脸自动检测是人脸跟踪、识别等研究的基础。提出了一种结合图像增强技术、gabor特征变 换和adaboost算法的视频序列人脸检测方法,其主要思想是使用图像增强技术对图像进行光照补偿,减轻不同的光 照条件(如局部的阴影和高亮等)对检测结果的影响。该方法首先通过高频增强滤波强化图像的边缘和细节信息,用 基于直方图的技术来调节图像的亮度,然后应用gabor小波变换进行特征抽取,最后采用adaboost方法训练样本,完 成人脸的检测。实验表明,该方法能够在不同的光照条件下准确检测出人脸,显示出较强的鲁棒性。