Research on Facial Expression Capture Based on Two-Stage Neural Network

To generate realistic three-dimensional animation of virtual character, capturing real facial expression is the primary task. Due to diverse facial expressions and complex background, facial landmarks recognized by existing strategies have the problem of deviations and low accuracy. Therefore, a method for facial expression capture based on two-stage neural network is proposed in this paper which takes advantage of improved multi-task cascaded convolutional networks (MTCNN) and high-resolution network. Firstly, the convolution operation of traditional MTCNN is improved. The face information in the input image is quickly filtered by feature fusion in the first stage and Octave Convolution instead of the original ones is introduced into in the second stage to enhance the feature extraction ability of the network, which further rejects a large number of false candidates. The model outputs more accurate facial candidate windows for better landmarks recognition and locates the faces. Then the images cropped after face detection are input into high-resolution network. Multi-scale feature fusion is realized by parallel connection of multi-resolution streams, and rich high-resolution heatmaps of facial landmarks are obtained. Finally, the changes of facial landmarks recognized are tracked in real-time. The expression parameters are extracted and transmitted to Unity3D engine to drive the virtual character's face, which can realize facial expression synchronous animation. Extensive experimental results obtained on the WFLW database demonstrate the superiority of the proposed method in terms of accuracy and robustness, especially for diverse expressions and complex background. The method can accurately capture facial expression and generate three-dimensional animation effects, making online entertainment and social interaction more immersive in shared virtual space.     

基于深度学习检测器的多角度人脸关键点检测

针对人脸关键点检测(人脸对齐)在应用场景下的速度和精度需求,首先在SSD基础之上融合更多分布均匀的特征层,对人脸框坐标进行级联预测,形成对于多尺度人脸信息均具有更加鲁棒响应的深度学习检测器MR-SSD。其次在局部二值特征LBF的级联形状回归方法基础上,提出了基于面部像素差值的多角度初始化算法。采用端正人脸正负90°倾斜范围内的五组特征点形状进行初始化,求取每组回归后形状的眼部特征点像素均方差值并以最大者对应方案作为最终回归形状,从而实现对多角度倾斜人脸优异的拟合效果。本文所提出的最优架构可以实时获得极具鲁棒性的人脸框坐标并且可实现对于多角度倾斜人脸的关键点检测。     

Face modeling and editing with statistical local feature control models

This article presents a novel method based on statistical facial feature control models for generating realistic controllable face models. The local feature control models are constructed based on the exemplar 3D face scans. We use a three‐step model fitting approach for the 3D registration problem. Once we have a common surface representation for examples, we form feature shape spaces by applying a principal component analysis (PCA) to the data sets of facial feature shapes. We compute a set of anthropometric measurements to parameterize the exemplar shapes of each facial feature in a measurement space. Using PCA coefficients as a compact shape representation, we approach the shape synthesis problem by forming scattered data interpolation functions that are devoted to the generation of desired shape by taking the anthropometric parameters as input. The correspondence among all exemplar face textures is obtained by parameterizing a 3D generic mesh over a 2D image domain. The new feature texture with desired attributes is synthesized by interpolating the exemplar textures. With the exception of an initial tuning of feature point positions and assignment of texture attribute values, our method is fully automated. In the resulting system, users are assisted in automatically generating or editing a face model by controlling the high‐level parameters. © 2008 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 17, 341–358, 2007     

级联网络和金字塔光流的旋转不变人脸检测

在无约束的开放空间中,由于面部姿态变化、背景环境复杂、运动模糊等,人脸检测仍是一个具有挑战性的任务。本文针对视频流中人脸检测存在的平面内旋转问题,将人脸关键点与金字塔光流相结合,提出了基于级联网络和金字塔光流的旋转不变人脸检测算法。首先利用级联渐进卷积神经网络对视频流中前一帧进行人脸位置和关键点的定位;其次为获取关键点与人脸候选框间光流映射,使用独立的关键点检测网络对当前帧进行再次定位;之后计算前后两帧之间关键点光流位移;最后通过关键点光流位移与人脸候选框的映射关系,对视频中检测到的人脸进行校正,从而完成平面内旋转人脸不变性检测。实验经FDDB公开数据集上测试,证明该方法精确度较高。并且,在Boston面部跟踪数据集上进行动态测试,证明该人脸检测算法能有效解决平面内旋转人脸检测问题。对比其它检测算法,该算法检测速度有较大优势,同时视频中窗口抖动问题得到了很好解决。     

基于深度数据的空间人脸旋转角度估计

提出一种基于三维人脸深度数据的人脸姿态计算方法。利用人脸的深度数据以及与其一一对应的灰度图像,根据微分几何原理和相应的曲率算法与人脸数据中的灰度特征对人脸面部关键特征点定位,进而计算出人脸姿态在三维空间中的3个姿态角。实验证明该方法能在姿态变化情况下实现对人脸旋转角的准确估计,为进一步的人脸识别和表情分析提供基础。     

Face Recognition Based on Gabor Feature Extraction Followed by FastICA and LDA

Over the past few decades, face recognition has become the most effective biometric technique in recognizing people’s identity, as it is widely used in many areas of our daily lives. However, it is a challenging technique since facial images vary in rotations, expressions, and illuminations. To minimize the impact of these challenges, exploiting information from various feature extraction methods is recommended since one of the most critical tasks in face recognition system is the extraction of facial features. Therefore, this paper presents a new approach to face recognition based on the fusion of Gabor-based feature extraction, Fast Independent Component Analysis (FastICA), and Linear Discriminant Analysis (LDA). In the presented method, first, face images are transformed to grayscale and resized to have a uniform size. After that, facial features are extracted from the aligned face image using Gabor, FastICA, and LDA methods. Finally, the nearest distance classifier is utilized to recognize the identity of the individuals. Here, the performance of six distance classifiers, namely Euclidean, Cosine, Bray-Curtis, Mahalanobis, Correlation, and Manhattan, are investigated. Experimental results revealed that the presented method attains a higher rank-one recognition rate compared to the recent approaches in the literature on four benchmarked face datasets: ORL, GT, FEI, and Yale. Moreover, it showed that the proposed method not only helps in better extracting the features but also in improving the overall efficiency of the facial recognition system.     

基于机器视觉的VVT发动机转子缺陷检测系统设计

针对目前工业生产线上的VVT（variable valve timing,可变气门正时）发动机转子存在尺寸误差和外观缺陷等问题,大多数工厂采用人工方式来测量尺寸和检测缺陷,但人工测量和检测的精度易受外部环境和主观意识的影响,从而产生过检和漏检。为此,设计了一种基于机器视觉的VVT发动机转子缺陷检测系统。首先,针对VVT发动机转子凸台外边缘磕碰点对外径测量的干扰,提出一种基于梯度特征和位置序列的磕碰点检测算法,先通过分析轮廓点的距离-位置序列、梯度-位置序列曲线来筛选并去除凸台外边缘的磕碰点,再采用最小二乘法对筛选后的轮廓点进行圆弧拟合以实现外径测量。然后,针对VVT发动机转子端面上的划痕、划伤等缺陷,提出一种基于改进HOG（histogram of oriented gradient,方向梯度直方图）特征的SVM（support vector machines,支持向量机）分类算法,先采用连通域分析方法得到待检测的目标区域,再提取目标区域的改进HOG特征,并利用SVM进行分类,以实现端面缺陷的检测。实验结果表明,所设计的缺陷检测系统在测量VVT发动机转子外径时的绝对精度可达到0.01 mm,且能够准确地筛选出凸台外边缘的磕碰点;因改进的HOG特征优于传统的HOG特征,所设计的缺陷检测系统在检测转子端面缺陷时具有较低的过检率和漏检率。综上可知,基于机器视觉的VVT发动机转子缺陷检测系统可实现外径的精确测量和外观缺陷的有效检测,基本满足工业检测要求,具有较高的实用价值。     

A Privacy-Preserving System Design for Digital Presence Protection

A person’s privacy has become a growing concern, given the nature of an expansive reliance on real-time video activities with video capture, stream, and storage. This paper presents an innovative system design based on a privacy-preserving model. The proposed system design is implemented by employing an enhanced capability that overcomes today’s single parameter-based access control protection mechanism for digital privacy preservation. The enhanced capability combines multiple access control parameters: facial expression, resource, environment, location, and time. The proposed system design demonstrated that a person’s facial expressions combined with a set of access control rules can achieve a person’s privacy-preserving preferences. The findings resulted in different facial expressions successfully triggering a person’s face to be blurred and a person’s privacy when using a real-time video conferencing service captured from a webcam or virtual webcam. A comparison analysis of capabilities between existing designs and the proposed system design shows enhancement of the capabilities of the proposed system. A series of experiments exercising the enhanced, real-time multi-parameter-based system was shown as a viable path forward for preserving a person’s privacy while using a webcam or virtual webcam to capture, stream, and store videos.     

A multimodal fusion system for people detection and tracking

Because a people detection system that considers only a single feature tends to be unstable, many people detection systems have been proposed to extract multiple features simultaneously. These detection systems usually integrate features using a heuristic method based on the designers' observations and induction. Whenever the number of features to be considered is changed, the designer must change and adjust the integration mechanism accordingly. To avoid this tedious process, we propose a multimodal fusion system that can detect and track people in a scalable, accurate, robust, and flexible manner. Each module considers a single feature and all modules operate independently at the same time. A depth module is constructed to detect people based on the depth‐from‐stereo method, and a novel approach is proposed to extract people by analyzing the vertical projection in each layer. A color module that detects the human face, and a motion module that detects human movement are also developed. The outputs from these individual modules are fused together and tracked over time, using a Kalman filter. © 2005 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 15, 131–142, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.20046     

基于特征点匹配的视频稳定算法

目的 解决检测摄像机在拍摄过程中因外界抖动导致视频扭曲失真等问题。方法 选取视频第1帧作为基准帧,检测视频每帧的特征点,并与第1帧特征点位置进行对比,经过仿射变换消除每一帧的扰动。通过FAST算法生成每一帧特征点,利用BRISK算法得到特征点的二进制特征描述符,比较特征点描述符的汉明距离来匹配相邻两帧间的特征点,利用匹配好的特征点对确定仿射变换模型,最后利用得到的仿射变换矩阵对每一帧进行校正。结果 实验原视频总帧数为23帧,时长为1 s,分辨率为320×240,视频中除第1帧外,其余22帧均得到修正。结论 FAST角点检测算法与BRISK匹配方法结合可以有效解决视频扭曲等问题。     

Real-time multi-face detection on FPGA for video surveillance applications

In the field of face recognition and in the establishment of a face database, face detection is a crucial step. In current security and surveillance systems, most of the face detection proposed now is focused on software algorithms to improve the detection rate and decrease false alarms. However, these more complex algorithms require more computation time, which hinders real-time applications. In this paper, we propose a real-time multi-face detection system based on hardware design to enhance processing time. The proposed hardware architecture is implemented on an Altera DE2-70 field-programmable gate array development board to test the feasibility of our hardware design. To implement, our system requires 15,223 logic elements. The proposed system can operate in real-time at a frame rate of 30?fps, and detects up to five faces simultaneously. Our experimental results show that the proposed face detection architecture provides a reliable real-time system operating at low cost and providing a high detection rate.     

Detection and characterization of transport barriers in complex flows via ridge extraction of the finite time Lyapunov exponent field

This paper proposes an algorithm to detect and characterize ridges in the finite time Lyapunov exponent (FTLE) field obtained from a continuous dynamical system or flow. These ridges represent time‐dependent separatrices of the flow and are also called Lagrangian coherent structures (LCS). LCS have been demonstrated to be an effective way to analyze realistic time‐chaotic flows, although they can be quite complex. Therefore, in order to exploit the information that LCS can provide it is important to locate and characterize these structures in a systematic way. This can be accomplished by interpreting the FTLE as a height field and detecting the LCS as ridges of this graph. Methodologies developed in the image processing framework are integrated with dynamical system inspired approaches in order to characterize ridge strength and location. The main novel contribution of the proposed algorithm is a scheme to connect sets of points into curves or surfaces (rather than distributions of points around a ridge axis) and classify these curves or surfaces using a dynamical systems measure of strength. This approach provides the capability to track ranked LCS in space and time. The results are presented for a simple analytical model and noisy LCS from realistic three‐dimensional geophysical fluid data. Copyright © 2010 John Wiley & Sons, Ltd.     

Deep Learning Based Face Detection and Identification of Criminal Suspects

Occurrence of crimes has been on the constant rise despite the emerging discoveries and advancements in the technological field in the past decade. One of the most tedious tasks is to track a suspect once a crime is committed. As most of the crimes are committed by individuals who have a history of felonies, it is essential for a monitoring system that does not just detect the person’s face who has committed the crime, but also their identity. Hence, a smart criminal detection and identification system that makes use of the OpenCV Deep Neural Network (DNN) model which employs a Single Shot Multibox Detector for detection of face and an auto-encoder model in which the encoder part is used for matching the captured facial images with the criminals has been proposed. After detection and extraction of the face in the image by face cropping, the captured face is then compared with the images in the Criminal Database. The comparison is performed by calculating the similarity value between each pair of images that are obtained by using the Cosine Similarity metric. After plotting the values in a graph to find the threshold value, we conclude that the confidence rate of the encoder model is 0.75 and above.     

Synaptic Device Network Architecture with Feature Extraction for Unsupervised Image Classification

For the efficient recognition and classification of numerous images, neuroinspired deep learning algorithms have demonstrated their substantial performance. Nevertheless, current deep learning algorithms that are performed on von Neumann machines face significant limitations due to their inherent inefficient energy consumption. Thus, alternative approaches (i.e., neuromorphic systems) are expected to provide more energy‐efficient computing units. However, the implementation of the neuromorphic system is still challenging due to the uncertain impacts of synaptic device specifications on system performance. Moreover, only few studies are reported how to implement feature extraction algorithms on the neuromorphic system. Here, a synaptic device network architecture with a feature extraction algorithm inspired by the convolutional neural network is demonstrated. Its pattern recognition efficacy is validated using a device‐to‐system level simulation. The network can classify handwritten digits at up to a 90% recognition rate despite using fewer synaptic devices than the architecture without feature extraction.     

Face Age Estimation Based on CSLBP and Lightweight Convolutional Neural Network

As the use of facial attributes continues to expand, research into facial age estimation is also developing. Because face images are easily affected by factors including illumination and occlusion, the age estimation of faces is a challenging process. This paper proposes a face age estimation algorithm based on lightweight convolutional neural network in view of the complexity of the environment and the limitations of device computing ability. Improving face age estimation based on Soft Stagewise Regression Network (SSR-Net) and facial images, this paper employs the Center Symmetric Local Binary Pattern (CSLBP) method to obtain the feature image and then combines the face image and the feature image as network input data. Adding feature images to the convolutional neural network can improve the accuracy as well as increase the network model robustness. The experimental results on IMDB-WIKI and MORPH 2 datasets show that the lightweight convolutional neural network method proposed in this paper reduces model complexity and increases the accuracy of face age estimations.     

基于机器视觉的玻璃瓶口缺陷检测方法

目的为提高玻璃瓶口缺陷检测精度,确保生产线包装效率。方法基于机器视觉设计一种瓶口缺陷检测方法,并简要介绍检测系统的整体框架。分别论述基于最大熵值法的图像分割方法、瓶口定位方法以及图像特征提取方法,其中图像特征主要包括周长、圆形度、相对圆心距离。利用BP神经网络实现瓶口缺陷的准确识别,将瓶口破损程度转换为具体数值,最后进行实验验证。结果文中检测方法对破损瓶口的检测成功率为99%,对于不同的破损类型均有较高的检测准确度。结论基于机器视觉的玻璃瓶口缺陷检测方法能够满足生产线对准确性和实时性的要求。     

Automatic facial expression recognition based on pixel‐pattern‐based texture feature

PCA, ICA, and Gabor wavelet are considered as the important and powerful face representation methods. In this article, we propose a new approach for face representation, which is called a pixel‐pattern‐based texture feature (PPBTF) and apply it to the real‐time facial expression recognition. A gray scale image is transformed into a pattern map where edges and lines are used for characterizing the facial texture information. Based on the pattern map, a feature vector is comprised of the numbers of the pixels belonging to each pattern. We use the image basis functions obtained by principal component analysis as the templates for pattern matching. Adaboost and Support Vector Machine are adopted to classify facial expression. Extensive experiments on the Cohn‐Kanade Database, PIE Database, and DUT Database illustrate that the PPBTF is quite effective and insensitive to illumination. The comparison with Gabor show the PPBTF is speedy. © 2010 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 20, 253–260, 2010     

基于高精度匹配点的对极几何估计

提出了一种以Harris算子和空间位置约束条件自动提取的初始特征点为引导,利用小面元模型进行灰度分布的拟合曲面求极值点的超精度特征点定位方法。并利用超精度的特征点估计立体图像对的对极几何约束关系。通过对真实图像的对比实验证明该方法能有效的提高基本矩阵估计精度,与传统的基于像素级特征点的估计算法相比,平均余差值降低了3.4%~21.4%,算法运行速度快,能在视觉测量中有效地提高测量精度。     

A discrete mechanics approach to the Cosserat rod theory—Part 1: static equilibria

A theory of discrete Cosserat rods is formulated in the language of discrete Lagrangian mechanics. By exploiting Kirchhoff's kinetic analogy, the potential energy density of a rod is a function on the tangent bundle of the configuration manifold and thus formally corresponds to the Lagrangian function of a dynamical system. The equilibrium equations are derived from a variational principle using a formulation that involves null‐space matrices. In this formulation, no Lagrange multipliers are necessary to enforce orthonormality of the directors. Noether's theorem relates first integrals of the equilibrium equations to Lie group actions on the configuration bundle, so‐called symmetries. The symmetries relevant for rod mechanics are frame‐indifference, isotropy, and uniformity. We show that a completely analogous and self‐contained theory of discrete rods can be formulated in which the arc‐length is a discrete variable ab initio. In this formulation, the potential energy density is defined directly on pairs of points along the arc‐length of the rod, in analogy to Veselov's discrete reformulation of Lagrangian mechanics. A discrete version of Noether's theorem then identifies exact first integrals of the discrete equilibrium equations. These exact conservation properties confer the discrete solutions accuracy and robustness, as demonstrated by selected examples of application. Copyright © 2010 John Wiley & Sons, Ltd.