Analysis of 3D face forms for proper sizing and CAD of spectacle frames

Three-dimensional morphological variations in the human face were analysed using digital models of the human face, and the usefulness of such analysis in designing industrial products was demonstrated by validating spectacle frame designs based on an original sizing system developed based on the analysis. A normalized model of the three-dimensional face form was made for each of 56 young adult Japanese males. The morphological distances between subjects were defined, and subjects were divided into four groups based on analysis of the distance matrix. A prototype spectacle frame was designed for the average form of each of the four groups. Tightening force of the prototype frames was adjusted using the materialized average forms with soft material placed at the nasal bridge and side of the head. Four prototype frames as well as a conventional frame were evaluated using sensory evaluation and physical measurement of the pressure and slip in 38 young adult male subjects. For each of the 38 subjects, prototype frames were ranked according to the morphological similarity of the subjects and the average form of the four groups: the frame designed for the average form of the group most similar to the subject was #1, the frame designed for the average form of the next most similar group was #2, and so on. For the groups with smaller or narrower faces, new frame #1 was most preferred and had the best overall fit, smallest slip sensation and largest pressure sensation. The groups with larger or wider faces preferred tighter frames than new frame #1, because they were concerned that the frames might slip, although the frames did not. Most of the subjects habitually wore spectacles, and the reason that groups with larger or wider faces preferred tighter frames was thought to be that they were accustomed to tighter fitting frames.     

智能配镜三维特征参数提取方法研究

摘 要：针对智能配镜中三维面部特征点提取算法复杂度较高的问题,提出一种将三维点 云转换为映射图像定位特征点的方法。采用 Voronoi 方法计算面部三角网格各顶点处的高斯曲 率、平均曲率。选取鼻尖、眼角等曲率特征明显的区域估计面部点云姿态。根据曲率旋转不变 性,使用初选的点云方向向量简化旋转矩阵的计算,使面部点云正面朝向视点。将点云映射转 换为图像,三维网格模型中三角面片一对一映射到图像中的三角形。搭建卷积神经网络,使用 Texas 3DFRD 数据集进行模型训练。进行人脸对齐,预测所得各面部特征点分别限制在图像某 三角形中。根据图像中三角形映射查找三维网格模型中对应三角面片,通过三角面片顶点坐标 计算配镜所需的面部特征点位置坐标,实现配镜特征参数的提取。     

Depth calculation by using face detection ASIC

Real-time processing in mobile devices is always a hot issue. Although face detection and stereo matching, in particular, are core elements for HRI applications, they are processes that impose the heaviest loads in real-time processing. In this article, we present novel methods of calculating disparities and face detection by using an Altair chip which can detect 32 frontal faces per frame at speed of 30 frames per second (f.p.s.). Altair can take two stereo images and process them in a frame-by-frame manner. If the left-hand image is input in the N-th time frame and then the right-hand image is input in the (N + 1)-th time frame, we can get information about left and right facial images with a one-frame delay. Then we can calculate the disparities between the faces, and can easily estimate the distance from a robot to a human with this information. In cases where several faces are detected, we classified various cases in order to identify corresponding faces. The experimental results show that we can detect faces and calculate distances in the range 0.5 m–3.5 m, with a 0.5%–8.9% error, at a processing speed of 15 f.p.s.     

Spatio-temporal filtering of thermal video sequences for heart rate estimation

In this paper, a novel method for non-contact measurement of heart rate using thermal imaging was proposed. Thermal videos were recorded from subjects’ faces. The measurements are performed on three different areas: the whole face, the upper half of the face and the supraorbital region. A tracker was used to track these regions to make the algorithm invulnerable to the subject's motion. After tracking, the videos were spatially filtered using a full Laplacian pyramid decomposition to increase the signal to noise ratio; next, the video frames were successively temporally filtered using an ideal bandpass filter for extracting the thermal variations caused by blood circulation. Finally, the heart rate was calculated by using two methods including zero crossing and Fast Fourier Transform. For evaluating the results, the complement of absolute normalized difference (CAND) index was used which was introduced by Pavlidis. This index was 99.42% in the best case and 92.472% in average for 22 subjects. These results showed a growth in CAND index in comparison with previous work. Zerocrossing outperformed FFT because of the nonstationary nature of thermal signals. Another benefit of our method is that, the videos are taken from the face unlike most of the studies that take it from the neck and Carotid. Neck and carotid are less accessible than faces. Finally, the optimum ROI for estimating the heart rate from face was identified.     

ARCosmetics: a real-time augmented reality cosmetics try-on system

A virtual cosmetics try-on system provides a realistic try-on experience for consumers and helps them efficiently choose suitable cosmetics. In this article, we propose a real-time augmented reality virtual cosmetics try-on system for smartphones (ARCosmetics), taking speed, accuracy, and stability into consideration at each step to ensure a better user experience. A novel and very fast face tracking method utilizes the face detection box and the average position of facial landmarks to estimate the faces in continuous frames. A dynamic weight Wing loss is introduced to assign a dynamic weight to every landmark by the estimated error during training. It balances the attention between small, medium, and large range error and thus increases the accuracy and robustness. We also designed a weighted average method to utilize the information of the adjacent frame for landmark refinement, guaranteeing the stability of the generated landmarks. Extensive experiments conducted on a large 106-point facial landmark dataset and the 300-VW dataset demonstrate the superior performance of the proposed method compared to other state-of-the-art methods. We also conducted user satisfaction studies further to verify the efficiency and effectiveness of our ARCosmetics system.     

Facial expression recognition of a speaker using front-view face judgment, vowel judgment, and thermal image processing

For facial expression recognition, we selected three images: (i) just before speaking, (ii) speaking the first vowel, and (iii) speaking the last vowel in an utterance. In this study, as a pre-processing module, we added a judgment function to distinguish a front-view face for facial expression recognition. A frame of the front-view face in a dynamic image is selected by estimating the face direction. The judgment function measures four feature parameters using thermal image processing, and selects the thermal images that have all the values of the feature parameters within limited ranges which were decided on the basis of training thermal images of front-view faces. As an initial investigation, we adopted the utterance of the Japanese name “Taro,” which is semantically neutral. The mean judgment accuracy of the front-view face was 99.5% for six subjects who changed their face direction freely. Using the proposed method, the facial expressions of six subjects were distinguishable with 84.0% accuracy when they exhibited one of the intentional facial expressions of “angry,” “happy,” “neutral,” “sad,” and “surprised.” We expect the proposed method to be applicable for recognizing facial expressions in daily conversation.     

基于MS-KCF模型的图像序列中人脸快速稳定检测

为快速稳定地检测图像序列中角度变化较大、遮挡较为严重的人脸，结合快速精确的目标检测模型MobileNet-SSD （MS）和快速跟踪模型核相关滤波（KCF），提出一种新的自动检测-跟踪-检测（DTD）模式，即MS-KCF人脸检测模型。首先，利用MS模型快速精确地对人脸进行检测，并且更新跟踪模型；其次，将检测到的人脸坐标信息输入到KCF跟踪模型中进行稳定的跟踪，并加快整体的检测速度；最后，为了防止跟踪丢失，跟踪数帧后再次更新检测模型，重新对人脸进行检测。实验显示，在FDDB人脸检测基准中，MS-KCF模型的召回率为93.60%；在WIDER FACE人脸检测基准的Easy、Medium和Hard数据集中，MS-KCF模型的召回率分别为93.11%、92.18%和82.97%，平均速度为193帧/s。实验结果表明，MS-KCF模型具有稳定性和快速性，在图像序列中对严重遮挡和角度变化大的人脸具有很好的检测效果。     

CFSM: a novel frame analyzing mechanism for real-time face recognition system on the embedded system

The development of web cameras and smart phones is mature, and more and more facial recognition-related applications are implemented on embedded systems. The demand for real-time face recognition on embedded systems is also increasing. In order to improve the accuracy of face recognition, most of the modern face recognition systems consist of multiple deep neural network models for recognition. However, in an embedded system, integrating these complex neural network models and execute simultaneously is not easy to achieve the goal of real-time recognition of human faces and their identities. In view of this, this study proposes a new frame analysis mechanism, continuous frames skipping mechanism (CFSM), which can analyze the frame in real time to determine whether it is necessary to perform face recognition on the current frame. Through the analysis of CFSM, the frames that do not need to be re-recognized for face are omitted. In this way, the workload of the face recognition system will be greatly reduced to achieve the goal of real-time face recognition in the embedded system. The experimental results show that the proposed CFSM mechanism can greatly increase the speed of face recognition in the video on the embedded system, achieving the goal of real-time face recognition.

     

基于深度学习的ARM平台实时人脸识别

针对ARM平台上人脸识别实时性不强和识别率低的问题，提出一种基于深度学习的实时人脸识别方法。首先基于MTCNN人脸检测算法设计了一种实时检测并追踪人脸的算法；然后在ARM平台上基于深度残差网络（ResNet）设计人脸特征提取网络；最后针对ARM平台的特点，使用Mali-GPU加速人脸特征提取网络的运算，分担CPU负荷，提高系统整体运行效率。算法部署在基于ARM的瑞芯微RK3399开发板上，运行速度达到22 帧/s。实验结果表明，与MobileFaceNet相比，该方法在MegaFace上的识别率提升了11个百分点。     

Design and VLSI implementation of a high-performance face detection engine

This paper proposes a novel hardware structure and field-programmable gate array (FPGA) implementation method for real-time detection of multiple human faces with robustness against illumination variations. These are designed to greatly improve face detection in various environments with using MCT techniques and the AdaBoost learning algorithm which is robust against variable illumination. We have designed, implemented, and verified the hardware architecture of the face detection engine for high-performance face detection and real-time processing. The face detection chip is developed by verifying and implementing it using a FPGA and an application-specific integrated circuit (ASIC). To verify and implement the chip, we used a Virtex5 LX330 FPGA board and a 0.18 μm 1-poly and 6-metal CMOS logic process. Performance results of the implementation and verification showed it is possible to detect at least 32 faces of a wide variety of sizes at a maximum speed of 147 frames per second.     

Extending Steinitz’s Theorem to Upward Star-Shaped Polyhedra and Spherical Polyhedra

In 1922, Steinitz’s theorem gave a complete characterization of the topological structure of the vertices, edges, and faces of convex polyhedra as triconnected planar graphs. In this paper, we generalize Steinitz’s theorem to non-convex polyhedra. More specifically, we introduce a new class of polyhedra, wider than convex polyhedra, called upward star-shaped polyhedra and spherical polyhedra, and present graph-theoretic characterization for both polyhedra. Upward star-shaped polyhedra are polyhedra where each face is star-shaped, all faces except the bottom face are visible from a view point, and any two faces sharing two vertices are non-coplanar. Spherical polyhedra are non-singular, non-coplanar polyhedra with no holes.     

Effect of gloss and diffuse reflectance of display frames on visual comfort

Abstract— A series of 12 display frames of different combinations of gloss levels and diffuse reflectance levels was mounted on a display to test the visual comfort of 31 test subjects. The frames were tested in a controlled environment where a light reflex was purposefully visible on the frame. The intensity of the light reflex was controllable by the user, and the dial setting was recorded for each frame. The frames were also shown in two different test office environments, one light and one dark. Qualitative comments about the frames were recorded. A gloss level of 30 gloss units corresponded to about 30% of test subjects that wanted no reflex at all to be visible on the frame. There was a significant effect of frame gloss on the acceptable light level of the reflex. When the acceptable‐light‐level dial setting was converted to luminance contrast, the acceptable luminance contrast on the frames decreased with increasing gloss, and this effect was more pronounced for black frames.     

A generic method of cleaning and enhancing handwritten data from business forms

The automation of business form processing is attracting intensive research interests due to its wide application and its reduction of the heavy workload due to manual processing. Preparing clean and clear images for the recognition engines is often taken for granted as a trivial task that requires little attention. In reality, handwritten data usually touch or cross the preprinted form frames and texts, creating tremendous problems for the recognition engines. In this paper, we contribute answers to two questions: “Why do we need cleaning and enhancement procedures in form processing systems?” and “How can we clean and enhance the hand-filled items with easy implementation and high processing speed?” Here, we propose a generic system including only cleaning and enhancing phases. In the cleaning phase, the system registers a template to the input form by aligning corresponding landmarks. A unified morphological scheme is proposed to remove the form frames and restore the broken handwriting from gray or binary images. When the handwriting is found touching or crossing preprinted texts, morphological operations based on statistical features are used to clean it. In applications where a black-and-white scanning mode is adopted, handwriting may contain broken or hollow strokes due to improper thresholding parameters. Therefore, we have designed a module to enhance the image quality based on morphological operations. Subjective and objective evaluations have been studied to show the effectiveness of the proposed procedures. Received January 19, 2000 / Revised March 20, 2001     

Robust3D: a robust 3D face reconstruction application

In the process of reconstructing a historical event such as a rock concert only from video, the reconstruction of faces and expressions of the musicians is obviously important. However, in the process of rebuilding appearance, because of the low quality of the video of the recorded concert, the result of the reconstruction may be far from the real appearance. In this paper, a robust 3D face reconstruction application is described that can be applied to a video recording. The application first uses DeblurGAN program to run anti-ambiguity calculation and removes the ambiguity in the concert video. Then, the super-resolution program is used to enlarge every frame of the concert video by four times, thus making every frame of the video clearer. Finally, the 3D faces are obtained after 3D reconstruction of the processed video frames via the 3DMM_CNN program.

     

Three-Dimensional Occlusion Detection and Restoration of Partially Occluded Faces

This paper presents an innovative three dimensional occlusion detection and restoration strategy for the recognition of three dimensional faces partially occluded by unforeseen, extraneous objects. The detection method considers occlusions as local deformations of the face that correspond to perturbations in a space designed to represent non-occluded faces. Once detected, occlusions represent missing information, or “holes” in the faces. The restoration module exploits the information provided by the non-occluded part of the face to recover the whole face, using an appropriate basis for the space in which non-occluded faces lie. The restoration strategy does not depend on the method used to detect occlusions and can also be applied to restore faces in the presence of noise and missing pixels due to acquisition inaccuracies. The strategy has been experimented on the occluded acquisitions taken from the Bosphorus 3D face database. A method for the generation of real-looking occlusions is also presented. Artificial occlusions, applied to the UND database, allowed for an in-depth analysis of the capabilities of our approach. Experimental results demonstrate the robustness and feasibility of our approach.     

Comparison between male and female subjective estimates of thermal effects and sensations

Thirty-one male subjects aged between 18 and 40 years and 15 female subjects aged between 18 and 24 years were used to compare male and female subjective estimates of the effect of heat on thermal sensation, drowsiness, boredom and fatigue. The subjects were exposed to climatic conditions between 74 degrees and 110 degrees F (23.3 degrees and 43.3 degrees C) Wet Bulb Globe Temperature (WBGT) while performing perceptual-motor tasks. Linear regression equations relating thermal sensation and ambient temperature were developed. These equations showed that the preferred temperature for males (71.6 degrees F (22 degrees C) WBGT) is significantly lower than that for females (77.1 degrees F (25 degrees C) WBGT). Females tend to feel more uncomfortable than males at both high and low temperature extremes. The subjective responses of thermal sensation take the same pattern as the subjective estimates of physiological responses to heat. Subjective estimates of thermal sensation in the heat relate directly to actual changes occurring in body temperature and heart rate. The subjective estimates by both males and females of drowsiness, boredom and fatigue form inverted U-curves when plotted against increasing temperature. The peaks of the inverted U or arousal curves for the two groups occur at 90 degrees F (32.2 degrees C) WBGT. The females reported higher levels of both drowsiness and boredom than the males at all temperatures, and the males reported higher fatigue than the females at the upper temperature levels.     

煤矿监控图像中脸肤色和模板检测

煤矿监控图像的对比度低、灰度不均匀,使监控图像的处理和识别难度增大.在巷道环境下采集多角度的矿工监控图像,根据脸部肤色区域像素的统计特征,用上下限阈值的方法在HSV颜色空间分割出各种角度的矿工脸部,用半径为2的圆形结构元素进行数学形态学的开启和闭合操作,去除被误分为脸部区域的背景像素;根据脸部灰度分布特征构造了平均脸模板,用相似度函数作为脸部模板匹配的判别函数,检测矿工脸部位置.结果表明,用肤色分割和平均脸部模板匹配方法,可以快速检测出矿工脸部的精确位置.     

基于人脸检测的人脸跟踪算法

文章提出了一种基于人脸检测技术的人脸跟踪算法。该算法利用前一帧的人脸检测结果预测当前帧中人脸可能的尺度与位置范围,在限定的范围内采用模板匹配与人工神经网分类的方法定位人脸,从而实现快速而可靠的人脸跟踪。由于使用了人脸检测技术,该方法可以自动定位初始人脸。实验表明该方法在具有复杂、动态变化背景的图象序列中是很有效的,速度为5－11Hz,可用于实时性系统。     

复杂背景下基于时间差分的人脸检测算法

本文主要结合序列图像的运动信息和肤色信息进行复杂背景下的人脸检测。首先利用连续三帧间的运动信息进行粗检测,采用自适应阈值法从差分图像中提取出运动区域,再对差分图像进行相与操作,限制搜索范围;然后利用人脸肤色信息,在YCbCr色彩空间内检测出肤色区域;最后利用人脸的几何特征信息对肤色区域作进一步验证,得到精确人脸。该方法综合了帧间和帧内的分析结果,实现简单、高效。实验证明,在复杂背景下对光照和其他噪声有较好的鲁棒性。     

Eliminating the effect of freezing frames on user perceptive by using a time interleaving technique

Streaming video over a wireless network faces several challenges such as high packet error rates, bandwidth variations, and delays, which could have negative effects on the video streaming and the viewer will perceive a frozen picture for certain durations due to loss of frames. In this study, we propose a Time Interleaving Robust Streaming (TIRS) technique to significantly reduce the frozen video problem and provide a satisfactory quality for the mobile viewer. This is done by reordering the streaming video frames as groups of even and odd frames. The objective of streaming the video in this way is to avoid the losses of a sequence of neighbouring frames in case of a long sequence interruption. We evaluate our approach by using a user panel and mean opinion score (MOS) measurements; where the users observe three levels of frame losses. The results show that our technique significantly improves the smoothness of the video on the mobile device in the presence of frame losses, while the transmitted data are only increased by almost 9% (due to reduced time locality).