Real-time shape tracking of facial landmarks

Detection of facial landmarks and accurate tracking of their shape are essential in real-time applications such as virtual makeup, where users can see the makeup’s effect by moving their face in diverse directions. Typical face tracking techniques detect facial landmarks and track them using a point tracker such as the Kanade-Lucas-Tomasi (KLT) point tracker. Typically, 5 or 64 points are used for tracking a face. Even though these points are enough to track the approximate locations of facial landmarks, they are not sufficient to track the exact shape of facial landmarks. In this paper, we propose a method that can track the exact shape of facial landmarks in real-time by combining a deep learning technique and a point tracker. We detect facial landmarks accurately using SegNet, which performs semantic segmentation based on deep learning. Edge points of detected landmarks are tracked using the KLT point tracker. In spite of its popularity, the KLT point tracker suffers from the point loss problem. We solve this problem by executing SegNet periodically to recalculate the shape of facial landmarks. That is, by combining the two techniques, we can avoid the computational overhead of SegNet and the point loss problem of the KLT point tracker, which leads to accurate real-time shape tracking. We performed several experiments to evaluate the performance of our method and report some of the results herein.

     

Landmark-free posture invariant human shape correspondence

We consider the problem of computing accurate point-to-point correspondences among a set of human bodies in varying postures using a landmark-free approach. The approach learns the locations of the anthropometric landmarks present in a database of human models in strongly varying postures and uses this knowledge to automatically predict the locations of these anthropometric landmarks on a newly available scan. The predicted landmarks are then used to compute point-to-point correspondences between a rigged template model and the newly available scan.     

Evaluating shape correspondence for statistical shape analysis: a benchmark study

This paper introduces a new benchmark study to evaluate the performance of landmark-based shape correspondence used for statistical shape analysis. Different from previous shape-correspondence evaluation methods, the proposed benchmark first generates a large set of synthetic shape instances by randomly sampling a given statistical shape model that defines a ground-truth shape space. We then run a test shape-correspondence algorithm on these synthetic shape instances to identify a set of corresponded landmarks. According to the identified corresponded landmarks, we construct a new statistical shape model, which defines a new shape space. We finally compare this new shape space against the ground-truth shape space to determine the performance of the test shape-correspondence algorithm. In this paper, we introduce three new performance measures that are landmark independent to quantify the difference between the ground-truth and the newly derived shape spaces. By introducing a ground-truth shape space that is defined by a statistical shape model and three new landmark-independent performance measures, we believe the proposed benchmark allows for a more objective evaluation of shape correspondence than previous methods. In this paper, we focus on developing the proposed benchmark for $2$D shape correspondence. However it can be easily extended to $3$D cases.     

Facial landmarks detection using improved active shape model on android platform

Detection of facial feature is fundamental for applications such as security, biometrics, 3D face modeling and personal authentication. Active Shape Model (ASM) is one of the most popular local texture models for face detection. This paper presents an issue related to face detection based on ASM, and proposes an efficient extraction algorithm for facial landmarks suitable for use on mobile devices. We modifies the original ASM to improve its performance with three changes; (1) Improving the initialization model using the center of the eyes by using a feature map of color information, (2) Constructing modified model definition and fitting more landmarks than the classical ASM, and (3) Extending and building a 2-D profile model for detecting faces in input image. The proposed method is evaluated on dataset containing over 700 images of faces, and experimental results reveal that the proposed algorithm exhibited a significant improvement of over 10.2 % in average success ratio, compared to the classic ASM, clearly outperforming on success rate and computing time.     

Robust shape extraction for automatically segmenting raw LiDAR data of outdoor scenes

ABSTRACT

Light detection and ranging (LiDAR) scanning has become a prevalent technique for digitalizing outdoor scenes with three-dimensional (3D) point clouds. An automatic segmentation of the LiDAR data is important for understanding and reconstructing the outdoor scenes. However, it is still a challenging task due to complex and various objects in the outdoor scenes and some scanning drawbacks in the LiDAR data. Observing that most of the objects, such as the ground, road, roof, and facade, can be locally described with a group of geometric shapes, e.g. plane, sphere, and cylinder, we propose an automatic method to segment raw LiDAR data by robustly extracting the shapes in this paper. Firstly, our method divides the raw LiDAR data into a number of supervoxels considering the geometric and spectral consistencies of LiDAR data. Secondly, we robustly extract shapes from each supervoxels using a random sample consensus (RANSAC)-based method and evaluates and optimises the extracted shapes with a density loss estimation technique. Finally, our method outputs the segmentation result by merging the extracted shapes into a group of complete shapes, each of them represents a meaningful object in the outdoor scene. Experiments show that the method is efficient and robust for extracting most of the shapes in the outdoor scenes from a given raw LiDAR point cloud, and no preprocessing is required.     

结合谱变换和马尔可夫模型的三维形状对齐

针对大尺度变形下的三维形状对齐问题,提出根据三维形状的等距性构造马尔可夫能量最小化模型,得到最优对齐结果。算法对三维模型进行谱变换,在变换空间中对三维模型进行初始化对齐。以谱距离和测地距离分别定义马尔可夫模型的单点势能函数和点对势能函数,形成可用于形状对齐的能量最小化模型。通过Alpha扩展算法对模型进行求解,得到最终的对齐结果。实验结果表明,算法在大尺度变形和拓扑变化等情况都能够输出很好的对齐结果。     

基于主动形状模型的人脸脸型自动分类算法

研究表明人脸脸型可以分为圆脸、椭圆脸、方脸、三角脸等.基于主动形状模型(ASM)提出了一种自动人脸脸型分类方法.首先利用各种脸形的样本进行训练以建立脸型形状模型库,然后运用ASM算法对测试样本自动定位正面人脸形状,比较其与各个脸型形状模型的距离,最后应用最近邻方法实现脸型的自动分类.仿真实验表明,该方法优于利用人脸轮廓曲率或下颌曲率的方法,能够充分挖掘人脸形状信息,分类结果稳定准确,可以有效提高大库人脸识别的速度和准确率.     

Biologically inspired collective construction with visual landmarks

We describe our research in using environmental visual landmarks as the basis for completing simple robot construction tasks.Inspired by honeybee visual navigation behavior,a visual template mechanism is proposed in which a natural landmark serves as a visual reference or template for distance determination as well as for navigation during collective construction.To validate our proposed mechanism,a wall construction problem is investigated and a minimalist solution is given.Experimental results show that,using the mechanism of a visual template,a collective robotic system can successfully build the desired structure in a decentralized fashion using only local sensing and no direct communication.In addition,a particular variable,which defines tolerance for alignment of the structure,is found to impact the system performance.By decreasing the value of the variable,system performance is improved at the expense of a longer construction time.The visual template mechanism is appealing in that it can use a reference point or salient object in a natural environment that is new or unexplored and it could be adapted to facilitate more complicated building tasks.     

Interactive optimization of 3D shape and 2D correspondence usingmultiple geometric constraints via POCS

The traditional approach of handling motion tracking and structure from motion (SFM) independently in successive steps exhibits inherent limitations in terms of achievable precision and incorporation of prior geometric constraints about the scene. This paper proposes a projections onto convex sets (POCS) framework for iterative refinement of the measurement matrix in the well-known factorization method to incorporate multiple geometric constraints about the scene, thereby improving the accuracy of both 2D feature point tracking and 3D structure estimates. Regularities in the scene, such as points on line and plane and parallel lines and planes, that can be interactively identified and marked at each POCS iteration, enforce rank and parallelism constraints on appropriately defined local measurement matrices, one for each constraint. The POCS framework allows for the integration of the information in each of these local measurement matrices into a single measurement matrix that is "closest" to the initial observed measurement matrix in Frobenius norm, which is then factored in the usual manner. Experimental results demonstrate that the proposed interactive POCS framework consistently improves both 2D correspondences and 3D shape/motion estimates and similar results cannot be achieved by enforcing these constraints as either post or preprocessing     

Multi-observation sensor resetting localization with ambiguous landmarks

Successful approaches to the robot localization problem include particle filters, which estimate non-parametric localization belief distributions. Particle filters are successful at tracking a robot’s pose, although they fare poorly at determining the robot’s global pose. The global localization problem has been addressed for robots that sense unambiguous visual landmarks with sensor resetting, by performing sensor-based resampling when the robot is lost. Unfortunately, for robots that make sparse, ambiguous and noisy observations, standard sensor resetting places new pose hypotheses across a wide region, in poses that may be inconsistent with previous observations. We introduce multi-observation sensor resetting (MOSR) to address the localization problem with sparse, ambiguous and noisy observations. MOSR merges observations from multiple frames to generate new hypotheses more effectively. We demonstrate experimentally on the NAO humanoid robots that MOSR converges more efficiently to the robot’s true pose than standard sensor resetting, and is more robust to systematic vision errors.     

Human body segmentation based on shape constraint

Human body segmentation is essential for many practical applications, e.g., video surveillance analysis in intelligent urban. However, existing methods mainly suffer from various human poses. In this paper, we try to address this issue by introducing human shape constraint. First, human pose estimation is performed, and locations of human body parts are determined. Contrast to the previous work, we just use the human body parts with high precision. Then we combines the star convexity and the human body parts’ locations as shape constraint. The final segmentation results are acquired through the optimization step. Comprehensive and comparative experimental results demonstrate that the proposed method achieves promising performance and outperforms many state-of-the-art methods over publicly available challenging datasets.     

基于变异技术的程序故障自动化修复方法

结合故障定位技术, 提出了一种基于变异的程序故障自动化修复方法, 并采用该方法阐释了C程序表达式故障的变异修复机制, 研制了C程序故障自动化修复辅助工具。通过对实际故障系统案例的分析, 证明了方法的有效性。     

Exploring contingency tables with correspondence analysis

An algorithm for correspondence analysis is described and implemented in SAS/IML (SAS Institute, 1985a). The technique is shown, through the analysis of several biological examples, to supplement the log-linear models approach to the analysis of contingency tables, both in the model identification and model interpretation stages of analysis. A simple two-way contingency table of tumor data is analyzed using correspondence analysis. This example emphasises the relationships between the parameters of the log-linear model for the table and the graphical correspondence analysis results. The technique is also applied to a three-way table of survey data concerning ulcer patients to demonstrate applications of simple correspondence analysis to higher dimensional tables with fixed margins. Finally, the diets and foraging behaviors of birds of the Hubbard Brook Forest are each analyzed and then a simultaneous display of the two separate but related tables is constructed to highlight relationships between the tables.     

Human fall detection using normalized shape aspect ratio

In video surveillance, automatic human fall detection is important to protect vulnerable groups such as the elderly. When the camera layout varies, the shape aspect ratio (SAR) of a human body may change substantially. In order to rectify these changes, in this paper, we propose an automatic human fall detection method using the normalized shape aspect ratio (NSAR). A calibration process and bicubic interpolation are implemented to generate the NSAR table for each camera. Compared with some representative fall detection methods using the SAR, the proposed method integrates the NSAR with the moving speed and direction information to robustly detect human fall, as well as being able to detect falls toward eight different directions for multiple humans. Moreover, while most of the existing fall detection methods were designed only for indoor environment, experimental results demonstrate that this newly proposed method can effectively detect human fall in both indoor and outdoor environments.

     

Human skin model capable of natural shape variation

In the production of character animation that treats living things moving at will, as in the case of humans and animals, it is important to express natural action and realistic body shape. If we can express these freely and easily, we will be able to apply character animation to many fields, such as the simulation of dances and sports, and electronic stand-ins. The computer graphics technique may be one of the most effective means to achieve such goals. We have developed human skin model capable of natural shape variation. This model has a skeleton structure, and free form surfaces cover the skeleton just like skin. The model permits continuous motion of every components of the skeleton according to actions. During such movements, the skin retains smoothness and naturalness. We are verifying the human skin model by producing several short animation pieces.     

Realistic wrinkle generation for 3D face modeling based on automatically extracted curves and improved shape control functions

Realistic wrinkles are extremely important for enhancing the realism of three-dimensional (3D) virtual face models. This paper proposes an approach for generating realistic wrinkle on a 3D face model based on a face image. It includes image preprocessing, automatically extracting wrinkle curves and generating wrinkles on a 3D surface. For image preprocessing, we use a linear transform method to conduct a grayscale conversion. We then use a transfinite-pixel neighborhood averaging method to reduce the noise, and a high pass filter to sharpen the image. For the automatic extraction of wrinkle curves, an improved Canny edge detector is employed. For wrinkle generation on a 3D surface, a number of novel techniques are employed. Some feature points are firstly defined both on the face image and on the 3D face model. By aligning these feature points, the extracted wrinkle curves are then projected onto the 3D face model. Finally, three shape control functions are used to produce more realistic properties of the 3D wrinkles. They are the proposed cross-section shape control function (CSCF) to determine the cross-section shape and size, the depth attenuation function (DAF) and the width attenuation function (WAF) to control the depth and width amplitude variations of the wrinkles, respectively. For better results, an adaptive subdivision is applied to the predefined influence region to adjust the resolution around the wrinkle mesh. The experiment results of applying this method to individualized 3D human models demonstrate the ability of our method to generate more natural wrinkles.     

用VB自动生成可发布的WEB电子相册

介绍了利用VB自动生成可直接用于网站发布的电子相册程序.