A Unified Gradient-Based Approach for Combining ASM into AAM

Active Appearance Model (AAM) framework is a very useful method that can fit the shape and appearance model to the input image for various image analysis and synthesis problems. However, since the goal of the AAM fitting algorithm is to minimize the residual error between the model appearance and the input image, it often fails to accurately converge to the landmark points of the input image. To alleviate this weakness, we have combined Active Shape Models (ASM) into AAMs, in which ASMs try to find correct landmark points using the local profile model. Since the original objective function of the ASM search is not appropriate for combining these methods, we derive a gradient based iterative method by modifying the objective function of the ASM search. Then, we propose a new fitting method that combines the objective functions of both ASM and AAM into a single objective function in a gradient based optimization framework. Experimental results show that the proposed fitting method reduces the average fitting error when compared with existing fitting methods such as ASM, AAM, and Texture Constrained-ASM (TC-ASM) and improves the performance of facial expression recognition significantly.     

一种鲁棒的全自动人脸特征点定位方法

人脸特征点定位的目标是能够对人脸进行全自动精确定位. 主动形状模型(Active shape modal, ASM)和主动表象模型(Active appearance modal, AAM)的发表为全自动人脸特征点定位工作提供了很好的思路和解决框架. 之后很多研究工作也都在ASM和AAM的框架下进行了改进. 但是目前的研究工作尚未很好地解决人脸表情、光照以及姿态变化情况下的人脸特征点定位问题, 本文基于ASM框架提出了全自动人脸特征点定位算法. 和传统ASM方法以及ASM的改进方法的不同在于: 1)引进有效的机器学习方法来建立局部纹理模型. 这部分工作改进了传统ASM方法中用灰度图像的梯度分布进行局部纹理建模的方法, 引入了基于随机森林分类器和点对比较特征的局部纹理建模方法. 这种方法基于大量样本的统计学习, 能够有效解决人脸特征点定位中光照和表情变化这些难点; 2)在人脸模型参数优化部分, 本文成功地将分类器输出的结果结合到人脸模型参数优化的目标函数当中, 并且加入形状限制项使得优化的目标函数更为合理. 本文在包含表情、光照以及姿态变化的人脸数据上进行实验, 实验结果证明本文提出的全自动人脸特征点定位方法能够有效地适应人脸的光照和表情变化. 在姿态数据库上的测试结果说明了本算法的有效性.     

Multi-task learning with Multi-view Weighted Fusion Attention for artery-specific calcification analysis

In general, artery-specific calcification analysis comprises the simultaneous calcification segmentation and quantification tasks. It can help provide a thorough assessment for calcification of different coronary arteries, and further allow for an efficient and rapid diagnosis of cardiovascular diseases (CVD). However, as a high-dimensional multi-type estimation problem, artery-specific calcification analysis has not been profoundly investigated due to the intractability of obtaining discriminative feature representations. In this work, we propose a Multi-task learning network with Multi-view Weighted Fusion Attention (MMWFAnet) to solve this challenging problem. The MMWFAnet first employs a Multi-view Weighted Fusion Attention (MWFA) module to extract discriminative feature representations by enhancing the collaboration of multiple views. Specifically, MWFA weights these views to improve multi-view learning for calcification features. Based on the fusion of these multiple views, the proposed approach takes advantage of multi-task learning to obtain accurate segmentation and quantification of artery-specific calcification simultaneously. We perform experimental studies on 676 non-contrast Computed Tomography scans, achieving state-of-the-art performance in terms of multiple evaluation metrics. These compelling results evince that the proposed MMWFAnet is capable of improving the effectivity and efficiency of clinical CVD diagnosis.     

基于AAM和T型结构的人脸3D姿态估计

在分析已有的人脸姿态估计方法基础上,提出了一种基于主动表观模型(AAM)和T型结构的人脸3D姿态估计方法。对多姿态的人脸样本进行训练,得到多姿态的AAM模板集;利用训练得到的多姿态的AAM模板集进行最佳模板匹配,并对人脸的特征点进行精确定位;用人脸的双眼和嘴部构建T型模型,进行人脸3D姿态的参数估计。实验结果表明,该方法能适应较大的姿态旋转角度,并具有良好的姿态估计精度。     

Pose-Robust Facial Expression Recognition Using View-Based 2D $+$ 3D AAM

This paper proposes a pose-robust face tracking and facial expression recognition method using a view-based 2D 3D active appearance model (AAM) that extends the 2D 3D AAM to the view-based approach, where one independent face model is used for a specific view and an appropriate face model is selected for the input face image. Our extension has been conducted in many aspects. First, we use principal component analysis with missing data to construct the 2D 3D AAM due to the missing data in the posed face images. Second, we develop an effective model selection method that directly uses the estimated pose angle from the 2D 3D AAM, which makes face tracking pose-robust and feature extraction for facial expression recognition accurate. Third, we propose a double-layered generalized discriminant analysis (GDA) for facial expression recognition. Experimental results show the following: 1) The face tracking by the view-based 2D 3D AAM, which uses multiple face models with one face model per each view, is more robust to pose change than that by an integrated 2D 3D AAM, which uses an integrated face model for all three views; 2) the double-layered GDA extracts good features for facial expression recognition; and 3) the view-based 2D 3D AAM outperforms other existing models at pose-varying facial expression recognition.     

两种超声颈动脉血管斑块图像分割方法比较与改进

针对颈动脉超声图像,实现了两种颈动脉血管斑块的分割方法—活动形状模型(Active Shape Models,ASM)和活动表观模型(Active Appearance Models,AAM),对38组颈动脉超声图像进行了内外轮廓分割,并比较了两类算法对颈动脉内外轮廓分割的有效性。在综合分析实验结果的基础上,结合颈动脉超声图像的特点,通过引入比例不变性改进了ASM算法。统计结果表明,在运行时间上,ASM和改进ASM的运行时间相近,AAM大约为ASM和改进ASM的16倍。同时,采用FOM和RAY两种方法对分割效果进行评价,结果表明,改进ASM算法的分割效果较ASM有了很大的提高,是最适合颈动脉血管斑块超声图像分割的算法。     

一种鲁棒高效的人脸特征点跟踪方法

人脸特征点跟踪能获取除粗略的人脸位置和运动轨迹以外的人脸部件的精确信息,对计算机视觉研究有重要作用.主动表象模型(Active appearance model, AAM)是描述人脸特征点位置的最有效的方法之一,但是其高维参数空间和梯度下降优化策略使得AAM对初始参数敏感,且易陷入局部极值. 因此,基于传统AAM的人脸特征点跟踪方法不能同时较好地解决大姿态、光照和表情的问题.本文在多视角AAM的框架下,提出一种结合随机森林和线性判别分析(Linear discriminate analysis, LDA)的实时姿态估计算法对跟踪的人脸进行姿态预估计和更新,从而有效地解决了视频人脸大姿态变化的问题.提出了一种改进的在线表象模型(Online appearance model, OAM)方法来评估跟踪的准确性,并自适应地通过增量主成分分析(Principle component analysis, PCA) 学习来更新AAM的纹理模型,极大地提高了跟踪的稳定性和模型应对光照和表情变化的能力.实验结果表明,本文算法在视频人脸特征点跟踪的准确性、鲁棒性和实时性方面都有良好的性能.     

Pose robust face tracking by combining view-based AAMs and temporal filters

Active appearance models (AAMs) are useful for face tracking for the advantages of detailed face interpretation, accurate alignment and high efficiency. However, they are sensitive to initial parameters and may easily be stuck in local minima due to the gradient-descent optimization, which makes the AAM based face tracker unstable in the presence of large pose deviation and fast motion. In this paper, we propose to combine the view-based AAMs with two novel temporal filters to overcome the limitations. First, we build a new view space based on the shape parameters of AAMs, instead of the model parameters controlling both the shape and appearance, for the purpose of pose estimation. Then the Kalman filter is used to simultaneously update the pose and shape parameters for a better fitting of each frame. Second, we propose a temporal matching filter which is twofold. The inter-frame local appearance constraint is incorporated into AAM fitting, where the mechanism of the active shape model (ASM) is also implemented in a unified framework to find more accurate matching points. Moreover, we propose to initialize the shape with correspondences found by a random forest based local feature matching. By introducing the local information and temporal correspondences, the twofold temporal matching filter improves the tracking stability when confronted with fast appearance changes. Experimental results show that our algorithm is more pose robust than basic AAMs and some state-of-art AAM based methods, and that it can also handle large expressions and non-extreme illumination changes in test video sequences.     

ASM的手骨提取方法研究

骨龄自动评估面临的困难是骨骼准确定位与骨骼兴趣区域提取。由于手骨X光图像存在光照不均及骨骼发育程度不规则等因素影响,传统的图像分割方法在骨骼上的分割效果不太理想;为了实现对手骨边缘的精确提取,结合AdaBoost级联分类器,提出基于ASM（主动形状模型）算法的手骨边缘提取方法,丰富了骨龄自动评价系统的应用研究。实验表明,基于ASM算法的手骨分割能有效对手骨X射线图像进行准确的定位,为骨龄自动化评价系统的下一步工作奠定基础。     

Multi-View AAM Fitting and Construction

Active Appearance Models (AAMs) are generative, parametric models that have been successfully used in the past to model deformable objects such as human faces. The original AAMs formulation was 2D, but they have recently been extended to include a 3D shape model. A variety of single-view algorithms exist for fitting and constructing 3D AAMs but one area that has not been studied is multi-view algorithms. In this paper we present multi-view algorithms for both fitting and constructing 3D AAMs. Fitting an AAM to an image consists of minimizing the error between the input image and the closest model instance; i.e. solving a nonlinear optimization problem. In the first part of the paper we describe an algorithm for fitting a single AAM to multiple images, captured simultaneously by cameras with arbitrary locations, rotations, and response functions. This algorithm uses the scaled orthographic imaging model used by previous authors, and in the process of fitting computes, or calibrates, the scaled orthographic camera matrices. In the second part of the paper we describe an extension of this algorithm to calibrate weak perspective (or full perspective) camera models for each of the cameras. In essence, we use the human face as a (non-rigid) calibration grid. We demonstrate that the performance of this algorithm is roughly comparable to a standard algorithm using a calibration grid. In the third part of the paper, we show how camera calibration improves the performance of AAM fitting. A variety of non-rigid structure-from-motion algorithms, both single-view and multi-view, have been proposed that can be used to construct the corresponding 3D non-rigid shape models of a 2D AAM. In the final part of the paper, we show that constructing a 3D face model using non-rigid structure-from-motion suffers from the Bas-Relief ambiguity and may result in a “scaled” (stretched/compressed) model. We outline a robust non-rigid motion-stereo algorithm for calibrated multi-view 3D AAM construction and show how using calibrated multi-view motion-stereo can eliminate the Bas-Relief ambiguity and yield face models with higher 3D fidelity. Electronic Supplementary Material The online version of this article () contains supplementary material, which is available to authorized users.     

基于MR-AAM双重拟合的人脸特征点定位方法

传统的主动表观模型(AAM)反向组合算法仅进行了单次拟合过程,当初始位置与目标对象偏移过大时,往往会陷入局部最小,难以收敛到正确位置。针对此问题,提出了一种基于多分辨率AAM(MR-AAM)的双重拟合方法,首先在低分辨率模型下进行第一次拟合以确定面部初始位置,然后在高分辨率模型下进行二次拟合。由于能够快速获得较准确的初始位置,进而取得较好的人脸特征标定结果。实验结果表明,所提方法与传统方法相比,在能保证实时的情况下,提高了拟合精度。     

Gaze direction estimation using support vector machine with active appearance model

In recent years, research on human-computer interaction is becoming popular, most of which uses body movements, gestures or eye gaze direction. Until now, gazing estimation is still an active research domain. We propose an efficient method to solve the problem of the eye gaze point. We first locate the eye region by modifying the characteristics of the Active Appearance Model (AAM). Then by employing the Support Vector Machine (SVM), we estimate the five gazing directions through classification. The original 68 facial feature points in AAM are modified into 36 eye feature points. According to the two-dimensional coordinates of feature points, we classify different directions of eye gazing. The modified 36 feature points describe the contour of eyes, iris size, iris location, and the position of pupils. In addition, the resolution of cameras does not affect our method to determine the direction of line of sight accurately. The final results show the independence of classifications, less classification errors, and more accurate estimation of the gazing directions.     

Multi-view clustering via simultaneous weighting on views and features

In big data era, more and more data are collected from multiple views, each of which reflect distinct perspectives of the data. Many multi-view data are accompanied by incompatible views and high dimension, both of which bring challenges for multi-view clustering. This paper proposes a strategy of simultaneous weighting on view and feature to discriminate their importance. Each feature of multi-view data is given bi-level weights to express its importance in feature level and view level, respectively. Furthermore, we implements the proposed weighting method in the classical k-means algorithm to conduct multi-view clustering task. An efficient gradient-based optimization algorithm is embedded into k-means algorithm to compute the bi-level weights automatically. Also, the convergence of the proposed weight updating method is proved by theoretical analysis. In experimental evaluation, synthetic datasets with varied noise and missing-value are created to investigate the robustness of the proposed approach. Then, the proposed approach is also compared with five state-of-the-art algorithms on three real-world datasets. The experiments show that the proposed method compares very favourably against the other methods.     

基于统计模型与Gabor小波的人脸对齐

将基于Gabor小波的人脸特征点跟踪算法与基于统计模型的主动外观模型AAM人脸特征点定位方法结合起来,实现视频中人脸的自动对齐。先利用Gabor小波进行特征点跟踪,其结果作为AAM的初始形状。利用AAM的全局形状和纹理信息作为约束,对Gabor小波的局部跟踪错误进行校正。实验表明,该方法是有效的。     

Tracking Multiple Occluding People by Localizing on Multiple Scene Planes

Occlusion and lack of visibility in crowded and cluttered scenes make it difficult to track individual people correctly and consistently, particularly in a single view. We present a multi-view approach to solving this problem. In our approach we neither detect nor track objects from any single camera or camera pair; rather evidence is gathered from all the cameras into a synergistic framework and detection and tracking results are propagated back to each view. Unlike other multi-view approaches that require fully calibrated views our approach is purely image-based and uses only 2D constructs. To this end we develop a planar homographic occupancy constraint that fuses foreground likelihood information from multiple views, to resolve occlusions and localize people on a reference scene plane. For greater robustness this process is extended to multiple planes parallel to the reference plane in the framework of plane to plane homologies. Our fusion methodology also models scene clutter using the Schmieder and Weathersby clutter measure, which acts as a confidence prior, to assign higher fusion weight to views with lesser clutter. Detection and tracking are performed simultaneously by graph cuts segmentation of tracks in the space-time occupancy likelihood data. Experimental results with detailed qualitative and quantitative analysis, are demonstrated in challenging multi-view, crowded scenes.     

多视角模糊双加权可能性聚类算法

为解决传统可能性聚类算法（PCM）无法满足多视角学习场景聚类的实际问题,并进一步考虑到现有多视角聚类算法尚未重视的视角权重及视角内特征权重优化问题,本文提出一种新的具备最佳视角及最优特征划分能力的多视角模糊双加权可能性聚类算法（MV-FDW-PCM）。该算法将基于传统的PCM算法,给出了详细的多视角聚类学习框架使得PCM算法具备多视角聚类能力,进而通过引入视角间模糊加权机制及视角内属性模糊加权机制解决视角间权重及视角内特征权重优化问题。实验结果表明,所提的MV-FDW-PCM算法在面对多视角聚类问题时较以往算法具有更佳的聚类效果。     

基于改进主动形状模型的人脸表情识别

主动形状模型（ASM）是面部特征定位、人脸识别和表情识别等模式识别领域中常用的一种方法。但受到初始情况、光照等诸多因素的影响,其性能会有所下降。研究了一种改进的主动形状模型,改进主要体现在两个方面：第一,首先用点轮廓检测算法（PCDM）检测到双眼的位置,为ASM中的点分布模型粗略地定位好初始位置;第二,从ASM原始的思想出发,充分挖掘标定点之间的联系,提出一种构建局部纹理模型的新方法。在JAFFE人脸数据库中进行验证,结果表明,改进ASM方法提高了搜索速度与特征点定位的精度。最终构造神经网络分类器进行人脸表情识别实验,得到了较好的识别率。     

基于改进联合模型的人脸表情识别

在联合主动表观模型和主动形状模型的基础上,充分挖掘标定点之间的联系,提出一种局部纹理模型构建方法。通过改进匹配算法提高特征点的定位精度和匹配速度。将该算法提取到的人脸表情特征输入最近邻分类器,分类结果表明其识别率较高。     

Multi-view correlation tracking with adaptive memory-improved update model

Recently, some researchers concentrate on applying multi-view learning to the correlation filter tracking to achieve both the efficiency and accuracy. However, most of them fail to effectively collaborate multiple views to deal with more complex environment. Moreover, their methods are prone to drift in case of long-term occlusion due to the memory loss. In this paper, we propose a novel multi-view correlation filters-based tracker for robust visual tracking. First, we present an adaptive multi-view collaboration strategy to highlight different contributions of different views by jointly considering the reliability and discrimination. Second, an effective memory-improved model update rule is introduced to avoid falling into a contaminated target model. Compared with the conventional linear interpolation update rule, it can effectively deal with long-term occlusion by improving the memory of historical models. Furthermore, instead of assigning a unified learning rate for all views in each frame, we design varying learning rates for different views according to their respective evaluations on the current tracking result, which can prevent the target models of all views from being contaminated at the same time. Finally, a failure-aware scale update scheme is developed to avoid noisy scale estimation in case of temporal tracking failure. Extensive experimental results on the recent benchmark demonstrate that our tracker performs favorably against other state-of-the-art trackers with a real-time performance.

     

Multi-view face recognition based on tensor subspace analysis and view manifold modeling

This paper aims to address the face recognition problem with a wide variety of views. We proposed a tensor subspace analysis and view manifold modeling based multi-view face recognition algorithm by improving the TensorFace based one. Tensor subspace analysis is applied to separate the identity and view information of multi-view face images. To model the nonlinearity in view subspace, a novel view manifold is introduced to TensorFace. Thus, a uniform multi-view face model is achieved to deal with the linearity in identity subspace as well as the nonlinearity in view subspace. Meanwhile, a parameter estimation algorithm is developed to solve the view and identity factors automatically. The new face model yields improved facial recognition rates against the traditional TensorFace based method.