Multi-view face segmentation using fusion of statistical shape and appearance models

This paper demonstrates how a weighted fusion of multiple Active Shape (ASM) or Active Appearance (AAM) models can be utilized to perform multi-view facial segmentation with only a limited number of views available for training the models. The idea is to construct models only from frontal and profile views and subsequently fuse these models with adequate weights to segment any facial view. This reduces the problem of multi-view facial segmentation to that of weight estimation, the algorithm for which is proposed as well. The evaluation is performed on a set of 280 landmarked static face images corresponding to seven different rotation angles and on several video sequences of the AV@CAR database. The evaluation demonstrates that the estimation of the weights does not have to be very accurate in the case of ASM, while in the case of AAM the influence of correct weight estimation is more critical. The segmentation with the proposed weight estimation method produced accurate segmentations in 91% of 280 testing images with the median point-to-point error varying from two to eight pixels (1.8–7.2% of average inter-eye distance).     

Dynamic shape and appearance models

We propose a model of the joint variation of shape and appearance of portions of an image sequence. The model is conditionally linear, and can be thought of as an extension of active appearance models to exploit the temporal correlation of adjacent image frames. Inference of the model parameters can be performed efficiently using established numerical optimization techniques borrowed from finite-element analysis and system identification techniques     

An information fusion framework for robust shape tracking

Existing methods for incorporating subspace model constraints in shape tracking use only partial information from the measurements and model distribution. We propose a unified framework for robust shape tracking, optimally fusing heteroscedastic uncertainties or noise from measurement, system dynamics, and a subspace model. The resulting nonorthogonal subspace projection and fusion are natural extensions of the traditional model constraint using orthogonal projection. We present two motion measurement algorithms and introduce alternative solutions for measurement uncertainty estimation. We build shape models offline from training data and exploit information from the ground truth initialization online through a strong model adaptation. Our framework is applied for tracking in echocardiograms where the motion estimation errors are heteroscedastic in nature, each heart has a distinct shape, and the relative motions of epicardial and endocardial borders reveal crucial diagnostic features. The proposed method significantly outperforms the existing shape-space-constrained tracking algorithm. Due to the complete treatment of heteroscedastic uncertainties, the strong model adaptation, and the coupled tracking of double-contours, robust performance is observed even on the most challenging cases.     

Metamorphs: deformable shape and appearance models

This paper presents a new deformable modeling strategy aimed at integrating shape and appearance in a unified space. If we think traditional deformable models as active contours or evolving curve fronts, the new deformable shape and appearance models we propose are deforming disks or volumes. Each model has not only boundary shape but also interior appearance. The model shape is implicitly embedded in a higher dimensional space of distance transforms, thus represented by a distance map image. In this way, both shape and appearance of the model are defined in the pixel space. A common deformation scheme, the Free Form Deformations (FFD), parameterizes warping deformations of the volumetric space in which the model is embedded in, hence deforming both model boundary and interior simultaneously.     

Measurement error in statistical models of shape

Active shape models (ASMs) are popular and sophisticated methods of extracting features in (especially medical) images. Here we analyse the error in placing ASM points on the boundary of the feature. By using replications, a corrected covariance matrix is presented that should reduce the effects of placement error. We show analytically and via simulations that the cumulative variability for a given number of eigenvalues retained in principal components analysis (PCA) ought to be reduced by increasing levels of point-placement error. Results for predicted errors are in excellent agreement with the set-up parameters of two simulated shapes and with anecdotal evidence from the trained experts for real data taken from the OSTEODENT project. We derive an equation for the reliability of placing the points and we find values of 0.79 and 0.85 (where 0 = bad and 1 = good) for the two clinical experts for the OSTEODENT data. These analyses help us to understand the sources and effects of measurement error in shape models.     

Building optimal 2D statistical shape models

Statistical shape models are used widely as a basis for segmenting and interpreting images. A major drawback of the approach is the need, during training, to establish a dense correspondence across a training set of segmented shapes. We show that model construction can be treated as an optimisation problem, automating the process and guaranteeing the effectiveness of the resulting models. This is achieved by optimising an objective function with respect to the correspondence. We use an information theoretic objective function that directly promotes desirable features of the model. This is coupled with an effective method of manipulating correspondence, based on re-parameterising each training shape, to build optimal statistical shape models. The method is evaluated on several training sets of shapes, showing that it constructs better models than alternative approaches.     

加权多特征外观表示的实时目标追踪

目的 目标跟踪是计算机视觉领域重点研究方向之一,在智能交通、人机交互等方面有着广泛应用。尽管目前基于相关滤波的方法由于其高效、鲁棒在该领域取得了显著进展,但特征的选择和表示一直是追踪过程中建立目标外观时的首要考虑因素。为了提高外观模型的鲁棒性,越来越多的跟踪器中引入梯度特征、颜色特征或其他组合特征代替原始灰度单一特征,但是该类方法没有结合特征本身考虑不同特征在模型中所占的比重。方法 本文重点研究特征的选取以及融合方式,通过引入权重向量对特征进行融合,设计了基于加权多特征外观模型的追踪器。根据特征的计算方式,构造了一项二元一次方程,将权重向量的求解转化为确定特征的比例系数,结合特征本身的维度信息,得到方程的有限组整数解集,最后通过实验确定最终的比例系数,并将其归一化得到权重向量,进而构建一种新的加权混合特征模型对目标外观建模。结果 采用OTB-100中的100个视频序列,将本文算法与其他7种主流算法,包括5种相关滤波类方法,以精确度、平均中心误差、实时性为评价指标进行了对比实验分析。在保证实时性的同时,本文算法在Basketball、DragonBaby、Panda、Lemming等多个数据集上均表现出了更好的追踪结果。在100个视频集上的平均结果与基于多特征融合的尺度自适应跟踪器相比,精确度提高了1.2%。结论 本文基于相关滤波的追踪框架在进行目标的外观描述时引入权重向量,进而提出了加权多特征融合追踪器,使得在复杂动态场景下追踪长度更长,提高了算法的鲁棒性。     

A weighted voting framework for classifiers ensembles

We propose a probabilistic framework for classifier combination, which gives rigorous optimality conditions (minimum classification error) for four combination methods: majority vote, weighted majority vote, recall combiner and the naive Bayes combiner. The framework is based on two assumptions: class-conditional independence of the classifier outputs and an assumption about the individual accuracies. The four combiners are derived subsequently from one another, by progressively relaxing and then eliminating the second assumption. In parallel, the number of the trainable parameters increases from one combiner to the next. Simulation studies reveal that if the parameter estimates are accurate and the first assumption is satisfied, the order of preference of the combiners is: naive Bayes, recall, weighted majority and majority. By inducing label noise, we expose a caveat coming from the stability-plasticity dilemma. Experimental results with 73 benchmark data sets reveal that there is no definitive best combiner among the four candidates, giving a slight preference to naive Bayes. This combiner was better for problems with a large number of fairly balanced classes while weighted majority vote was better for problems with a small number of unbalanced classes.     

A shape analysis framework for neuromorphometry

This paper addresses in an integrated and systematic fashion the relatively overlooked but increasingly important issue of measuring and characterizing the geometrical properties of nerve cells and structures, an area often called neuromorphology. After discussing the main motivation for such an endeavour, a comprehensive mathematical framework for characterizing neural shapes, capable of expressing variations over time, is presented and used to underline the main issues in neuromorphology. Three particularly powerful and versatile families of neuromorphological approaches, including differential measures, symmetry axes/skeletons, and complexity, are presented and their respective potentials for applications in neuroscience are identified. Examples of applications of such measures are provided based on experimental investigations related to automated dendrogram extraction, mental retardation characterization, and axon growth analysis.     

Online learning and fusion of orientation appearance models for robust rigid object tracking

We introduce a robust framework for learning and fusing of orientation appearance models based on both texture and depth information for rigid object tracking. Our framework fuses data obtained from a standard visual camera and dense depth maps obtained by low-cost consumer depth cameras such as the Kinect. To combine these two completely different modalities, we propose to use features that do not depend on the data representation: angles. More specifically, our framework combines image gradient orientations as extracted from intensity images with the directions of surface normals computed from dense depth fields. We propose to capture the correlations between the obtained orientation appearance models using a fusion approach motivated by the original Active Appearance Models (AAMs). To incorporate these features in a learning framework, we use a robust kernel based on the Euler representation of angles which does not require off-line training, and can be efficiently implemented online. The robustness of learning from orientation appearance models is presented both theoretically and experimentally in this work. This kernel enables us to cope with gross measurement errors, missing data as well as other typical problems such as illumination changes and occlusions. By combining the proposed models with a particle filter, the proposed framework was used for performing 2D plus 3D rigid object tracking, achieving robust performance in very difficult tracking scenarios including extreme pose variations.     

A statistical framework for mining substitution rules

In this paper, a new mining capability, called mining of substitution rules, is explored. A substitution refers to the choice made by a customer to replace the purchase of some items with that of others. The mining of substitution rules in a transaction database, the same as that of association rules, will lead to very valuable knowledge in various aspects, including market prediction, user behaviour analysis and decision support. The process of mining substitution rules can be decomposed into two procedures. The first procedure is to identify concrete itemsets among a large number of frequent itemsets, where a concrete itemset is a frequent itemset whose items are statistically dependent. The second procedure is then on the substitution rule generation. In this paper, we first derive theoretical properties for the model of substitution rule mining and devise a technique on the induction of positive itemset supports to improve the efficiency of support counting for negative itemsets. Then, in light of these properties, the SRM (substitution rule mining) algorithm is designed and implemented to discover the substitution rules efficiently while attaining good statistical significance. Empirical studies are performed to evaluate the performance of the SRM algorithm proposed. It is shown that the SRM algorithm not only has very good execution efficiency but also produces substitution rules of very high quality.     

Surface completion for shape and appearance

In this paper, we present a new surface content completion system that can effectively repair both shape and appearance from scanned, incomplete point set inputs. First, geometric holes can be robustly identified from noisy and defective data sets without the need for any normal or orientation information. The geometry and texture information of the holes can then be determined either automatically from the models’ context, or interactively from users’ selection. We use local parameterizations to align patches in order to extract their curvature-driven digital signature. After identifying the patch that most resembles each hole region, the geometry and texture information can be completed by warping the candidate region and gluing it onto the hole area. The displacement vector field for the exact alignment process is computed by solving a Poisson equation with boundary conditions. Our experiments show that the unified framework, founded upon the techniques of deformable models, local parameterization, and PDE modeling, can provide a robust and elegant solution for content completion of defective, complex point surfaces.     

A statistical framework for few-shot action recognition

Along with the exponential growth of online video creation platforms such as Tik Tok and Instagram, state of the art research involving quick and effective action/gesture recognition remains crucial. This work addresses the challenge of classifying short video clips, using a domain-specific feature design approach, capable of performing significantly well using as little as one training example per action. The method is based on Gunner Farneback’s dense optical flow (GF-OF) estimation strategy, Gaussian mixture models, and information divergence. We first aim to obtain accurate representations of the human movements/actions by clustering the results given by GF-OF using K-means method of vector quantization. We then proceed by representing the result of one instance of each action by a Gaussian mixture model. Furthermore, using Kullback-Leibler divergence (KL-divergence), we attempt to find similarities between the trained actions and the ones in the test videos. Classification is done by matching each test video to the trained action with the highest similarity (a.k.a lowest KL-divergence). We have performed experiments on the KTH and Weizmann Human Action datasets using One-Shot and K-Shot learning approaches, and the results reveal the discriminative nature of our proposed methodology in comparison with state-of-the-art techniques.

     

Human action recognition: a framework of statistical weighted segmentation and rank correlation-based selection

Pattern Analysis and Applications - Human action recognition from a video sequence has received much attention lately in the field of computer vision due to its range of applications in...     

A unifying framework for automatic and semi-automatic segmentation of vertebrae from radiographs using sample-driven active shape models

Segmentation of vertebral contours is an essential task in the design of imaging biomarkers for osteoporosis based on vertebra shape or texture. In this paper, we propose a novel automatic segmentation technique which can optionally be constrained by the user. The proposed technique solves the segmentation problem in a hierarchical manner. In the first phase, a coarse estimate of the overall spine alignment and the vertebra locations is computed using a sampling scheme. These samples are used to initialize a second phase of active shape model search, under a nonlinear model of vertebra appearance. The search is constrained by a conditional shape model, based on the variability of the coarse spine location estimates. In supplement, we describe an approach for manual initialization of the segmentation procedure as a simple set of constraints on the fully automatic technique. The technique is evaluated on a data base of 157 manually annotated lumbar radiographs, resulting in a final mean point-to-contour error of $0.81~\pm ~0.98$  mm for automatic segmentation. The results outperform the previous work in automatic vertebra segmentation in terms of both segmentation accuracy and failure rate, offering a both automatic and semi-automatic approach in one unifying framework.