Active Appearance Models Revisited

Active Appearance Models (AAMs) and the closely related concepts of Morphable Models and Active Blobs are generative models of a certain visual phenomenon. Although linear in both shape and appearance, overall, AAMs are nonlinear parametric models in terms of the pixel intensities. Fitting an AAM to an image consists of minimising the error between the input image and the closest model instance; i.e. solving a nonlinear optimisation problem. We propose an efficient fitting algorithm for AAMs based on the inverse compositional image alignment algorithm. We show that the effects of appearance variation during fitting can be precomputed (projected out) using this algorithm and how it can be extended to include a global shape normalising warp, typically a 2D similarity transformation. We evaluate our algorithm to determine which of its novel aspects improve AAM fitting performance.Supplementary material to this paper is available in electronic form at http://dx.doi.org/10.1023/B:VISI.0000029666.37597.d3     

Video-based face model fitting using Adaptive Active Appearance Model

Active Appearance Model (AAM) represents the shape and appearance of an object via two low-dimensional subspaces, one for shape and one for appearance. AAM for facial images is currently receiving considerable attention from the computer vision community. However, most existing work focuses on fitting an AAM to a single image. For many applications, effectively fitting an AAM to video sequences is of critical importance and challenging, especially considering the varying quality of real-world video content. This paper proposes an Adaptive Active Appearance Model (AAAM) to address this problem, where both a generic AAM component and a subject-specific appearance model component are employed simultaneously in the proposed fitting scheme. While the generic AAM component is held fixed, the subject-specific model component is updated during the fitting process by selecting the frames that can be best explained by the generic model. Experimental results from both indoor and outdoor representative video sequences demonstrate the faster fitting convergence and improved fitting accuracy.     

Phonetic segmentation using multiple speech features

In this paper we propose a method for improving the performance of the segmentation of speech waveforms to phonetic units. The proposed method is based on the well known Viterbi time-alignment algorithm and utilizes the phonetic boundary predictions from multiple speech parameterization techniques. Specifically, we utilize the most appropriate, with respect to boundary type, phone transition position prediction as initial point to start Viterbi time-alignment for the prediction of the successor phonetic boundary. The proposed method was evaluated on the TIMIT database, with the exploitation of several, well known in the area of speech processing, Fourier-based and wavelet-based speech parameterization algorithms. The experimental results for the tolerance of 20 milliseconds indicated an improvement of the absolute segmentation accuracy of approximately 0.70%, when compared to the baseline speech segmentation scheme.     

Object segmentation using graph cuts based active contours

In this paper we present a graph cuts based active contours (GCBAC) approach to object segmentation. GCBAC approach is a combination of the iterative deformation idea of active contours and the optimization tool of graph cuts. It differs from traditional active contours in that it uses graph cuts to iteratively deform the contour and its cost function is defined as the summation of edge weights on the cut. The resulting contour at each iteration is the global optimum within a contour neighborhood (CN) of the previous result. Since this iterative algorithm is shown to converge, the final contour is the global optimum within its own CN. The use of contour neighborhood alleviates the well-known bias of the minimum cut in favor of a shorter boundary. GCBAC approach easily extends to the segmentation of three and higher dimensional objects, and is suitable for interactive correction. Experimental results on selected data sets and performance analysis are provided.     

The Painful Face - Pain Expression Recognition Using Active Appearance Models

Pain is typically assessed by patient self-report. Self-reported pain, however, is difficult to interpret and may be impaired or in some circumstances (i.e., young children and the severely ill) not even possible. To circumvent these problems behavioral scientists have identified reliable and valid facial indicators of pain. Hitherto, these methods have required manual measurement by highly skilled human observers. In this paper we explore an approach for automatically recognizing acute pain without the need for human observers. Specifically, our study was restricted to automatically detecting pain in adult patients with rotator cuff injuries. The system employed video input of the patients as they moved their affected and unaffected shoulder. Two types of ground truth were considered. Sequence-level ground truth consisted of Likert-type ratings by skilled observers. Frame-level ground truth was calculated from presence/absence and intensity of facial actions previously associated with pain. Active appearance models (AAM) were used to decouple shape and appearance in the digitized face images. Support vector machines (SVM) were compared for several representations from the AAM and of ground truth of varying granularity. We explored two questions pertinent to the construction, design and development of automatic pain detection systems. First, at what level (i.e., sequence- or frame-level) should datasets be labeled in order to obtain satisfactory automatic pain detection performance? Second, how important is it, at both levels of labeling, that we non-rigidly register the face?     

Painful monitoring: Automatic pain monitoring using the UNBC-McMaster shoulder pain expression archive database

In intensive care units in hospitals, it has been recently shown that enormous improvements in patient outcomes can be gained from the medical staff periodically monitoring patient pain levels. However, due to the burden/stress that the staff are already under, this type of monitoring has been difficult to sustain so an automatic solution could be an ideal remedy. Using an automatic facial expression system to do this represents an achievable pursuit as pain can be described via a number of facial action units (AUs). To facilitate this work, the “University of Northern British Columbia-McMaster Shoulder Pain Expression Archive Database” was collected which contains video of participant's faces (who were suffering from shoulder pain) while they were performing a series of range-of-motion tests. Each frame of this data was AU coded by certified FACS coders, and self-report and observer measures at the sequence level were taken as well. To promote and facilitate research into pain and augmentcurrent datasets, we have publicly made available a portion of this database, which includes 200 sequences across 25 subjects, containing more than 48,000 coded frames of spontaneous facial expressions with 66-point AAM tracked facial feature landmarks. In addition to describing the data distribution, we give baseline pain and AU detection results on a frame-by-frame basis at the binary-level (i.e. AU vs. no-AU and pain vs. no-pain) using our AAM/SVM system. Another contribution we make is classifying pain intensities at the sequence-level by using facial expressions and 3D head pose changes.     

A novel active contour model for image segmentation using distance regularization term

In this paper, a novel active contour model (R-DRLSE model) based on level set method is proposed for image segmentation. The R-DRLSE model is a variational level set approach that utilizes the region information to find image contours by minimizing the presented energy functional. To avoid the time-consuming re-initialization step, the distance regularization term is used to penalize the deviation of the level set function from a signed distance function. The numerical implementation scheme of the model can significantly reduce the iteration number and computation time. The results of experiments performed on some synthetic and real images show that the R-DRLSE model is effective and efficient. In particular, our method has been applied to MR kidney image segmentation with desirable results.     

Active contours with selective local or global segmentation: A new formulation and level set method

A novel region-based active contour model (ACM) is proposed in this paper. It is implemented with a special processing named Selective Binary and Gaussian Filtering RegularizedLevel Set(SBGFRLS) method, which first selectively penalizes the level set function to be binary, and then uses a Gaussian smoothing kernel to regularize it. The advantages of our method are as follows. First, a new region-based signed pressure force (SPF) function is proposed, which can efficiently stop the contours at weak or blurred edges. Second, the exterior and interior boundaries can be automatically detected with the initial contour being anywhere in the image. Third, the proposed ACM with SBGFRLS has the property of selective local or global segmentation. It can segment not only the desired object but also the other objects. Fourth, the level set function can be easily initialized with a binary function, which is more efficient to construct than the widely used signed distance function (SDF). The computational cost for traditional re-initialization can also be reduced. Finally, the proposed algorithm can be efficiently implemented by the simple finite difference scheme. Experiments on synthetic and real images demonstrate the advantages of the proposed method over geodesic active contours (GAC) and Chan–Vese (C–V) active contours in terms of both efficiency and accuracy.     

Active Learning with Bootstrapped Dendritic Classifier applied to medical image segmentation

We perform the segmentation of medical images following an Active Learning approach that allows quick interactive segmentation minimizing the requirements for intervention of the human operator. The basic classifier is the Bootstrapped Dendritic Classifier (BDC), which combine the output of an ensemble of weak Dendritic Classifiers by majority voting. Weak Dendritic Classifiers are trained on bootstrapped samples of the train data setting a limit on the number of dendrites. We validate the approach on the segmentation of the thrombus in 3D Computed Tomography Angiography (CTA) data of Abdominal Aortic Aneurysm (AAA) patients simulating the human oracle by the provided ground truth. The generalization results in terms of accuracy and true positive ratio of the classification of the entire volume by the classifier trained on one slice confirm that the approach is worth its consideration for clinical practice.     

Combinatorial Optimization of the piecewise constant Mumford-Shah functional with application to scalar/vector valued and volumetric image segmentation

Front propagation models represent an important category of image segmentation techniques in the current literature. These models are normally formulated in a continuous level sets framework and optimized using gradient descent methods. Such formulations result in very slow algorithms that get easily stuck in local solutions and are highly sensitive to initialization.In this paper, we reformulate one of the most influential front propagation models, the Chan-Vese model, in the discrete domain. The graph representability and submodularity of the discrete energy function is established and then max-flow/min-cut approach is applied to perform the optimization of the discrete energy function. Our results show that this formulation is much more robust than the level sets formulation. Our approach is not sensitive to initialization and provides much faster solutions than level sets. The results also depict that our segmentation approach is robust to topology changes, noise and ill-defined edges, i.e., it preserves all the advantages associated with level sets methods.     

A hybrid semi-automatic method for liver segmentation based on level-set methods using multiple seed points

The present study developed a hybrid semi-automatic method to extract the liver from abdominal computerized tomography (CT) images. The proposed hybrid method consists of a customized fast-marching level-set method for detection of an optimal initial liver region from multiple seed points selected by the user and a threshold-based level-set method for extraction of the actual liver region based on the initial liver region. The performance of the hybrid method was compared with those of the 2D region growing method implemented in OsiriX using abdominal CT datasets of 15 patients. The hybrid method showed a significantly higher accuracy in liver extraction (similarity index, SI = 97.6 ± 0.5%; false positive error, FPE = 2.2 ± 0.7%; false negative error, FNE = 2.5 ± 0.8%; average symmetric surface distance, ASD = 1.4 ± 0.5 mm) than the 2D (SI = 94.0 ± 1.9%; FPE = 5.3 ± 1.1%; FNE = 6.5 ± 3.7%; ASD = 6.7 ± 3.8 mm) region growing method. The total liver extraction time per CT dataset of the hybrid method (77 ± 10 s) is significantly less than the 2D region growing method (575 ± 136 s). The interaction time per CT dataset between the user and a computer of the hybrid method (28 ± 4 s) is significantly shorter than the 2D region growing method (484 ± 126 s). The proposed hybrid method was found preferred for liver segmentation in preoperative virtual liver surgery planning.     

Segmentation using Appearance of Mesostructure Roughness

This paper introduces mesostructure roughness as an effective cue in image segmentation. Mesostructure roughness corresponds to small-scale bumps on the macrostructure (i.e., geometry) of objects. Specifically, the focus is on the texture that is created by the projection of the mesostructure roughness on the camera plane. Three intrinsic images are derived: reflectance, smooth-surface shading and mesostructure roughness shading (meta-texture images). A constructive approach is proposed for computing a meta-texture image by preserving, equalizing and enhancing the underlying surface-roughness across color, brightness and illumination variations. We evaluate the performance on sample images and illustrate quantitatively that different patches of the same material, in an image, are normalized in their statistics despite variations in color, brightness and illumination. We develop an algorithm for segmentation of an image into patches that share salient mesostructure roughness. Finally, segmentation by line-based boundary-detection is proposed and results are provided and compared to known algorithms.     

作用域扩展函数在主动轮廓线模型中的应用

为了改进主动轮廓线模型求解过程中轮廓线初始位置的敏感性,通常要采用一些方法扩大图像力的作用域,但是,这些方法往往需要根据具体场合来设计,没有比较好的普适性,而且在扩大作用域的过程中不能很好地控制作用域的大小。文章提出一种作用域扩展函数,既能方便地应用于各种需要扩展图像力作用域的场合,还能够控制图像力的作用范围。     

串并行分割法在分形图像压缩的应用

分形理论是20世纪70年代美国Benoit B.Mandelbrot提出的,在图像压缩领域中得到了迅速的发展与应用,分形编码压缩的两大难点是如何进行图像分割和构造迭代。介于现阶段的分形压缩算法复杂,编码时间长的缺点,本文通过细化图像分割以减轻迭代时计算量的思想,采用串行边界分割与并行区域分割相合的一种改进方法。     

Multiple piecewise constant with geodesic active contours (MPC-GAC) framework for interactive image segmentation using graph cut optimization

This paper proposes an improved variational model, multiple piecewise constant with geodesic active contour (MPC-GAC) model, which generalizes the region-based active contour model by Chan and Vese, 2001 [11] and merges the edge-based active contour by Caselles et al., 1997 [7] to inherit the advantages of region-based and edge-based image segmentation models. We show that the new MPC-GAC energy functional can be iteratively minimized by graph cut algorithms with high computational efficiency compared with the level set framework. This iterative algorithm alternates between the piecewise constant functional learning and the foreground and background updating so that the energy value gradually decreases to the minimum of the energy functional. The k-means method is used to compute the piecewise constant values of the foreground and background of image. We use a graph cut method to detect and update the foreground and background. Numerical experiments show that the proposed interactive segmentation method based on the MPC-GAC model by graph cut optimization can effectively segment images with inhomogeneous objects and background.     

基于AAM的人脸图像描述与编码

利用主动表观模型(AAM)来对人脸图像进行描述和编码,经过一定次数迭代,进行模型和人脸匹配,合成人脸图像.方法基于统计信息建模来实现对目标图像的描述.由于采用了优化算法,经过迭代运算使合成的模型与目标图像不断接近,最终能得到反应目标图像纹理及形状的合成模型.实验表明AAM方法进行人脸描述和编码的有效性.方法在人脸图像编码有重要的意义.     

Speech segmentation using regression fusion of boundary predictions

In the present work we study the appropriateness of a number of linear and non-linear regression methods, employed on the task of speech segmentation, for combining multiple phonetic boundary predictions which are obtained through various segmentation engines. The proposed fusion schemes are independent of the implementation of the individual segmentation engines as well as from their number. In order to illustrate the practical significance of the proposed approach, we employ 112 speech segmentation engines based on hidden Markov models (HMMs), which differ in the setup of the HMMs and in the speech parameterization techniques they employ. Specifically we relied on sixteen different HMMs setups and on seven speech parameterization techniques, four of which are recent and their performance on the speech segmentation task have not been evaluated yet. In the evaluation experiments we contrast the performance of the proposed fusion schemes for phonetic boundary predictions against some recently reported methods. Throughout this comparison, on the established for the phonetic segmentation task TIMIT database, we demonstrate that the support vector regression scheme is capable of achieving more accurate predictions, when compared to other fusion schemes reported so far.     

Spatially enabled customer segmentation using a data classification method with uncertain predicates

Spatial attributes are important factors for predicting customer behavior. However, thorough studies on this subject have never been carried out. This paper presents a new idea that incorporates spatial predicates describing the spatial relationships between customer locations and surrounding objects into customer attributes. More specifically, we developed two algorithms in order to achieve spatially enabled customer segmentation. First, a novel filtration algorithm is proposed that can select more relevant predicates from the huge amounts of spatial predicates than existing filtration algorithms. Second, since spatial predicates fundamentally involve some uncertainties, a rough set-based spatial data classification algorithm is developed to handle the uncertainties and therefore provide effective spatial data classification. A series of experiments were conducted and the results indicate that our proposed methods are superior to existing methods for data classification.