Circulant dissimilarity-based shape registration for object segmentation

ABSTRACT

A shape prior-based object segmentation is developed in this paper by using a shape transformation distance to constrain object contour evolution. In the proposed algorithm, the transformation distance measures the dissimilarity between two unaligned shapes by cyclic shift, which is called ‘circulant dissimilarity’. This dissimilarity with respect to transformation of the object shape is represented by circular convolution, which could be efficiently computed by using fast Fourier transform. Given a set of training shapes, the kernel density estimation is adopted to model shape prior. By integrating low-level image feature, high-level shape prior and transformation distance, a variational segmentation model is proposed to solve the transformation invariance of shape prior. Numerical experiments demonstrate that circulant dissimilarity-based shape registration outperforms the iterative optimization on explicit pose parameters, and show promising results and highlight the potential of the method for object registration and segmentation.     

Moment-based alignment for shape prior with variational B-spline level set

This paper presents a new shape prior-based implicit active contour model for image segmentation. The paper proposes an energy functional including a data term and a shape prior term. The data term, inspired from the region-based active contour approach, evolves the contour based on the region information of the image to segment. The shape prior term, defined as the distance between the evolving shape and a reference shape, constraints the evolution of the contour with respect to the reference shape. Especially, in this paper, we present shapes via geometric moments, and utilize the shape normalization procedure, which takes into account the affine transformation, to align the evolving shape with the reference one. By this way, we could directly calculate the shape transformation, instead of solving a set of coupled partial differential equations as in the gradient descent approach. In addition, we represent the level-set function in the proposed energy functional as a linear combination of continuous basic functions expressed on a B-spline basic. This allows a fast convergence to the segmentation solution. Experiment results on synthetic, real, and medical images show that the proposed model is able to extract object boundaries even in the presence of clutter and occlusion.     

On Symmetry, Perspectivity, and Level-Set-Based Segmentation

We introduce a novel variational method for the extraction of objects with either bilateral or rotational symmetry in the presence of perspective distortion. Information on the symmetry axis of the object and the distorting transformation is obtained as a by--product of the segmentation process. The key idea is the use of a flip or a rotation of the image to segment as if it were another view of the object. We call this generated image the symmetrical counterpart image. We show that the symmetrical counterpart image and the source image are related by planar projective homography. This homography is determined by the unknown planar projective transformation that distorts the object symmetry. The proposed segmentation method uses a level-set-based curve evolution technique. The extraction of the object boundaries is based on the symmetry constraint and the image data. The symmetrical counterpart of the evolving level-set function provides a dynamic shape prior. It supports the segmentation by resolving possible ambiguities due to noise, clutter, occlusions, and assimilation with the background. The homography that aligns the symmetrical counterpart to the source level-set is recovered via a registration process carried out concurrently with the segmentation. Promising segmentation results of various images of approximately symmetrical objects are shown.     

Affine and projective active contour models

We investigate the evolution of active contours in terms of progressive modification of an initial contour following the chosen Lie group of object-to-image transformations. Because of non-fronto-parallel viewing of an object or due to relative motion between the camera and the object, the resultant image may undergo affine or projective object-to-image transformations. In a recent paper we have shown that in the case of object tracking, frame-to-frame deformations of an initial curve obtained through Euler-Lagrange descent equations of a curve functional can be used to enact a desired Lie group of plane transformations [A.-R. Mansouri, D.P. Mukherjee, S.T. Acton, Constraining active contour evolution via Lie groups of transformation, IEEE Trans. Image Process. 13 (2004) 853-863]. In this work, we propose an energy functional that encodes the Lie group transformation parameters, which in turn guide shape distortion due to oblique viewing. Additional constraints, such as transformation smoothness, are imposed on the active contour by modifying the energy functional. The functional is minimized using numerical schemes similar to the conjugate gradient technique, and the convergence properties are discussed. The success of the technique for affine and projective scenes is demonstrated with both synthetic and real image examples and compared with the related approaches.     

A Projective Framework for Polyhedral Mesh Modelling

We present a novel framework for polyhedral mesh editing with face‐based projective maps that preserves planarity by definition. Such meshes are essential in the field of architectural design and rationalization. By using homogeneous coordinates to describe vertices, we can parametrize the entire shape space of planar‐preserving deformations with bilinear equations. The generality of this space allows for polyhedral geometric processing methods to be conducted with ease. We demonstrate its usefulness in planar‐quadrilateral mesh subdivision, a resulting multi‐resolution editing algorithm, and novel shape‐space exploration with prescribed transformations. Furthermore, we show that our shape space is a discretization of a continuous space of conjugate‐preserving projective transformation fields on surfaces. Our shape space directly addresses planar‐quad meshes, on which we put a focus, and we further show that our framework naturally extends to meshes with faces of more than four vertices as well.     

结合先验形状和Mumford-Shah模型的活动轮廓分割

基于先验形状和Mumford-Shah模型的活动轮廓分割是一种抗噪声干扰、稳定的图像分割方法。该模型采用水平集方法,并结合活动轮廓模型、先验形状和Mumford-Shah模型来控制曲线演化。特定目标的先验知识可以有效地指导目标准确分割,经过主成分分析（PCA）法可以得到感兴趣对象形状的主要信息。通过对不同图片分割实验表明,针对特定的形状,该方法对杂乱背景、部分遮挡、缺失和强噪声的图片依然能得到满意的结果。     

Shape-Based Mutual Segmentation

We present a novel variational approach for simultaneous segmentation of two images of the same object taken from different viewpoints. Due to noise, clutter and occlusions, neither of the images contains sufficient information for correct object-background partitioning. The evolving object contour in each image provides a dynamic prior for the segmentation of the other object view. We call this process mutual segmentation. The foundation of the proposed method is a unified level-set framework for region and edge based segmentation, associated with a shape similarity term. The suggested shape term incorporates the semantic knowledge gained in the segmentation process of the image pair, accounting for excess or deficient parts in the estimated object shape. Transformations, including planar projectivities, between the object views are accommodated by a registration process held concurrently with the segmentation. The proposed segmentation algorithm is demonstrated on a variety of image pairs. The homography between each of the image pairs is estimated and its accuracy is evaluated.     

A Coupled Minimization Problem for Medical Image Segmentation with Priors

We present a coupled minimization problem for image segmentation using prior shape and intensity profile. One part of the model minimizes a shape related energy and the energy of geometric active contour with a parameter that balances the influence from these two. The minimizer corresponding to a fixed parameter in this minimization gives a segmentation and an alignment between the segmentation and prior shape. The second part of this model optimizes the selection of the parameter by maximizing the mutual information of image geometry between the prior and the aligned novel image over all the alignments corresponding to different parameters in the first part. By this coupling the segmentation arrives at higher image gradient, forms a shape similar to the prior, and captures the prior intensity profile. We also propose using mutual information of image geometry to generate intensity model from a set of training images. Experimental results on cardiac ultrasound images are presented. These results indicate that the proposed model provides close agreement with expert traced borders, and the parameter determined in this model for one image can be used for images with similar properties.     

Watershed segmentation using prior shape and appearance knowledge

Watershed transformation is a common technique for image segmentation. However, its use for automatic medical image segmentation has been limited particularly due to oversegmentation and sensitivity to noise. Employing prior shape knowledge has demonstrated robust improvements to medical image segmentation algorithms. We propose a novel method for enhancing watershed segmentation by utilizing prior shape and appearance knowledge. Our method iteratively aligns a shape histogram with the result of an improved k-means clustering algorithm of the watershed segments. Quantitative validation of magnetic resonance imaging segmentation results supports the robust nature of our method.     

Recognition Using Region Correspondences

Recognition systems attempt to recover information about the identity of observed objects and their location in the environment. A fundamental problem in recognition is pose estimation. This is the problem of using a correspondence between some portions of an object model and some portions of an image to determine whether the image contains an instance of the object, and, in case it does, to determine the transformation that relates the model to the image. The current approaches to this problem are divided into methods that use global properties of the object (e.g., centroid and moments of inertia) and methods that use local properties of the object (e.g., corners and line segments). Global properties are sensitive to occlusion and, specifically, to self occlusion. Local properties are difficult to locate reliably, and their matching involves intensive computation.We present a novel method for recognition that uses region information. In our approach the model and the image are divided into regions. Given a match between subsets of regions (without any explicit correspondence between different pieces of the regions) the alignment transformation is computed. The method applies to planar objects under similarity, affine, and projective transformations and to projections of 3-D objects undergoing affine and projective transformations. The new approach combines many of the advantages of the previous two approaches, while avoiding some of their pitfalls. Like the global methods, our approach makes use of region information that reflects the true shape of the object. But like local methods, our approach can handle occlusion.     

OBJCUT: Efficient Segmentation Using Top-Down and Bottom-Up Cues

We present a probabilistic method for segmenting instances of a particular object category within an image. Our approach overcomes the deficiencies of previous segmentation techniques based on traditional grid conditional random fields (CRF), namely that 1) they require the user to provide seed pixels for the foreground and the background and 2) they provide a poor prior for specific shapes due to the small neighborhood size of grid CRF. Specifically, we automatically obtain the pose of the object in a given image instead of relying on manual interaction. Furthermore, we employ a probabilistic model which includes shape potentials for the object to incorporate top-down information that is global across the image, in addition to the grid clique potentials which provide the bottom-up information used in previous approaches. The shape potentials are provided by the pose of the object obtained using an object category model. We represent articulated object categories using a novel layered pictorial structures model. Nonarticulated object categories are modeled using a set of exemplars. These object category models have the advantage that they can handle large intraclass shape, appearance, and spatial variation. We develop an efficient method, OBJCUT, to obtain segmentations using our probabilistic framework. Novel aspects of this method include: 1) efficient algorithms for sampling the object category models of our choice and 2) the observation that a sampling-based approximation of the expected log-likelihood of the model can be increased by a single graph cut. Results are presented on several articulated (e.g., animals) and nonarticulated (e.g., fruits) object categories. We provide a favorable comparison of our method with the state of the art in object category specific image segmentation, specifically the methods of Leibe and Schiele and Schoenemann and Cremers.     

基于形状先验和图割的彩色图像分割

基于图割理论的图像分割方法在二值标号问题中可以获取全局最优解,而在多标号问题中可以获取带有很强特征的局部最优解。但对于含有噪声或遮挡物等复杂的图像,分割结果不完整,效果并不令人满意,提出了一种基于形状先验和图割的图像分割方法。以图割算法为基础,加入形状先验知识,使该算法包含更多约束信息,从而限制感兴趣区域的搜寻空间,能够更好地分割出完整的目标,增加了算法的精确度。针对形状的仿射变换,运用特征匹配算法进行处理,使算法更加具有灵活性,能够应对不同类型的情况。实验表明了该算法的有效性。     

基于图割的矩形目标交互式分割方法

为得到更加精确的图像分割结果,可以在基于图割的分割框架中引入形状先验指导分割。针对矩形目标提出了一种目标/背景交互式分割方法。分割能量用马尔可夫随机场最大后验概率描述,通过限制有向图中的流向引入形状先验,可以保证图割优化后的分割结果为矩形形状。对仿真图像与真实图像的实验结果证明了该算法的有效性。     

一种DCT变换的三维网格物体盲水印方法

提出了一种基于DCT变换的3D网格物体鲁棒性盲水印方案。首先将3D物体模型转换到仿射不变空间,抽取三维物体重心到顶点的距离生成一个一维的离散信号,将该离散信号进行DCT变换,改变其系数以嵌入水印。然后经过逆DCT变换生成带水印的3D物体模型。在仿射不变空间下,实现了3D物体模型对平移、旋转、比例变换的鲁棒性,采用DCT变换使3D模型具有很强的水印不可见性,而且具有一定的噪声鲁棒性。试验结果也表明该方法不仅对于旋转、平移、比例变换具有很强的鲁棒性,而且具有良好的水印不可见性。     

Continuous Generalized Procrustes analysis

Two-dimensional shape models have been successfully applied to solve many problems in computer vision, such as object tracking, recognition, and segmentation. Typically, 2D shape models are learned from a discrete set of image landmarks (corresponding to projection of 3D points of an object), after applying Generalized Procustes Analysis (GPA) to remove 2D rigid transformations. However, the standard GPA process suffers from three main limitations. Firstly, the 2D training samples do not necessarily cover a uniform sampling of all the 3D transformations of an object. This can bias the estimate of the shape model. Secondly, it can be computationally expensive to learn the shape model by sampling 3D transformations. Thirdly, standard GPA methods use only one reference shape, which can might be insufficient to capture large structural variability of some objects.To address these drawbacks, this paper proposes continuous generalized Procrustes analysis (CGPA). CGPA uses a continuous formulation that avoids the need to generate 2D projections from all the rigid 3D transformations. It builds an efficient (in space and time) non-biased 2D shape model from a set of 3D model of objects. A major challenge in CGPA is the need to integrate over the space of 3D rotations, especially when the rotations are parameterized with Euler angles. To address this problem, we introduce the use of the Haar measure. Finally, we extended CGPA to incorporate several reference shapes. Experimental results on synthetic and real experiments show the benefits of CGPA over GPA.     

A New Method of Image Mosaicking and Its Application to Cultural Heritage Representation

This paper presents an original two-step procedure for estimating projective transformations between pairs of images: first, the transformation between the images is approximated as an affine transformation; second, this estimate is refined into that of a projective transformation. This strategy for matching projective views is computationally very efficient. The proposed method can be applied both to mosaicking of high resolution images of planar textured objects (e.g., frescoes and paintings), with subpixel accuracy, and to construction of panoramic images. Practical examples of mosaicking of cultural heritage imagery obtained by using the presented procedure are discussed in the paper.     

Procrustes Analysis with Deformations: A Closed-Form Solution by Eigenvalue Decomposition

Generalized Procrustes Analysis (GPA) is the problem of bringing multiple shapes into a common reference by estimating transformations. GPA has been extensively studied for the Euclidean and affine transformations. We introduce GPA with deformable transformations, which forms a much wider and difficult problem. We specifically study a class of transformations called the Linear Basis Warps, which contains the affine transformation and most of the usual deformation models, such as the Thin-Plate Spline (TPS). GPA with deformations is a nonconvex underconstrained problem. We resolve the fundamental ambiguities of deformable GPA using two shape constraints requiring the eigenvalues of the shape covariance. These eigenvalues can be computed independently as a prior or posterior. We give a closed-form and optimal solution to deformable GPA based on an eigenvalue decomposition. This solution handles regularization, favoring smooth deformation fields. It requires the transformation model to satisfy a fundamental property of free-translations, which asserts that the model can implement any translation. We show that this property fortunately holds true for most common transformation models, including the affine and TPS models. For the other models, we give another closed-form solution to GPA, which agrees exactly with the first solution for models with free-translation. We give pseudo-code for computing our solution, leading to the proposed DefGPA method, which is fast, globally optimal and widely applicable. We validate our method and compare it to previous work on six diverse 2D and 3D datasets, with special care taken to choose the hyperparameters from cross-validation.

     

Notes on the Mathematics of the PHIGS Viewing Pipeline

It is well known that homogeneous linear matrix transformations of the projective space may be efficiently used for the representation and the execution of common geometrical transformations but the general class of such transformations include also matrices which may cause numerical problems by transforming certain finite geometrical objects to infinity Different methods have been developed for handling such cases and this paper presents a new one called'UW clipping'which is based on some interesting properties of projective transformations. Furthermore the paper introduces die concept of 'conic sectors' as a generalisation to half-planes and halfspaces respectively, with the invariance property that such sectors are mapped onto other such sectors by projective transformations, and thus enable the transformation of clipping halfplanes and halfspaces. Finally the possibility of transforming die rectangular clipping box into the object space is investigated.     

Robust Region Detection via Consensus Segmentation of Deformable Shapes

We consider the problem of stable region detection and segmentation of deformable shapes. We pursue this goal by determining a consensus segmentation from a heterogeneous ensemble of putative segmentations, which are generated by a clustering process on an intrinsic embedding of the shape. The intuition is that the consensus segmentation, which relies on aggregate statistics gathered from the segmentations in the ensemble, can reveal components in the shape that are more stable to deformations than the single baseline segmentations. Compared to the existing approaches, our solution exhibits higher robustness and repeatability throughout a wide spectrum of non‐rigid transformations. It is computationally efficient, naturally extendible to point clouds, and remains semantically stable even across different object classes. A quantitative evaluation on standard datasets confirms the potentiality of our method as a valid tool for deformable shape analysis.