The role of model-based segmentation in the recovery of volumetricparts from range data

We present a method for segmenting and estimating the shape of 3D objects from range data. The technique uses model views, or aspects, to constrain the fitting of deformable models to range data. Based on an initial region segmentation of a range image, regions are grouped into aspects corresponding to the volumetric parts that make up an object. The qualitative segmentation of the range image into a set of volumetric parts not only captures the coarse shape of the parts, but qualitatively encodes the orientation of each part through its aspect. Knowledge of a part's coarse shape, its orientation, as well as the mapping between the faces in its aspect and the surfaces on the part provides strong constraints on the fitting of a deformable model (supporting both global and local deformations) to the data. Unlike previous work in physics-based deformable model recovery from range data, the technique does not require presegmented data. Furthermore, occlusion is handled at segmentation time and does not complicate the fitting process, as only 3D points known to belong to a part participate in the fitting of a model to the part. We present the approach in detail and apply it to the recovery of objects from range data     

Motion estimation of elastic articulated objects from points and contours with volume invariable constraint

This paper presents a model of elastic articulated objects based on revolving conic surface and a method of model-based motion estimation. The model includes 3D object skeleton and deformable surfaces that can represent the deformation of human body surfaces. In each limb, surface deformation is represented by adjusting one or two deformation parameters. Then, the 3D deformation parameters are determined by corresponding 2D image points and contours with volume invariable constraint from a sequence of stereo images. The 3D motion parameters are estimated based on the 3D model. The algorithm presented in this paper includes model-based parameter estimation of motion and parameter determination of deformable surfaces.     

Globally minimal surfaces by continuous maximal flows

In this paper, we address the computation of globally minimal curves and surfaces for image segmentation and stereo reconstruction. We present a solution, simulating a continuous maximal flow by a novel system of partial differential equations. Existing methods are either grid-biased (graph-based methods) or suboptimal (active contours and surfaces). The solution simulates the flow of an ideal fluid with isotropic velocity constraints. Velocity constraints are defined by a metric derived from image data. An auxiliary potential function is introduced to create a system of partial differential equations. It is proven that the algorithm produces a globally maximal continuous flow at convergence, and that the globally minimal surface may be obtained trivially from the auxiliary potential. The bias of minimal surface methods toward small objects is also addressed. An efficient implementation is given for the flow simulation. The globally minimal surface algorithm is applied to segmentation in 2D and 3D as well as to stereo matching. Results in 2D agree with an existing minimal contour algorithm for planar images. Results in 3D segmentation and stereo matching demonstrate that the new algorithm is robust and free from grid bias.     

Simplex Mesh Diffusion Snakes: Integrating 2D and 3D Deformable Models and Statistical Shape Knowledge in a Variational Framework

In volumetric medical imaging the boundaries of structures are frequently blurred due to insufficient resolution. This artefact is particularly serious in structures such as articular joints, where different cartilage surfaces appear to be linked at the contact regions. Traditional image segmentation techniques fail to separate such erroneously linked structures, and a sensible approach has been the introduction of prior-knowledge to the segmentation process. Although several 3D prior-knowledge based techniques that could successfully segment these structures have been published, most of them are pixel-labelling schemes that generate pixellated images with serious geometric distortions. The Simplex Mesh Diffusion Snakes segmentation technique presented here is an extension of the two dimensional Diffusion Snakes, but without any restriction on the number of dimensions of the data set. This technique integrates a Simplex Mesh, a region-based deformable model and Statistical Shape Knowledge into a single energy functional, so that it takes into account both the image information available directly from the data set, and the shape statistics obtained from a training process. The resulting segmentations converge correctly to well defined boundaries and provide a feasible location for those removed boundaries. The algorithm has been evaluated using 2D and 3D data sets obtained with Magnetic Resonance Imaging (MRI) and has proved to be robust to most of the MRI artefacts, providing continuous and smooth curves or surfaces with sub-pixel resolution. Additionally, this novel technique opens a wide range of opportunities for segmentation and tracking time-dependent 3D structures or data sets with more than three dimensions, due to its non-restrictive mathematical formulation.     

Extracting a valid boundary representation from a segmented rangeimage

A new approach is presented for extracting an explicit 3D shape model from a single range image. One novel aspect is that the model represents both observed object surfaces, and surfaces which bound the volume of occluded space. Another novel aspect is that the approach does not require that the range image segmentation be perfect. The low-level segmentation may be such that the model-building process encounters topology versus geometry conflicts. The model-building process is designed to be “fail soft” in the face of such problems. The portion of the 3D model where a problem presents itself is “glued” together in a manner meant to minimize the disturbance in the 3D shape. The goal is to produce a valid boundary-representation which can be processed by higher-level routines. A third novel aspect of this work is that the implementation has been evaluated on over 200 real range images of polyhedral objects, with no operator intervention and all parameters held constant, and obtained a 97% success rate in creating valid b-reps     

Volumetric Shapes of Solids of Revolution from a Single-View Range Image

Many man-made objects such as industrial parts are partially constructed of surfaces of revolution, as well as planar surfaces. We have studied the problem of finding and recovering solids of revolution in range data which are potentially useful for modeling and recognizing 3D objects. We propose an approach to the problem which is based on the fact that at least one of two focal surfaces for a surface of revolution degenerates into the axis of rotation. First, by computing the surface normal and principal curvatures, the centers of principal curvature which construct the focal surfaces are obtained for each point in the range image. Then, using the Hough transform, the axes of rotation are detected by finding the centers of principal curvature which lie on straight lines in space. Finally, the solid of revolution is completely determined by estimating the radius function of cross-section along each rotational axis. The proposed method can be used even in situations where occlusion or truncation is a problem because it does not require the visibility of entire surfaces. Experiments have been successfully carried out with real range data obtained from laser rangefinders.     

人体心脏的4维图像建模和参数分析

研究一种基于4维医学图像的活动形状模型方法,用于构建人类心脏的3维柔性模型并进行自动化参数分析,以得到心脏各方位的量化功能指标。首先,通过造影技术获取心脏周期内各时刻的立体图像,根据时间序列形成一组4维图像。在大量医学图像的基础上,用统计方法分析得到心脏的一般形状,局部的变化范围和分布概率密度,为人类心脏建立一个数字化的计算模型。在此基础上,针对具体病人进行图像分割和形状拟合以计算其心脏静态和动态形状参数,然后分析得到与心脏功能相关的一些重要参数。研究内容包括心脏模型的建立,心脏静态参数分析,心脏动态功能分析,疾病分析等,对病人心脏在特定时刻的每组图像生成其3维结构模型,为医生提供丰富有力的诊断和治疗依据。     

Range image segmentation based on differential geometry: a hybridapproach

The authors describe a hybrid approach to the problem of image segmentation in range data analysis, where hybrid refers to a combination of both region- and edge-based considerations. The range image of 3-D objects is divided into surface primitives which are homogeneous in their intrinsic differential geometric properties and do not contain discontinuities in either depth of surface orientation. The method is based on the computation of partial derivatives, obtained by a selective local biquadratic surface fit. Then, by computing the Gaussian and mean curvatures, an initial region-gased segmentation is obtained in the form of a curvature sign map. Two additional initial edge-based segmentations are also computed from the partial derivatives and depth values, namely, jump and roof-edge maps. The three image maps are then combined to produce the final segmentation. Experimental results obtained for both synthetic and real range data of polyhedral and curved objects are given     

基于CV／CAD的三维物体几何建模

在虚拟现实等技术领域中,都涉及到由现实世界中的实际景物建立对应的计算机描述的虚拟景物的问题,为此提出了利用计算机视觉与CAD几何建模技术相结合的三维珠体建模途径,首先通过编码光栅方法获取三维物体的深度图象,并采用数学形态学的方法加以分割,然后利用代数曲面拟合手段对分割后的三维曲面片进行重建,并使用CAD几何建模工具由重建的曲面片构成物体的几何模型,该文给出了初步的实验结果,证明所提出的技术途径基本可行。     

Narrow band region-based active contours and surfaces for 2D and 3D segmentation

We describe a narrow band region approach for deformable curves and surfaces in the perspective of 2D and 3D image segmentation. Basically, we develop a region energy involving a fixed-width band around the curve or surface. Classical region-based methods, like the Chan–Vese model, often make strong assumptions on the intensity distributions of the searched object and background. In order to be less restrictive, our energy achieves a trade-off between local features of gradient-like terms and global region features. Relying on the theory of parallel curves and surfaces, we perform a mathematical derivation to express the region energy in a curvature-based form allowing efficient computation on explicit models. We introduce two different region terms, each one being dedicated to a particular configuration of the target object. Evolution of deformable models is performed by means of energy minimization using gradient descent. We provide both explicit and implicit implementations. The explicit models are a parametric snake in 2D and a triangular mesh in 3D, whereas the implicit models are based on the level set framework, regardless of the dimension. Experiments are carried out on MRI and CT medical images, in 2D and 3D, as well as 2D color photographs.     

Fast deformable matching of 3D images over multiscale nested subspaces. Application to atlas-based MRI segmentation

This paper presents a fast method to perform dense deformable matching of 3D images, applied to the registration of inter-subject brain MR images. To recover the complex morphological variations in neuroanatomy, the registration method uses a hierarchy of 3D deformations fields that are estimated, by minimizing a global energy function over a sequence of nested subspaces. The resulting deformable matching method shows low sensitivity to local minima and is able to track large non-linear deformations, with moderate computational load. The performances of the approach are assessed both on simulated 3D transformations and on a real data base of 3D brain MR images from different individuals. An application of the deformable image matching method to 3D atlas-based image segmentation is presented. This atlas-based segmentation is used at Strasbourg Hospital, in daily clinical applications, in order to extract regions of interest from 3D MR images of patients suffering from epilepsy.     

Anisotropic diffusion in vector field visualization on Euclideandomains and surfaces

Vector field visualization is an important topic in scientific visualization. Its aim is to graphically represent field data on two and three-dimensional domains and on surfaces in an intuitively understandable way. Here, a new approach based on anisotropic nonlinear diffusion is introduced. It enables an easy perception of vector field data and serves as an appropriate scale space method for the visualization of complicated flow pattern. The approach is closely related to nonlinear diffusion methods in image analysis where images are smoothed while still retaining and enhancing edges. Here, an initial noisy image intensity is smoothed along integral lines, whereas the image is sharpened in the orthogonal direction. The method is based on a continuous model and requires the solution of a parabolic PDE problem. It is discretized only in the final implementational step. Therefore, many important qualitative aspects can already be discussed on a continuous level. Applications are shown for flow fields in 2D and 3D, as well as for principal directions of curvature on general triangulated surfaces. Furthermore, the provisions for flow segmentation are outlined     

Shape differentiation of freeform surfaces using a similarity measure based on an integral of Gaussian curvature

Freeform surfaces are popularly used to design and model complex 3D objects. These 3D models are stored as computerized models in databases. To facilitate data retrieval and shape matching, a major challenge lies in defining and computing the level of similarity between two or more freeform surfaces. In order to explore the useful 3D information associated with the surfaces, an integrated approach based on the integral of Gaussian curvature is proposed to develop the measures of similarity of freeform surfaces. Specifically, the integral of Gaussian curvature is mapped into the 2D space, and a shape-based measure is developed using statistical methods to compute the level of similarity. For smooth surfaces, a fast approximation algorithm is developed to calculate the curvature of individual subregions. In cases where the target surface has a complex topological structure or a smooth surface is not available, the integral of Gaussian curvature for the discrete surface is first calculated at each vertex, followed by mapping onto a 2D spherical coordinate. The distance measure focuses on the local geometry, which is critical to investigate models with a certain level of resemblance such as products in a family. This proposed approach can be applied to surfaces under various transformations, as well as 3D data from various sources.     

Topological active volumes: A topology-adaptive deformable model for volume segmentation

This paper proposes a generic methodology for segmentation and reconstruction of volumetric datasets based on a deformable model, the topological active volumes (TAV). This model, based on a polyhedral mesh, integrates features of region-based and boundary-based segmentation methods in order to fit the contours of the objects and model its inner topology. Moreover, it implements automatic procedures, the so-called topological changes, that alter the mesh structure and allow the segmentation of complex features such as pronounced curvatures or holes, as well as the detection of several objects in the scene. This work presents the TAV model and the segmentation methodology and explains how the changes in the TAV structure can improve the adjustment process. In particular, it is focused on the increase of the mesh density in complex image areas in order to improve the adjustment to object surfaces. The suitability of the mesh structure and the segmentation methodology is analyzed and the accuracy of the proposed model is proved with both synthetic and real images.     

Segmentation of kidney from ultrasound B-mode images with texture-based classification

The segmentation of anatomical structures from sonograms can help physicians evaluate organ morphology and realize quantitative measurement. It is an important but difficult issue in medical image analysis. In this paper, we propose a new method based on Laws' microtexture energies and maximum a posteriori (MAP) estimation to construct a probabilistic deformable model for kidney segmentation. First, using texture image features and MAP estimation, we classify each image pixel as inside or outside the boundary. Then, we design a deformable model to locate the actual boundary and maintain the smooth nature of the organ. Using gradient information subject to a smoothness constraint, the optimal contour is obtained by the dynamic programming technique. Experiments on different datasets are described. We find this method to be an effective approach.     

An efficient and robust algorithm for 3D mesh segmentation

This paper presents an efficient and robust algorithm for 3D mesh segmentation. Segmentation is one of the main areas of 3D object modeling. Most segmentation methods decompose 3D objects into parts based on curvature analysis. Most of the existing curvature estimation algorithms are computationally costly. The proposed algorithm extracts features using Gaussian curvature and concaveness estimation to partition a 3D model into meaningful parts. More importantly, this algorithm can process highly detailed objects using an eXtended Multi-Ring (XMR) neighborhood based feature extraction. After feature extraction, we also developed a fast marching watershed-based segmentation algorithm followed by an efficient region merging scheme. Experimental results show that this segmentation algorithm is efficient and robust.     

Software for interactive segmentation of the carotid artery from 3D black blood magnetic resonance images

A semiautomatic algorithm for segmenting organ surfaces from 3D medical images is presented in this work. The algorithm is based on a deformable model, and allows the user to initialize the model by combining and molding primitive shapes such as cylinders and spheres to form an initial approximate model of the organ surface. The initial model is automatically deformed to better fit organ boundaries. The algorithm was applied to segment the carotid bifurcation from 3D black blood magnetic resonance (MR) images of 5 subjects. The algorithm-segmented surfaces were compared to surfaces segmented manually by an experienced user. On average, approximately 3 min were required to segment an image using the algorithm, whereas 1h was required for manual segmentation. The average distance between corresponding points on the manually and algorithm-segmented surfaces was 0.37 mm, whereas the average maximum distance was 2.03 mm. Moreover, algorithm-segmented surfaces exhibited less intra-operator variability than those segmented manually.     

The Alignment of Objects with Smooth Surfaces

This paper examines the recognition of rigid objects bounded by smooth surfaces, using an alignment approach. The projected image of such an object changes during rotation in a manner that is generally difficult to predict. An approach to this problem is suggested, using the 3D surface curvature at the points along the silhouette. The curvature information requires a single number for each point along the object′s silhouette, the radial curvature at the point. We have implemented this method and tested it on images of complex 3D objects. Models of the viewed objects were acquired using three images of each object. The implemented scheme was found to give accurate predictions of the objects′ appearances for large transformations. Using this method, a small number of (viewer-centered) models can be used to predict the new appearance of an object from any given viewpoint.     

New paradigms for interactive 3D volume segmentation

We present a new virtual reality‐based interaction metaphor for semi‐automatic segmentation of medical 3D volume data. The mouse‐based, manual initialization of deformable surfaces in 3D represents a major bottleneck in interactive segmentation. In our multi‐modal system we enhance this process with additional sensory feedback. A 3D haptic device is used to extract the centreline of a tubular structure. Based on the obtained path a cylinder with varying diameter is generated, which in turn is used as the initial guess for a deformable surface. Copyright © 2002 John Wiley & Sons, Ltd.     

Inflatable Models

A physically-based model is presented for the simulation of a new type of deformable objects-inflatable objects, such as shaped balloons, which consist of pressurized air enclosed by an elastic surface. These objects have properties inherent in both 3D and 2D elastic bodies, as they demonstrate the behaviour of 3D shapes using 2D formulations. As there is no internal structure in them, their behaviour is substantially different from the behaviour of deformable solid objects. We use one of the few available models for deformable surfaces, and enhance it to include the forces of internal and external pressure. These pressure forces may also incorporate buoyancy forces, to allow objects filled with a low density gas to float in denser media. The obtained models demonstrate rich dynamic behaviour, such as bouncing, floating, deflation and inflation.