Prior shape level set segmentation on multistep generated probability maps of MR datasets for fully automatic kidney parenchyma volumetry

Fully automatic 3-D segmentation techniques for clinical applications or epidemiological studies have proven to be a very challenging task in the domain of medical image analysis. 3-D organ segmentation on magnetic resonance (MR) datasets requires a well-designed segmentation strategy due to imaging artifacts, partial volume effects, and similar tissue properties of adjacent tissues. We developed a 3-D segmentation framework for fully automatic kidney parenchyma volumetry that uses Bayesian concepts for probability map generation. The probability map quality is improved in a multistep refinement approach. An extended prior shape level set segmentation method is then applied on the refined probability maps. The segmentation quality is improved by incorporating an exterior cortex edge alignment technique using cortex probability maps. In contrast to previous approaches, we combine several relevant kidney parenchyma features in a sequence of segmentation techniques for successful parenchyma delineation on native MR datasets. Furthermore, the proposed method is able to recognize and exclude parenchymal cysts from the parenchymal volume. We analyzed four different quality measures showing better results for right parenchymal tissue than for left parenchymal tissue due to an incorporated liver part removal in the segmentation framework. The results show that the outer cortex edge alignment approach successfully improves the quality measures.     

Hierarchical scale-based multiobject recognition of 3-D anatomical structures

Segmentation of anatomical structures from medical images is a challenging problem, which depends on the accurate recognition (localization) of anatomical structures prior to delineation. This study generalizes anatomy segmentation problem via attacking two major challenges: 1) automatically locating anatomical structures without doing search or optimization, and 2) automatically delineating the anatomical structures based on the located model assembly. For 1), we propose intensity weighted ball-scale object extraction concept to build a hierarchical transfer function from image space to object (shape) space such that anatomical structures in 3-D medical images can be recognized without the need to perform search or optimization. For 2), we integrate the graph-cut (GC) segmentation algorithm with prior shape model. This integrated segmentation framework is evaluated on clinical 3-D images consisting of a set of 20 abdominal CT scans. In addition, we use a set of 11 foot MR images to test the generalizability of our method to the different imaging modalities as well as robustness and accuracy of the proposed methodology. Since MR image intensities do not possess a tissue specific numeric meaning, we also explore the effects of intensity nonstandardness on anatomical object recognition. Experimental results indicate that: 1) effective recognition can make the delineation more accurate; 2) incorporating a large number of anatomical structures via a model assembly in the shape model improves the recognition and delineation accuracy dramatically; 3) ball-scale yields useful information about the relationship between the objects and the image; 4) intensity variation among scenes in an ensemble degrades object recognition performance.     

3-D active appearance models: segmentation of cardiac MR and ultrasound images

A model-based method for three-dimensional image segmentation was developed and its performance assessed in segmentation of volumetric cardiac magnetic resonance (MR) images and echocardiographic temporal image sequences. Comprehensive design of a three-dimensional (3-D) active appearance model (AAM) is reported for the first time as an involved extension of the AAM framework introduced by Cootes et al. The model's behavior is learned from manually traced segmentation examples during an automated training stage. Information about shape and image appearance of the cardiac structures is contained in a single model. This ensures a spatially and/or temporally consistent segmentation of three-dimensional cardiac images. The clinical potential of the 3-D AAM is demonstrated in short-axis cardiac MR images and four-chamber echocardiographic sequences. The method's performance was assessed by comparison with manually identified independent standards in 56 clinical MR and 64 clinical echo image sequences. The AAM method showed good agreement with the independent standard using quantitative indexes of border positioning errors, endo- and epicardial volumes, and left ventricular mass. In MR, the endocardial volumes, epicardial volumes, and left ventricular wall mass correlation coefficients between manual and AAM were R2 = 0.94, 0.97, 0.82, respectively. For echocardiographic analysis, the area correlation was R2 = 0.79. The AAM method shows high promise for successful application to MR and echocardiographic image analysis in a clinical setting.     

Segmentation of whole cells and cell nuclei from 3-D optical microscope images using dynamic programming

Communications between cells in large part drive tissue development and function, as well as disease-related processes such as tumorigenesis. Understanding the mechanistic bases of these processes necessitates quantifying specific molecules in adjacent cells or cell nuclei of intact tissue. However, a major restriction on such analyses is the lack of an efficient method that correctly segments each object (cell or nucleus) from 3-D images of an intact tissue specimen. We report a highly reliable and accurate semi-automatic algorithmic method for segmenting fluorescence-labeled cells or nuclei from 3-D tissue images. Segmentation begins with semi-automatic, 2-D object delineation in a user-selected plane, using dynamic programming (DP) to locate the border with an accumulated intensity per unit length greater that any other possible border around the same object. Then the two surfaces of the object in planes above and below the selected plane are found using an algorithm that combines DP and combinatorial searching. Following segmentation, any perceived errors can be interactively corrected. Segmentation accuracy is not significantly affected by intermittent labeling of object surfaces, diffuse surfaces, or spurious signals away from surfaces. The unique strength of the segmentation method was demonstrated on a variety of biological tissue samples where all cells, including irregularly shaped cells, were accurately segmented based on visual inspection.     

多层时空切片联合的快速人体跟踪算法

针对传统人体跟踪方法中目标模型复杂、计算量大等问题,该文提出一种无目标模型的多层时空切片联合的人体跟踪算法。用多层时空切片中的多个动态区域表示人体,区域的选择无需使用任何预定义的目标区域模型。使用时空切片方法在图像序列空间中提取多层水平时空切片图像,在每层时空切片图像中,检测和跟踪潜在的运动区域,并根据区域运动一致性和空间一致性关系,将多个区域关联成不同的人体目标,实现多个人体目标的跟踪,从而将XYT 3维空间中的人体跟踪问题转化为多个XT 2维空间的区域联合跟踪问题。实验表明,该算法降低了跟踪的轨迹误差,满足实时性跟踪要求,同时通过多区域的联合增强了跟踪算法的抗干扰能力,即在人体部分区域丢失的情况下仍能有效跟踪。     

Automatic body segmentation with graph cut and self-adaptive initialization level set (SAILS)

In this paper, we propose an automatic human body segmentation system which mainly consists of human body detection and object segmentation. Firstly, an automatic human body detector is designed to provide hard constraints on the object and background for segmentation. And a coarse-to-fine segmentation strategy is employed to deal with the situation of partly detected object. Secondly, background contrast removal (BCR) and self-adaptive initialization level set (SAILS) are proposed to solve the tough segmentation problems of the high contrast at object boundary and/or similar colors existing in the object and background. Finally, an object updating scheme is proposed to detect and segment new object when it appears in the scene. Experimental results demonstrate that our body segmentation system works very well in the live video and standard sequences with complex background.     

网格运动分析与形态学滤波相结合的视频对象分割

提出了一种基于二维网格运动分析与改进形态学滤波空域自动分割策略相结合的视频对象时空分割算法。该算法首先利用高阶统计方法对视频图像的二维网格表示进行运动分析，快速得到前景对象区域，通过后处理有效获得前景对象运动检测掩膜。然后，用一种结合交变序列重建滤波算法和自适应阈值判别算法的改进分水岭分割策略有效获得前景对象的精确边缘。最后，用区域基时空融合算法将时域分割结果和空域分割结果结合起来提取出边缘精细的视频对象。实验结果表明，本算法综合了多种算法的优点，主客观分割效果理想。     

Background removal of multiview images by learning shape priors.

Image-based rendering has been successfully used to display 3-D objects for many applications. A well-known example is the object movie, which is an image-based 3-D object composed of a collection of 2-D images taken from many different viewpoints of a 3-D object. In order to integrate image-based 3-D objects into a chosen scene (e.g., a panorama), one has to meet a hard challenge--to efficiently and effectively remove the background from the foreground object. This problem is referred to as multiview images (MVIs) segmentation. Another task requires MVI segmentation is image-based 3-D reconstruction using multiview images. In this paper, we propose a new method for segmenting MVI, which integrates some useful algorithms, including the well-known graph-cut image segmentation and volumetric graph-cut. The main idea is to incorporate the shape prior into the image segmentation process. The shape prior introduced into every image of the MVI is extracted from the 3-D model reconstructed by using the volumetric graph cuts algorithm. Here, the constraint obtained from the discrete medial axis is adopted to improve the reconstruction algorithm. The proposed MVI segmentation process requires only a small amount of user intervention, which is to select a subset of acceptable segmentations of the MVI after the initial segmentation process. According to our experiments, the proposed method can provide not only good MVI segmentation, but also provide acceptable 3-D reconstructed models for certain less-demanding applications.     

Automatic active model initialization via Poisson inverse gradient

Active models have been widely used in image processing applications. A crucial stage that affects the ultimate active model performance is initialization. This paper proposes a novel automatic initialization approach for parametric active models in both 2-D and 3-D. The PIG initialization method exploits a novel technique that essentially estimates the external energy field from the external force field and determines the most likely initial segmentation. Examples and comparisons with two state-of-the- art automatic initialization methods are presented to illustrate the advantages of this innovation, including the ability to choose the number of active models deployed, rapid convergence, accommodation of broken edges, superior noise robustness, and segmentation accuracy.     

Volume delineation by fusion of fuzzy sets obtained frommultiplanar tomographic images

Techniques of three-dimensional (3-D) volume delineation from tomographic medical imaging are usually based on 2-D contour definition. For a given structure, several different contours can be obtained depending on the segmentation method used or the user's choice. The goal of this work is to develop a new method that reduces the inaccuracies generally observed. A minimum volume that is certain to be included in the volume concerned (membership degree mu = 1), and a maximum volume outside which no part of the volume is expected to be found (membership degree mu = 0), are defined semi-automatically. The intermediate fuzziness region (0 < mu < 1) is processed using the theory of possibility. The resulting fuzzy volume is obtained after data fusion from multiplanar slices. The influence of the contrast-to-noise ratio was tested on simulated images. The influence of slice thickness as well as the accuracy of the method were studied on phantoms. The absolute volume error was less than 2% for phantom volumes of 2-8 cm3, whereas the values obtained with conventional methods were much larger than the actual volumes. Clinical experiments were conducted, and the fuzzy logic method gave a volume lower than that obtained with the conventional method. Our fuzzy logic method allows volumes to be determined with better accuracy and reproducibility.     

A 3-D liver segmentation method with parallel computing for selective internal radiation therapy

This study describes a new 3-D liver segmentation method in support of the selective internal radiation treatment as a treatment for liver tumors. This 3-D segmentation is based on coupling a modified k-means segmentation method with a special localized contouring algorithm. In the segmentation process, five separate regions are identified on the computerized tomography image frames. The merit of the proposed method lays in its potential to provide fast and accurate liver segmentation and 3-D rendering as well as in delineating tumor region(s), all with minimal user interaction. Leveraging of multicore platforms is shown to speed up the processing of medical images considerably, making this method more suitable in clinical settings. Experiments were performed to assess the effect of parallelization using up to 442 slices. Empirical results, using a single workstation, show a reduction in processing time from 4.5 h to almost 1 h for a 78% gain. Most important is the accuracy achieved in estimating the volumes of the liver and tumor region(s), yielding an average error of less than 2% in volume estimation over volumes generated on the basis of the current manually guided segmentation processes. Results were assessed using the analysis of variance statistical analysis.     

Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features

We propose an automatic four-chamber heart segmentation system for the quantitative functional analysis of the heart from cardiac computed tomography (CT) volumes. Two topics are discussed: heart modeling and automatic model fitting to an unseen volume. Heart modeling is a nontrivial task since the heart is a complex nonrigid organ. The model must be anatomically accurate, allow manual editing, and provide sufficient information to guide automatic detection and segmentation. Unlike previous work, we explicitly represent important landmarks (such as the valves and the ventricular septum cusps) among the control points of the model. The control points can be detected reliably to guide the automatic model fitting process. Using this model, we develop an efficient and robust approach for automatic heart chamber segmentation in 3-D CT volumes. We formulate the segmentation as a two-step learning problem: anatomical structure localization and boundary delineation. In both steps, we exploit the recent advances in learning discriminative models. A novel algorithm, marginal space learning (MSL), is introduced to solve the 9-D similarity transformation search problem for localizing the heart chambers. After determining the pose of the heart chambers, we estimate the 3-D shape through learning-based boundary delineation. The proposed method has been extensively tested on the largest dataset (with 323 volumes from 137 patients) ever reported in the literature. To the best of our knowledge, our system is the fastest with a speed of 4.0 s per volume (on a dual-core 3.2-GHz processor) for the automatic segmentation of all four chambers.      

Temporal stabilization of Video Object Segmentation for 3D-TV applications

Our aim is to insert depth information into an existing 2D video sequence to provide content for 3D-TV applications, which we try to achieve through segmentation of the objects in the given 2D video sequence. To this effect, we present a method for temporal stabilization of video object segmentation algorithms for 3D-TV applications. First, two quantitative measures to evaluate temporal stability without ground-truth are discussed. Then, a pseudo-3D curve evolution method, which spatio-temporally stabilizes the estimated segmentation of a video object is introduced. Temporal stability is achieved by re-distributing existing object segmentation errors such that they will be less disturbing when the scene is rendered and viewed in 3D. Our starting point is the hypothesis that if making segmentation errors is inevitable, these errors should be made in a temporally consistent way for 3D-TV applications. This hypothesis is supported by the experiments, which show that there is significant improvement in segmentation quality both in terms of the objective quantitative measures and in terms of the viewing comfort in subjective perceptual tests. Therefore, it is possible to increase the perceptual object segmentation quality without increasing the actual segmentation accuracy.     

Volumetric object reconstruction using the 3D-MRF model-basedsegmentation [magnetic resonance imaging]

A number of segmentation algorithms have been developed, but those algorithms are not effective on volume reconstruction because they are limited to operating only on two-dimensional (2-D) images. Here, the authors propose the volumetric object reconstruction method using the three-dimensional Markov random field (3D-MRF) model-based segmentation. The 3D-MRF model is known to be one of the most efficient ways to model spatial contextual information. The method is compared with the 2-D region growing scheme under three types of interpolation. The results show that the proposed method is better in terms of image quality than the other methods     

Automatic determination of LV orientation from SPECT data

Presents a new method to determine the orientation or pose of the left ventricle (LV) of the heart from cardiac SPECT (single photon emission computed tomography) data. This proposed approach offers an accurate, fast, and robust delineation of the LV long-axis. The location and shape of the generated long-axis can then be utilized to define automatically the tomographic slices for enhanced visualization and quantification of the clinical data. The methodology is broadly composed of two main steps: (1) volume segmentation of cardiac SPECT data; and (2) topological goniometry, a novel approach incorporating volume visualization and computer graphics ideas to determine the overall shape of 3-D objects. The outcome of the algorithm is a 3-D curve representing the overall pose of the LV long-axis. Experimental results on both phantom and clinical data (50 technetium-99m and 74 thallium-201) are presented. An interactive graphical interface to visualize the volume (3-D) data, the left ventricle, and its pose is an integral part of the overall methodology. This technique is completely data driven and expeditious, making it viable for routine clinical use.     

Novel multistage three-dimensional medical image segmentation: methodology and validation.

In this paper, we propose a novel multistage method for three-dimensional (3-D) segmentation of medical images and a new radial distance-based segmentation validation approach. For the 3-D segmentation method, we first employ a morphological recursive erosion operation to reduce the connectivity between the region of interest and its surrounding neighborhood; then we design a hybrid segmentation method to achieve an initial result. The hybrid approach integrates an improved fast marching method and a morphological reconstruction algorithm. Finally, a morphological recursive dilation is employed to recover any lost structure from the first stage of the multistage method. This approach is tested on 12 CT and 3 MRI images of the brain, heart, and kidney, to demonstrate the effectiveness and accuracy of this technique across a variety of imaging modalities and organ systems. In order to validate the multistage segmentation method, a novel radial distance-based validation method is proposed that uses a global accuracy (GA) measure. The GA is calculated based on local radial distance errors (LRDE), where LRDE are calculated on the radii emitted from points along the skeleton of the object rather than the centroid, in order to accommodate more complicated organ structures. The experimental results demonstrate that the proposed multistage segmentation method is fast and accurate, with comparable performance to existing segmentation methods, but with a significantly higher execution speed.     

2-D mesh-based video object segmentation and tracking with occlusion resolution

This paper integrates fully automatic video object segmentation and tracking including detection and assignment of uncovered regions in a 2-D mesh-based framework. Particular contributions of this work are (i) a novel video object segmentation method that is posed as a constrained maximum contrast path search problem along the edges of a 2-D triangular mesh, and (ii) a 2-D mesh-based uncovered region detection method along the object boundary as well as within the object. At the first frame, an optimal number of feature points are selected as nodes of a 2-D content-based mesh. These points are classified as moving (foreground) and stationary nodes based on multi-frame node motion analysis, yielding a coarse estimate of the foreground object boundary. Color differences across triangles near the coarse boundary are employed for a maximum contrast path search along the edges of the 2-D mesh to refine the boundary of the video object. Next, we propagate the refined boundary to the subsequent frame by using motion vectors of the node points to form the coarse boundary at the next frame. We detect occluded regions by using motion-compensated frame differences and range filtered edge maps. The boundaries of detected uncovered regions are then refined by using the search procedure. These regions are either appended to the foreground object or tracked as new objects. The segmentation procedure is re-initialized when unreliable motion vectors exceed a certain number. The proposed scheme is demonstrated on several video sequences.     

Video representation with three-dimensional entities

Very low bit-rate coding requires new paradigms that go well beyond pixel- and frame-based video representations. We introduce a novel content-based video representation using tridimensional entities: textured object models and pose estimates. The multiproperty object models carry stochastic information about the shape and texture of each object present in the scene. The pose estimates define the position and orientation of the objects for each frame. This representation is compact. It provides alternative means for handling video by manipulating and compositing three-dimensional (3-D) entities. We call this representation tridimensional video compositing, or 3DVC for short. We present the 3DVC framework and describe the methods used to construct incrementally the object models and the pose estimates from unregistered noisy depth and texture measurements. We also describe a method for video frame reconstruction based on 3-D scene assembly, and discuss potential applications of 3DVC to video coding and content-based handling. 3DVC assumes that the objects in the scene are rigid and segmented. By assuming segmentation, we do not address the difficult questions of nonrigid segmentation and multiple object segmentation. In our experiments, segmentation is obtained via depth thresholding. It is important to notice that 3DVC is independent of the segmentation technique adopted. Experimental results with synthetic and real video sequences where compression ratios in the range of 1:150-1:2700 are achieved demonstrate the applicability of the proposed representation to very low bit-rate coding     

Interpolation of 3-D binary images based on morphological skeletonization

In this paper, the morphological skeleton interpolation (MSI) algorithm is presented. It is an efficient, shape-based interpolation method used for interpolating slices in a three-dimensional (3-D) binary object. It is based on morphological skeletonization, which is used for two-dimensional (2-D) slice representation. The proposed morphological skeleton matching process provides translation, rotation, and scaling information at the same time. The interpolated slices preserve the shape of the original object slices, when the slices have similar shapes. It can also modify the shape of an object when the successive slices do not have similar shapes. Applications on artificial and real data are also presented.     

Improving shape from focus using defocus cue.

The shape-from-focus (SFF) method uses a sequence of frames to estimate the structure of a 3-D object. Its accuracy depends on the step size by which the translational table is moved while capturing the images. Existing SFF algorithms use an ad hoc interpolation strategy to account for the error due to the finite step size. We propose an improved SFF method that uses relative defocus blur derived from actual image data to arrive at the final estimates of the structure of the object. A space-variant image restoration scheme is also proposed to obtain a focused image of the 3-D object. The reconstructed 3-D structure as well as the quality of the restored image are superior for the proposed method in comparison to traditional SFF.