Model evaluation and calibration for prospective respiratory motion correction in coronary MR angiography based on 3-D image registration

Image processing was used as a fundamental tool to derive motion information from magnetic resonance (MR) images, which was fed back into prospective respiratory motion correction during subsequent data acquisition to improve image quality in coronary MR angiography (CMRA) scans. This reduces motion artifacts in the images and, in addition, enables the usage of a broader gating window than commonly used today to increase the scan efficiency. The aim of the study reported in this paper was to find a suitable motion model to be used for respiratory motion correction in cardiac imaging and to develop a calibration procedure to adapt the motion model to the individual patient. At first, the performance of three motion models [one-dimensional translation in feet-head (FH) direction, three-dimensional (3-D) translation, and 3-D affine transformation] was tested in a small volunteer study. An elastic image registration algorithm was applied to 3-D MR images of the coronary vessels obtained at different respiratory levels. A strong intersubject variability was observed. The 3-D translation and affine transformation model were found to be superior over the conventional FH translation model used today. Furthermore, a new approach is presented, which utilizes a fast model-based image registration to extract motion information from time series of low-resolution 3-D MR images, which reflects the respiratory motion of the heart. The registration is based on a selectable global 3-D motion model (translation, rigid, or affine transformation). All 3-D MR images were registered with respect to end expiration. The resulting time series of model parameters were analyzed in combination with additionally acquired motion information from a diaphragmatic MR pencil-beam navigator to calibrate the respiratory motion model. To demonstrate the potential of a calibrated motion model for prospective motion correction in coronary imaging, the approach was tested in CMRA examinations in five volunteers.     

Deformable triangular surfaces using fast 1-D radial Lagrangian dynamics--segmentation of 3-D MR and CT images of the wrist

We developed a new triangulated deformable surface model, which is used to detect the boundary of the bones in three-dimensional magnetic resonance (MR) and computed tomography (CT) images of the wrist. This surface model is robust to initialization and provides wide geometrical coverage and quantitative power. The surface is deformed by applying one-dimensional (1-D) radial Lagrangian dynamics. For initialization a tetrahedron is placed within the bone to be segmented. This initial surface is inflated to a binary approximation of the boundary. During inflation, the surface is refined by the addition of vertices. After the surface is fully inflated, a detailed, accurate boundary detection is obtained by the application of radial scale-space relaxation. In this optimization stage, the image intensity is filtered with a series of 1-D second-order Gaussian filters. The resolution of the triangulated mesh is adapted to the width of the Gaussian filter. To maintain the coherence between the vertices, a resampling technique is applied which is based on collapsing and splitting of edges. We regularized the triangulated mesh by a combination of volume-preserving vertex averaging and equi-angulation of edges. In this paper, we present both qualitative and quantitative results of the surface segmentations in eight MR and ten CT images.     

Tooth segmentation of dental study models using range images

The accurate segmentation of the teeth from the digitized representation of a dental study model is an important component in computer-based algorithms for orthodontic feature detection and measurement and in the simulation of orthodontic procedures such as tooth rearrangement. This paper presents an automated method for tooth segmentation from the three-dimensional (3-D) digitized image captured by a laser scanner. We avoid the complexity of directly processing 3-D mesh data by proposing the innovative idea of detecting features on two range images computed from the 3-D image. The dental arch is first obtained from the plan-view range image. Using the arch as the reference, a panoramic range image of the dental model can be computed. The interstices between the teeth are detected separately in the two range images, and results from both views are combined for a determination of interstice locations and orientations. Finally, the teeth are separated from the gums by delineating the gum margin. The algorithm was tested on 34 dental models representing a variety of malocclusions and was found to be robust and accurate.     

一种基于多特征的距离正则化水平集快速分割方法

现有的图像分割模型存在对初始化信息敏感,分割速率慢,图像弱边界区的泄露等现象.提出了一种混合快速分割方法.该方法利用偏压场近似估计图像的局部统计信息,并结合全局信息相容性及改进的距离正则化方法建立模型,最后将模型嵌入水平集框架中,与此同时,引入双重终止准则以提高分割的速度.最后利用合成图像和真实图像进行分割实验,并与CV（Chan-Vese）模型、非线性自适应水平集方法以及局部尺度拟合模型对比,表明本方法不仅对初始化信息敏感度降低,而且分割速度提高3~5倍.     

Automatic active model initialization via Poisson inverse gradient

Active models have been widely used in image processing applications. A crucial stage that affects the ultimate active model performance is initialization. This paper proposes a novel automatic initialization approach for parametric active models in both 2-D and 3-D. The PIG initialization method exploits a novel technique that essentially estimates the external energy field from the external force field and determines the most likely initial segmentation. Examples and comparisons with two state-of-the- art automatic initialization methods are presented to illustrate the advantages of this innovation, including the ability to choose the number of active models deployed, rapid convergence, accommodation of broken edges, superior noise robustness, and segmentation accuracy.     

A physics-based coordinate transformation for 3-D image matching

Many image matching schemes are based on mapping coordinate locations, such as the locations of landmarks, in one image to corresponding locations in a second image. A new approach to this mapping (coordinate transformation), called the elastic body spline (EBS), is described. The spline is based on a physical model of a homogeneous, isotropic three-dimensional (3-D) elastic body. The model can approximate the way that some physical objects deform. The EBS as well as the affine transformation, the thin plate spline and the volume spline are used to match 3-D magnetic resonance images (MRI's) of the breast that are used in the diagnosis and evaluation of breast cancer. These coordinate transformations are evaluated with different types of deformations and different numbers of corresponding (paired) coordinate locations. In all but one of the cases considered, using the EBS yields more similar images than the other methods     

Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features

We propose an automatic four-chamber heart segmentation system for the quantitative functional analysis of the heart from cardiac computed tomography (CT) volumes. Two topics are discussed: heart modeling and automatic model fitting to an unseen volume. Heart modeling is a nontrivial task since the heart is a complex nonrigid organ. The model must be anatomically accurate, allow manual editing, and provide sufficient information to guide automatic detection and segmentation. Unlike previous work, we explicitly represent important landmarks (such as the valves and the ventricular septum cusps) among the control points of the model. The control points can be detected reliably to guide the automatic model fitting process. Using this model, we develop an efficient and robust approach for automatic heart chamber segmentation in 3-D CT volumes. We formulate the segmentation as a two-step learning problem: anatomical structure localization and boundary delineation. In both steps, we exploit the recent advances in learning discriminative models. A novel algorithm, marginal space learning (MSL), is introduced to solve the 9-D similarity transformation search problem for localizing the heart chambers. After determining the pose of the heart chambers, we estimate the 3-D shape through learning-based boundary delineation. The proposed method has been extensively tested on the largest dataset (with 323 volumes from 137 patients) ever reported in the literature. To the best of our knowledge, our system is the fastest with a speed of 4.0 s per volume (on a dual-core 3.2-GHz processor) for the automatic segmentation of all four chambers.      

Real-time volume rendering visualization of dual-modality PET/CT images with interactive fuzzy thresholding segmentation.

Three-dimensional (3-D) visualization has become an essential part for imaging applications, including image-guided surgery, radiotherapy planning, and computer-aided diagnosis. In the visualization of dual-modality positron emission tomography and computed tomography (PET/CT), 3-D volume rendering is often limited to rendering of a single image volume and by high computational demand. Furthermore, incorporation of segmentation in volume rendering is usually restricted to visualizing the presegmented volumes of interest. In this paper, we investigated the integration of interactive segmentation into real-time volume rendering of dual-modality PET/CT images. We present and validate a fuzzy thresholding segmentation technique based on fuzzy cluster analysis, which allows interactive and real-time optimization of the segmentation results. This technique is then incorporated into a real-time multi-volume rendering of PET/CT images. Our method allows a real-time fusion and interchangeability of segmentation volume with PET or CT volumes, as well as the usual fusion of PET/CT volumes. Volume manipulations such as window level adjustments and lookup table can be applied to individual volumes, which are then fused together in real time as adjustments are made. We demonstrate the benefit of our method in integrating segmentation with volume rendering in its application to PET/CT images. Responsive frame rates are achieved by utilizing a texture-based volume rendering algorithm and the rapid transfer capability of the high-memory bandwidth available in low-cost graphic hardware.     

Hierarchical scale-based multiobject recognition of 3-D anatomical structures

Segmentation of anatomical structures from medical images is a challenging problem, which depends on the accurate recognition (localization) of anatomical structures prior to delineation. This study generalizes anatomy segmentation problem via attacking two major challenges: 1) automatically locating anatomical structures without doing search or optimization, and 2) automatically delineating the anatomical structures based on the located model assembly. For 1), we propose intensity weighted ball-scale object extraction concept to build a hierarchical transfer function from image space to object (shape) space such that anatomical structures in 3-D medical images can be recognized without the need to perform search or optimization. For 2), we integrate the graph-cut (GC) segmentation algorithm with prior shape model. This integrated segmentation framework is evaluated on clinical 3-D images consisting of a set of 20 abdominal CT scans. In addition, we use a set of 11 foot MR images to test the generalizability of our method to the different imaging modalities as well as robustness and accuracy of the proposed methodology. Since MR image intensities do not possess a tissue specific numeric meaning, we also explore the effects of intensity nonstandardness on anatomical object recognition. Experimental results indicate that: 1) effective recognition can make the delineation more accurate; 2) incorporating a large number of anatomical structures via a model assembly in the shape model improves the recognition and delineation accuracy dramatically; 3) ball-scale yields useful information about the relationship between the objects and the image; 4) intensity variation among scenes in an ensemble degrades object recognition performance.     

Medical image segmentation by combining graph cuts and oriented active appearance models

In this paper, we propose a novel method based on a strategic combination of the active appearance model (AAM), live wire (LW), and graph cuts (GCs) for abdominal 3-D organ segmentation. The proposed method consists of three main parts: model building, object recognition, and delineation. In the model building part, we construct the AAM and train the LW cost function and GC parameters. In the recognition part, a novel algorithm is proposed for improving the conventional AAM matching method, which effectively combines the AAM and LW methods, resulting in the oriented AAM (OAAM). A multiobject strategy is utilized to help in object initialization. We employ a pseudo-3-D initialization strategy and segment the organs slice by slice via a multiobject OAAM method. For the object delineation part, a 3-D shape-constrained GC method is proposed. The object shape generated from the initialization step is integrated into the GC cost computation, and an iterative GC-OAAM method is used for object delineation. The proposed method was tested in segmenting the liver, kidneys, and spleen on a clinical CT data set and also on the MICCAI 2007 Grand Challenge liver data set. The results show the following: 1) The overall segmentation accuracy of true positive volume fraction TPVF > 94.3% and false positive volume fraction can be achieved; 2) the initialization performance can be improved by combining the AAM and LW; 3) the multiobject strategy greatly facilitates initialization; 4) compared with the traditional 3-D AAM method, the pseudo-3-D OAAM method achieves comparable performance while running 12 times faster; and 5) the performance of the proposed method is comparable to state-of-the-art liver segmentation algorithm. The executable version of the 3-D shape-constrained GC method with a user interface can be downloaded from http://xinjianchen.wordpress.com/research/.     

Real-time fusion of endoscopic views with dynamic 3-D cardiac images: a phantom study

Minimally invasive robotically assisted cardiac surgical systems currently do not routinely employ 3-D image guidance. However, preoperative magnetic resonance and computed tomography (CT) images have the potential to be used in this role, if appropriately registered with the patient anatomy and animated synchronously with the motion of the actual heart. This paper discusses the fusion of optical images of a beating heart phantom obtained from an optically tracked endoscope, with volumetric images of the phantom created from a dynamic CT dataset. High quality preoperative dynamic CT images are created by first extracting the motion parameters of the heart from the series of temporal frames, and then applying this information to animate a high-quality heart image acquired at end systole. Temporal synchronization of the endoscopic and CT model is achieved by selecting the appropriate CT image from the dynamic set, based on an electrocardiographic trigger signal. The spatial error between the optical and virtual images is 1.4 +/- 1.1 mm, while the time discrepancy is typically 50-100 ms. Index Terms-Image guidance, image warping, minimally invasive cardiac surgery, virtual endoscopy, virtual reality.     

3-D/2-D registration of CT and MR to X-ray images

A crucial part of image-guided therapy is registration of preoperative and intraoperative images, by which the precise position and orientation of the patient's anatomy is determined in three dimensions. This paper presents a novel approach to register three-dimensional (3-D) computed tomography (CT) or magnetic resonance (MR) images to one or more two-dimensional (2-D) X-ray images. The registration is based solely on the information present in 2-D and 3-D images. It does not require fiducial markers, intraoperative X-ray image segmentation, or timely construction of digitally reconstructed radiographs. The originality of the approach is in using normals to bone surfaces, preoperatively defined in 3-D MR or CT data, and gradients of intraoperative X-ray images at locations defined by the X-ray source and 3-D surface points. The registration is concerned with finding the rigid transformation of a CT or MR volume, which provides the best match between surface normals and back projected gradients, considering their amplitudes and orientations. We have thoroughly validated our registration method by using MR, CT, and X-ray images of a cadaveric lumbar spine phantom for which "gold standard" registration was established by means of fiducial markers, and its accuracy assessed by target registration error. Volumes of interest, containing single vertebrae L1-L5, were registered to different pairs of X-ray images from different starting positions, chosen randomly and uniformly around the "gold standard" position. CT/X-ray (MR/ X-ray) registration, which is fast, was successful in more than 91% (82% except for L1) of trials if started from the "gold standard" translated or rotated for less than 6 mm or 17 degrees (3 mm or 8.6 degrees), respectively. Root-mean-square target registration errors were below 0.5 mm for the CT to X-ray registration and below 1.4 mm for MR to X-ray registration.     

Background removal of multiview images by learning shape priors.

Image-based rendering has been successfully used to display 3-D objects for many applications. A well-known example is the object movie, which is an image-based 3-D object composed of a collection of 2-D images taken from many different viewpoints of a 3-D object. In order to integrate image-based 3-D objects into a chosen scene (e.g., a panorama), one has to meet a hard challenge--to efficiently and effectively remove the background from the foreground object. This problem is referred to as multiview images (MVIs) segmentation. Another task requires MVI segmentation is image-based 3-D reconstruction using multiview images. In this paper, we propose a new method for segmenting MVI, which integrates some useful algorithms, including the well-known graph-cut image segmentation and volumetric graph-cut. The main idea is to incorporate the shape prior into the image segmentation process. The shape prior introduced into every image of the MVI is extracted from the 3-D model reconstructed by using the volumetric graph cuts algorithm. Here, the constraint obtained from the discrete medial axis is adopted to improve the reconstruction algorithm. The proposed MVI segmentation process requires only a small amount of user intervention, which is to select a subset of acceptable segmentations of the MVI after the initial segmentation process. According to our experiments, the proposed method can provide not only good MVI segmentation, but also provide acceptable 3-D reconstructed models for certain less-demanding applications.     

Segmentation and analysis of the human airway tree from three-dimensional X-ray CT images

The lungs exchange air with the external environment via the pulmonary airways. Computed tomography (CT) scanning can be used to obtain detailed images of the pulmonary anatomy, including the airways. These images have been used to measure airway geometry, study airway reactivity, and guide surgical interventions. Prior to these applications, airway segmentation can be used to identify the airway lumen in the CT images. Airway tree segmentation can be performed manually by an image analyst, but the complexity of the tree makes manual segmentation tedious and extremely time-consuming. We describe a fully automatic technique for segmenting the airway tree in three-dimensional (3-D) CT images of the thorax. We use grayscale morphological reconstruction to identify candidate airways on CT slices and then reconstruct a connected 3-D airway tree. After segmentation, we estimate airway branchpoints based on connectivity changes in the reconstructed tree. Compared to manual analysis on 3-mm-thick electron-beam CT images, the automatic approach has an overall airway branch detection sensitivity of approximately 73%.     

Automatic registration and alignment on a template of cardiacstress and rest reoriented SPECT images

Single photon emission computed tomography (SPECT) imaging with ²⁰¹Tl or ^99mTc agent is used to assess the location or the extent of myocardial infarction or ischemia. A method is proposed to decrease the effect of operator variability in the visual or quantitative interpretation of scintigraphic myocardial perfusion studies. To effect this, the patient's myocardial images (target cases) are registered automatically over a template image, utilizing a nonrigid transformation. The intermediate steps are: 1) Extraction of feature points in both stress and rest three-dimensional (3-D) images. The images are resampled in a polar geometry to detect edge points, which in turn are filtered by the use of a priori constraints. The remaining feature points are assumed to be points on the edges of the left ventricular myocardium. 2) Registration of stress and rest images with a global affine transformation. The matching method is an adaptation of the iterative closest point algorithm. 3) Registration and morphological matching of both stress and rest images on a template using a nonrigid local spline transformation following a global affine transformation. 4) Resampling of both stress and rest images in the geometry of the template. Optimization of the method was performed on a database of 40 pairs of stress and rest images selected to obtain a wide variation of images and abnormalities. Further testing was performed on 250 cases selected from the same database on the basis of the availability of angiographic results and patient stratification     

Automatic matching of homologous histological sections

The role of neuroanatomical atlases is undergoing a significant redefinition as digital atlases become available. These have the potential to serve as more than passive guides and to hold the role of directing segmentation and multimodal fusion of experimental data. Key elements needed to support these new tasks are registration algorithms. For images derived from histological procedures, the need is for techniques to map the two-dimensional (2-D) images of the sectional material into the reference atlas which may be a full three-dimensional (3-D) data set or one consisting of a series of 2-D images. A variety of 2-D-2-D registration methods are available to align experimental images with the atlas once the corresponding plane of section through the atlas has been identified. Methods to automate the identification of the homologous plane, however, have not been previously reported. Here, the authors use the external section contour to drive the identification and registration procedure. For this purpose, the authors model the contours by B-splines because of their attractive properties the most important of which are: (1) smoothness and continuity; (2) local controllability which implies that local changes in shape are confined to the B-spline parameters local to that change; (3) shape invariance under affine transformation, which means that the affine transformed curve is still a B-spline whose control points are related to the object control points through the transformation. The authors present a fast algorithm for estimating the control points of the B-spline which is robust to nonuniform sampling, noise, and local deformations. Curve matching is achieved by using a similarity measure that depends directly on the parameters of the B-spline. Performance tests are reported using histological material from rat brains     

Registration of 3-D CT and 2-D Flat Images of Mouse via Affine Transformation

It is difficult to directly coregister the 3-D fluorescence molecular tomography (FMT) image of a small tumor in a mouse whose maximal diameter is only a few millimeters with a larger CT image of the entire animal that spans about 10 cm. This paper proposes a new method to register 2-D flat and 3-D CT image first to facilitate the registration between small 3-D FMT images and large 3-D CT images. A novel algorithm combining differential evolution and improved simplex method for the registration between the 2-D flat and 3-D CT images is introduced and validated with simulated images and real images of mice. The visualization of the alignment of the 3-D FMT and CT image through 2-D registration shows promising results.      

Concurrent 3-D motion segmentation and 3-D interpretation of temporal sequences of monocular images.

The purpose of this study is to investigate a variational method for joint multiregion three-dimensional (3-D) motion segmentation and 3-D interpretation of temporal sequences of monocular images. Interpretation consists of dense recovery of 3-D structure and motion from the image sequence spatiotemporal variations due to short-range image motion. The method is direct insomuch as it does not require prior computation of image motion. It allows movement of both viewing system and multiple independently moving objects. The problem is formulated following a variational statement with a functional containing three terms. One term measures the conformity of the interpretation within each region of 3-D motion segmentation to the image sequence spatiotemporal variations. The second term is of regularization of depth. The assumption that environmental objects are rigid accounts automatically for the regularity of 3-D motion within each region of segmentation. The third and last term is for the regularity of segmentation boundaries. Minimization of the functional follows the corresponding Euler-Lagrange equations. This results in iterated concurrent computation of 3-D motion segmentation by curve evolution, depth by gradient descent, and 3-D motion by least squares within each region of segmentation. Curve evolution is implemented via level sets for topology independence and numerical stability. This algorithm and its implementation are verified on synthetic and real image sequences. Viewers presented with anaglyphs of stereoscopic images constructed from the algorithm's output reported a strong perception of depth.     

基于图像内在特征的图像自动拼接方法

针对传统图像拼接方法需要人工干预的不足,提出了基于图像内在特征的图像自动拼接方法.本方法应用由粗到精的技术实现图像之间像素精度的拼接.图像粗配准阶段采用仿射变换模型,利用Fourier-Mellin变换的频域分析技术同图像多分辨率技术相结合的方法求取仿射变换模型下的粗配准参数.图像的精配准阶段采用投影变换模型,将粗配准参数作为图像投影变换模型下的初始值,相邻图像中重叠区域的灰度差平方和作为误差指标函数,利用Levenberg-Marquardt方法进行图像投影模型参数估计的优化,得到全局意义条件下的精确变换参数.最后,将此方法应用于实际拍摄的可见光和红外图像序列进行试验分析,验证了本方法的有效性.     

Initial results for automated computational modeling ofpatient-specific electromagnetic hyperthermia

Developments in finite-difference time-domain (FD-TD) computational modeling of Maxwell's equations, super-computer technology, and computed tomography (CT) imagery open the possibility of accurate numerical simulation of electromagnetic (EM) wave interactions with specific, complex, biological tissue structures. One application of this technology is in the area of treatment planning for EM hyperthermia. In this paper, we report the first highly automated CT image segmentation and interpolation scheme applied to model patient-specific EM hyperthermia. This novel system is based on sophisticated tools from the artificial intelligence, computer vision, and computer graphics disciplines. It permits CT-based patient-specific hyperthermia models to be constructed without tedious manual contouring on digitizing pads or CRT screens. The system permits in principle near real-time assistance in hyperthermia treatment planning. We apply this system to interpret actual patient CT data, reconstructing a 3-D model of the human thigh from a collection of 29 serial CT images at 10 mm intervals. Then, using FD-TD, we obtain 2-D and 3-D models of EM hyperthermia of this thigh due to a waveguide applicator. We find that different results are obtained from the 2-D and 3-D models, and conclude that full 3-D tissue models are required for future clinical usage.