Landmark-based elastic registration using approximating thin-platesplines

We consider elastic image registration based on a set of corresponding anatomical point landmarks and approximating thin-plate splines. This approach is an extension of the original interpolating thin-plate spline approach and allows to take into account landmark localization errors. The extension is important for clinical applications since landmark extraction is always prone to error. Our approach is based on a minimizing functional and can cope with isotropic as well as anisotropic landmark errors. In particular, in the latter case it is possible to include different types of landmarks, e.g., unique point landmarks as well as arbitrary edge points. Also, the scheme is general with respect to the image dimension and the order of smoothness of the underlying functional. Optimal affine transformations as well as interpolating thin-plate splines are special cases of this scheme. To localize landmarks we use a semi-automatic approach which is based on three-dimensional (3-D) differential operators. Experimental results are presented for two-dimensional as well as 3-D tomographic images of the human brain.     

Compensation of sound speed deviations in 3-D B-mode ultrasound for intraoperative determination of the anterior pelvic plane

An accurate determination of the pelvic orientation is inevitable for the correct cup prosthesis placement of navigated total hip arthroplasties. Conventionally, this step is accomplished by percutaneous palpation of anatomic landmarks. Sterility issues and an increased landmark localization error for obese patients lead to the application of B-mode ultrasound imaging in the field of computer-assisted orthopedic surgery. Many approaches have been proposed in the literature to replace the percutaneous digitization by 3-D B-mode ultrasound imaging. However, the correct depth localization of the pelvic landmarks could be significantly affected by the acoustic properties of the penetrated tissues. Imprecise depth estimation could lead to a miscalculation of the pelvic orientation and subsequently to a misalignment of the acetabular cup implant. But so far, no solution has been presented, which compensates for acoustic property differences for correct depth estimation. In this paper, we present a novel approach to determine pelvic orientation from ultrasound images by applying a hierarchical registration scheme based on patch statistical shape models to compensate for differences in speed of sound. The method was validated based on plastic bones and a cadaveric specimen.     

Consistent landmark and intensity-based image registration

Two new consistent image registration algorithms are presented: one is based on matching corresponding landmarks and the other is based on matching both landmark and intensity information. The consistent landmark and intensity registration algorithm produces good correspondences between images near landmark locations by matching corresponding landmarks and away from landmark locations by matching the image intensities. In contrast to similar unidirectional algorithms, these new consistent algorithms jointly estimate the forward and reverse transformation between two images while minimizing the inverse consistency error-the error between the forward (reverse) transformation and the inverse of the the reverse (forward) transformation. This reduces the ambiguous correspondence between the forward and reverse transformations associated with large inverse consistency errors. In both algorithms a thin-plate spline (TPS) model is used to regularize the estimated transformations. Two-dimensional (2-D) examples are presented that show the inverse consistency error produced by the traditional unidirectional landmark TPS algorithm can be relatively large and that this error is minimized using the consistent landmark algorithm. Results using 2-D magnetic resonance imaging data are presented that demonstrate that using landmark and intensity information together produce better correspondence between medical images than using either landmarks or intensity information alone.     

Registration of real and CT-derived virtual bronchoscopic images toassist transbronchial biopsy

This paper describes research work motivated by an innovative medical application: computer-assisted transbronchial biopsy. This project involves the registration, with no external localization device, of a preoperative three-dimensional (3-D) computed tomography (CT) scan of the thoracic cavity (showing a tumor that requires a needle biopsy), and an intraoperative endoscopic two-dimensional (2-D) image sequence, in order to provide assistance in transbronchial puncture of the tumor. Because of the specific difficulties resulting from the data being processed, a multilevel strategy was introduced. For each analysis level, the relevant information to process and the corresponding algorithms were defined. This multilevel strategy, thus, provides the best possible accuracy. Original image processing methods were elaborated, dealing with segmentation, registration and 3-D reconstruction of the bronchoscopic images. In particular, these methods involve adapted mathematical morphology tools, a “daemon-based” registration algorithm, and a model-based shape-from-shading algorithm. This pilot study presents the application of these algorithms to recorded bronchoscopic video sequences for five patients. The preliminary results presented here demonstrate that it is possible to precisely localize the endoscopic camera within the CT data coordinate system. The computer can thus synthesize in near real-time the CT-derived virtual view that corresponds to the actual endoscopic view     

Automatic construction of multiple-object three-dimensional statistical shape models: application to cardiac modeling

A novel method is introduced for the generation of landmarks for three-dimensional (3-D) shapes and the construction of the corresponding 3-D statistical shape models. Automatic landmarking of a set of manual segmentations from a class of shapes is achieved by 1) construction of an atlas of the class, 2) automatic extraction of the landmarks from the atlas, and 3) subsequent propagation of these landmarks to each example shape via a volumetric nonrigid registration technique using multiresolution B-spline deformations. This approach presents some advantages over previously published methods: it can treat multiple-part structures and requires less restrictive assumptions on the structure's topology. In this paper, we address the problem of building a 3-D statistical shape model of the left and right ventricle of the heart from 3-D magnetic resonance images. The average accuracy in landmark propagation is shown to be below 2.2 mm. This application demonstrates the robustness and accuracy of the method in the presence of large shape variability and multiple objects.     

Minimal shape and intensity cost path segmentation

A new generic model-based segmentation algorithm is presented, which can be trained from examples akin to the active shape model (ASM) approach in order to acquire knowledge about the shape to be segmented and about the gray-level appearance of the object in the image. Whereas ASM alternates between shape and intensity information during search, the proposed approach optimizes for shape and intensity characteristics simultaneously. Local gray-level appearance information at the landmark points extracted from feature images is used to automatically detect a number of plausible candidate locations for each landmark. The shape information is described by multiple landmark-specific statistical models that capture local dependencies between adjacent landmarks on the shape. The shape and intensity models are combined in a single cost function that is optimized noniteratively using dynamic programming, without the need for initialization. The algorithm was validated for segmentation of anatomical structures in chest and hand radiographs. In each experiment, the presented method had a significant higher performance when compared to the ASM schemes. As the method is highly effective, optimally suited for pathological cases and easy to implement, it is highly useful for many medical image segmentation tasks.     

Fast parametric elastic image registration

We present an algorithm for fast elastic multidimensional intensity-based image registration with a parametric model of the deformation. It is fully automatic in its default mode of operation. In the case of hard real-world problems, it is capable of accepting expert hints in the form of soft landmark constraints. Much fewer landmarks are needed and the results are far superior compared to pure landmark registration. Particular attention has been paid to the factors influencing the speed of this algorithm. The B-spline deformation model is shown to be computationally more efficient than other alternatives. The algorithm has been successfully used for several two-dimensional (2-D) and three-dimensional (3-D) registration tasks in the medical domain, involving MRI, SPECT, CT, and ultrasound image modalities. We also present experiments in a controlled environment, permitting an exact evaluation of the registration accuracy. Test deformations are generated automatically using a random hierarchical fractional wavelet-based generator.     

Computer-aided kidney segmentation on abdominal CT images.

In this paper, an effective model-based approach for computer-aided kidney segmentation of abdominal CT images with anatomic structure consideration is presented. This automatic segmentation system is expected to assist physicians in both clinical diagnosis and educational training. The proposed method is a coarse to fine segmentation approach divided into two stages. First, the candidate kidney region is extracted according to the statistical geometric location of kidney within the abdomen. This approach is applicable to images of different sizes by using the relative distance of the kidney region to the spine. The second stage identifies the kidney by a series of image processing operations. The main elements of the proposed system are: 1) the location of the spine is used as the landmark for coordinate references; 2) elliptic candidate kidney region extraction with progressive positioning on the consecutive CT images; 3) novel directional model for a more reliable kidney region seed point identification; and 4) adaptive region growing controlled by the properties of image homogeneity. In addition, in order to provide different views for the physicians, we have implemented a visualization tool that will automatically show the renal contour through the method of second-order neighborhood edge detection. We considered segmentation of kidney regions from CT scans that contain pathologies in clinical practice. The results of a series of tests on 358 images from 30 patients indicate an average correlation coefficient of up to 88% between automatic and manual segmentation.     

Differences between pelvic skin and bone landmark identification in different seated positions on spinal-cord injured subjects

The purpose of this paper was to determine the differences between internal and external pelvic landmark locations in different seating positions. A computer tool developed for the registration of two series of images was used to obtain the internal geometry. First, images of the pelvis were acquired by magnetic resonance imaging (MRI) for each subject, in a supine position; internal landmarks were then identified on the images. Second, ultrasound images of the iliac crests were acquired in four seated positions. A registration algorithm was applied to obtain the transformation matrix between the two image reference systems. The MRI anatomical landmarks were, therefore, transferred into the ultrasound referential, to obtain their three-dimensional (3-D) location in the different seating positions. The external landmarks in those seated positions were identified with a 3-D digitizer. The results revealed that generally the internal and external coordinates of corresponding landmarks are statistically different. The differences are not only due to soft tissue thickness but also to different interpretations of the landmarks' locations between the supine and the seated postures. However, these differences generally did not affect significantly the accuracy with which orientation indexes can be estimated (pelvic tilt, obliquity, transverse rotation). Correlations were found between the internal and external coordinates, implying that linear regressions can be established.     

Nonrigid registration of 3-d multichannel microscopy images of cell nuclei.

We present an intensity-based nonrigid registration approach for the normalization of 3-D multichannel microscopy images of cell nuclei. A main problem with cell nuclei images is that the intensity structure of different nuclei differs very much; thus, an intensity-based registration scheme cannot be used directly. Instead, we first perform a segmentation of the images from the cell nucleus channel, smooth the resulting images by a Gaussian filter, and then apply an intensity-based registration algorithm. The obtained transformation is applied to the images from the nucleus channel as well as to the images from the other channels. To improve the convergence rate of the algorithm, we propose an adaptive step length optimization scheme and also employ a multiresolution scheme. Our approach has been successfully applied using 2-D cell-like synthetic images, 3-D phantom images as well as 3-D multichannel microscopy images representing different chromosome territories and gene regions. We also describe an extension of our approach, which is applied for the registration of 3D + t (4-D) image series of moving cell nuclei.     

Automatic active model initialization via Poisson inverse gradient

Active models have been widely used in image processing applications. A crucial stage that affects the ultimate active model performance is initialization. This paper proposes a novel automatic initialization approach for parametric active models in both 2-D and 3-D. The PIG initialization method exploits a novel technique that essentially estimates the external energy field from the external force field and determines the most likely initial segmentation. Examples and comparisons with two state-of-the- art automatic initialization methods are presented to illustrate the advantages of this innovation, including the ability to choose the number of active models deployed, rapid convergence, accommodation of broken edges, superior noise robustness, and segmentation accuracy.     

Automated 2-D cephalometric analysis on X-ray images by a model-based approach

Craniofacial landmark localization and anatomical structure tracing on cephalograms are two important ways to obtain the cephalometric analysis. In order to computerize them in parallel, a model-based approach is proposed to locate 262 craniofacial feature points, including 90 landmarks and 172 auxiliary points. In model training, 12 landmarks are selected as reference points and used to divide every training shape to 10 regions according to the anatomical knowledge; principle components analysis is employed to characterize the region shape variations and the statistical grey profile of every feature point. Locating feature points on an input image is a two-stage procedure. First, we identify the reference landmarks by image processing and pattern matching techniques, so that the shape partition is performed on the input image. Then, for each region, its feature points are located by a modified active shape model. All craniofacial anatomical structures can be traced out by connecting the located points with subdivision curves according to the prior knowledge. Users are permitted to modify the results interactively in many different ways. Experimental results show the advantage and reliability of the proposed method.     

Registration of multiview real-time 3-D echocardiographic sequences

Real-time 3-D echocardiography opens up the possibility of interactive, fast 3-D analysis of cardiac anatomy and function. However, at the present time its quantitative power cannot be fully exploited due to the limited quality of the images. In this paper, we present an algorithm to register apical and parasternal echocardiographic datasets that uses a new similarity measure, based on local orientation and phase differences. By using phase and orientation to guide registration, the effect of artifacts intrinsic to ultrasound images is minimized. The presented method is fully automatic except for initialization. The accuracy of the method was validated qualitatively, resulting in 85% of the cardiac segments estimated having a registration error smaller than 2 mm, and no segments with an error larger than 5 mm. Robustness with respect to landmark initialization was validated quantitatively, with average errors smaller than 0.2 mm and 0.5 degrees for initialization landmarks rotations of up to 15 degrees and translations of up to 10 mm.     

A comparison framework for breathing motion estimation methods from 4-D imaging

Motion estimation is an important issue in radiation therapy of moving organs. In particular, motion estimates from 4-D imaging can be used to compute the distribution of an absorbed dose during the therapeutic irradiation. We propose a strategy and criteria incorporating spatiotemporal information to evaluate the accuracy of model-based methods capturing breathing motion from 4-D CT images. This evaluation relies on the identification and tracking of landmarks on the 4-D CT images by medical experts. Three different experts selected more than 500 landmarks within 4-D CT images of lungs for three patients. Landmark tracking was performed at four instants of the expiration phase. Two metrics are proposed to evaluate the tracking performance of motion-estimation models. The first metric cumulates over the four instants the errors on landmark location. The second metric integrates the error over a time interval according to an a priori breathing model for the landmark spatiotemporal trajectory. This latter metric better takes into account the dynamics of the motion. A second aim of this paper is to estimate the impact of considering several phases of the respiratory cycle as compared to using only the extreme phases (end-inspiration and end-expiration). The accuracy of three motion estimation models (two image registration-based methods and a biomechanical method) is compared through the proposed metrics and statistical tools. This paper points out the interest of taking into account more frames for reliably tracking the respiratory motion.     

A comprehensive cardiac motion estimation framework using both untagged and 3-D tagged MR images based on nonrigid registration

In this paper, we present a novel technique based on nonrigid image registration for myocardial motion estimation using both untagged and 3-D tagged MR images. The novel aspect of our technique is its simultaneous usage of complementary information from both untagged and 3-D tagged MR images. To estimate the motion within the myocardium, we register a sequence of tagged and untagged MR images during the cardiac cycle to a set of reference tagged and untagged MR images at end-diastole. The similarity measure is spatially weighted to maximize the utility of information from both images. In addition, the proposed approach integrates a valve plane tracker and adaptive incompressibility into the framework. We have evaluated the proposed approach on 12 subjects. Our results show a clear improvement in terms of accuracy compared to approaches that use either 3-D tagged or untagged MR image information alone. The relative error compared to manually tracked landmarks is less than 15% throughout the cardiac cycle. Finally, we demonstrate the automatic analysis of cardiac function from the myocardial deformation fields.     

Constrained registration for motion compensation in atrial fibrillation ablation procedures

Fluoroscopic overlay images rendered from preoperative volumetric data can provide additional anatomical details to guide physicians during catheter ablation procedures for treatment of atrial fibrillation (AFib). As these overlay images are often compromised by cardiac and respiratory motion, motion compensation methods are needed to keep the overlay images in sync with the fluoroscopic images. So far, these approaches have either required simultaneous biplane imaging for 3-D motion compensation, or in case of monoplane X-ray imaging, provided only a limited 2-D functionality. To overcome the downsides of the previously suggested methods, we propose an approach that facilitates a full 3-D motion compensation even if only monoplane X-ray images are available. To this end, we use a training phase that employs a biplane sequence to establish a patient specific motion model. Afterwards, a constrained model-based 2-D/3-D registration method is used to track a circumferential mapping catheter. This device is commonly used for AFib catheter ablation procedures. Based on the experiments on real patient data, we found that our constrained monoplane 2-D/3-D registration outperformed the unconstrained counterpart and yielded an average 2-D tracking error of 0.6 mm and an average 3-D tracking error of 1.6 mm. The unconstrained 2-D/3-D registration technique yielded a similar 2-D performance, but the 3-D tracking error increased to 3.2 mm mostly due to wrongly estimated 3-D motion components in X-ray view direction. Compared to the conventional 2-D monoplane method, the proposed method provides a more seamless workflow by removing the need for catheter model re-initialization otherwise required when the C-arm view orientation changes. In addition, the proposed method can be straightforwardly combined with the previously introduced biplane motion compensation technique to obtain a good trade-off between accuracy and radiation dose reduction.     

Volume registration using needle paths and point landmarks for evaluation of interventional MRI treatments

We created a method for three-dimensional (3-D) registration of medical images (e.g., magnetic resonance imaging (MRI) or computed tomography) to images of physical tissue sections or to other medical images and evaluated its accuracy. Our method proved valuable for evaluation of animal model experiments on interventional-MRI guided thermal ablation and on a new localized drug delivery system. The method computes an optimum set of rigid body registration parameters by minimization of the Euclidean distances between automatically chosen correspondence points, along manually selected fiducial needle paths, and optional point landmarks, using the iterative closest point algorithm. For numerically simulated experiments, using two needle paths over a range of needle orientations, mean voxel displacement errors depended mostly on needle localization error when the angle between needles was at least 20 degrees. For parameters typical of our in vivo experiments, the mean voxel displacement error was < 0.35 mm. In addition, we determined that the distance objective function was a useful diagnostic for predicting registration quality. To evaluate the registration quality of physical specimens, we computed the misregistration for a needle not considered during the optimization procedure. We registered an ex vivo sheep brain MR volume with another MR volume and tissue section photographs, using various combinations of needle and point landmarks. Mean registration error was always < or = 0.54 mm for MR-to-MR registrations and < or = 0.52 mm for MR to tissue section registrations. We also applied the method to correlate MR volumes of radio-frequency induced thermal ablation lesions with actual tissue destruction. In this case, in vivo rabbit thigh volumes were registered to photographs of ex vivo tissue sections using two needle paths. Mean registration errors were between 0.7 and 1.36 mm over all rabbits, the largest error less than two MR voxel widths. We conclude that our method provides sufficient spatial correspondence to facilitate comparison of 3-D image data with data from gross pathology tissue sections and histology.     

3-D/2-D registration of CT and MR to X-ray images

A crucial part of image-guided therapy is registration of preoperative and intraoperative images, by which the precise position and orientation of the patient's anatomy is determined in three dimensions. This paper presents a novel approach to register three-dimensional (3-D) computed tomography (CT) or magnetic resonance (MR) images to one or more two-dimensional (2-D) X-ray images. The registration is based solely on the information present in 2-D and 3-D images. It does not require fiducial markers, intraoperative X-ray image segmentation, or timely construction of digitally reconstructed radiographs. The originality of the approach is in using normals to bone surfaces, preoperatively defined in 3-D MR or CT data, and gradients of intraoperative X-ray images at locations defined by the X-ray source and 3-D surface points. The registration is concerned with finding the rigid transformation of a CT or MR volume, which provides the best match between surface normals and back projected gradients, considering their amplitudes and orientations. We have thoroughly validated our registration method by using MR, CT, and X-ray images of a cadaveric lumbar spine phantom for which "gold standard" registration was established by means of fiducial markers, and its accuracy assessed by target registration error. Volumes of interest, containing single vertebrae L1-L5, were registered to different pairs of X-ray images from different starting positions, chosen randomly and uniformly around the "gold standard" position. CT/X-ray (MR/ X-ray) registration, which is fast, was successful in more than 91% (82% except for L1) of trials if started from the "gold standard" translated or rotated for less than 6 mm or 17 degrees (3 mm or 8.6 degrees), respectively. Root-mean-square target registration errors were below 0.5 mm for the CT to X-ray registration and below 1.4 mm for MR to X-ray registration.     

Segmentation and quantification of human vessels using a 3-D cylindrical intensity model.

We introduce a new approach for 3-D segmentation and quantification of vessels. The approach is based on a 3-D cylindrical parametric intensity model, which is directly fitted to the image intensities through an incremental process based on a Kalman filter. Segmentation results are the vessel centerline and shape, i.e., we estimate the local vessel radius, the 3-D position and 3-D orientation, the contrast, as well as the fitting error. We carried out an extensive validation using 3-D synthetic images and also compared the new approach with an approach based on a Gaussian model. In addition, the new model has been successfully applied to segment vessels from 3-D MRA and computed tomography angiography image data. In particular, we compared our approach with an approach based on the randomized Hough transform. Moreover, a validation of the segmentation results based on ground truth provided by a radiologist confirms the accuracy of the new approach. Our experiments show that the new model yields superior results in estimating the vessel radius compared to previous approaches based on a Gaussian model as well as the Hough transform.