Deformation analysis to detect and quantify active lesions inthree-dimensional medical image sequences

Evaluating precisely the temporal variations of lesion volumes is very important for at least three types of practical applications: pharmaceutical trials, decision making for drug treatment or surgery, and patient follow-up. In this paper we present a volumetric analysis technique, combining precise rigid registration of three-dimensional (3-D) (volumetric) medical images, nonrigid deformation computation, and flow-field analysis. Our analysis technique has two outcomes: the detection of evolving lesions and the quantitative measurement of volume variations. The originality of our approach is that no precise segmentation of the lesion is needed but the approximative designation of a region of interest (ROI) which can be automated. We distinguish between tissue transformation (image intensity changes without deformation) and expansion or contraction effects reflecting a change of mass within the tissue. A real lesion is generally the combination of both effects. The method is tested with synthesized volumetric image sequences and applied, in a first attempt to quantify in vivo a mass effect, to the analysis of a real patient case with multiple sclerosis (MS).     

Investigation of intraoperative brain deformation using a 1.5-Tinterventional MR system: preliminary results

All image-guided neurosurgical systems that the authors are aware of assume that the head and its contents behave as a rigid body. It is important to measure intraoperative brain deformation (brain shift) to provide some indication of the application accuracy of image-guided surgical systems, and also to provide data to develop and validate nonrigid registration algorithms to correct for such deformation. The authors are collecting data from patients undergoing neurosurgery in a high-field (1.5 T) interventional magnetic resonance (MR) scanner. High-contrast and high-resolution gradient-echo MR image volumes are collected immediately prior to surgery, during surgery, and at the end of surgery, with the patient intubated and lying on the operating table in the operative position. Here, the authors report initial results from six patients: one freehand biopsy, one stereotactic functional procedure, and four resections. The authors investigate intraoperative brain deformation by examining threshold boundary overlays and difference images and by measuring ventricular volume. They also present preliminary results obtained using a nonrigid registration algorithm to quantify deformation. They found that some cases had much greater deformation than others, and also that, regardless of the procedure, there was very little deformation of the midline, the tentorium, the hemisphere contralateral to the procedure, and ipsilateral structures except those that are within 1 cm of the lesion or are gravitationally above the surgical site     

Quantification of small cerebral ventricular volume changes in treated growth hormone patients using nonrigid registration

Nonrigid registration can automatically quantify small changes in volume of anatomical structures over time by means of segmentation propagation. Here, we use a nonrigid registration algorithm based on optimising normalized mutual information to quantify small changes in brain ventricle volume in magnetic resonance (MR) images of a group of five patients treated with growth hormone replacement therapy and a control group of six volunteers. The lateral ventricles are segmented from each subject image by registering with the brainweb image which has this structure delineated. The mean (standard deviation) volume change measurements are 1.09 (0.73) cm3 for the patient group and 0.08 (0.62) cm3 for the volunteer group; this difference is statistically significant at the 1% level. We validate our volume measurements by determining the precision from three consecutive scans of five volunteers and also comparing the measurements to previously published volume change estimates obtained by visual inspection of difference images. Results demonstrate a precision of sigma < or = 0.52 cm3 (n = 5) and a rank correlation coefficient with assessed difference images of p = 0.7 (n = 11). To determine the level of shape correspondence we manually segmented subject's ventricles and compared them to the propagations using a voxel overlap similarity index, this gave a mean similarity index of 0.81 (n = 7).     

Voxel similarity measures for 3-D serial MR brain image registration

The authors have evaluated eight different similarity measures used for rigid body registration of serial magnetic resonance (MR) brain scans. To assess their accuracy the authors used 33 clinical three-dimensional (3-D) serial MR images, with deformable extradural tissue excluded by manual segmentation and simulated 3-D MR images with added intensity distortion. For each measure the authors determined the consistency of registration transformations for both sets of segmented and unsegmented data. They have shown that of the eight measures tested, the ones based on joint entropy produced the best consistency. In particular, these measures seemed to be least sensitive to the presence of extradural tissue. For these data the difference in accuracy of these joint entropy measures, with or without brain segmentation, was within the threshold of visually detectable change in the difference images     

Volume-preserving nonrigid registration of MR breast images using free-form deformation with an incompressibility constraint

In this paper, we extend a previously reported intensity-based nonrigid registration algorithm by using a novel regularization term to constrain the deformation. Global motion is modeled by a rigid transformation while local motion is described by a free-form deformation based on B-splines. An information theoretic measure, normalized mutual information, is used as an intensity-based image similarity measure. Registration is performed by searching for the deformation that minimizes a cost function consisting of a weighted combination of the image similarity measure and a regularization term. The novel regularization term is a local volume-preservation (incompressibility) constraint, which is motivated by the assumption that soft tissue is incompressible for small deformations and short time periods. The incompressibility constraint is implemented by penalizing deviations of the Jacobian determinant of the deformation from unity. We apply the nonrigid registration algorithm with and without the incompressibility constraint to precontrast and post-contrast magnetic resonance (MR) breast images from 17 patients. Without using a constraint, the volume of contrast-enhancing lesions decreases by 1%-78% (mean 26%). Image improvement (motion artifact reduction) obtained using the new constraint is compared with that obtained using a smoothness constraint based on the bending energy of the coordinate grid by blinded visual assessment of maximum intensity projections of subtraction images. For both constraints, volume preservation improves, and motion artifact correction worsens, as the weight of the constraint penalty term increases. For a given volume change of the contrast-enhancing lesions (2% of the original volume), the incompressibility constraint reduces motion artifacts better than or equal to the smoothness constraint in 13 out of 17 cases (better in 9, equal in 4, worse in 4). The preliminary results suggest that incorporation of the incompressibility regularization term improves intensity-based free-form nonrigid registration of contrast-enhanced MR breast images by greatly reducing the problem of shrinkage of contrast-enhancing structures while simultaneously allowing motion artifacts to be substantially reduced.     

Nonrigid registration using free-form deformations: application tobreast MR images

In this paper we present a new approach for the nonrigid registration of contrast-enhanced breast MRI. A hierarchical transformation model of the motion of the breast has been developed. The global motion of the breast is modeled by an affine transformation while the local breast motion is described by a free-form deformation (FFD) based on B-splines. Normalized mutual information is used as a voxel-based similarity measure which is insensitive to intensity changes as a result of the contrast enhancement. Registration is achieved by minimizing a cost function, which represents a combination of the cost associated with the smoothness of the transformation and the cost associated with the image similarity. The algorithm has been applied to the fully automated registration of three-dimensional (3-D) breast MRI in volunteers and patients. In particular, we have compared the results of the proposed nonrigid registration algorithm to those obtained using rigid and affine registration techniques. The results clearly indicate that the nonrigid registration algorithm is much better able to recover the motion and deformation of the breast than rigid or affine registration algorithms.     

Simultaneous nonrigid registration, segmentation, and tumor detection in MRI guided cervical cancer radiation therapy

External beam radiation therapy (EBRT) for the treatment of cancer enables accurate placement of radiation dose on the cancerous region. However, the deformation of soft tissue during the course of treatment, such as in cervical cancer, presents significant challenges for the delineation of the target volume and other structures of interest. Furthermore, the presence and regression of pathologies such as tumors may violate registration constraints and cause registration errors. In this paper, automatic segmentation, nonrigid registration and tumor detection in cervical magnetic resonance (MR) data are addressed simultaneously using a unified Bayesian framework. The proposed novel method can generate a tumor probability map while progressively identifying the boundary of an organ of interest based on the achieved nonrigid transformation. The method is able to handle the challenges of significant tumor regression and its effect on surrounding tissues. The new method was compared to various currently existing algorithms on a set of 36 MR data from six patients, each patient has six T2-weighted MR cervical images. The results show that the proposed approach achieves an accuracy comparable to manual segmentation and it significantly outperforms the existing registration algorithms. In addition, the tumor detection result generated by the proposed method has a high agreement with manual delineation by a qualified clinician.     

A "twisting and bending" model-based nonrigid image registration technique for 3-D ultrasound carotid images

Atherosclerosis at the carotid bifurcation resulting in cerebral emboli is a major cause of ischemic stroke. Most strokes associated with carotid atherosclerosis can be prevented by lifestyle/dietary changes and pharmacological treatments if identified early by monitoring carotid plaque changes. Registration of 3-D ultrasound (US) images of carotid plaque obtained at different time points is essential for sensitive monitoring of plaque changes in volume and surface morphology. This registration technique should be nonrigid, since different head positions during image acquisition sessions cause relative bending and torsion in the neck, producing nonlinear deformations between the images. We modeled the movement of the neck using a “twisting and bending” model with only six parameters for nonrigid registration. We evaluated the algorithm using 3-D US carotid images acquired at two different head positions to simulate images acquired at different times. We calculated the mean registration error (MRE) between the segmented vessel surfaces in the target image and the registered image using a distance-based error metric after applying our “twisting and bending” model-based nonrigid registration algorithm. We achieved an average registration error of $0.80 pm 0.26$ mm using our nonrigid registration technique, which was a significant improvement in registration accuracy over rigid registration, even with reduced degrees-of-freedom compared to the other nonrigid registration algorithms.      

Automatic segmentation of hippocampal subfields based on multi-atlas image segmentation techniques

The volume of hippocampal subfields is closely related with early diagnosis of Alzheimer＇s disease. Due to the anatomical complexity of hippocampal subfields, automatic segmentation merely on the content of MR images is extremely difficult. We presented a method which combines multi-atlas image segmentation with extreme learning machine based bias detection and correction technique to achieve a fully automatic segmentation of hippocampal subfields. Symmetric diffeomorphic registration driven by symmetric mutual information energy was implemented in atlas registration, which allows multi-modal image registration and accelerates execution time. An exponential function based label fusion strategy was proposed for the normalized similarity measure case in segmentation combination, which yields better combination accuracy. The test results show that this method is effective, especially for the larger subfields with an overlap of more than 80%, which is competitive with the current methods and is of potential clinical significance.     

An automatic nonrigid registration for stained histological sections.

Automatic computer-based analyses of histological sections which are differently stained require that they are related to each other. Most registration methods are only able to perform rigid-body motion and are sensitive to noise and artifacts. Histological images, however, are accompanied by several artifacts and different contrasts, which require a nonrigid registration. In this paper, we present a hierarchical nonrigid registration algorithm able to align images, which contain minor image artifacts. The algorithm requires no a priori knowledge of the true image. The hierarchical design of the algorithm enhances robustness and accuracy, and saves computational costs. The proposed algorithm is decomposed into a fast, coarse, rigid registration step and a slower, but finer, nonrigid step. For the coarse registration, we use image pyramids, while for the second step, we combine a point-based registration with an elastic thin-plate spline interpolation. Accuracy tests, performed for 20 histological images obtained from human arteries, have shown that the error measure is acceptable, and that the image noise does not cause a problem. The associated convergence rate of the mean pixel displacement error during the rigid and nonrigid registrations is satisfying. The algorithm can be applied to various multicontrast elastic registration problems in medical imaging and may be extended to three dimensions.     

Accounting for signal loss due to dephasing in the correction of distortions in gradient-echo EPI via nonrigid registration

Gradient-echo (GE) echo planar imaging (EPI) is susceptible to both geometric distortions and signal loss. This paper presents a retrospective correction approach based on nonrigid image registration. A new physics-based intensity correction factor derived to compensate for intravoxel dephasing in GE EPI images is incorporated into a previously reported nonrigid registration algorithm. Intravoxel dephasing causes signal loss and thus intensity attenuation in the images. The new rephasing factor we introduce, which changes the intensity of a voxel in images during the registration, is used to improve the accuracy of the intensity-based nonrigid registration method and mitigate the intensity attenuation effect. Simulation-based experiments are first used to evaluate the method. A magnetic resonance (MR) simulator and a real field map are used to generate a realistic GE EPI image. The geometric distortion computed from the field map is used as the ground truth to which the estimated nonrigid deformation is compared. We then apply the algorithm to a set of real human brain images. The results show that, after registration, alignment between EPI and multi-shot, spin-echo images, which have relatively long acquisition times but negligible distortion, is improved and that signal loss caused by dephasing can be recovered.     

Multiscale modeling for image analysis of brain tumor studies

Image-based modeling of tumor growth combines methods from cancer simulation and medical imaging. In this context, we present a novel approach to adapt a healthy brain atlas to MR images of tumor patients. In order to establish correspondence between a healthy atlas and a pathologic patient image, tumor growth modeling in combination with registration algorithms is employed. In a first step, the tumor is grown in the atlas based on a new multiscale, multiphysics model including growth simulation from the cellular level up to the biomechanical level, accounting for cell proliferation and tissue deformations. Large-scale deformations are handled with an Eulerian approach for finite element computations, which can operate directly on the image voxel mesh. Subsequently, dense correspondence between the modified atlas and patient image is established using nonrigid registration. The method offers opportunities in atlas-based segmentation of tumor-bearing brain images as well as for improved patient-specific simulation and prognosis of tumor progression.     

Nonrigid image registration in shared-memory multiprocessor environments with application to brains, breasts, and bees

One major problem with nonrigid image registration techniques is their high computational cost. Because of this, these methods have found limited application to clinical situations where fast execution is required, e.g., intraoperative imaging. This paper presents a parallel implementation of a nonrigid image registration algorithm. It takes advantage of shared-memory multiprocessor computer architectures using multithreaded programming by partitioning of data and partitioning of tasks, depending on the computational subproblem. For three different biomedical applications (intraoperative brain deformation, contrast-enhanced MR mammography, intersubject brain registration), the scaling behavior of the algorithm is quantitatively analyzed. The method is demonstrated to perform the computation of intra-operative brain deformation in less than a minute using 64 CPUs on a 128-CPU shared-memory supercomputer (SGI Origin 3800). It is shown that its serial component is no more than 2% of the total computation time, allowing a speedup of at least a factor of 50. In most cases, the theoretical limit of the speedup is substantially higher (up to 132-fold in the application examples presented in this paper). The parallel implementation of our algorithm is, therefore, capable of solving nonrigid registration problems with short execution time requirements and may be considered an important step in the application of such techniques to clinically important problems such as the computation of brain deformation during cranial image-guided surgery.     

3-D/2-D registration of CT and MR to X-ray images

A crucial part of image-guided therapy is registration of preoperative and intraoperative images, by which the precise position and orientation of the patient's anatomy is determined in three dimensions. This paper presents a novel approach to register three-dimensional (3-D) computed tomography (CT) or magnetic resonance (MR) images to one or more two-dimensional (2-D) X-ray images. The registration is based solely on the information present in 2-D and 3-D images. It does not require fiducial markers, intraoperative X-ray image segmentation, or timely construction of digitally reconstructed radiographs. The originality of the approach is in using normals to bone surfaces, preoperatively defined in 3-D MR or CT data, and gradients of intraoperative X-ray images at locations defined by the X-ray source and 3-D surface points. The registration is concerned with finding the rigid transformation of a CT or MR volume, which provides the best match between surface normals and back projected gradients, considering their amplitudes and orientations. We have thoroughly validated our registration method by using MR, CT, and X-ray images of a cadaveric lumbar spine phantom for which "gold standard" registration was established by means of fiducial markers, and its accuracy assessed by target registration error. Volumes of interest, containing single vertebrae L1-L5, were registered to different pairs of X-ray images from different starting positions, chosen randomly and uniformly around the "gold standard" position. CT/X-ray (MR/ X-ray) registration, which is fast, was successful in more than 91% (82% except for L1) of trials if started from the "gold standard" translated or rotated for less than 6 mm or 17 degrees (3 mm or 8.6 degrees), respectively. Root-mean-square target registration errors were below 0.5 mm for the CT to X-ray registration and below 1.4 mm for MR to X-ray registration.     

Robust white matter lesion segmentation in FLAIR MRI

This paper discusses a white matter lesion (WML) segmentation scheme for fluid attenuation inversion recovery (FLAIR) MRI. The method computes the volume of lesions with subvoxel precision by accounting for the partial volume averaging (PVA) artifact. As WMLs are related to stroke and carotid disease, accurate volume measurements are most important. Manual volume computation is laborious, subjective, time consuming, and error prone. Automated methods are a nice alternative since they quantify WML volumes in an objective, efficient, and reliable manner. PVA is initially modeled with a localized edge strength measure since PVA resides in the boundaries between tissues. This map is computed in 3-D and is transformed to a global representation to increase robustness to noise. Significant edges correspond to PVA voxels, which are used to find the PVA fraction α (amount of each tissue present in mixture voxels). Results on simulated and real FLAIR images show high WML segmentation performance compared to ground truth (98.9% and 83% overlap, respectively), which outperforms other methods. Lesion load studies are included that automatically analyze WML volumes for each brain hemisphere separately. This technique does not require any distributional assumptions/parameters or training samples and is applied on a single MR modality, which is a major advantage compared to the traditional methods.     

GGO nodule volume-preserving nonrigid lung registration using GLCM texture analysis

In lung cancer screening, benign and malignant nodules can be classified through nodule growth assessment by the registration and, then, subtraction between follow-up computed tomography scans. During the registration, the volume of nodule regions in the floating image should be preserved, whereas the volume of other regions in the floating image should be aligned to that in the reference image. However, ground glass opacity (GGO) nodules are very elusive to automatically segment due to their inhomogeneous interior. In other words, it is difficult to automatically define the volume-preserving regions of GGO nodules. In this paper, we propose an accurate and fast nonrigid registration method. It applies the volume-preserving constraint to candidate regions of GGO nodules, which are automatically detected by gray-level cooccurrence matrix (GLCM) texture analysis. Considering that GGO nodules can be characterized by their inner inhomogeneity and high intensity, we identify the candidate regions of GGO nodules based on the homogeneity values calculated by the GLCM and the intensity values. Furthermore, we accelerate our nonrigid registration by using Compute Unified Device Architecture (CUDA). In the nonrigid registration process, the computationally expensive procedures of the floating-image transformation and the cost-function calculation are accelerated by using CUDA. The experimental results demonstrated that our method almost perfectly preserves the volume of GGO nodules in the floating image as well as effectively aligns the lung between the reference and floating images. Regarding the computational performance, our CUDA-based method delivers about 20× faster registration than the conventional method. Our method can be successfully applied to a GGO nodule follow-up study and can be extended to the volume-preserving registration and subtraction of specific diseases in other organs (e.g., liver cancer).     

Medical image registration using mutual information

Analysis of multispectral or multitemporal images requires proper geometric alignment of the images to compare corresponding regions in each image volume. Retrospective three-dimensional alignment or registration of multimodal medical images based on features intrinsic to the image data itself is complicated by their different photometric properties, by the complexity of the anatomical objects in the scene and by the large variety of clinical applications in which registration is involved. While the accuracy of registration approaches based on matching of anatomical landmarks or object surfaces suffers from segmentation errors, voxel-based approaches consider all voxels in the image without the need for segmentation. The recent introduction of the criterion of maximization of mutual information, a basic concept from information theory, has proven to be a breakthrough in the field. While solutions for intrapatient affine registration based on this concept are already commercially available, current research in the field focuses on interpatient nonrigid matching.     

Gradient-based 2-D/3-D rigid registration of fluoroscopic X-ray to CT

We present a gradient-based method for rigid registration of a patient preoperative computed tomography (CT) to its intraoperative situation with a few fluoroscopic X-ray images obtained with a tracked C-arm. The method is noninvasive, anatomy-based, requires simple user interaction, and includes validation. It is generic and easily customizable for a variety of routine clinical uses in orthopaedic surgery. Gradient-based registration consists of three steps: 1) initial pose estimation; 2) coarse geometry-based registration on bone contours, and; 3) fine gradient projection registration (GPR) on edge pixels. It optimizes speed, accuracy, and robustness. Its novelty resides in using volume gradients to eliminate outliers and foreign objects in the fluoroscopic X-ray images, in speeding up computation, and in achieving higher accuracy. It overcomes the drawbacks of intensity-based methods, which are slow and have a limited convergence range, and of geometry-based methods, which depend on the image segmentation quality. Our simulated, in vitro, and cadaver experiments on a human pelvis CT, dry vertebra, dry femur, fresh lamb hip, and human pelvis under realistic conditions show a mean 0.5-1.7 mm (0.5-2.6 mm maximum) target registration accuracy.     

Image registration by maximization of combined mutual information and gradient information

Mutual information has developed into an accurate measure for rigid and affine monomodality and multimodality image registration. The robustness of the measure is questionable, however. A possible reason for this is the absence of spatial information in the measure. The present paper proposes to include spatial information by combining mutual information with a term based on the image gradient of the images to be registered. The gradient term not only seeks to align locations of high gradient magnitude, but also aims for a similar orientation of the gradients at these locations. Results of combining both standard mutual information as well as a normalized measure are presented for rigid registration of three-dimensional clinical images [magnetic resonance (MR), computed tomography (CT), and positron emission tomography (PET)]. The results indicate that the combined measures yield a better registration function does mutual information or normalized mutual information per se. The registration functions are less sensitive to low sampling resolution, do not contain incorrect global maxima that are sometimes found in the mutual information function, and interpolation-induced local minima can be reduced. These characteristics yield the promise of more robust registration measures. The accuracy of the combined measures is similar to that of mutual information-based methods.     

Retrospective evaluation of intersubject brain registration

Although numerous methods to register brains of different individuals have been proposed, no work has been done, as far as we know, to evaluate and objectively compare the performances of different nonrigid (or elastic) registration methods on the same database of subjects. In this paper, we propose an evaluation framework, based on global and local measures of the relevance of the registration. We have chosen to focus more particularly on the matching of cortical areas, since intersubject registration methods are dedicated to anatomical and functional normalization, and also because other groups have shown the relevance of such registration methods for deep brain structures. Experiments were conducted using 6 methods on a database of 18 subjects. The global measures used show that the quality of the registration is directly related to the transformation's degrees of freedom. More surprisingly, local measures based on the matching of cortical sulci did not show significant differences between rigid and non rigid methods.