Multiscale modeling for image analysis of brain tumor studies

Image-based modeling of tumor growth combines methods from cancer simulation and medical imaging. In this context, we present a novel approach to adapt a healthy brain atlas to MR images of tumor patients. In order to establish correspondence between a healthy atlas and a pathologic patient image, tumor growth modeling in combination with registration algorithms is employed. In a first step, the tumor is grown in the atlas based on a new multiscale, multiphysics model including growth simulation from the cellular level up to the biomechanical level, accounting for cell proliferation and tissue deformations. Large-scale deformations are handled with an Eulerian approach for finite element computations, which can operate directly on the image voxel mesh. Subsequently, dense correspondence between the modified atlas and patient image is established using nonrigid registration. The method offers opportunities in atlas-based segmentation of tumor-bearing brain images as well as for improved patient-specific simulation and prognosis of tumor progression.     

Automatic quantification of changes in bone in serial MR images of joints

Recent innovations in drug therapies have made it highly desirable to obtain sensitive biomarkers of disease progression that can be used to quantify the performance of candidate disease modifying drugs. In order to measure potential image-based biomarkers of disease progression in an experimental model of rheumatoid arthritis (RA), we present two different methods to automatically quantify changes in a bone in in-vivo serial magnetic resonance (MR) images from the model. Both methods are based on rigid and nonrigid image registration to perform the analysis. The first method uses segmentation propagation to delineate a bone from the serial MR images giving a global measure of temporal changes in bone volume. The second method uses rigid body registration to determine intensity change within a bone, and then maps these into a reference coordinate system using nonrigid registration. This gives a local measure of temporal changes in bone lesion volume. We detected significant temporal changes in local bone lesion volume in five out of eight identified candidate bone lesion regions, and significant difference in local bone lesion volume between male and female subjects in three out of eight candidate bone lesion regions. But the global bone volume was found to be fluctuating over time. Finally, we compare our findings with histology of the subjects and the manual segmentation of bone lesions.     

A new validation method for X-ray mammogram registration algorithms using a projection model of breast X-ray compression

Establishing spatial correspondence between features visible in X-ray mammograms obtained at different times has great potential to aid assessment and quantitation of change in the breast indicative of malignancy. The literature contains numerous nonrigid registration algorithms developed for this purpose, but existing approaches are flawed by the assumption of inappropriate 2-D transformation models and quantitative estimation of registration accuracy is limited. In this paper, we describe a novel validation method which simulates plausible mammographic compressions of the breast using a magnetic resonance imaging (MRI) derived finite element model. By projecting the resulting known 3-D displacements into 2-D and generating pseudo-mammograms from these same compressed magnetic resonance (MR) volumes, we can generate convincing images with known 2-D displacements with which to validate a registration algorithm. We illustrate this approach by computing the accuracy for two conventional nonrigid 2-D registration algorithms applied to mammographic test images generated from three patient MR datasets. We show that the accuracy of these algorithms is close to the best achievable using a 2-D one-to-one correspondence model but that new algorithms incorporating more representative transformation models are required to achieve sufficiently accurate registrations for this application.     

Accounting for signal loss due to dephasing in the correction of distortions in gradient-echo EPI via nonrigid registration

Gradient-echo (GE) echo planar imaging (EPI) is susceptible to both geometric distortions and signal loss. This paper presents a retrospective correction approach based on nonrigid image registration. A new physics-based intensity correction factor derived to compensate for intravoxel dephasing in GE EPI images is incorporated into a previously reported nonrigid registration algorithm. Intravoxel dephasing causes signal loss and thus intensity attenuation in the images. The new rephasing factor we introduce, which changes the intensity of a voxel in images during the registration, is used to improve the accuracy of the intensity-based nonrigid registration method and mitigate the intensity attenuation effect. Simulation-based experiments are first used to evaluate the method. A magnetic resonance (MR) simulator and a real field map are used to generate a realistic GE EPI image. The geometric distortion computed from the field map is used as the ground truth to which the estimated nonrigid deformation is compared. We then apply the algorithm to a set of real human brain images. The results show that, after registration, alignment between EPI and multi-shot, spin-echo images, which have relatively long acquisition times but negligible distortion, is improved and that signal loss caused by dephasing can be recovered.     

Nonrigid image registration in shared-memory multiprocessor environments with application to brains, breasts, and bees

One major problem with nonrigid image registration techniques is their high computational cost. Because of this, these methods have found limited application to clinical situations where fast execution is required, e.g., intraoperative imaging. This paper presents a parallel implementation of a nonrigid image registration algorithm. It takes advantage of shared-memory multiprocessor computer architectures using multithreaded programming by partitioning of data and partitioning of tasks, depending on the computational subproblem. For three different biomedical applications (intraoperative brain deformation, contrast-enhanced MR mammography, intersubject brain registration), the scaling behavior of the algorithm is quantitatively analyzed. The method is demonstrated to perform the computation of intra-operative brain deformation in less than a minute using 64 CPUs on a 128-CPU shared-memory supercomputer (SGI Origin 3800). It is shown that its serial component is no more than 2% of the total computation time, allowing a speedup of at least a factor of 50. In most cases, the theoretical limit of the speedup is substantially higher (up to 132-fold in the application examples presented in this paper). The parallel implementation of our algorithm is, therefore, capable of solving nonrigid registration problems with short execution time requirements and may be considered an important step in the application of such techniques to clinically important problems such as the computation of brain deformation during cranial image-guided surgery.     

Nonrigid 2D/3D registration of coronary artery models with live fluoroscopy for guidance of cardiac interventions

A 2D/3D nonrigid registration method is proposed that brings a 3D centerline model of the coronary arteries into correspondence with bi-plane fluoroscopic angiograms. The registered model is overlaid on top of interventional angiograms to provide surgical assistance during image-guided chronic total occlusion procedures, thereby reducing the uncertainty inherent in 2D interventional images. The proposed methodology is divided into two parts: global structural alignment and local nonrigid registration. In both cases, vessel centerlines are automatically extracted from the 2D fluoroscopic images, and serve as the basis for the alignment and registration algorithms. In the first part, an energy minimization method is used to estimate a global affine transformation that aligns the centerline with the angiograms. The performance of nine general purpose optimizers has been assessed for this problem, and detailed results are presented. In the second part, a fully nonrigid registration method is proposed and used to compensate for any local shape discrepancy. This method is based on a variational framework, and uses a simultaneous matching and reconstruction process to compute a nonrigid registration. With a typical run time of less than 3 s, the algorithms are fast enough for interactive applications. Experiments on five different subjects are presented and show promising results.     

Multichannel image registration by feature-based information fusion

This paper proposes a novel nonrigid inter-subject multichannel image registration method which combines information from different modalities/channels to produce a unified joint registration. Multichannel images are created using co-registered multimodality images of the same subject to utilize information across modalities comprehensively. Contrary to the existing methods which combine the information at the image/intensity level, the proposed method uses feature-level information fusion method to spatio-adaptively combine the complementary information from different modalities that characterize different tissue types, through Gabor wavelets transformation and Independent Component Analysis (ICA), to produce a robust inter-subject registration. Experiments on both simulated and real multichannel images illustrate the applicability and robustness of the proposed registration method that combines information across modalities. This inter-subject registration is expected to pave the way for subsequent unified population-based multichannel studies.     

Noise estimation in infrared image sequences: a tool for the quantitative evaluation of the effectiveness of registration algorithms

Dynamic infrared imaging has been proposed in literature as an adjunctive technique to mammography in breast cancer diagnosis. It is based on the acquisition of hundreds of consecutive thermal images with a frame rate ranging from 50 to 200 frames/s, followed by the harmonic analysis of temperature time series at each image pixel. However, the temperature fluctuation due to blood perfusion, which is the signal of interest, is small compared to the signal fluctuation due to subject movements. Hence, before extracting the time series describing temperature fluctuations, it is fundamental to realign the thermal images to attenuate motion artifacts. In this paper, we describe a method for the quantitative evaluation of any kind of feature-based registration algorithm on thermal image sequences, provided that an estimation of local velocities of reference points on the skin is available. As an example of evaluation of a registration algorithm, we report the evaluation of the SNR improvement obtained by applying a nonrigid piecewise linear algorithm.     

Image registration based on lifting process: an application to digital subtraction radiography.

In this paper, a digital subtraction radiology scheme is presented based on a new method for the automatic registration of dental radiographs acquired with or without rigorous a priori standardization. The scheme is comprised of an automatic registration method and a subtraction process. The proposed registration method can be considered as an object-based registration method without imposing the prerequisite of image segmentation in order to detect the boundary of the objects of interest or the automatic detection of matching landmarks. This is achieved by augmenting the dimensionality of the problem from two-dimensional gray-level matching to three-dimensional surface matching using the process of lifting in combination with a surface-matching technique. The pseudo three-dimensional affine transformation that matches the lifted images incorporates advantageous characteristics including spatial alignment of the surfaces, anisotropic correction of brightness/contrast differences, and stable convergence of the similarity function to its optimal value. The performance of the proposed automatic registration method is assessed against a manual method based on the projective transformation. The qualitative and quantitative assessments of the experiments have shown advantageous performance of the proposed automatic registration method against the manual one. Finally, the proposed registration method has been further improved in terms of execution time by the implementation of a surface decimation process.     

GGO nodule volume-preserving nonrigid lung registration using GLCM texture analysis

In lung cancer screening, benign and malignant nodules can be classified through nodule growth assessment by the registration and, then, subtraction between follow-up computed tomography scans. During the registration, the volume of nodule regions in the floating image should be preserved, whereas the volume of other regions in the floating image should be aligned to that in the reference image. However, ground glass opacity (GGO) nodules are very elusive to automatically segment due to their inhomogeneous interior. In other words, it is difficult to automatically define the volume-preserving regions of GGO nodules. In this paper, we propose an accurate and fast nonrigid registration method. It applies the volume-preserving constraint to candidate regions of GGO nodules, which are automatically detected by gray-level cooccurrence matrix (GLCM) texture analysis. Considering that GGO nodules can be characterized by their inner inhomogeneity and high intensity, we identify the candidate regions of GGO nodules based on the homogeneity values calculated by the GLCM and the intensity values. Furthermore, we accelerate our nonrigid registration by using Compute Unified Device Architecture (CUDA). In the nonrigid registration process, the computationally expensive procedures of the floating-image transformation and the cost-function calculation are accelerated by using CUDA. The experimental results demonstrated that our method almost perfectly preserves the volume of GGO nodules in the floating image as well as effectively aligns the lung between the reference and floating images. Regarding the computational performance, our CUDA-based method delivers about 20× faster registration than the conventional method. Our method can be successfully applied to a GGO nodule follow-up study and can be extended to the volume-preserving registration and subtraction of specific diseases in other organs (e.g., liver cancer).     

Investigation of intraoperative brain deformation using a 1.5-Tinterventional MR system: preliminary results

All image-guided neurosurgical systems that the authors are aware of assume that the head and its contents behave as a rigid body. It is important to measure intraoperative brain deformation (brain shift) to provide some indication of the application accuracy of image-guided surgical systems, and also to provide data to develop and validate nonrigid registration algorithms to correct for such deformation. The authors are collecting data from patients undergoing neurosurgery in a high-field (1.5 T) interventional magnetic resonance (MR) scanner. High-contrast and high-resolution gradient-echo MR image volumes are collected immediately prior to surgery, during surgery, and at the end of surgery, with the patient intubated and lying on the operating table in the operative position. Here, the authors report initial results from six patients: one freehand biopsy, one stereotactic functional procedure, and four resections. The authors investigate intraoperative brain deformation by examining threshold boundary overlays and difference images and by measuring ventricular volume. They also present preliminary results obtained using a nonrigid registration algorithm to quantify deformation. They found that some cases had much greater deformation than others, and also that, regardless of the procedure, there was very little deformation of the midline, the tentorium, the hemisphere contralateral to the procedure, and ipsilateral structures except those that are within 1 cm of the lesion or are gravitationally above the surgical site     

Validation of nonrigid image registration using finite-element methods: application to breast MR images

This paper presents a novel method for validation of nonrigid medical image registration. This method is based on the simulation of physically plausible, biomechanical tissue deformations using finite-element methods. Applying a range of displacements to finite-element models of different patient anatomies generates model solutions which simulate gold standard deformations. From these solutions, deformed images are generated with a range of deformations typical of those likely to occur in vivo. The registration accuracy with respect to the finite-element simulations is quantified by co-registering the deformed images with the original images and comparing the recovered voxel displacements with the biomechanically simulated ones. The functionality of the validation method is demonstrated for a previously described nonrigid image registration technique based on free-form deformations using B-splines and normalized mutual information as a voxel similarity measure, with an application to contrast-enhanced magnetic resonance mammography image pairs. The exemplar nonrigid registration technique is shown to be of subvoxel accuracy on average for this particular application. The validation method presented here is an important step toward more generic simulations of biomechanically plausible tissue deformations and quantification of tissue motion recovery using nonrigid image registration. It will provide a basis for improving and comparing different nonrigid registration techniques for a diversity of medical applications, such as intrasubject tissue deformation or motion correction in the brain, liver or heart.     

A comprehensive cardiac motion estimation framework using both untagged and 3-D tagged MR images based on nonrigid registration

In this paper, we present a novel technique based on nonrigid image registration for myocardial motion estimation using both untagged and 3-D tagged MR images. The novel aspect of our technique is its simultaneous usage of complementary information from both untagged and 3-D tagged MR images. To estimate the motion within the myocardium, we register a sequence of tagged and untagged MR images during the cardiac cycle to a set of reference tagged and untagged MR images at end-diastole. The similarity measure is spatially weighted to maximize the utility of information from both images. In addition, the proposed approach integrates a valve plane tracker and adaptive incompressibility into the framework. We have evaluated the proposed approach on 12 subjects. Our results show a clear improvement in terms of accuracy compared to approaches that use either 3-D tagged or untagged MR image information alone. The relative error compared to manually tracked landmarks is less than 15% throughout the cardiac cycle. Finally, we demonstrate the automatic analysis of cardiac function from the myocardial deformation fields.     

A novel algorithm for computer-assisted measurement of cervical length from transvaginal ultrasound images

The cervical length measured by transvaginal ultrasound is a proven clinical tool for predicting premature birth. The standard manual measurement of the cervix is limited by variability in the technique. In this research, we develop the first computer algorithm that is able to identify the anatomic landmarks of the cervix on a transvaginal ultrasound image and determine the standard cervical length. The system is composed of four stages: The first stage is adaptive speckle suppression using variable length sticks algorithm. The second stage is the location of the internal cervical opening or "os" using a region-based segmentation. The third stage is delineation of the cervical canal. The fourth stage uses gray level summation patterns and prior knowledge to first localize the tissue boundary of the external cervix, and then use a template to determine the specific location of the external os. The cervical length is determined and calculated to image scale. To validate the proposed algorithm, 101 cervical ultrasound images were selected from a series of 37 examinations performed on 17 patients over an eight-month period. Repeated measurements of cervical length using the computer-assisted method were compared with those carried out by two experienced sonographers. The median intraobserver variability for the 101 images using the computer-assisted method was significantly smaller than that of the manual method by either sonographer. In a pairwise comparison, the mean cervical length for the computer method matches with the mean manual cervical length.     

Automatic construction of parts+geometry models for initializing groupwise registration

Groupwise nonrigid image registration is a powerful tool to automatically establish correspondences across sets of images. Such correspondences are widely used for constructing statistical models of shape and appearance. As existing techniques usually treat registration as an optimization problem, a good initialization is required. Although the standard initialization-affine transformation-generally works well, it is often inadequate when registering images of complex structures. In this paper we present a more sophisticated method that uses the sparse matches of a parts+geometry model as the initialization. We show that both the model and its matches can be automatically obtained, and that the matches are able to effectively initialize a groupwise nonrigid registration algorithm, leading to accurate dense correspondences. We also show that the dense mesh models constructed during the groupwise registration process can be used to accurately annotate new images. We demonstrate the efficacy of the approach on three datasets of increasing difficulty, and report on a detailed quantitative evaluation of its performance.     

Statistical deformable model-based segmentation of image motion

We present a statistical method for the motion-based segmentation of deformable structures undergoing nonrigid movements. The proposed approach relies on two models describing the shape of interest, its variability, and its movement. The first model corresponds to a statistical deformable template that constrains the shape and its deformations. The second model is introduced to represent the optical flow field inside the deformable template. These two models are combined within a single probability distribution, which enables to derive shape and motion estimates using a maximum likelihood approach. The method requires no manual initialization and is demonstrated on synthetic data and on a medical X-ray image sequence.     

A "twisting and bending" model-based nonrigid image registration technique for 3-D ultrasound carotid images

Atherosclerosis at the carotid bifurcation resulting in cerebral emboli is a major cause of ischemic stroke. Most strokes associated with carotid atherosclerosis can be prevented by lifestyle/dietary changes and pharmacological treatments if identified early by monitoring carotid plaque changes. Registration of 3-D ultrasound (US) images of carotid plaque obtained at different time points is essential for sensitive monitoring of plaque changes in volume and surface morphology. This registration technique should be nonrigid, since different head positions during image acquisition sessions cause relative bending and torsion in the neck, producing nonlinear deformations between the images. We modeled the movement of the neck using a “twisting and bending” model with only six parameters for nonrigid registration. We evaluated the algorithm using 3-D US carotid images acquired at two different head positions to simulate images acquired at different times. We calculated the mean registration error (MRE) between the segmented vessel surfaces in the target image and the registered image using a distance-based error metric after applying our “twisting and bending” model-based nonrigid registration algorithm. We achieved an average registration error of $0.80 pm 0.26$ mm using our nonrigid registration technique, which was a significant improvement in registration accuracy over rigid registration, even with reduced degrees-of-freedom compared to the other nonrigid registration algorithms.      

A method to track cortical surface deformations using a laser range scanner

This paper reports a novel method to track brain shift using a laser-range scanner (LRS) and nonrigid registration techniques. The LRS used in this paper is capable of generating textured point-clouds describing the surface geometry/intensity pattern of the brain as presented during cranial surgery. Using serial LRS acquisitions of the brain's surface and two-dimensional (2-D) nonrigid image registration, we developed a method to track surface motion during neurosurgical procedures. A series of experiments devised to evaluate the performance of the developed shift-tracking protocol are reported. In a controlled, quantitative phantom experiment, the results demonstrate that the surface shift-tracking protocol is capable of resolving shift to an accuracy of approximately 1.6 mm given initial shifts on the order of 15 mm. Furthermore, in a preliminary in vivo case using the tracked LRS and an independent optical measurement system, the automatic protocol was able to reconstruct 50% of the brain shift with an accuracy of 3.7 mm while the manual measurement was able to reconstruct 77% with an accuracy of 2.1 mm. The results suggest that a LRS is an effective tool for tracking brain surface shift during neurosurgery.     

Nonrigid point set matching of white matter tracts for diffusion tensor image analysis

Patient studies based on diffusion tensor images (DTI) require spatial correspondence between subjects. We propose to obtain the correspondence from white matter tracts, by introducing a new method for nonrigid matching of white matter fiber tracts in DTI. The method boils down to point set registration that involves simultaneously clustering and matching of the data points. The tracts are implicitly warped to a common frame of reference to avoid the potential bias toward one of the datasets. The algorithm gradually refines from global to local registration, which is implemented through deterministic annealing. Special care was taken to incorporate the spatial relation between fiber points and the uncertainty in principal diffusion orientation. As a result, the computed clusters are oriented along the fiber tracts and discriminate between adjacent but distinct fiber tracts. This is validated on synthetic and clinical data. The root-mean-squared distance with respect to expert-annotated landmarks is low (3 mm). In contrast to a state-of-the-art nonrigid registration technique, the proposed method is more robust to residual misalignments in terms of measured fractional anisotropy values.     

Nonrigid registration using free-form deformations: application tobreast MR images

In this paper we present a new approach for the nonrigid registration of contrast-enhanced breast MRI. A hierarchical transformation model of the motion of the breast has been developed. The global motion of the breast is modeled by an affine transformation while the local breast motion is described by a free-form deformation (FFD) based on B-splines. Normalized mutual information is used as a voxel-based similarity measure which is insensitive to intensity changes as a result of the contrast enhancement. Registration is achieved by minimizing a cost function, which represents a combination of the cost associated with the smoothness of the transformation and the cost associated with the image similarity. The algorithm has been applied to the fully automated registration of three-dimensional (3-D) breast MRI in volunteers and patients. In particular, we have compared the results of the proposed nonrigid registration algorithm to those obtained using rigid and affine registration techniques. The results clearly indicate that the nonrigid registration algorithm is much better able to recover the motion and deformation of the breast than rigid or affine registration algorithms.