Registration of 3-D intraoperative MR images of the brain using a finite-element biomechanical model

We present a new algorithm for the nonrigid registration of three-dimensional magnetic resonance (MR) intraoperative image sequences showing brain shift. The algorithm tracks key surfaces of objects (cortical surface and the lateral ventricles) in the image sequence using a deformable surface matching algorithm. The volumetric deformation field of the objects is then inferred from the displacements at the boundary surfaces using a linear elastic biomechanical finite-element model. Two experiments on synthetic image sequences are presented, as well as an initial experiment on intraoperative MR images showing brain shift. The results of the registration algorithm show a good correlation of the internal brain structures after deformation, and a good capability of measuring surface as well as subsurface shift. We measured distances between landmarks in the deformed initial image and the corresponding landmarks in the target scan. Cortical surface shifts of up to 10 mm and subsurface shifts of up to 6 mm were recovered with an accuracy of 1 mm or less and 3 mm or less respectively.     

Volume registration using needle paths and point landmarks for evaluation of interventional MRI treatments

We created a method for three-dimensional (3-D) registration of medical images (e.g., magnetic resonance imaging (MRI) or computed tomography) to images of physical tissue sections or to other medical images and evaluated its accuracy. Our method proved valuable for evaluation of animal model experiments on interventional-MRI guided thermal ablation and on a new localized drug delivery system. The method computes an optimum set of rigid body registration parameters by minimization of the Euclidean distances between automatically chosen correspondence points, along manually selected fiducial needle paths, and optional point landmarks, using the iterative closest point algorithm. For numerically simulated experiments, using two needle paths over a range of needle orientations, mean voxel displacement errors depended mostly on needle localization error when the angle between needles was at least 20 degrees. For parameters typical of our in vivo experiments, the mean voxel displacement error was < 0.35 mm. In addition, we determined that the distance objective function was a useful diagnostic for predicting registration quality. To evaluate the registration quality of physical specimens, we computed the misregistration for a needle not considered during the optimization procedure. We registered an ex vivo sheep brain MR volume with another MR volume and tissue section photographs, using various combinations of needle and point landmarks. Mean registration error was always < or = 0.54 mm for MR-to-MR registrations and < or = 0.52 mm for MR to tissue section registrations. We also applied the method to correlate MR volumes of radio-frequency induced thermal ablation lesions with actual tissue destruction. In this case, in vivo rabbit thigh volumes were registered to photographs of ex vivo tissue sections using two needle paths. Mean registration errors were between 0.7 and 1.36 mm over all rabbits, the largest error less than two MR voxel widths. We conclude that our method provides sufficient spatial correspondence to facilitate comparison of 3-D image data with data from gross pathology tissue sections and histology.     

A comparative study of biomechanical simulators in deformable registration of brain tumor images

Simulating the brain tissue deformation caused by tumor growth has been found to aid the deformable registration of brain tumor images. In this paper, we evaluate the impact that different biomechanical simulators have on the accuracy of deformable registration. We use two alternative frameworks for biomechanical simulations of mass effect in 3-D magnetic resonance (MR) brain images. The first one is based on a finite-element model of nonlinear elasticity and unstructured meshes using the commercial software package ABAQUS. The second one employs incremental linear elasticity and regular grids in a fictitious domain method. In practice, biomechanical simulations via the second approach may be at least ten times faster. Landmarks error and visual examination of the coregistered images indicate that the two alternative frameworks for biomechanical simulations lead to comparable results of deformable registration. Thus, the computationally less expensive biomechanical simulator offers a practical alternative for registration purposes.     

Adaptive fuzzy segmentation of magnetic resonance images

An algorithm is presented for the fuzzy segmentation of two-dimensional (2-D) and three-dimensional (3-D) multispectral magnetic resonance (MR) images that have been corrupted by intensity inhomogeneities, also known as shading artifacts. The algorithm is an extension of the 2-D adaptive fuzzy C-means algorithm (2-D AFCM) presented in previous work by the authors. This algorithm models the intensity inhomogeneities as a gain field that causes image intensities to smoothly and slowly vary through the image space. It iteratively adapts to the intensity inhomogeneities and is completely automated. In this paper, we fully generalize 2-D AFCM to three-dimensional (3-D) multispectral images. Because of the potential size of 3-D image data, we also describe a new faster multigrid-based algorithm for its implementation. We show, using simulated MR data, that 3-D AFCM yields lower error rates than both the standard fuzzy C-means (FCM) algorithm and two other competing methods, when segmenting corrupted images. Its efficacy is further demonstrated using real 3-D scalar and multispectral MR brain images.     

Semiautomatic Registration of Pre- and Postbrain Tumor Resection Laser Range Data: Method and Validation

This paper presents a semiautomatic method for the registration of images acquired during surgery with a tracked laser range scanner (LRS). This method, which relies on the registration of vessels that can be visualized in the pre- and the post-resection images, is a component of a larger system designed to compute brain shift that occurs during tumor resection cases. Because very large differences between pre- and postresection images are typically observed, the development of fully automatic methods to register these images is difficult. The method presented herein is semiautomatic and requires only the identification of a number of points along the length of the vessels. Vessel segments joining these points are then automatically identified using an optimal path finding algorithm that relies on intensity features extracted from the images. Once vessels are identified, they are registered using a robust point-based nonrigid registration algorithm. The transformation computed with the vessels is then applied to the entire image. This permits establishment of a complete correspondence between the pre- and post-3-D LRS data. Experiments show that the method is robust to operator errors in localizing homologous points and a quantitative evaluation performed on ten surgical cases shows submillimetric registration accuracy.     

3-D/2-D registration by integrating 2-D information in 3-D

In image-guided therapy, high-quality preoperative images serve for planning and simulation, and intraoperatively as "background", onto which models of surgical instruments or radiation beams are projected. The link between a preoperative image and intraoperative physical space of the patient is established by image-to-patient registration. In this paper, we present a novel 3-D/2-D registration method. First, a 3-D image is reconstructed from a few 2-D X-ray images and next, the preoperative 3-D image is brought into the best possible spatial correspondence with the reconstructed image by optimizing a similarity measure (SM). Because the quality of the reconstructed image is generally low, we introduce a novel SM, which is able to cope with low image quality as well as with different imaging modalities. The novel 3-D/2-D registration method has been evaluated and compared to the gradient-based method (GBM) using standardized evaluation methodology and publicly available 3-D computed tomography (CT), 3-D rotational X-ray (3DRX), and magnetic resonance (MR) and 2-D X-ray images of two spine phantoms, for which gold standard registrations were known. For each of the 3DRX, CT, or MR images and each set of X-ray images, 1600 registrations were performed from starting positions, defined as the mean target registration error (mTRE), randomly generated and uniformly distributed in the interval of 0-20 mm around the gold standard. The capture range was defined as the distance from gold standard for which the final TRE was less than 2 mm in at least 95% of all cases. In terms of success rate, as the function of initial misalignment and capture range the proposed method outperformed the GBM. TREs of the novel method and the GBM were approximately the same. For the registration of 3DRX and CT images to X-ray images as few as 2-3 X-ray views were sufficient to obtain approximately 0.4 mm TREs, 7-9 mm capture range, and 80%-90% of successful registrations. To obtain similar results for MR to X-ray registrations, an image, reconstructed from at least 11 X-ray images was required. Reconstructions from more than 11 images had no effect on the registration results.     

A new validation method for X-ray mammogram registration algorithms using a projection model of breast X-ray compression

Establishing spatial correspondence between features visible in X-ray mammograms obtained at different times has great potential to aid assessment and quantitation of change in the breast indicative of malignancy. The literature contains numerous nonrigid registration algorithms developed for this purpose, but existing approaches are flawed by the assumption of inappropriate 2-D transformation models and quantitative estimation of registration accuracy is limited. In this paper, we describe a novel validation method which simulates plausible mammographic compressions of the breast using a magnetic resonance imaging (MRI) derived finite element model. By projecting the resulting known 3-D displacements into 2-D and generating pseudo-mammograms from these same compressed magnetic resonance (MR) volumes, we can generate convincing images with known 2-D displacements with which to validate a registration algorithm. We illustrate this approach by computing the accuracy for two conventional nonrigid 2-D registration algorithms applied to mammographic test images generated from three patient MR datasets. We show that the accuracy of these algorithms is close to the best achievable using a 2-D one-to-one correspondence model but that new algorithms incorporating more representative transformation models are required to achieve sufficiently accurate registrations for this application.     

Entropy-based dual-portal-to-3-DCT registration incorporating pixel correlation

For patient setup verification in external beam radiotherapy (EBRT) of prostate cancer, we developed an information theoretic registration framework, called the minimax entropy registration framework, to simultaneously and iteratively segment portal images and register them to three-dimensional (3-D) computed tomography (CT) image data. The registration framework has two steps, the max step and the min step, and evaluates appropriate entropies to estimate segmentations of the portal images and to find the transformation parameters. In the initial version of the algorithm (Bansal et al. 1999), we assumed image pixels to be independently distributed, an assumption not true in general. Thus, to better segment the portal images and to improve the accuracy of the estimated registration parameters, in this initial formulation of the problem, the correlation among pixel intensities is modeled using a one-dimensional Markov random process. Line processes are incorporated into the model to improve the estimation of segmentation of the portal images. In the max step, the principle of maximum entropy is invoked to estimate the probability distribution on the segmentations. The estimated distribution is then incorporated into the min step to estimate the registration parameters. Performance of the proposed framework is evaluated and compared to that of a mutual information-based registration algorithm using both simulated and real patient data. In the proposed registration framework, registration of the 3-D CT image and the portal images is guided by an estimated segmentation of the pelvic bone. However, as the prostate can move with respect to the pelvic structure, further localization of the prostate using ultrasound image data is required, an issue to be further explored in future.     

Rapid Dynamic Image Registration of the Beating Heart for Diagnosis and Surgical Navigation

Dynamic cardiac magnetic resonance imaging (MR) and computed tomography (CT) provide cardiologists and cardiac surgeons with high-quality 4-D images for diagnosis and therapy, yet the effective use of these high-quality anatomical models remains a challenge. Ultrasound (US) is a flexible imaging tool, but the US images produced are often difficult to interpret unless they are placed within their proper 3-D anatomical context. The ability to correlate real-time 3-D US volumes (RT3D US) with dynamic MR/CT images would offer a significant contribution to improve the quality of cardiac procedures. In this paper, we present a rapid two-step method for registering RT3D US to high-quality dynamic 3-D MR/CT images of the beating heart. This technique overcomes some major limitations of image registration (such as the correct registration result not necessarily occurring at the maximum of the mutual information (MI) metric) using the MI metric. We demonstrate the effectiveness of our method in a dynamic heart phantom (DHP) study and a human subject study. The achieved mean target registration error of CT+US images in the phantom study is 2.59 mm. Validation using human MR/US volumes shows a target registration error of 1.76 mm. We anticipate that this technique will substantially improve the quality of cardiac diagnosis and therapies.      

Model-updated image guidance: initial clinical experiences with gravity-induced brain deformation

Image-guided neurosurgery relies on accurate registration of the patient, the preoperative image series, and the surgical instruments in the same coordinate space. Recent clinical reports have documented the magnitude of gravity-induced brain deformation in the operating room and suggest these levels of tissue motion may compromise the integrity of such systems. We are investigating a model-based strategy which exploits the wealth of readily-available preoperative information in conjunction with intraoperatively acquired data to construct and drive a three dimensional (3-D) computational model which estimates volumetric displacements in order to update the neuronavigational image set. Using model calculations, the preoperative image database can be deformed to generate a more accurate representation of the surgical focus during an operation. In this paper, we present a preliminary study of four patients that experienced substantial brain deformation from gravity and correlate cortical shift measurements with model predictions. Additionally, we illustrate our image deforming algorithm and demonstrate that preoperative image resolution is maintained. Results over the four cases show that the brain shifted, on average, 5.7 mm in the direction of gravity and that model predictions could reduce this misregistration error to an average of 1.2 mm.     

Nonrigid registration of 3-d multichannel microscopy images of cell nuclei.

We present an intensity-based nonrigid registration approach for the normalization of 3-D multichannel microscopy images of cell nuclei. A main problem with cell nuclei images is that the intensity structure of different nuclei differs very much; thus, an intensity-based registration scheme cannot be used directly. Instead, we first perform a segmentation of the images from the cell nucleus channel, smooth the resulting images by a Gaussian filter, and then apply an intensity-based registration algorithm. The obtained transformation is applied to the images from the nucleus channel as well as to the images from the other channels. To improve the convergence rate of the algorithm, we propose an adaptive step length optimization scheme and also employ a multiresolution scheme. Our approach has been successfully applied using 2-D cell-like synthetic images, 3-D phantom images as well as 3-D multichannel microscopy images representing different chromosome territories and gene regions. We also describe an extension of our approach, which is applied for the registration of 3D + t (4-D) image series of moving cell nuclei.     

A "twisting and bending" model-based nonrigid image registration technique for 3-D ultrasound carotid images

Atherosclerosis at the carotid bifurcation resulting in cerebral emboli is a major cause of ischemic stroke. Most strokes associated with carotid atherosclerosis can be prevented by lifestyle/dietary changes and pharmacological treatments if identified early by monitoring carotid plaque changes. Registration of 3-D ultrasound (US) images of carotid plaque obtained at different time points is essential for sensitive monitoring of plaque changes in volume and surface morphology. This registration technique should be nonrigid, since different head positions during image acquisition sessions cause relative bending and torsion in the neck, producing nonlinear deformations between the images. We modeled the movement of the neck using a “twisting and bending” model with only six parameters for nonrigid registration. We evaluated the algorithm using 3-D US carotid images acquired at two different head positions to simulate images acquired at different times. We calculated the mean registration error (MRE) between the segmented vessel surfaces in the target image and the registered image using a distance-based error metric after applying our “twisting and bending” model-based nonrigid registration algorithm. We achieved an average registration error of $0.80 pm 0.26$ mm using our nonrigid registration technique, which was a significant improvement in registration accuracy over rigid registration, even with reduced degrees-of-freedom compared to the other nonrigid registration algorithms.      

Iterative principal axes registration method for analysis of MR-PETbrain images

Computerized automatic registration of MR-PET images of the brain is of significant interest for multimodality brain image analysis. Here, the authors discuss the principal axes transformation for registration of three-dimensional MR and PET images. A new brain phantom designed to test MR-PET registration accuracy determines that the principal axes registration (PAR) method is accurate to within an average of 1.37 mm with a standard deviation of 0.78 mm. Often the PET scans are not complete in the sense that the PET volume does not match the respective MR volume. The authors have developed an iterative PAR (IPAR) algorithm for such cases. Partial volumes of PET can be accurately registered to the complete MR volume using the new iterative algorithm. The quantitative and qualitative analyses of MR-PET image registration are presented and discussed. Results show that the new PAR algorithm is accurate and practical in MR-PET correlation studies     

Robust Gradient-Based 3-D/2-D Registration of CT and MR to X-Ray Images

One of the most important technical challenges in image-guided intervention is to obtain a precise transformation between the intrainterventional patient's anatomy and corresponding preinterventional 3-D image on which the intervention was planned. This goal can be achieved by acquiring intrainterventional 2-D images and matching them to the preinterventional 3-D image via 3-D/2-D image registration. A novel 3-D/2-D registration method is proposed in this paper. The method is based on robustly matching 3-D preinterventional image gradients and coarsely reconstructed 3-D gradients from the intrainterventional 2-D images. To improve the robustness of finding the correspondences between the two sets of gradients, hypothetical correspondences are searched for along normals to anatomical structures in 3-D images, while the final correspondences are established in an iterative process, combining the robust random sample consensus algorithm (RANSAC) and a special gradient matching criterion function. The proposed method was evaluated using the publicly available standardized evaluation methodology for 3-D/2-D registration, consisting of 3-D rotational X-ray, computed tomography, magnetic resonance (MR), and 2-D X-ray images of two spine segments, and standardized evaluation criteria. In this way, the proposed method could be objectively compared to the intensity, gradient, and reconstruction-based registration methods. The obtained results indicate that the proposed method performs favorably both in terms of registration accuracy and robustness. The method is especially superior when just a few X-ray images and when MR preinterventional images are used for registration, which are important advantages for many clinical applications.      

3-D/2-D registration of CT and MR to X-ray images

A crucial part of image-guided therapy is registration of preoperative and intraoperative images, by which the precise position and orientation of the patient's anatomy is determined in three dimensions. This paper presents a novel approach to register three-dimensional (3-D) computed tomography (CT) or magnetic resonance (MR) images to one or more two-dimensional (2-D) X-ray images. The registration is based solely on the information present in 2-D and 3-D images. It does not require fiducial markers, intraoperative X-ray image segmentation, or timely construction of digitally reconstructed radiographs. The originality of the approach is in using normals to bone surfaces, preoperatively defined in 3-D MR or CT data, and gradients of intraoperative X-ray images at locations defined by the X-ray source and 3-D surface points. The registration is concerned with finding the rigid transformation of a CT or MR volume, which provides the best match between surface normals and back projected gradients, considering their amplitudes and orientations. We have thoroughly validated our registration method by using MR, CT, and X-ray images of a cadaveric lumbar spine phantom for which "gold standard" registration was established by means of fiducial markers, and its accuracy assessed by target registration error. Volumes of interest, containing single vertebrae L1-L5, were registered to different pairs of X-ray images from different starting positions, chosen randomly and uniformly around the "gold standard" position. CT/X-ray (MR/ X-ray) registration, which is fast, was successful in more than 91% (82% except for L1) of trials if started from the "gold standard" translated or rotated for less than 6 mm or 17 degrees (3 mm or 8.6 degrees), respectively. Root-mean-square target registration errors were below 0.5 mm for the CT to X-ray registration and below 1.4 mm for MR to X-ray registration.     

最优控制点选取的遥感图像亚像素配准算法

分析了已有图像配准算法应用遥感图像配准方面的面临的问题,针对提高不同模态遥感图像配准精度问题,提出了一种人工辅助多模态图像配准算法。该算法首先由人工对待配准图像(测试图像)和参考图像输入控制点,利用高斯差分算子确定测试图像极值点;其次利用投影变换和最小线性平方差算法计算双边平均配准误差;最后,根据配准误差自动对控制点进行亚像素调整,取得亚像素级控制点匹配,实现遥感图像精确配准。实验结果表明,该算法具备更高的配准精度。      

Slice-to-volume registration and its potential application to interventional MRI-guided radio-frequency thermal ablation of prostate cancer

In this study, we registered live-time interventional magnetic resonance imaging (iMRI) slices with a previously obtained high-resolution MRI volume that in turn can be registered with a variety of functional images, e.g., PET, SPECT, for tumor targeting. We created and evaluated a slice-to-volume (SV) registration algorithm with special features for its potential use in iMRI-guided radio-frequency (RF) thermal ablation of prostate cancer. The algorithm features included a multiresolution approach, two similarity measures, and automatic restarting to avoid local minima. Imaging experiments were performed on volunteers using a conventional 1.5-T MR scanner and a clinical 0.2-T C-arm iMRI system under realistic conditions. Both high-resolution MR volumes and actual iMRI image slices were acquired from the same volunteers. Actual and simulated iMRI images were used to test the dependence of SV registration on image noise, receive coil inhomogeneity, and RF needle artifacts. To quantitatively assess registration, we calculated the mean voxel displacement over a volume of interest between SV registration and volume-to-volume registration, which was previously shown to be quite accurate. More than 800 registration experiments were performed. For transverse image slices covering the prostate, the SV registration algorithm was 100% successful with an error of <2 mm, and the average and standard deviation was only 0.4 mm +/- 0.2 mm. Visualizations such as combined sector display and contour overlay showed excellent registration of the prostate and other organs throughout the pelvis. Error was greater when an image slice was obtained at other orientations and positions, mostly because of inconsistent image content such as that from variable rectal and bladder filling. These preliminary experiments indicate that MR SV registration is sufficiently accurate to aid image-guided therapy.     

Registration of 3-D CT and 2-D Flat Images of Mouse via Affine Transformation

It is difficult to directly coregister the 3-D fluorescence molecular tomography (FMT) image of a small tumor in a mouse whose maximal diameter is only a few millimeters with a larger CT image of the entire animal that spans about 10 cm. This paper proposes a new method to register 2-D flat and 3-D CT image first to facilitate the registration between small 3-D FMT images and large 3-D CT images. A novel algorithm combining differential evolution and improved simplex method for the registration between the 2-D flat and 3-D CT images is introduced and validated with simulated images and real images of mice. The visualization of the alignment of the 3-D FMT and CT image through 2-D registration shows promising results.      

Multiscale modeling for image analysis of brain tumor studies

Image-based modeling of tumor growth combines methods from cancer simulation and medical imaging. In this context, we present a novel approach to adapt a healthy brain atlas to MR images of tumor patients. In order to establish correspondence between a healthy atlas and a pathologic patient image, tumor growth modeling in combination with registration algorithms is employed. In a first step, the tumor is grown in the atlas based on a new multiscale, multiphysics model including growth simulation from the cellular level up to the biomechanical level, accounting for cell proliferation and tissue deformations. Large-scale deformations are handled with an Eulerian approach for finite element computations, which can operate directly on the image voxel mesh. Subsequently, dense correspondence between the modified atlas and patient image is established using nonrigid registration. The method offers opportunities in atlas-based segmentation of tumor-bearing brain images as well as for improved patient-specific simulation and prognosis of tumor progression.     

Registration of images with varying topology using embedded maps

This paper presents registration via embedded maps (REM), a deformable registration algorithm for images with varying topology. The algorithm represents 3-D images as 4-D manifolds in a Riemannian space (referred to as embedded maps). Registration is performed as a surface evolution matching one embedded map to another using a diffusion process. The approach differs from those existing in that it takes an a priori estimation of image regions where topological changes are present, for example lesions, and generates a dense vector field representing both the shape and intensity changes necessary to match the images. The algorithm outputs both a diffeomorphic deformation field and an intensity displacement which corrects the intensity difference caused by topological changes. Multiple sets of experiments are conducted on magnetic resonance imaging (MRI) with lesions from OASIS and ADNI datasets. These images are registered to either a brain template or images of healthy individuals. An exemplar case registering a template to an MRI with tumor is also given. The resulting deformation fields were compared with those obtained using diffeomorphic demons, where topological changes are not modeled. These sets of experiments demonstrate the efficacy of our proposed REM method for registration of brain MRI with severe topological differences.