Voxel-based 2-D/3-D registration of fluoroscopy images and CT scansfor image-guided surgery

Registration of intraoperative fluoroscopy images with preoperative 3D CT images can he used for several purposes in image-guided surgery. On the one hand, it can be used to display the position of surgical instruments, which are being tracked by a localizer, in the preoperative CT scan. On the other hand, the registration result can be used to project preoperative planning information or important anatomical structures visible in the CT image on to the fluoroscopy image. For this registration task, a novel voxel-based method in combination with a new similarity measure (pattern intensity) has been developed. The basic concept of the method is explained at the example of 2D/3D registration of a vertebra in an X-ray fluoroscopy image with a 3D CT image. The registration method is described, and the results for a spine phantom are presented and discussed. Registration has been carried out repeatedly with different starting estimates to study the capture range. Information about registration accuracy has been obtained by comparing the registration results with a highly accurate “ground-truth” registration, which has been derived from fiducial markers attached to the phantom prior to imaging. In addition, registration results for different vertebrae have been compared. The results show that the rotation parameters and the shifts parallel to the projection plane can accurately be determined from a single projection. Because of the projection geometry, the accuracy of the height above the projection plane is significantly lower     

3-D/2-D registration of CT and MR to X-ray images

A crucial part of image-guided therapy is registration of preoperative and intraoperative images, by which the precise position and orientation of the patient's anatomy is determined in three dimensions. This paper presents a novel approach to register three-dimensional (3-D) computed tomography (CT) or magnetic resonance (MR) images to one or more two-dimensional (2-D) X-ray images. The registration is based solely on the information present in 2-D and 3-D images. It does not require fiducial markers, intraoperative X-ray image segmentation, or timely construction of digitally reconstructed radiographs. The originality of the approach is in using normals to bone surfaces, preoperatively defined in 3-D MR or CT data, and gradients of intraoperative X-ray images at locations defined by the X-ray source and 3-D surface points. The registration is concerned with finding the rigid transformation of a CT or MR volume, which provides the best match between surface normals and back projected gradients, considering their amplitudes and orientations. We have thoroughly validated our registration method by using MR, CT, and X-ray images of a cadaveric lumbar spine phantom for which "gold standard" registration was established by means of fiducial markers, and its accuracy assessed by target registration error. Volumes of interest, containing single vertebrae L1-L5, were registered to different pairs of X-ray images from different starting positions, chosen randomly and uniformly around the "gold standard" position. CT/X-ray (MR/ X-ray) registration, which is fast, was successful in more than 91% (82% except for L1) of trials if started from the "gold standard" translated or rotated for less than 6 mm or 17 degrees (3 mm or 8.6 degrees), respectively. Root-mean-square target registration errors were below 0.5 mm for the CT to X-ray registration and below 1.4 mm for MR to X-ray registration.     

Three-dimensional cardiac strain estimation using spatio-temporal elastic registration of ultrasound images: a feasibility study

Current ultrasound methods for measuring myocardial strain are often limited to measurements in one or two dimensions. Cardiac motion and deformation however are truly 3-D. With the introduction of matrix transducer technology, 3-D ultrasound imaging of the heart has become feasible but suffers from low temporal and spatial resolution, making 3-D strain estimation challenging. In this paper, it is shown that automatic intensity-based spatio–temporal elastic registration of currently available 3-D volumetric ultrasound data sets can be used to measure the full 3-D strain tensor. The method was validated using simulated 3-D ultrasound data sets of the left ventricle (LV). Three types of data sets were simulated: a normal and symmetric LV with different heart rates, a more realistic asymmetric normal LV and an infarcted LV. The absolute error in the estimated displacement was between $0.47 pm 0.23$ and $1.00 pm 0.59$ mm, depending on heart rate and amount of background noise. The absolute error on the estimated strain was 9%–21% for the radial strain and 1%–4% for the longitudinal and circumferential strains. No large differences were found between the different types of data sets. The shape of the strain curves was estimated properly and the position of the infarcts could be identified correctly. Preliminary results on clinical data taken in vivo from three healthy volunteers and one patient with an apical aneurism confirmed these findings in a qualitative manner as the strain curves obtained with the proposed method have an amplitude and shape similar to what could be expected.      

Automatic registration of CT and MR brain images using correlation of geometrical features

Describes an automated approach to register CT and MR brain images. Differential operators in scale space are applied to each type of image data, so as to produce feature images depicting "ridgeness". The resulting CT and MR feature images show similarities which can be used for matching. No segmentation is needed and the method is devoid of human interaction. The matching is accomplished by hierarchical correlation techniques. Results of 2-D and 3-D matching experiments are presented. The correlation function ensures an accurate match even if the scanned volumes to be matched do not completely overlap, or if some of the features in the images are not similar.     

ORBIT: a multiresolution framework for deformable registration of brain tumor images

A deformable registration method is proposed for registering a normal brain atlas with images of brain tumor patients. The registration is facilitated by first simulating the tumor mass effect in the normal atlas in order to create an atlas image that is as similar as possible to the patient's image. An optimization framework is used to optimize the location of tumor seed as well as other parameters of the tumor growth model, based on the pattern of deformation around the tumor region. In particular, the optimization is implemented in a multiresolution and hierarchical scheme, and it is accelerated by using a principal component analysis (PCA)-based model of tumor growth and mass effect, trained on a computationally more expensive biomechanical model. Validation on simulated and real images shows that the proposed registration framework, referred to as ORBIT (optimization of tumor parameters and registration of brain images with tumors), outperforms other available registration methods particularly for the regions close to the tumor, and it has the potential to assist in constructing statistical atlases from tumor-diseased brain images.      

A moment-based approach to registration of images with affinegeometric distortion

A new method is described for automatic control point selection and matching. First, reference and sensed images are segmented and closed-boundary regions are extracted. Each region is represented by a set of affine-invariant moment-based features. Correspondence between the regions is then established by a two-stage matching algorithm that works both in the feature space and in the image space. Centers of gravity of corresponding regions are used as control points. A practical use of the proposed method is demonstrated by registration of SPOT and Landsat TM images. It is shown that the authors' method can produce subpixel registration accuracy     

Automated segmentation of the lumbar pedicle in CT images for spinal fusion surgery

Exact information about the shape of a lumbar pedicle can increase operation accuracy and safety during computer-aided spinal fusion surgery, which requires extreme caution on the part of the surgeon, due to the complexity and delicacy of the procedure. In this paper, a robust framework for segmenting the lumbar pedicle in computed tomography (CT) images is presented. The framework that has been designed takes a CT image, which includes the lumbar pedicle as input, and provides the segmented lumbar pedicle in the form of 3-D voxel sets. This multistep approach begins with 2-D dynamic thresholding using local optimal thresholds, followed by procedures to recover the spine geometry in a high curvature environment. A subsequent canal reference determination using proposed thinning-based integrated cost is then performed. Based on the obtained segmented vertebra and canal reference, the edge of the spinal pedicle is segmented. This framework has been tested on 84 lumbar vertebrae of 19 patients requiring spinal fusion. It was successfully applied, resulting in an average success rate of 93.22 % and a final mean error of 0.14 ± 0.05 mm. Precision errors were smaller than 1 % for spine pedicle volumes. Intra- and interoperator precision errors were not significantly different.     

Validation of nonrigid image registration using finite-element methods: application to breast MR images

This paper presents a novel method for validation of nonrigid medical image registration. This method is based on the simulation of physically plausible, biomechanical tissue deformations using finite-element methods. Applying a range of displacements to finite-element models of different patient anatomies generates model solutions which simulate gold standard deformations. From these solutions, deformed images are generated with a range of deformations typical of those likely to occur in vivo. The registration accuracy with respect to the finite-element simulations is quantified by co-registering the deformed images with the original images and comparing the recovered voxel displacements with the biomechanically simulated ones. The functionality of the validation method is demonstrated for a previously described nonrigid image registration technique based on free-form deformations using B-splines and normalized mutual information as a voxel similarity measure, with an application to contrast-enhanced magnetic resonance mammography image pairs. The exemplar nonrigid registration technique is shown to be of subvoxel accuracy on average for this particular application. The validation method presented here is an important step toward more generic simulations of biomechanically plausible tissue deformations and quantification of tissue motion recovery using nonrigid image registration. It will provide a basis for improving and comparing different nonrigid registration techniques for a diversity of medical applications, such as intrasubject tissue deformation or motion correction in the brain, liver or heart.     

A comparative study of biomechanical simulators in deformable registration of brain tumor images

Simulating the brain tissue deformation caused by tumor growth has been found to aid the deformable registration of brain tumor images. In this paper, we evaluate the impact that different biomechanical simulators have on the accuracy of deformable registration. We use two alternative frameworks for biomechanical simulations of mass effect in 3-D magnetic resonance (MR) brain images. The first one is based on a finite-element model of nonlinear elasticity and unstructured meshes using the commercial software package ABAQUS. The second one employs incremental linear elasticity and regular grids in a fictitious domain method. In practice, biomechanical simulations via the second approach may be at least ten times faster. Landmarks error and visual examination of the coregistered images indicate that the two alternative frameworks for biomechanical simulations lead to comparable results of deformable registration. Thus, the computationally less expensive biomechanical simulator offers a practical alternative for registration purposes.     

Retrospective intermodality registration techniques for images of the head: surface-based versus volume-based

The primary objective of this study is to perform a blinded evaluation of two groups of retrospective image registration techniques, using as a gold standard a prospective marker-based registration method, and to compare the performance of one group with the other. These techniques have already been evaluated individually [27]. In this paper, however, we find that by grouping the techniques as volume based or surface based, we can make some interesting conclusions which were not visible in the earlier study. In order to ensure blindness, all retrospective registrations were performed by participants who had no knowledge of the gold-standard results until after their results had been submitted. Image volumes of three modalities: X-ray computed tomography (CT), magnetic resonance (MR), and positron emission tomography (PET) were obtained from patients undergoing neurosurgery at Vanderbilt University Medical Center on whom bone-implanted fiducial markers were mounted. These volumes had all traces of the markers removed and were provided via the Internet to project collaborators outside Vanderbilt, who then performed retrospective registrations on the volumes, calculating transformations from CT to MR and/or from PET to MR. These investigators communicated their transformations, again via the Internet, to Vanderbilt, where the accuracy of each registration was evaluated. In this evaluation, the accuracy is measured at multiple volumes of interest (VOI's). Our results indicate that the volume-based techniques in this study tended to give substantially more accurate and reliable results than the surface-based ones for the CT-to-MR registration tasks, and slightly more accurate results for the PET-to-MR tasks. Analysis of these results revealed that the rotational component of error was more pronounced for the surface-based group. It was also apparent that all of the registration techniques we examined have the potential to produce satisfactory results much of the time, but that visual inspection is necessary to guard against large errors.     

极限学习机方法在经纬仪空间配准中的应用

针对光电经纬仪测量中多传感器的空间配准问题,提出了一种基于极限学习机（ELM）的空间配准建模方法。首先介绍了ELM算法和ELM空间配准模型的建立步骤,然后使用星体测量数据建立ELM空间配准模型,最后将该模型与单项差修正模型、球谐函数修正模型进行了对比验证。实验结果表明:ELM空间配准模型可以使光电经纬仪的测量精度从17左右提高到1以内,与单项差修正模型、球谐函数修正模型相比精度提高35%以上。由此可见,与单项差修正模型和球谐函数修正模型相比,采用ELM算法所建立的光电经纬仪空间配准模型具有更高的精度和更强的泛化能力。     

Explicit incorporation of prior anatomical information into a nonrigid registration of thoracic and abdominal CT and 18-FDG whole-body emission PET images

The aim of this paper is to develop a registration methodology in order to combine anatomical and functional information provided by thoracic/abdominal computed tomography (CT) and whole-body positron emission tomography (PET) images. The proposed procedure is based on the incorporation of prior anatomical information in an intensity-based nonrigid registration algorithm. This incorporation is achieved in an explicit way, initializing the intensity-based registration stage with the solution obtained by a nonrigid registration of corresponding anatomical structures. A segmentation algorithm based on a hierarchically ordered set of anatomy-specific rules is used to obtain anatomical structures in CT and emission PET scans. Nonrigid deformations are modeled in both registration stages by means of free-form deformations, the optimization of the control points being achieved by means of an original vector field-based approach instead of the classical gradient-based techniques, considerably reducing the computational time of the structure registration stage. We have applied the proposed methodology to 38 sets of images (33 provided by standalone machines and five by hybrid systems) and an assessment protocol has been developed to furnish a qualitative evaluation of the algorithm performance.     

Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge

EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.     

Nonrigid registration of 3-D free-hand ultrasound images of the breast

Three-dimensional (3-D) ultrasound imaging of the breast enables better assessment of diseases than conventional two-dimensional (2-D) imaging. Free-hand techniques are often used for generating 3-D data from a sequence of 2-D slice images. However, the breast deforms substantially during scanning because it is composed primarily of soft tissue. This often causes tissue mis-registration in spatial compounding of multiple scan sweeps. To overcome this problem, in this paper, instead of introducing additional constraints on scanning conditions, we use image processing techniques. We present a fully automatic algorithm for 3-D nonlinear registration of free-hand ultrasound data. It uses a block matching scheme and local statistics to estimate local tissue deformation. A Bayesian regularization method is applied to the sample displacement field. The final deformation field is obtained by fitting a B-spline approximating mesh to the sample displacement field. Registration accuracy is evaluated using phantom data and similar registration errors are achieved with (0.19 mm) and without (0.16 mm) gaps in the data. Experimental results show that registration is crucial in spatial compounding of different sweeps. The execution time of the method on moderate hardware is sufficiently fast for fairly large research studies.     

Rigid registration of 3-D ultrasound with MR images: a new approachcombining intensity and gradient information

We present a new image-based technique to rigidly register intraoperative three-dimensional ultrasound (US) with preoperative magnetic resonance (MR) images. Automatic registration is achieved by maximization of a similarity measure which generalizes the correlation ratio, and whose novelty is to incorporate multivariate information from the MR data (intensity and gradient). In addition, the similarity measure is built upon a robust intensity-based distance measure, which makes it possible to handle a variety of US artifacts. A cross-validation study has been carried out using a number of phantom and clinical data. This indicates that the method is quite robust and that the worst registration errors are of the order of the MR image resolution.     

A robust method for registration of three-dimensional knee implant models to two-dimensional fluoroscopy images

A method was developed for registering three-dimensional knee implant models to single plane X-ray fluoroscopy images. We use a direct image-to-image similarity measure, taking advantage of the speed of modern computer graphics workstations to quickly render simulated (predicted) images. As a result, the method does not require an accurate segmentation of the implant silhouette in the image (which can be prone to errors). A robust optimization algorithm (simulated annealing) is used that can escape local minima and find the global minimum (true solution). Although we focus on the analysis of total knee arthroplasty (TKA) in this paper, the method can be (and has been) applied to other implanted joints, including, but not limited to, hips, ankles, and temporomandibular joints. Convergence tests on an in vivo image show that the registration method can reliably find poses that are very close to the optimal (i.e., within 0.4 degrees and 0.1 mm), even from starting poses with large initial errors. However, the precision of translation measurement in the Z (out-of-plane) direction is not as good. We also show that the method is robust with respect to image noise and occlusions. However, a small amount of user supervision and intervention is necessary to detect cases when the optimization algorithm falls into a local minimum. Intervention is required less than 5% of the time when the initial starting pose is reasonably close to the correct answer, but up to 50% of the time when the initial starting pose is far away. Finally, extensive evaluations were performed on cadaver images to determine accuracy of relative pose measurement. Comparing against data derived from an optical sensor as a "gold standard," the overall root-mean-square error of the registration method was approximately 1.5 degrees and 0.65 mm (although Z translation error was higher). However, uncertainty in the optical sensor data may account for a large part of the observed error.