Interactive 3D registration of ultrasound and magnetic resonanceimages based on a magnetic position sensor

The use of stereotactic systems has been one of the main approaches for image-based guidance of the surgical tool within the brain. The main limitation of stereotactic systems is that they are based on preoperative images that might become outdated and invalid during the course of surgery. Ultrasound (US) is considered the most practical and cost-effective intraoperative imaging modality, but US images inherently have a low signal-to-noise ratio. Integrating intraoperative US with stereotactic systems has recently been attempted. In this paper, we present a new system for interactively registering two-dimensional US and three-dimensional magnetic resonance (MR) images. This registration is based on tracking the US probe with a DC magnetic position sensor. We have performed an extensive analysis of the errors of our system by using a custom-built phantom. The registration error between the MR and the position sensor space was found to have a mean value of 1.78 mm and a standard deviation of 0.18 mm. The registration error between US and MR space was dependent on the distance of the target point from the US probe face. For a 3.5-MHz phased one-dimensional array transducer and a depth of 6 cm, the mean value of the registration error was 2.00 mm and the standard deviation was 0.75 mm. The registered MR images were reconstructed using either zeroth-order or first-order interpolation     

Rigid registration of 3-D ultrasound with MR images: a new approachcombining intensity and gradient information

We present a new image-based technique to rigidly register intraoperative three-dimensional ultrasound (US) with preoperative magnetic resonance (MR) images. Automatic registration is achieved by maximization of a similarity measure which generalizes the correlation ratio, and whose novelty is to incorporate multivariate information from the MR data (intensity and gradient). In addition, the similarity measure is built upon a robust intensity-based distance measure, which makes it possible to handle a variety of US artifacts. A cross-validation study has been carried out using a number of phantom and clinical data. This indicates that the method is quite robust and that the worst registration errors are of the order of the MR image resolution.     

3-D/2-D registration of CT and MR to X-ray images

A crucial part of image-guided therapy is registration of preoperative and intraoperative images, by which the precise position and orientation of the patient's anatomy is determined in three dimensions. This paper presents a novel approach to register three-dimensional (3-D) computed tomography (CT) or magnetic resonance (MR) images to one or more two-dimensional (2-D) X-ray images. The registration is based solely on the information present in 2-D and 3-D images. It does not require fiducial markers, intraoperative X-ray image segmentation, or timely construction of digitally reconstructed radiographs. The originality of the approach is in using normals to bone surfaces, preoperatively defined in 3-D MR or CT data, and gradients of intraoperative X-ray images at locations defined by the X-ray source and 3-D surface points. The registration is concerned with finding the rigid transformation of a CT or MR volume, which provides the best match between surface normals and back projected gradients, considering their amplitudes and orientations. We have thoroughly validated our registration method by using MR, CT, and X-ray images of a cadaveric lumbar spine phantom for which "gold standard" registration was established by means of fiducial markers, and its accuracy assessed by target registration error. Volumes of interest, containing single vertebrae L1-L5, were registered to different pairs of X-ray images from different starting positions, chosen randomly and uniformly around the "gold standard" position. CT/X-ray (MR/ X-ray) registration, which is fast, was successful in more than 91% (82% except for L1) of trials if started from the "gold standard" translated or rotated for less than 6 mm or 17 degrees (3 mm or 8.6 degrees), respectively. Root-mean-square target registration errors were below 0.5 mm for the CT to X-ray registration and below 1.4 mm for MR to X-ray registration.     

3-D/2-D registration by integrating 2-D information in 3-D

In image-guided therapy, high-quality preoperative images serve for planning and simulation, and intraoperatively as "background", onto which models of surgical instruments or radiation beams are projected. The link between a preoperative image and intraoperative physical space of the patient is established by image-to-patient registration. In this paper, we present a novel 3-D/2-D registration method. First, a 3-D image is reconstructed from a few 2-D X-ray images and next, the preoperative 3-D image is brought into the best possible spatial correspondence with the reconstructed image by optimizing a similarity measure (SM). Because the quality of the reconstructed image is generally low, we introduce a novel SM, which is able to cope with low image quality as well as with different imaging modalities. The novel 3-D/2-D registration method has been evaluated and compared to the gradient-based method (GBM) using standardized evaluation methodology and publicly available 3-D computed tomography (CT), 3-D rotational X-ray (3DRX), and magnetic resonance (MR) and 2-D X-ray images of two spine phantoms, for which gold standard registrations were known. For each of the 3DRX, CT, or MR images and each set of X-ray images, 1600 registrations were performed from starting positions, defined as the mean target registration error (mTRE), randomly generated and uniformly distributed in the interval of 0-20 mm around the gold standard. The capture range was defined as the distance from gold standard for which the final TRE was less than 2 mm in at least 95% of all cases. In terms of success rate, as the function of initial misalignment and capture range the proposed method outperformed the GBM. TREs of the novel method and the GBM were approximately the same. For the registration of 3DRX and CT images to X-ray images as few as 2-3 X-ray views were sufficient to obtain approximately 0.4 mm TREs, 7-9 mm capture range, and 80%-90% of successful registrations. To obtain similar results for MR to X-ray registrations, an image, reconstructed from at least 11 X-ray images was required. Reconstructions from more than 11 images had no effect on the registration results.     

Rapid Dynamic Image Registration of the Beating Heart for Diagnosis and Surgical Navigation

Dynamic cardiac magnetic resonance imaging (MR) and computed tomography (CT) provide cardiologists and cardiac surgeons with high-quality 4-D images for diagnosis and therapy, yet the effective use of these high-quality anatomical models remains a challenge. Ultrasound (US) is a flexible imaging tool, but the US images produced are often difficult to interpret unless they are placed within their proper 3-D anatomical context. The ability to correlate real-time 3-D US volumes (RT3D US) with dynamic MR/CT images would offer a significant contribution to improve the quality of cardiac procedures. In this paper, we present a rapid two-step method for registering RT3D US to high-quality dynamic 3-D MR/CT images of the beating heart. This technique overcomes some major limitations of image registration (such as the correct registration result not necessarily occurring at the maximum of the mutual information (MI) metric) using the MI metric. We demonstrate the effectiveness of our method in a dynamic heart phantom (DHP) study and a human subject study. The achieved mean target registration error of CT+US images in the phantom study is 2.59 mm. Validation using human MR/US volumes shows a target registration error of 1.76 mm. We anticipate that this technique will substantially improve the quality of cardiac diagnosis and therapies.      

Dynamic 2D Ultrasound and 3D CT Image Registration of the Beating Heart

Two-dimensional ultrasound (US) is widely used in minimally invasive cardiac procedures due to its convenience of use and noninvasive nature. However, the low quality of US images often limits their utility as a means for guiding procedures, since it is often difficult to relate the images to their anatomical context. To improve the interpretability of the US images while maintaining US as a flexible anatomical and functional real-time imaging modality, we describe a multimodality image navigation system that integrates 2D US images with their 3D context by registering them to high quality preoperative models based on magnetic resonance imaging (MRI) or computed tomography (CT) images. The mapping from such a model to the patient is completed using spatial and temporal registrations. Spatial registration is performed by a two-step rapid registration method that first approximately aligns the two images as a starting point to an automatic registration procedure. Temporal alignment is performed with the aid of electrocardiograph (ECG) signals and a latency compensation method. Registration accuracy is measured by calculating the TRE. Results show that the error between the US and preoperative images of a beating heart phantom is $1.7 pm 0.4$ mm, with a similar performance being observed in in vivo animal experiments.      

Surface-based registration of CT images to physical space forimage-guided surgery of the spine: a sensitivity study

This paper presents a method designed to register preoperative computed tomography (CT) images to vertebral surface points acquired intraoperatively from ultrasound (US) images or via a tracked probe. It also presents a comparison of the registration accuracy achievable with surface points acquired from the entire posterior surface of the vertebra to the accuracy achievable with points acquired only from the spinous process and central laminar regions. Using a marker-based method as a reference, this work shows that submillimetric registration accuracy can be obtained even when a small portion of the posterior vertebral surface is used for registration. It also shows that when selected surface patches are used, CT slice thickness is not a critical parameter in the registration process. Furthermore, the paper includes qualitative results of registering vertebral surface points in US images to multiple CT slices. The method has been tested with US points and physical points on a plastic spine phantom and with simulated data on a patient CT scan     

A "twisting and bending" model-based nonrigid image registration technique for 3-D ultrasound carotid images

Atherosclerosis at the carotid bifurcation resulting in cerebral emboli is a major cause of ischemic stroke. Most strokes associated with carotid atherosclerosis can be prevented by lifestyle/dietary changes and pharmacological treatments if identified early by monitoring carotid plaque changes. Registration of 3-D ultrasound (US) images of carotid plaque obtained at different time points is essential for sensitive monitoring of plaque changes in volume and surface morphology. This registration technique should be nonrigid, since different head positions during image acquisition sessions cause relative bending and torsion in the neck, producing nonlinear deformations between the images. We modeled the movement of the neck using a “twisting and bending” model with only six parameters for nonrigid registration. We evaluated the algorithm using 3-D US carotid images acquired at two different head positions to simulate images acquired at different times. We calculated the mean registration error (MRE) between the segmented vessel surfaces in the target image and the registered image using a distance-based error metric after applying our “twisting and bending” model-based nonrigid registration algorithm. We achieved an average registration error of $0.80 pm 0.26$ mm using our nonrigid registration technique, which was a significant improvement in registration accuracy over rigid registration, even with reduced degrees-of-freedom compared to the other nonrigid registration algorithms.      

Self-calibrating 3D-ultrasound-based bone registration for minimally invasive orthopedic surgery

Intraoperative freehand three-dimensional (3-D) ultrasound (3D-US) has been proposed as a noninvasive method for registering bones to a preoperative computed tomography image or computer-generated bone model during computer-aided orthopedic surgery (CAOS). In this technique, an US probe is tracked by a 3-D position sensor and acts as a percutaneous device for localizing the bone surface. However, variations in the acoustic properties of soft tissue, such as the average speed of sound, can introduce significant errors in the bone depth estimated from US images, which limits registration accuracy. We describe a new self-calibrating approach to US-based bone registration that addresses this problem, and demonstrate its application within a standard registration scheme. Using realistic US image data acquired from 6 femurs and 3 pelves of intact human cadavers, and accurate Gold Standard registration transformations calculated using bone-implanted fiducial markers, we show that self-calibrating registration is significantly more accurate than a standard method, yielding an average root mean squared target registration error of 1.6 mm. We conclude that self-calibrating registration results in significant improvements in registration accuracy for CAOS applications over conventional approaches where calibration parameters of the 3D-US system remain fixed to values determined using a preoperative phantom-based calibration.     

Registration of 3-D intraoperative MR images of the brain using a finite-element biomechanical model

We present a new algorithm for the nonrigid registration of three-dimensional magnetic resonance (MR) intraoperative image sequences showing brain shift. The algorithm tracks key surfaces of objects (cortical surface and the lateral ventricles) in the image sequence using a deformable surface matching algorithm. The volumetric deformation field of the objects is then inferred from the displacements at the boundary surfaces using a linear elastic biomechanical finite-element model. Two experiments on synthetic image sequences are presented, as well as an initial experiment on intraoperative MR images showing brain shift. The results of the registration algorithm show a good correlation of the internal brain structures after deformation, and a good capability of measuring surface as well as subsurface shift. We measured distances between landmarks in the deformed initial image and the corresponding landmarks in the target scan. Cortical surface shifts of up to 10 mm and subsurface shifts of up to 6 mm were recovered with an accuracy of 1 mm or less and 3 mm or less respectively.     

In vivo quantification of a homogeneous brain deformation model for updating preoperative images during surgery

Clinicians using image-guidance for neurosurgical procedures have recently recognized that intraoperative deformation from surgical loading can compromise the accuracy of patient registration in the operating room. While whole brain intraoperative imaging is conceptually appealing it presents significant practical limitations. Alternatively, a promising approach may be to combine incomplete intraoperatively acquired data with a computational model of brain deformation to update high resolution preoperative images during surgery. The success of such an approach is critically dependent on identifying a valid model of brain deformation physics. Towards this end, we evaluate a three-dimensional finite element consolidation theory model for predicting brain deformation in vivo through a series of controlled repeat-experiments. This database is used to construct an interstitial pressure boundary condition calibration curve which is prospectively tested in a fourth validation experiment. The computational model is found to recover 75%-85% of brain motion occurring under loads comparable to clinical conditions. Additionally, the updating of preoperative images using the model calculations is presented and demonstrates that model-updated image-guided neurosurgery may be a viable option for addressing registration errors related to intraoperative tissue motion.     

Registration of 3-D images using weighted geometrical features

The authors present a weighted geometrical feature (WGF) registration algorithm. Its efficacy is demonstrated by combining points and a surface. The technique is an extension of Besl and McKay's (1992) iterative closest point (ICP) algorithm. The authors use the WGF algorithm to register X-ray computed tomography (CT) and T2-weighted magnetic resonance (MR) volume head images acquired from eleven patients that underwent craniotomies in a neurosurgical clinical trial. Each patient had five external markers attached to transcutaneous posts screwed into the outer table of the skull. The authors define registration error as the distance between positions of corresponding markers that are not used for registration. The CT and MR images are registered using fiducial paints (marker positions) only, a surface only, and various weighted combinations of points and a surface. The CT surface is derived from contours corresponding to the inner surface of the skull. The MR surface is derived from contours corresponding to the cerebrospinal fluid (CSF)-dura interface. Registration using points and a surface is found to be significantly more accurate then registration using only points or a surface     

Compensating for intraoperative soft-tissue deformations using incomplete surface data and finite elements

Image-guided liver surgery requires the ability to identify and compensate for soft tissue deformation in the organ. The predeformed state is represented as a complete three-dimensional surface of the organ, while the intraoperative data is a range scan point cloud acquired from the exposed liver surface. The first step is to rigidly align the coordinate systems of the intraoperative and preoperative data. Most traditional rigid registration methods minimize an error metric over the entire data set. In this paper, a new deformation-identifying rigid registration (DIRR) is reported that identifies and aligns minimally deformed regions of the data using a modified closest point distance cost function. Once a rigid alignment has been established, deformation is accounted for using a linearly elastic finite element model (FEM) and implemented using an incremental framework to resolve geometric nonlinearities. Boundary conditions for the incremental formulation are generated from intraoperatively acquired range scan surfaces of the exposed liver surface. A series of phantom experiments is presented to assess the fidelity of the DIRR and the combined DIRR/FEM approaches separately. The DIRR approach identified deforming regions in 90% of cases under conditions of realistic surgical exposure. With respect to the DIRR/FEM algorithm, subsurface target errors were correctly located to within 4 mm in phantom experiments.     

Multiple-object 2-D-3-D registration for noninvasive pose identification of fracture fragments

This paper presents a multiple-object 2-D-3-D registration technique for noninvasively identifying the poses of fracture fragments in the space of a preoperative treatment plan. The plan is made by manipulating and aligning computer models of individual fracture fragments that are segmented from a diagnostic computed tomography. The registration technique iteratively updates the treatment plan and matches its digitally reconstructed radiographs to a small number of intraoperative fluoroscopic images. The proposed approach combines an image similarity metric that integrates edge information with mutual information, and a global-local optimization scheme, to deal with challenges associated with the registration of multiple small fragments and limited imaging orientations in the operating room. The method is easy to use as minimum user interaction is required. Experiments on simulated fractures and two distal radius fracture phantoms demonstrate clinically acceptable target registration errors with capture range as large as 10 mm.     

Robust nonrigid registration to capture brain shift from intraoperative MRI

We present a new algorithm to register 3-D preoperative magnetic resonance (MR) images to intraoperative MR images of the brain which have undergone brain shift. This algorithm relies on a robust estimation of the deformation from a sparse noisy set of measured displacements. We propose a new framework to compute the displacement field in an iterative process, allowing the solution to gradually move from an approximation formulation (minimizing the sum of a regularization term and a data error term) to an interpolation formulation (least square minimization of the data error term). An outlier rejection step is introduced in this gradual registration process using a weighted least trimmed squares approach, aiming at improving the robustness of the algorithm. We use a patient-specific model discretized with the finite element method in order to ensure a realistic mechanical behavior of the brain tissue. To meet the clinical time constraint, we parallelized the slowest step of the algorithm so that we can perform a full 3-D image registration in 35 s (including the image update time) on a heterogeneous cluster of 15 personal computers. The algorithm has been tested on six cases of brain tumor resection, presenting a brain shift of up to 14 mm. The results show a good ability to recover large displacements, and a limited decrease of accuracy near the tumor resection cavity.     

Simultaneous ultrasound and MRI system for breast biopsy: compatibility assessment and demonstration in a dual modality phantom

Simultaneous capturing of ultrasound (US) and magnetic resonance (MR) images allows fusion of information obtained from both modalities. We propose an MR-compatible US system where MR images are acquired in a known orientation with respect to the US imaging plane and concurrent real-time imaging can be achieved. Compatibility of the two imaging devices is a major issue in the physical setup. Tests were performed to quantify the radio frequency (RF) noise introduced in MR and US images, with the US system used in conjunction with MRI scanner of different field strengths (0.5 T and 3 T). Furthermore, simultaneous imaging was performed on a dual modality breast phantom in the 0.5 T open bore and 3 T close bore MRI systems to aid needle-guided breast biopsy. Fiducial based passive tracking and electromagnetic based active tracking were used in 3 T and 0.5 T, respectively, to establish the location and orientation of the US probe inside the magnet bore. Our results indicate that simultaneous US and MR imaging are feasible with properly-designed shielding, resulting in negligible broadband noise and minimal periodic RF noise in both modalities. US can be used for real time display of the needle trajectory, while MRI can be used to confirm needle placement.     

Biomechanical modeling of the human head for physically based, nonrigid image registration

The accuracy of image-guided neurosurgery generally suffers from brain deformations due to intraoperative changes. These deformations cause significant changes of the anatomical geometry (organ shape and spatial interorgan relations), thus making intraoperative navigation based on preoperative images error prone. In order to improve the navigation accuracy, we developed a biomechanical model of the human head based on the finite element method, which can be employed for the correction of preoperative images to cope with the deformations occurring during surgical interventions. At the current stage of development, the two-dimensional (2-D) implementation of the model comprises two different materials, though the theory holds for the three-dimensional (3-D) case and is capable of dealing with an arbitrary number of different materials. For the correction of a preoperative image, a set of homologous landmarks must be specified which determine correspondences. These correspondences can be easily integrated into the model and are maintained throughout the computation of the deformation of the preoperative image. The necessary material parameter values have been determined through a comprehensive literature study. Our approach has been tested for the case of synthetic images and yields physically plausible deformation results. Additionally, we carried out registration experiments with a preoperative MR image of the human head and a corresponding postoperative image simulating an intraoperative image. We found that our approach yields good prediction results, even in the case when correspondences are given in a relatively small area of the image only.     

Accounting for signal loss due to dephasing in the correction of distortions in gradient-echo EPI via nonrigid registration

Gradient-echo (GE) echo planar imaging (EPI) is susceptible to both geometric distortions and signal loss. This paper presents a retrospective correction approach based on nonrigid image registration. A new physics-based intensity correction factor derived to compensate for intravoxel dephasing in GE EPI images is incorporated into a previously reported nonrigid registration algorithm. Intravoxel dephasing causes signal loss and thus intensity attenuation in the images. The new rephasing factor we introduce, which changes the intensity of a voxel in images during the registration, is used to improve the accuracy of the intensity-based nonrigid registration method and mitigate the intensity attenuation effect. Simulation-based experiments are first used to evaluate the method. A magnetic resonance (MR) simulator and a real field map are used to generate a realistic GE EPI image. The geometric distortion computed from the field map is used as the ground truth to which the estimated nonrigid deformation is compared. We then apply the algorithm to a set of real human brain images. The results show that, after registration, alignment between EPI and multi-shot, spin-echo images, which have relatively long acquisition times but negligible distortion, is improved and that signal loss caused by dephasing can be recovered.     

Registration and tracking to integrate X-ray and MR images in an XMR facility

We describe a registration and tracking technique to integrate cardiac X-ray images and cardiac magnetic resonance (MR) images acquired from a combined X-ray and MR interventional suite (XMR). Optical tracking is used to determine the transformation matrices relating MR image coordinates and X-ray image coordinates. Calibration of X-ray projection geometry and tracking of the X-ray C-arm and table enable three-dimensional (3-D) reconstruction of vessel centerlines and catheters from bi-plane X-ray views. We can, therefore, combine single X-ray projection images with registered projection MR images from a volume acquisition, and we can also display 3-D reconstructions of catheters within a 3-D or multi-slice MR volume. Registration errors were assessed using phantom experiments. Errors in the combined projection images (two-dimensional target registration error--TRE) were found to be 2.4 to 4.2 mm, and the errors in the integrated volume representation (3-D TRE) were found to be 4.6 to 5.1 mm. These errors are clinically acceptable for alignment of images of the great vessels and the chambers of the heart. Results are shown for two patients. The first involves overlay of a catheter used for invasive pressure measurements on an MR volume that provides anatomical context. The second involves overlay of invasive electrode catheters (including a basket catheter) on a tagged MR volume in order to relate electrophysiology to myocardial motion in a patient with an arrhythmia. Visual assessment of these results suggests the errors were of a similar magnitude to those obtained in the phantom measurements.     

Displacement estimation with co-registered ultrasound for image guided neurosurgery: a quantitative in vivo porcine study

Brain shift during open cranial surgery presents a challenge for maintaining registration with image-guidance systems. Ultrasound (US) is a convenient intraoperative imaging modality that may be a useful tool in detecting tissue shift and updating preoperative images based on intraoperative measurements of brain deformation. We have quantitatively evaluated the ability of spatially tracked freehand US to detect displacement of implanted markers in a series of three in vivo porcine experiments, where both US and computed tomography (CT) image acquisitions were obtained before and after deforming the brain. Marker displacements ranged from 0.5 to 8.5 mm. Comparisons between CT and US measurements showed a mean target localization error of 1.5 mm, and a mean vector error for displacement of 1.1 mm. Mean error in the magnitude of displacement was 0.6 mm. For one of the animals studied, the US data was used in conjunction with a biomechanical model to nonrigidly re-register a baseline CT to the deformed brain. The mean error between the actual and deformed CT's was found to be on average 1.2 and 1.9 mm at the marker locations depending on the extent of the deformation induced. These findings indicate the potential accuracy in coregistered freehand US displacement tracking in brain tissue and suggest that the resulting information can be used to drive a modeling re-registration strategy to comparable levels of agreement.