PET-CT image registration in the chest using free-form deformations

We have implemented and validated an algorithm for three-dimensional positron emission tomography transmission-to-computed tomography registration in the chest, using mutual information as a similarity criterion. Inherent differences in the two imaging protocols produce significant nonrigid motion between the two acquisitions. A rigid body deformation combined with localized cubic B-splines is used to capture this motion. The deformation is defined on a regular grid and is parameterized by potentially several thousand coefficients. Together with a spline-based continuous representation of images and Parzen histogram estimates, our deformation model allows closed-form expressions for the criterion and its gradient. A limited-memory quasi-Newton optimization algorithm is used in a hierarchical multiresolution framework to automatically align the images. To characterize the performance of the method, 27 scans from patients involved in routine lung cancer staging were used in a validation study. The registrations were assessed visually by two expert observers in specific anatomic locations using a split window validation technique. The visually reported errors are in the 0- to 6-mm range and the average computation time is 100 min on a moderate-performance workstation.     

Nonrigid registration using free-form deformations: application tobreast MR images

In this paper we present a new approach for the nonrigid registration of contrast-enhanced breast MRI. A hierarchical transformation model of the motion of the breast has been developed. The global motion of the breast is modeled by an affine transformation while the local breast motion is described by a free-form deformation (FFD) based on B-splines. Normalized mutual information is used as a voxel-based similarity measure which is insensitive to intensity changes as a result of the contrast enhancement. Registration is achieved by minimizing a cost function, which represents a combination of the cost associated with the smoothness of the transformation and the cost associated with the image similarity. The algorithm has been applied to the fully automated registration of three-dimensional (3-D) breast MRI in volunteers and patients. In particular, we have compared the results of the proposed nonrigid registration algorithm to those obtained using rigid and affine registration techniques. The results clearly indicate that the nonrigid registration algorithm is much better able to recover the motion and deformation of the breast than rigid or affine registration algorithms.     

A nonrigid registration framework using spatially encoded mutual information and free-form deformations

Mutual information (MI) registration including spatial information has been shown to perform better than the traditional MI measures for certain nonrigid registration tasks. In this work, we first provide new insight to problems of the MI-based registration and propose to use the spatially encoded mutual information (SEMI) to tackle these problems. To encode spatial information, we propose a hierarchical weighting scheme to differentiate the contribution of sample points to a set of entropy measures, which are associated to spatial variable values. By using free-form deformations (FFDs) as the transformation model, we can first define the spatial variable using the set of FFD control points, and then propose a local ascent optimization scheme for nonrigid SEMI registration. The proposed SEMI registration can improve the registration accuracy in the nonrigid cases where the traditional MI is challenged due to intensity distortion, contrast enhancement, or different imaging modalities. It also has a similar computation complexity to the registration using traditional MI measures, improving up to two orders of magnitude of computation time compared to the traditional schemes. We validate our algorithms using phantom brain MRI, simulated dynamic contrast enhanced mangetic resonance imaging (MRI) of the liver, and in vivo cardiac MRI. The results show that the SEMI registration significantly outperforms the traditional MI registration.     

Reconstruction of attenuation map using discrete consistency conditions

Methods of quantitative emission computed tomography require compensation for linear photon attenuation. A current trend in single-photon emission computed tomography (SPECT) and positron emission tomography (PET) is to employ transmission scanning to reconstruct the attenuation map. Such an approach, however, considerably complicates both the scanner design and the data acquisition protocol. A dramatic simplification could be made if the attenuation map could be obtained directly from the emission projections, without the use of a transmission scan. This can be done by applying the consistency conditions that enable us to identify the operator of the problem and, thus, to reconstruct the attenuation map. In this paper, we propose a new approach based on the discrete consistency conditions. One of the main advantages of the suggested method over previously used continuous conditions is that it can easily be applied in various scanning configurations, including fully three-dimensional (3-D) data acquisition protocols. Also, it provides a stable numerical implementation, allowing us to avoid the crosstalk between the attenuation map and the source function. A computationally efficient algorithm is implemented by using the QR and Cholesky decompositions. Application of the algorithm to computer-generated and experimentally measured SPECT data is considered.     

Three-dimensional tumor perfusion reconstruction using fractal interpolation functions

It has been shown that the perfusion of blood in tumor tissue can be approximated using the relative perfusion index determined from dynamic contrast-enhanced magnetic resonance imaging (DE-MRI) of the tumor blood pool. Also, it was concluded in a previous report that the blood perfusion in a two-dimensional (2-D) tumor vessel network has a fractal structure and that the evolution of the perfusion front can be characterized using invasion percolation. In this paper, the three-dimensional (3-D) tumor perfusion is reconstructed from the 2-D slices using the method of fractal interpolation functions (FIF), i.e., the piecewise self-affine fractal interpolation model (PSAFIM) and the piecewise hidden variable fractal interpolation model (PHVFIM). The fractal models are compared to classical interpolation techniques (linear, spline, polynomial) by means of determining the 2-D fractal dimension of the reconstructed slices. Using FIFs instead of classical interpolation techniques better conserves the fractal-like structure of the perfusion data. Among the two FIF methods, PHVFIM conserves the 3-D fractality better due to the cross correlation that exists between the data in the 2-D slices and the data along the reconstructed direction. The 3-D structures resulting from PHVFIM have a fractal dimension within 3%-5% of the one reported in literature for 3-D percolation. It is, thus, concluded that the reconstructed 3-D perfusion has a percolation-like scaling. As the perfusion term from bio-heat equation is possibly better described by reconstruction via fractal interpolation, a more suitable computation of the temperature field induced during hyperthermia treatments is expected.     

Three-dimensional reconstruction of live embryos using roboticmacroscope images

To determine the three-dimensional (3-D) shape of a live embryo is a technically challenging task. The authors show that reconstructions of live embryos can be done by collecting images from different viewing angles using a robotic macroscope, establishing point correspondences between these views by block matching, and using a new 3-D reconstruction algorithm that accommodates camera positioning errors. The algorithm assumes that the images are orthographic projections of the object and that the camera scaling factors are known. Point positions and camera errors are found simultaneously. Reconstructions of test objects and embryos show that meaningful reconstructions are possible only when camera positioning and alignment errors are accommodated since these errors can be substantial. Reconstructions of early-stage axolotl embryos were made from sets of 33 images. In a typical reconstruction, 781 points, each visible in at least three different views, were used to form 1511 triangles to represent the embryo surface. The resulting reconstruction had a mean radius of error of 0.27 pixels (1.1 μm). Mathematical properties of the reconstruction algorithm are identified and discussed     

Three-dimensional surface reconstruction using optical flow formedical imaging

The recovery of a three-dimensional (3-D) model from a sequence of two-dimensional (2-D) images is very useful in medical image analysis. Image sequences obtained from the relative motion between the object and the camera or the scanner contain more 3-D information than a single image. Methods to visualize the computed tomograms can be divided into two approaches: the surface rendering approach and the volume rendering approach. In this paper, a new surface rendering method using optical flow is proposed. Optical flow is the apparent motion in the image plane produced by the projection of real 3-D motion onto the 2-D image. The 3-D motion of an object can be recovered from the optical-flow field using additional constraints. By extracting the surface information from 3-D motion, it is possible to obtain an accurate 3-D model of the object. Both synthetic and real image sequences have been used to illustrate the feasibility of the proposed method. The experimental results suggest that the proposed method is suitable for the reconstruction of 3-D models from ultrasound medical images as well as other computed tomograms     

Three-dimensional reconstruction using homocentric spherical spatiotemporal image analysis

This paper presents a new algorithm for reconstructing a scene of three-dimensional structures from an image sequence. Three-dimensional reconstruction using an image sequence, called the spatiotemporal image method, is robust against image noises. But in this method, camera motion is limited to only one direction translation. Our algorithm makes allowances for camera rotation in spatiotemporal image analysis. With this technique, the whole spatiotemporal image is transformed to spherical projection and three-dimensional structures are determined robustly using the Hough transformation. We call the technique Homocentric Spherical Spatiotemporal Image (HSSI) analysis. With HSSI, it is possible to distinguish objects with a rotating camera from a longer baseline and to measure them with much greater accuracy than previously possible. This algorithm is demonstrated through simulations and experiments with real images from a translating and rotating camera, and the three-dimensional structures in a static scene are reconstructed.     

Three-dimensional biplanar reconstruction of scoliotic rib cage using the estimation of a mixture of probabilistic prior models

In this paper, we present an original method for the three-dimensional (3-D) reconstruction of the scoliotic rib cage from a planar and a conventional pair of calibrated radiographic images (postero-anterior with normal incidence and lateral). To this end, we first present a robust method for estimating the model parameters in a mixture of probabilistic principal component analyzers (PPCA). This method is based on the stochastic expectation maximization (SEM) algorithm. Parameters of this mixture model are used to constrain the 3-D biplanar reconstruction problem of scoliotic rib cage. More precisely, the proposed PPCA mixture model is exploited for dimensionality reduction and to obtain a set of probabilistic prior models associated with each detected class of pathological deformations observed on a representative training scoliotic rib cage population. By using an appropriate likelihood, for each considered class-conditional prior model, the proposed 3-D reconstruction is stated as an energy function minimization problem, which is solved with an exploration/selection algorithm. The optimal 3-D reconstruction then corresponds to the class of deformation and parameters leading to the minimal energy. This 3-D method of reconstruction has been successfully tested and validated on a database of 20 pairs of biplanar radiographic images of scoliotic patients, yielding very promising results. As an alternative to computed tomography-scan 3-D reconstruction this scheme has the advantage of low radiation for the patient, and may also be used for diagnosis and evaluation of deformity of a scoliotic rib cage. The proposed method remains sufficiently general to be applied to other reconstruction problems for which a database of objects to be reconstructed is available (with two or more radiographic views).     

Three-dimensional image reconstruction by digital tomo-synthesis using inverse filtering

Conventional X-ray tomosynthesis with film can provide a sagittal slice image with a single scan. This technique has the advantage of enabling reconstruction of a sagittal slice which is difficult to obtain from the X-ray CT system. However, only an image on the focal plane is obtained by a single scan. Furthermore, the image is degraded by superimpositions of the structures outside of the focal plane. A new three-dimensional image reconstruction method is proposed. This method utilizes a three-dimensional convolution process with an inverse filter function which is derived analytically by the point spread function of the projection and backprojection geometry. A digital tomosynthesis system has also been constructed for the purpose of evaluating the proposed method. This system was used in phantom experiments and clinical evaluations, and it was confirmed that the proposed method was able to reconstruct a better three-dimensional image with less artifacts from outside of the focused slice.     

Acceleration of maximum likelihood reconstruction, using frequency amplification and attenuation compensation

Algorithms that calculate maximum likelihood (ML) and maximum a posteriori solutions using expectation-maximization have been successfully applied to SPECT and PET. These algorithms are appealing because of their solid theoretical basis and their guaranteed convergence. A major drawback is the slow convergence, which results in-long processing times. The authors present 2 new heuristic acceleration methods for maximum likelihood reconstruction of ECT images. The first method incorporates a frequency-dependent amplification in the calculations, to compensate for the low pass filtering of the backprojection operation. In the second method, an amplification factor is incorporated that suppresses the effect of attenuation on the updating factors. Both methods are compared to the 1-dimensional line search method proposed by Lewitt. All 3 methods accelerate the ML algorithm. On the authors' test images, Lewitt's method produced the strongest acceleration of the three individual methods. However, the combination of the frequency amplification with the line search method results in a new algorithm with still better performance. Under certain conditions, an effective frequency amplification can be already achieved by skipping some of the calculations required for ML.     

Three-dimensional myocardial strain reconstruction from tagged MRI using a cylindrical B-spline model

In this paper, we present a new method for reconstructing three-dimensional (3-D) left ventricular myocardial strain from tagged magnetic resonance (MR) image data with a 3-D B-spline deformation model. The B-spline model is based on a cylindrical coordinate system that more closely fits the morphology of the myocardium than previously proposed Cartesian B-spline models and does not require explicit regularization. Our reconstruction method first fits a spatial coordinate B-spline displacement field to the tag line data. This displacement field maps each tag line point in the deformed myocardium back to its reference position (end-diastole). The spatial coordinate displacement field is then converted to material coordinates with another B-spline fit. Finally, strain is computed by analytically differentiating the material coordinate B-spline displacement field with respect to space. We tested our method with strains reconstructed from an analytically defined mathematical left ventricular deformation model and ten human imaging studies. Our results demonstrate that a quadratic cylindrical B-spline with a fixed number of control points can accurately fit a physiologically realistic range of deformations. The average 3-D reconstruction computation time is 20 seconds per time frame on a 450 MHz Sun Ultra80 workstation.     

Three-dimensional facial model reconstruction and plastic surgerysimulation

Facial model reconstruction and surgical simulation are essential to plastic surgery in today's medicine. Both can help surgeons to design appropriate repair plans and procedures prior to actual surgery. In this paper, we exploit a metamorphosis technique in our new design. First, using metamorphosis and vision techniques, we can establish 3D facial models from a given photograph. Second, we design several morphing operators, including augmentation, cutting and lacerating. Experiments show that the proposed algorithms can successfully create acceptable facial models and can generate realistic visual effects of surgical simulation     

Three-dimensional (3-D) reconstruction of the spine from a single X-ray image and prior vertebra models

The lateral bending test is routinely used by clinicians for the preoperative assessment of spinal mobility. The evaluation of bending motion is usually based on the qualitative analysis of a two-dimensional (2-D) antero-posterior X-ray image. The aim of this paper is to introduce a novel three-dimensional (3-D) reconstruction technique that is a prerequisite for the quantitative 3-D analysis of lateral bending motion. An algorithm was developed for the 3-D reconstruction of the spine from a single X-ray image. The X-ray is calibrated using a small calibration object and an explicit calibration algorithm. The information contained in the single X-ray is completed by registering a priori 3-D geometric models of individual vertebrae. Part of the error yielded by the 3-D/2-D registration is corrected by a vertebral alignment constraint that aims to minimize intervertebral dislocations. Three-dimensional models of 15 different scoliosis patients, obtained from a standard stereo-radiographic 3-D reconstruction, were used in simulation and validation experiments. Experimental results show that the new method is robust and accurate. With pessimistic levels of simulated noise, the average root mean square reconstruction error is 2.89 mm, which is appropriate for common clinical applications.     

Three-dimensional reconstruction of the digestive wall in capsule endoscopy videos using elastic video interpolation

Wireless capsule endoscopy is a revolutionary technology that allows physicians to examine the digestive tract of a human body in the minimum invasive way. Physicians can detect diseases such as blood-based abnormalities, polyps, ulcers, and Crohn's disease. Although this technology is really a marvel of our modern times, currently it suffers from two serious drawbacks: 1) frame rate is low (3 frames/s) and 2) no 3-D representation of the objects is captured from the camera of the capsule. In this paper we offer solutions (methodologies) that deal with each of the above issues improving the current technology without forcing hardware upgrades. These methodologies work synergistically to create smooth and visually friendly interpolated images from consecutive frames, while preserving the structure of the observed objects. They also extract and represent the texture of the surface of the digestive tract in 3-D. Thus the purpose of our methodology is not to reduce the time that the gastroenterologists need to spend to examine the video. On the contrary, the purpose is to enhance the video and therefore improve the viewing of the digestive tract leading to a more qualitative and efficient examination. The proposed work introduces 3-D capsule endoscopy textured results that have been welcomed by Digestive Specialists, Inc., Dayton, OH. Finally, illustrative results are given at the end of the paper.     

Simultaneous reconstruction of activity and attenuation for PET/MR

Medical investigations targeting a quantitative analysis of the position emission tomography (PET) images require the incorporation of additional knowledge about the photon attenuation distribution in the patient. Today, energy range adapted attenuation maps derived from computer tomography (CT) scans are used to effectively compensate for image quality degrading effects, such as attenuation and scatter. Replacing CT by magnetic resonance (MR) is considered as the next evolutionary step in the field of hybrid imaging systems. However, unlike CT, MR does not measure the photon attenuation and thus does not provide an easy access to this valuable information. Hence, many research groups currently investigate different technologies for MR-based attenuation correction (MR-AC). Typically, these approaches are based on techniques such as special acquisition sequences (alone or in combination with subsequent image processing), anatomical atlas registration, or pattern recognition techniques using a data base of MR and corresponding CT images. We propose a generic iterative reconstruction approach to simultaneously estimate the local tracer concentration and the attenuation distribution using the segmented MR image as anatomical reference. Instead of applying predefined attenuation values to specific anatomical regions or tissue types, the gamma attenuation at 511 keV is determined from the PET emission data. In particular, our approach uses a maximum-likelihood estimation for the activity and a gradient-ascent based algorithm for the attenuation distribution. The adverse effects of scattered and accidental gamma coincidences on the quantitative accuracy of PET, as well as artifacts caused by the inherent crosstalk between activity and attenuation estimation are efficiently reduced using enhanced decay event localization provided by time-of-flight PET, accurate correction for accidental coincidences, and a reduced number of unknown attenuation coefficients. First results achieved with measured whole body PET data and reference segmentation from CT showed an absolute mean difference of 0.005 cm?1 (< 20%) in the lungs, 0.0009 cm?1 (< 2%) in case of fat, and 0.0015 cm?1 (< 2%) for muscles and blood. The proposed method indicates a robust and reliable alternative to other MR-AC approaches targeting patient specific quantitative analysis in time-of-flight PET/MR.     

结合小波域马尔可夫树模型的压缩采样图像重建

已有的研究表明基于模型的压缩采样信号重建可以取得更好的重建效果。本文提出一种结合小波域马尔可夫树模型的压缩采样图像重建方法。马尔可夫树模型很好的匹配了图像小波变换后的系数在尺度间的持续性。这种统计特性可以在正交匹配追踪算法中协助原子的选取,从而更准确的选取具有大幅值系数的原子。在本文提出的新算法中,每次迭代新增的原子是从与残差信号较匹配的候选原子中选取。候选原子中使模型的状态似然函数最大的原子被选出。实验结果表明,新算法可以更准确选出具有大系数原子,重建的图像质量好于其它传统方法。     

Bayesian image reconstruction in SPECT using higher order mechanical models as priors

While the ML-EM algorithm for reconstruction for emission tomography is unstable due to the ill-posed nature of the problem. Bayesian reconstruction methods overcome this instability by introducing prior information, often in the form of a spatial smoothness regularizer. More elaborate forms of smoothness constraints may be used to extend the role of the prior beyond that of a stabilizer in order to capture actual spatial information about the object. Previously proposed forms of such prior distributions were based on the assumption of a piecewise constant source distribution. Here, the authors propose an extension to a piecewise linear model-the weak plate-which is more expressive than the piecewise constant model. The weak plate prior not only preserves edges but also allows for piecewise ramplike regions in the reconstruction. Indeed, for the authors' application in SPECT, such ramplike regions are observed in ground-truth source distributions in the form of primate autoradiographs of rCBF radionuclides. To incorporate the weak plate prior in a MAP approach, the authors model the prior as a Gibbs distribution and use a GEM formulation for the optimization. They compare quantitative performance of the ML-EM algorithm, a GEM algorithm with a prior favoring piecewise constant regions, and a GEM algorithm with their weak plate prior. Pointwise and regional bias and variance of ensemble image reconstructions are used as indications of image quality. The authors' results show that the weak plate and membrane priors exhibit improved bias and variance relative to ML-EM techniques.     

Three-dimensional surface reconstruction and fluorescent visualization of cardiac activation

Optical imaging of transmembrane potentials in cardiac tissue is a rapidly growing technique in cardiac electrophysiology. Traditional studies typically use a monocular imaging setup, thus limiting investigation to a restricted region of tissue. However, studies of large-scale wavefront dynamics, especially those during fibrillation and defibrillation, would benefit from visualization of the entire epicardial surface. To solve this problem, a panoramic cardiac visualization algorithm was developed which performs the two tasks of reconstruction of the surface geometry of the heart, and representation of the panoramic fluorescence information as a texture mapping onto the geometry that was previously created. This system permits measurement of epicardial electrodynamics over a geometrically realistic representation of the actual heart being studied. To verify the accuracy of the algorithm, the procedure was applied to synthetic images of a patterned ball; further verification was provided by application of the algorithm to a model heart placed in the experimental setup. Both sets of images produced mean registration image errors on the order of 2 pixels, corresponding to roughly 3 mm on the geometry. We demonstrate the algorithm by visualizing epicardial wavefronts on an isolated, perfused rabbit heart.