Automatic construction of multiple-object three-dimensional statistical shape models: application to cardiac modeling

A novel method is introduced for the generation of landmarks for three-dimensional (3-D) shapes and the construction of the corresponding 3-D statistical shape models. Automatic landmarking of a set of manual segmentations from a class of shapes is achieved by 1) construction of an atlas of the class, 2) automatic extraction of the landmarks from the atlas, and 3) subsequent propagation of these landmarks to each example shape via a volumetric nonrigid registration technique using multiresolution B-spline deformations. This approach presents some advantages over previously published methods: it can treat multiple-part structures and requires less restrictive assumptions on the structure's topology. In this paper, we address the problem of building a 3-D statistical shape model of the left and right ventricle of the heart from 3-D magnetic resonance images. The average accuracy in landmark propagation is shown to be below 2.2 mm. This application demonstrates the robustness and accuracy of the method in the presence of large shape variability and multiple objects.     

A statistical parts-based model of anatomical variability

In this paper, we present a statistical parts-based model (PBM) of appearance, applied to the problem of modeling intersubject anatomical variability in magnetic resonance (MR) brain images. In contrast to global image models such as the active appearance model (AAM), the PBM consists of a collection of localized image regions, referred to as parts, whose appearance, geometry and occurrence frequency are quantified statistically. The parts-based approach explicitly addresses the case where one-to-one correspondence does not exist between all subjects in a population due to anatomical differences, as model parts are not required to appear in all subjects. The model is constructed through a fully automatic machine learning algorithm, identifying image patterns that appear with statistical regularity in a large collection of subject images. Parts are represented by generic scale-invariant features, and the model can, therefore, be applied to a wide variety of image domains. Experimentation based on 2-D MR slices shows that a PBM learned from a set of 102 subjects can be robustly fit to 50 new subjects with accuracy comparable to 3 human raters. Additionally, it is shown that unlike global models such as the AAM, PBM fitting is stable in the presence of unexpected, local perturbation.     

Automatic construction of parts+geometry models for initializing groupwise registration

Groupwise nonrigid image registration is a powerful tool to automatically establish correspondences across sets of images. Such correspondences are widely used for constructing statistical models of shape and appearance. As existing techniques usually treat registration as an optimization problem, a good initialization is required. Although the standard initialization-affine transformation-generally works well, it is often inadequate when registering images of complex structures. In this paper we present a more sophisticated method that uses the sparse matches of a parts+geometry model as the initialization. We show that both the model and its matches can be automatically obtained, and that the matches are able to effectively initialize a groupwise nonrigid registration algorithm, leading to accurate dense correspondences. We also show that the dense mesh models constructed during the groupwise registration process can be used to accurately annotate new images. We demonstrate the efficacy of the approach on three datasets of increasing difficulty, and report on a detailed quantitative evaluation of its performance.     

Biomechanical modeling of the human head for physically based, nonrigid image registration

The accuracy of image-guided neurosurgery generally suffers from brain deformations due to intraoperative changes. These deformations cause significant changes of the anatomical geometry (organ shape and spatial interorgan relations), thus making intraoperative navigation based on preoperative images error prone. In order to improve the navigation accuracy, we developed a biomechanical model of the human head based on the finite element method, which can be employed for the correction of preoperative images to cope with the deformations occurring during surgical interventions. At the current stage of development, the two-dimensional (2-D) implementation of the model comprises two different materials, though the theory holds for the three-dimensional (3-D) case and is capable of dealing with an arbitrary number of different materials. For the correction of a preoperative image, a set of homologous landmarks must be specified which determine correspondences. These correspondences can be easily integrated into the model and are maintained throughout the computation of the deformation of the preoperative image. The necessary material parameter values have been determined through a comprehensive literature study. Our approach has been tested for the case of synthetic images and yields physically plausible deformation results. Additionally, we carried out registration experiments with a preoperative MR image of the human head and a corresponding postoperative image simulating an intraoperative image. We found that our approach yields good prediction results, even in the case when correspondences are given in a relatively small area of the image only.     

A framework for predictive modeling of anatomical deformations

A framework for modeling and predicting anatomical deformations is presented, and tested on simulated images. Although a variety of deformations can be modeled in this framework, emphasis is placed on surgical planning, and particularly on modeling and predicting changes of anatomy between preoperative and intraoperative positions, as well as on deformations induced by tumor growth. Two methods are examined. The first is purely shape-based and utilizes the principal modes of co-variation between anatomy and deformation in order to statistically represent deformability. When a patient's anatomy is available, it is used in conjunction with the statistical model to predict the way in which the anatomy will/can deform. The second method is related, and it uses the statistical model in conjunction with a biomechanical model of anatomical deformation. It examines the principal modes of co-variation between shape and forces, with the latter driving the biomechanical model, and thus predicting deformation. Results are shown on simulated images, demonstrating that systematic deformations, such as those resulting from change in position or from tumor growth, can be estimated very well using these models. Estimation accuracy will depend on the application, and particularly on how systematic a deformation of interest is.     

A comprehensive cardiac motion estimation framework using both untagged and 3-D tagged MR images based on nonrigid registration

In this paper, we present a novel technique based on nonrigid image registration for myocardial motion estimation using both untagged and 3-D tagged MR images. The novel aspect of our technique is its simultaneous usage of complementary information from both untagged and 3-D tagged MR images. To estimate the motion within the myocardium, we register a sequence of tagged and untagged MR images during the cardiac cycle to a set of reference tagged and untagged MR images at end-diastole. The similarity measure is spatially weighted to maximize the utility of information from both images. In addition, the proposed approach integrates a valve plane tracker and adaptive incompressibility into the framework. We have evaluated the proposed approach on 12 subjects. Our results show a clear improvement in terms of accuracy compared to approaches that use either 3-D tagged or untagged MR image information alone. The relative error compared to manually tracked landmarks is less than 15% throughout the cardiac cycle. Finally, we demonstrate the automatic analysis of cardiac function from the myocardial deformation fields.     

Validation of nonrigid image registration using finite-element methods: application to breast MR images

This paper presents a novel method for validation of nonrigid medical image registration. This method is based on the simulation of physically plausible, biomechanical tissue deformations using finite-element methods. Applying a range of displacements to finite-element models of different patient anatomies generates model solutions which simulate gold standard deformations. From these solutions, deformed images are generated with a range of deformations typical of those likely to occur in vivo. The registration accuracy with respect to the finite-element simulations is quantified by co-registering the deformed images with the original images and comparing the recovered voxel displacements with the biomechanically simulated ones. The functionality of the validation method is demonstrated for a previously described nonrigid image registration technique based on free-form deformations using B-splines and normalized mutual information as a voxel similarity measure, with an application to contrast-enhanced magnetic resonance mammography image pairs. The exemplar nonrigid registration technique is shown to be of subvoxel accuracy on average for this particular application. The validation method presented here is an important step toward more generic simulations of biomechanically plausible tissue deformations and quantification of tissue motion recovery using nonrigid image registration. It will provide a basis for improving and comparing different nonrigid registration techniques for a diversity of medical applications, such as intrasubject tissue deformation or motion correction in the brain, liver or heart.     

Parameter distribution models for estimation of population based left ventricular deformation using sparse fiducial markers

We present a method to estimate left ventricular (LV) motion based on three-dimensional (3-D) images that can be derived from any anatomical tomographic or 3-D modality, such as echocardiography, computed tomography, or magnetic resonance imaging. A finite element mesh of the LV was constructed to fit the geometry of the wall. The mesh was deformed by optimizing the nodal parameters to the motion of a sparse number of fiducial markers that were manually tracked in the images through the cardiac cycle. A parameter distribution model (PDM) of LV deformations was obtained from a database of MR tagging studies. This was used to filter the calculated deformation and incorporate a priori information on likely motions. The estimated deformation obtained from 13 normal untagged studies was compared with the deformation obtained from MR tagging. The end systolic (ES) circumferential and longitudinal strain values matched well with a mean difference of 0.1 +/- 3.2% and 0.3 +/- 3.0%, respectively. The calculated apex-base twist angle at ES had a mean difference of 1.0 +/- 2.3 degrees. We conclude that fiducial marker fitting in conjunction with a PDM provides accurate reconstruction of LV deformation in normal subjects.     

Robust Gradient-Based 3-D/2-D Registration of CT and MR to X-Ray Images

One of the most important technical challenges in image-guided intervention is to obtain a precise transformation between the intrainterventional patient's anatomy and corresponding preinterventional 3-D image on which the intervention was planned. This goal can be achieved by acquiring intrainterventional 2-D images and matching them to the preinterventional 3-D image via 3-D/2-D image registration. A novel 3-D/2-D registration method is proposed in this paper. The method is based on robustly matching 3-D preinterventional image gradients and coarsely reconstructed 3-D gradients from the intrainterventional 2-D images. To improve the robustness of finding the correspondences between the two sets of gradients, hypothetical correspondences are searched for along normals to anatomical structures in 3-D images, while the final correspondences are established in an iterative process, combining the robust random sample consensus algorithm (RANSAC) and a special gradient matching criterion function. The proposed method was evaluated using the publicly available standardized evaluation methodology for 3-D/2-D registration, consisting of 3-D rotational X-ray, computed tomography, magnetic resonance (MR), and 2-D X-ray images of two spine segments, and standardized evaluation criteria. In this way, the proposed method could be objectively compared to the intensity, gradient, and reconstruction-based registration methods. The obtained results indicate that the proposed method performs favorably both in terms of registration accuracy and robustness. The method is especially superior when just a few X-ray images and when MR preinterventional images are used for registration, which are important advantages for many clinical applications.      

A study of the motion and deformation of the heart due to respiration

This paper describes a quantitative assessment of respiratory motion of the heart and the construction of a model of respiratory motion. Three-dimensional magnetic resonance scans were acquired on eight normal volunteers and ten patients. The volunteers were imaged at multiple positions in the breathing cycle between full exhalation and full inhalation while holding their breath. The exhalation volume was segmented and used as a template to which the other volumes were registered using an intensity-based rigid registration algorithm followed by nonrigid registration. The patients were imaged at inhale and exhale only. The registration results were validated by visual assessment and consistency measurements indicating subvoxel registration accuracy. For all subjects, we assessed the nonrigid motion of the heart at the right coronary artery, right atrium, and left ventricle. We show that the rigid-body motion of the heart is primarily in the craniocaudal direction with smaller displacements in the right-left and anterior-posterior directions; this is in agreement with previous studies. Deformation was greatest for the free wall of the right atrium and the left ventricle; typical deformations were 3-4 mm with deformations of up to 7 mm observed in some subjects. Using the registration results, landmarks on the template surface were mapped to their correct positions through the breathing cycle. Principal component analysis produced a statistical model of the motion and deformation of the heart. We discuss how this model could be used to assist motion correction.     

Three-dimensional anatomical model-based segmentation of MR brainimages through principal axes registration

Model-based segmentation and analysis of brain images depends on anatomical knowledge which may be derived from conventional atlases. Classical anatomical atlases are based on the rigid spatial distribution provided by a single cadaver. Their use to segment internal anatomical brain structures in a high-resolution MR brain image does not provide any knowledge about the subject variability, and therefore they are not very efficient in analysis. The authors present a method to develop three-dimensional computerized composite models of brain structures to build a computerized anatomical atlas. The composite models are developed using the real MR brain images of human subjects which are registered through the principal axes transformation. The composite models provide probabilistic spatial distributions, which represent the variability of brain structures and can be easily updated for additional subjects. The authors demonstrate the use of such a composite model of ventricular structure to help segmentation of the ventricles and cerebrospinal fluid of MR brain images. Here, a composite model of ventricles using a set of 22 human subjects is developed and used in a model-based segmentation of ventricles, sulci, and white matter lesions. To illustrate the clinical usefulness, automatic volumetric measurements on ventricular size and cortical atrophy for an additional eight alcoholics and 10 normal subjects were made. The volumetric quantitative results indicated regional brain atrophy in chronic alcoholics     

A surface-based technique for warping three-dimensional images of the brain

The authors have devised, implemented, and tested a fast, spatially accurate technique for calculating the high-dimensional deformation field relating the brain anatomies of an arbitrary pair of subjects. The resulting three-dimensional (3-D) deformation map can be used to quantify anatomic differences between subjects or within the same subject over time and to transfer functional information between subjects or integrate that information on a single anatomic template. The new procedure is based on developmental processes responsible for variations in normal human anatomy and is applicable to 3-D brain images in general, regardless of modality. Hybrid surface models known as Chen surfaces (based on superquadrics and spherical harmonics) are used to efficiently initialize 3-D active surfaces, and these then extract from both scans the developmentally fundamental surfaces of the ventricles and cortex. The construction of extremely complex surface deformation maps on the internal cortex is made easier by building a generic surface structure to model it. Connected systems of parametric meshes model several deep sulci whose trajectories represent critical functional boundaries. These sulci are sufficiently extended inside the brain to reflect subtle and distributed variations in neuroanatomy between subjects. The algorithm then calculates the high-dimensional volumetric warp (typically with 3842x256x3 approximately 0.1 billion degrees of freedom) deforming one 3-D scan into structural correspondence with the other. Integral distortion functions are used to extend the deformation field required to elastically transform nested surfaces to their counterparts in the target scan. The algorithm's accuracy is tested, by warping 3-D magnetic resonance imaging (MRI) volumes from normal subjects and Alzheimer's patients, and by warping full-color 1024(3 ) digital cryosection volumes of the human head onto MRI volumes. Applications are discussed, including the transfer of multisubject 3-D functional, vascular, and histologic maps onto a single anatomic template; the mapping of 3-D brain atlases onto the scans of new subjects; and the rapid detection, quantification, and mapping of local shape changes in 3-D medical images in disease and during normal or abnormal growth and development.     

A new validation method for X-ray mammogram registration algorithms using a projection model of breast X-ray compression

Establishing spatial correspondence between features visible in X-ray mammograms obtained at different times has great potential to aid assessment and quantitation of change in the breast indicative of malignancy. The literature contains numerous nonrigid registration algorithms developed for this purpose, but existing approaches are flawed by the assumption of inappropriate 2-D transformation models and quantitative estimation of registration accuracy is limited. In this paper, we describe a novel validation method which simulates plausible mammographic compressions of the breast using a magnetic resonance imaging (MRI) derived finite element model. By projecting the resulting known 3-D displacements into 2-D and generating pseudo-mammograms from these same compressed magnetic resonance (MR) volumes, we can generate convincing images with known 2-D displacements with which to validate a registration algorithm. We illustrate this approach by computing the accuracy for two conventional nonrigid 2-D registration algorithms applied to mammographic test images generated from three patient MR datasets. We show that the accuracy of these algorithms is close to the best achievable using a 2-D one-to-one correspondence model but that new algorithms incorporating more representative transformation models are required to achieve sufficiently accurate registrations for this application.     

Brain surface conformal parameterization using Riemann surface structure

In medical imaging, parameterized 3-D surface models are useful for anatomical modeling and visualization, statistical comparisons of anatomy, and surface-based registration and signal processing. Here we introduce a parameterization method based on Riemann surface structure, which uses a special curvilinear net structure (conformal net) to partition the surface into a set of patches that can each be conformally mapped to a parallelogram. The resulting surface subdivision and the parameterizations of the components are intrinsic and stable (their solutions tend to be smooth functions and the boundary conditions of the Dirichlet problem can be enforced). Conformal parameterization also helps transform partial differential equations (PDEs) that may be defined on 3-D brain surface manifolds to modified PDEs on a two-dimensional parameter domain. Since the Jacobian matrix of a conformal parameterization is diagonal, the modified PDE on the parameter domain is readily solved. To illustrate our techniques, we computed parameterizations for several types of anatomical surfaces in 3-D magnetic resonance imaging scans of the brain, including the cerebral cortex, hippocampi, and lateral ventricles. For surfaces that are topologically homeomorphic to each other and have similar geometrical structures, we show that the parameterization results are consistent and the subdivided surfaces can be matched to each other. Finally, we present an automatic sulcal landmark location algorithm by solving PDEs on cortical surfaces. The landmark detection results are used as constraints for building conformal maps between surfaces that also match explicitly defined landmarks.     

Left ventricular deformation recovery from cine MRI using an incompressible model

This paper presents a method for 3-D deformation recovery of the left ventricular (LV) wall from anatomical cine magnetic resonance imaging (MRI). The method is based on a deformable model that is incompressible, a desired property since the myocardium has been shown to be nearly incompressible. The LV wall needs to be segmented in an initial frame after which the method automatically determines the deformation everywhere in the LV wall throughout the cardiac cycle. Two studies were conducted to validate the method. In the first study, the deformation recovered from a 3-D anatomical cine MRI of a healthy volunteer was compared against the manual segmentation of the LV wall and against the corresponding 3-D tagged cine MRI. The average volume agreement between the model and the manual segmentation had a false positive rate of 3%, false negative rate of 3%, and true positive rate of 93%. The average distance between the model and manually determined intersections of perpendicular tag planes was 1.6 mm (1.1 pixel). Another set of 3-D anatomical and tagged MRI scans was taken of the same volunteer four months later. The method was applied to the second set and the recovered deformation was very similar to the one obtained from the first set. In the second study, the method was applied to 3-D anatomical cine MRI scans of three patients with ventricular dyssynchrony and three age-matched healthy volunteers. The LV wall deformations recovered for the three normals agreed well and the recovered strains were similar to those reported by other researchers for normal subjects. Strains and displacements of the three patients were clearly smaller than those of the three normals indicating reduced cardiac function. The deformation recovered for the three normals and the three patients was validated against manual segmentation and corresponding tag cine MRI scans and the agreement was similar to that of the first validation study.     

Brain surface conformal parameterization with the Ricci flow

In brain mapping research, parameterized 3-D surface models are of great interest for statistical comparisons of anatomy, surface-based registration, and signal processing. Here, we introduce the theories of continuous and discrete surface Ricci flow, which can create Riemannian metrics on surfaces with arbitrary topologies with user-defined Gaussian curvatures. The resulting conformal parameterizations have no singularities and they are intrinsic and stable. First, we convert a cortical surface model into a multiple boundary surface by cutting along selected anatomical landmark curves. Secondly, we conformally parameterize each cortical surface to a parameter domain with a user-designed Gaussian curvature arrangement. In the parameter domain, a shape index based on conformal invariants is computed, and inter-subject cortical surface matching is performed by solving a constrained harmonic map. We illustrate various target curvature arrangements and demonstrate the stability of the method using longitudinal data. To map statistical differences in cortical morphometry, we studied brain asymmetry in 14 healthy control subjects. We used a manifold version of Hotelling's T(2) test, applied to the Jacobian matrices of the surface parameterizations. A permutation test, along with the cumulative distribution of p-values, were used to estimate the overall statistical significance of differences. The results show our algorithm's power to detect subtle group differences in cortical surfaces.     

Statistical deformable model-based segmentation of image motion

We present a statistical method for the motion-based segmentation of deformable structures undergoing nonrigid movements. The proposed approach relies on two models describing the shape of interest, its variability, and its movement. The first model corresponds to a statistical deformable template that constrains the shape and its deformations. The second model is introduced to represent the optical flow field inside the deformable template. These two models are combined within a single probability distribution, which enables to derive shape and motion estimates using a maximum likelihood approach. The method requires no manual initialization and is demonstrated on synthetic data and on a medical X-ray image sequence.     

Consistent landmark and intensity-based image registration

Two new consistent image registration algorithms are presented: one is based on matching corresponding landmarks and the other is based on matching both landmark and intensity information. The consistent landmark and intensity registration algorithm produces good correspondences between images near landmark locations by matching corresponding landmarks and away from landmark locations by matching the image intensities. In contrast to similar unidirectional algorithms, these new consistent algorithms jointly estimate the forward and reverse transformation between two images while minimizing the inverse consistency error-the error between the forward (reverse) transformation and the inverse of the the reverse (forward) transformation. This reduces the ambiguous correspondence between the forward and reverse transformations associated with large inverse consistency errors. In both algorithms a thin-plate spline (TPS) model is used to regularize the estimated transformations. Two-dimensional (2-D) examples are presented that show the inverse consistency error produced by the traditional unidirectional landmark TPS algorithm can be relatively large and that this error is minimized using the consistent landmark algorithm. Results using 2-D magnetic resonance imaging data are presented that demonstrate that using landmark and intensity information together produce better correspondence between medical images than using either landmarks or intensity information alone.