A comprehensive cardiac motion estimation framework using both untagged and 3-D tagged MR images based on nonrigid registration

In this paper, we present a novel technique based on nonrigid image registration for myocardial motion estimation using both untagged and 3-D tagged MR images. The novel aspect of our technique is its simultaneous usage of complementary information from both untagged and 3-D tagged MR images. To estimate the motion within the myocardium, we register a sequence of tagged and untagged MR images during the cardiac cycle to a set of reference tagged and untagged MR images at end-diastole. The similarity measure is spatially weighted to maximize the utility of information from both images. In addition, the proposed approach integrates a valve plane tracker and adaptive incompressibility into the framework. We have evaluated the proposed approach on 12 subjects. Our results show a clear improvement in terms of accuracy compared to approaches that use either 3-D tagged or untagged MR image information alone. The relative error compared to manually tracked landmarks is less than 15% throughout the cardiac cycle. Finally, we demonstrate the automatic analysis of cardiac function from the myocardial deformation fields.     

Exploratory analysis of the spatio-temporal deformation of the myocardium during systole from tagged MRI

Myocardial contractile function is, with perfusion, one of the main affected factors in ischemic heart diseases. In this paper, we propose an original framework based on functional data analysis for the quantitative study of spatio-temporal parameters related to the myocardial contraction mechanics. The mechanical strains in the left-ventricular (LV) myocardium are computed from tagged magnetic resonance imaging cardiac sequences. A statistical functional model of the normal contractile function of the LV is build from the study of eight examinations on healthy subjects. We show that it is possible to detect abnormal strain patterns comparatively to this model, by generating distance maps at rest and under pharmacological stress. We demonstrate the consistency of the results for the circumferential deformation parameter on healthy and pathological data sets.     

Analysis of 3-D myocardial motion in tagged MR images using nonrigid image registration

Tagged magnetic resonance imaging (MRI) is unique in its ability to noninvasively image the motion and deformation of the heart in vivo, but one of the fundamental reasons limiting its use in the clinical environment is the absence of automated tools to derive clinically useful information from tagged MR images. In this paper, we present a novel and fully automated technique based on nonrigid image registration using multilevel free-form deformations (MFFDs) for the analysis of myocardial motion using tagged MRI. The novel aspect of our technique is its integrated nature for tag localization and deformation field reconstruction using image registration and voxel based similarity measures. To extract the motion field within the myocardium during systole we register a sequence of images taken during systole to a set of reference images taken at end-diastole, maximizing the normalized mutual information between the images. We use both short-axis and long-axis images of the heart to estimate the full four-dimensional motion field within the myocardium. We also present validation results from data acquired from twelve volunteers.     

Cardiac motion and deformation recovery from MRI: a review

Magnetic resonance imaging (MRI) is a highly advanced and sophisticated imaging modality for cardiac motion tracking and analysis, capable of providing 3D analysis of global and regional cardiac function with great accuracy and reproducibility. In the past few years, numerous efforts have been devoted to cardiac motion recovery and deformation analysis from MR image sequences. Many approaches have been proposed for tracking cardiac motion and for computing deformation parameters and mechanical properties of the heart from a variety of cardiac MR imaging techniques. In this paper, an updated and critical review of cardiac motion tracking methods including major references and those proposed in the past ten years is provided. The MR imaging and analysis techniques surveyed are based on cine MRI, tagged MRI, phase contrast MRI, DENSE, and SENC. This paper can serve as a tutorial for new researchers entering the field.     

Fast tracking of cardiac motion using 3D-HARP

Magnetic resonance (MR) tagging is capable of accurate, noninvasive quantification of regional myocardial function. Routine clinical use, however, is hindered by cumbersome and time-consuming postprocessing procedures. We propose a fast, semiautomatic method for tracking three-dimensional (3-D) cardiac motion from a temporal sequence of short- and long-axis tagged MR images. The new method, called 3-D-HARmonic Phase (3D-HARP), extends the HARP approach, previously described for two-dimensional (2-D) tag analysis, to 3-D. A 3-D material mesh model is built to represent a collection of material points inside the left ventricle (LV) wall at a reference time. Harmonic phase, a material property that is time-invariant, is used to track the motion of the mesh through a cardiac cycle. Various motion-related functional properties of the myocardium, such as circumferential strain and left ventricular twist, are computed from the tracked mesh. The correlation analysis of 3D-HARP and FINDTAGS + Tag Strain(E) Analysis (TEA), which are well-established tag analysis techniques, shows that the regression coefficients of circumferential strain (E(CC)) and twist angle are r2 = 0.8605 and r2 = 0.8645, respectively. The total time required for tracking 3-D cardiac motion is approximately 10 min in a 9 timeframe tagged MRI dataset and has the potential to be much faster.     

Imaging myocardial strain

Measuring the local mechanical activity of the heart has lagged behind the measurement of electrical activity due to a lack of measurement tools. Myocardial wall motion abnormalities have been studied for years in the context of regional ischemia. Implanted beads and screws have been used to measure the mechanical activity of the heart in a few isolated regions. Over the past decade, precise and accurate methods for measuring local three-dimensional (3-D) myocardial motion with magnetic resonance imaging (MRI) have been developed using presaturation tagging patterns, velocity encoded phase maps, and displacement encoded phase maps. Concurrently, the quality of cardiac MRI images improved greatly with the use of customized receiver coils and the speed of acquisition has increased dramatically with the advent of undersampling techniques and new generations of MR machines with faster switching gradient coils. The use of these cardiac MRI techniques to produce an image of the local deformation of the heart in the form of a myocardial strain image is described. Using these images, the “mechanical activation” of the heart are defined, that is, the time of onset of contraction. A map of the mechanical activation over the heart is a direct analogy to an electrical activation map of the heart     

Segmentation of the left ventricle of the heart in 3-D+t MRI data using an optimized nonrigid temporal model

Modern medical imaging modalities provide large amounts of information in both the spatial and temporal domains and the incorporation of this information in a coherent algorithmic framework is a significant challenge. In this paper, we present a novel and intuitive approach to combine 3-D spatial and temporal (3-D + time) magnetic resonance imaging (MRI) data in an integrated segmentation algorithm to extract the myocardium of the left ventricle. A novel level-set segmentation process is developed that simultaneously delineates and tracks the boundaries of the left ventricle muscle. By encoding prior knowledge about cardiac temporal evolution in a parametric framework, an expectation-maximization algorithm optimally tracks the myocardial deformation over the cardiac cycle. The expectation step deforms the level-set function while the maximization step updates the prior temporal model parameters to perform the segmentation in a nonrigid sense.     

Myocardial kinematics from tagged MRI based on a 4-D B-spline model

Current research investigating the modeling of left ventricular dynamics for accurate clinical assessment of cardiac function is extensive. Magnetic resonance (MR) tagging is a functional imaging method which allows for encoding of a grid of signal voids on cardiac MR images, providing a mechanism for noninvasive measurement of intramural tissue deformations, in vivo. We present a novel technique of employing a four-dimensional (4-D) B-spline model which permits concurrent determination of myocardial beads and myocardial strains. The method entails fitting the knot planes of the 4-D B-spline model for fixed times to a sequence of triplets of orthogonal sets of tag surfaces for all imaged volumetric frames within the constraints of the model's spatio-temporal internal energy. From a three-dimensional (3-D) displacement field, the corresponding long and short-axis Lagrangian normal, shear, and principal strain maps are produced. As an important byproduct, the points defined by the 3-D intersections of the triplets of orthogonal tag planes, which we refer to as myocardial beads, can easily be determined by our model. Displaying the beads as a movie loop allows for the visualization of the nonrigid movement of the left ventricle in 3-D.     

Tag surface reconstruction and tracking of myocardial beads from SPAMM-MRI with parametric B-spline surfaces

Magnetic resonance imaging (MRI) is unique in its ability to noninvasively and selectively alter tissue magnetization, and create tag planes intersecting image slices. The resulting grid of signal voids allows for tracking deformations of tissues in otherwise homogeneous-signal myocardial regions. In this paper, we propose a specific spatial modulation of magnetization (SPAMM) imaging protocol together with efficient techniques for measurement of three-dimensional (3-D) motion of material points of the human heart (referred to as myocardial beads) from images collected with the SPAMM method. The techniques make use of tagged images in orthogonal views by explicitly reconstructing 3-D B-spline surface representation of tag planes (tag planes in two orthogonal orientations intersecting the short-axis (SA) image slices and tag planes in an orientation orthogonal to the short-axis tag planes intersecting long-axis (LA) image slices). The developed methods allow for viewing deformations of 3-D tag surfaces, spatial correspondence of long-axis and short-axis image slice and tag positions, as well as nonrigid movement of myocardial beads as a function of time.     

Normal and pathological NCAT image and phantom data based on physiologically realistic left ventricle finite-element models

The four-dimensional (4-D) NURBS-based cardiac-torso (NCAT) phantom, which provides a realistic model of the normal human anatomy and cardiac and respiratory motions, is used in medical imaging research to evaluate and improve imaging devices and techniques, especially dynamic cardiac applications. One limitation of the phantom is that it lacks the ability to accurately simulate altered functions of the heart that result from cardiac pathologies such as coronary artery disease (CAD). The goal of this work was to enhance the 4-D NCAT phantom by incorporating a physiologically based, finite-element (FE) mechanical model of the left ventricle (LV) to simulate both normal and abnormal cardiac motions. The geometry of the FE mechanical model was based on gated high-resolution X-ray multislice computed tomography (MSCT) data of a healthy male subject. The myocardial wall was represented as a transversely isotropic hyperelastic material, with the fiber angle varying from -90 degrees at the epicardial surface, through 0 degrees at the midwall, to 90 degrees at the endocardial surface. A time-varying elastance model was used to simulate fiber contraction, and physiological intraventricular systolic pressure-time curves were applied to simulate the cardiac motion over the entire cardiac cycle. To demonstrate the ability of the FE mechanical model to accurately simulate the normal cardiac motion as well as the abnormal motions indicative of CAD, a normal case and two pathologic cases were simulated and analyzed. In the first pathologic model, a subendocardial anterior ischemic region was defined. A second model was created with a transmural ischemic region defined in the same location. The FE-based deformations were incorporated into the 4-D NCAT cardiac model through the control points that define the cardiac structures in the phantom which were set to move according to the predictions of the mechanical model. A simulation study was performed using the FE-NCAT combination to investigate how the differences in contractile function between the subendocardial and transmural infarcts manifest themselves in myocardial Single photon emission computed tomography (SPECT) images. The normal FE model produced strain distributions that were consistent with those reported in the literature and a motion consistent with that defined in the normal 4-D NCAT beating heart model based on tagged magnetic resonance imaging (MRI) data. The addition of a subendocardial ischemic region changed the average transmural circumferential strain from a contractile value of -0.09 to a tensile value of 0.02. The addition of a transmural ischemic region changed average circumferential strain to a value of 0.13, which is consistent with data reported in the literature. Model results demonstrated differences in contractile function between subendocardial and transmural infarcts and how these differences in function are documented in simulated myocardial SPECT images produced using the 4-D NCAT phantom. Compared with the original NCAT beating heart model, the FE mechanical model produced a more accurate simulation for the cardiac motion abnormalities. Such a model, when incorporated into the 4-D NCAT phantom, has great potential for use in cardiac imaging research. With its enhanced physiologically based cardiac model, the 4-D NCAT phantom can be used to simulate realistic, predictive imaging data of a patient population with varying whole-body anatomy and with varying healthy and diseased states of the heart that will provide a known truth from which to evaluate and improve existing and emerging 4-D imaging techniques used in the diagnosis of cardiac disease.     

Imaging heart motion using harmonic phase MRI

This paper describes a new image processing technique for rapid analysis and visualization of tagged cardiac magnetic resonance (MR) images. The method is based on the use of isolated spectral peaks in spatial modulation of magnetization (SPAMM)-tagged magnetic resonance images. We call the calculated angle of the complex image corresponding to one of these peaks a harmonic phase (HARP) image and show that HARP images can be used to synthesize conventional tag lines, reconstruct displacement fields for small motions, and calculate two-dimensional (2-D) strain. The performance of this new approach is demonstrated using both real and simulated tagged MR images. Potential for use of HARP images in fast imaging techniques and three-dimensional (3-D) analyses are discussed.     

Tag and contour detection in tagged MR images of the left ventricle

Tracking magnetic resonance tags in myocardial tissue promises to be an effective tool for the assessment of myocardial motion. The authors describe a hierarchy of image processing steps which rapidly detects both the contours of the myocardial boundaries of the left ventricle and the tags within the myocardium. The method works on both short axis and long axis images containing radial and parallel tag patterns, respectively. Left ventricular boundaries are detected by first removing the tags using morphological closing and then selecting candidate edge points. The best inner and outer boundaries are found using a dynamic program that minimizes a nonlinear combination of several local cost functions. Tags are tracked by matching a template of their expected profile using a least squares estimate. Since blood pooling, contiguous and adjacent tissue, and motion artifacts sometimes cause detection errors, a graphical user interface was developed to allow user correction of anomalous points. The authors present results on several tagged images of a human. A fully automated run generally finds the endocardial boundary and the tag lines extremely well, requiring very little manual correction. The epicardial boundary sometimes requires more intervention to obtain an acceptable result. These methods are currently being used in the analysis of cardiac strain and as a basis for the analysis of alternate tag geometries.     

Neural network methods for volumetric magnetic resonance imaging ofthe human brain

Brain magnetic resonance (MR) images contain massive information requiring lengthy and complex interpretation (as in the identification of significant portions of the image), quantitative evaluation (as in the determination of the size of certain significant regions), and sophisticated interpretation (as in determining any image portions which indicate signs of lesions or of disease). In this paper we first survey the clinical and research needs for brain imaging. We present the state-of-the-art in relevant image analysis techniques. We then discuss our recent work on the use of novel artificial neural networks which have a recurrent structure to extract precise morphometric information from MRI scans of the human brain. Finally, experimental data using our novel approach is presented and suggestions are made for future research     

Accuracy of automatically determined borders in digital two-dimensional echocardiography using a cardiac phantom

Although two-dimensional echocardiography (2-D echo) is a useful technique for evaluation of global and regional left ventricular function, the main limitation is the inability to easily extract reliable and accurate quantitative information throughout all phases of the cardiac cycle. We sought to develop suitable automated techniques for the objective determination of endocardial and epicardial borders in two-dimensional echocardiographic images. To test algorithms for the automatic detection of myocardial borders we constructed a cardiac ultrasound phantom consisting of 16 echogenic annuli of known dimensions embedded in a material of low echogenicity which allowed imaging without partial volume effects. An algorithm based on Gaussian filtering followed by a difference gradient operator was applied to detect edges in the 2-D echo images of these annuli. The radii of the automatically determined inner borders were within 0.44 mm root meansquared error over a range of 15-25 mm true radius. This lower boundary for the error in our approach to automatic placement of myocardial borders in 2-D echocardiograms suggests the potential to extract more information concerning left ventricular function than is available with current techniques.     

Three-dimensional mapping of acute ischemic regions usingartificial neural networks and tagged MRI

Many methods for mapping ischemic myocardial regions by functional analysis have been suggested. However, the complicated relationship between myocardial function and perfusion, and the inherent limitations of the imaging techniques used, have led to a generally low mapping accuracy. The authors show herein, that highly accurate mapping can be obtained by combining tagged magnetic resonance imaging (MRI), three-dimensional (3-D) analysis, and artificial neural networks. Nine canine hearts with acute ischemia were studied using multiplanar tagged MRI. Twenty-four myocardial cuboids were tagged in each heart and reconstructed in 3-D at end diastole (ED) and end systole (ES). The cuboids were arranged in three slices approximately 1 cm thick and covered most of the left ventricle (LV). Transmural thickening and endocardial area strain were calculated for each cuboid. Applying a post-mortem (PM) analysis, the percent ischemia in each cuboid was estimated using monastral blue dye; the PM analysis served as a “gold standard”. An artificial neural network (ANN), designed to estimate the percent ischemia in each cuboid from the functional indexes, was then created. The ANN “learned” the function-ischemia relationship in 192 cuboids taken from eight of the hearts and was asked to estimate the percent ischemia in the 24 cuboids of the ninth heart. The process was repeated nine times, each time using a different heart as test case. The average accuracy of mapping, i.e., the accuracy with which the ANN has mapped the normal and ischemic cuboids using the functional parameters, was 87.5%±7.8 (s.d.). This accuracy was superior to the accuracy obtained by optimal thresholding of the same thickening (80.1%) and endocardial strain (76.9%) data     

Detection and visualization method of dynamic state transition for biological spatio-temporal imaging data

In the statistical analysis of functional brain imaging data, regression analysis and cross correlation analysis between time series data on each grid point have been widely used. The results can be graphically represented as an activation map on an anatomical image, but only activation signal, whose temporal pattern resembles the predefined reference function, can be detected. In the present study, we propose a fusion method comprising innovation approach in time series analysis and statistical test. Autoregressive (AR) models were fitted to time series data of each pixel for the range sufficiently before or after the state transition. Then, the remaining time series data were filtered using these AR parameters to obtain its innovation (filter output). The proposed method could extract brain neural activation as a phase transition of dynamics in the system without employing external information such as the reference function. The activation could be detected as temporal transitions of statistical test values. We evaluated this method by applying to optical imaging data obtained from the mammalian brain and the cardiac sino-atrial node (SAN), and demonstrated that our method can precisely detect spatio-temporal activation profiles in the brain or SAN.     

Anomaly3D: Video anomaly detection based on 3D-normality clusters

Abnormal behavior detection in surveillance videos is necessary for public monitoring and safety. In human-based surveillance systems, it requires continuous human attention and observation, which is a difficult task. The autonomous detection of such events is of essential significance. However, due to the scarcity of labeled data and the low occurrence probability of these events, abnormal event detection is a challenging vision problem. In this paper, we introduce a novel two-stage architecture for detecting anomalous behavior in videos. In the first stage, we propose a 3D Convolutional Autoencoder (3D-CAE) architecture to extract spatio-temporal features from normal event training videos. In 3D-CAE, the encoder and decoder architectures are based on 3D convolutions, which can learn both appearance and the motion features effectively in an unsupervised manner. In the second stage, we group the 3D spatio-temporal features into different normality clusters, and then remove the sparse clusters to represent a stronger pattern of normality. From these clusters, one-class SVM classifier is used to distinguish between normal and abnormal events based on the normality scores. Experimental results on four different benchmark datasets show significant performance improvement compared to state-of-the-art approaches while providing results in real-time.     

Spatio-temporal tracking of myocardial deformations with a 4-D B-spline model from tagged MRI

Accurate delineation of the volumetric motion of the left ventricle (LV) of the heart from tagged magnetic resonance imaging (MRI) is an important area of research. We have built a system that takes extracted tag line features from short axis (SA) and long axis (LA) image sequences as input and fits a four-dimensional (4-D) time-varying B-spline model to the data by simultaneously fitting the model knot solids to MRI frames via matching three sequences of solid knot planes to the LV tag planes for 4-D tracking. Important advantages of the model are that reconstruction of tag surfaces, three-dimensional (3-D) material point localization, as well as displacement reconstruction are all achieved in a single step. The generated 3-D displacement fields are validated with a cardiac motion simulator, and 3-D motion fields capturing in vivo deformations in a porcine model with posterolateral myocardial infarction are illustrated.     

Incorporation of a left ventricle finite element model defining infarction into the XCAT imaging phantom

The 4D extended cardiac-torso (XCAT) phantom was developed to provide a realistic and flexible model of the human anatomy and cardiac and respiratory motions for use in medical imaging research. A prior limitation to the phantom was that it did not accurately simulate altered functions of the heart that result from cardiac pathologies such as coronary artery disease (CAD). We overcame this limitation in a previous study by combining the phantom with a finite-element (FE) mechanical model of the left ventricle (LV) capable of more realistically simulating regional defects caused by ischemia. In the present work, we extend this model giving it the ability to accurately simulate motion abnormalities caused by myocardial infarction (MI), a far more complex situation in terms of altered mechanics compared with the modeling of acute ischemia. The FE model geometry is based on high resolution CT images of a normal male subject. An anterior region was defined as infarcted and the material properties and fiber distribution were altered, according to the bio-physiological properties of two types of infarction, i.e., fibrous and remodeled infarction (30% thinner wall than fibrous case). Compared with the original, surface-based 4D beating heart model of the XCAT, where regional abnormalities are modeled by simply scaling down the motion in those regions, the FE model was found to provide a more accurate representation of the abnormal motion of the LV due to the effects of fibrous infarction as well as depicting the motion of remodeled infarction. In particular, the FE models allow for the accurate depiction of dyskinetic motion. The average circumferential strain results were found to be consistent with measured dyskinetic experimental results. Combined with the 4D XCAT phantom, the FE model can be used to produce realistic multimodality sets of imaging data from a variety of patients in which the normal or abnormal cardiac function is accurately represented.     

Spatio-temporal visibility function for image sequences

The conventional visibility function of a still image is extended to a spatio-temporal visibility function for image sequences. The visibility of impairment noise is subjectively measured as a function of both spatial and temporal gradients by an A/B sequence test. Some characteristics of the spatio-temporal visibility function are presented such as approximated forms and comparisons with spatial visibility functions.<>