Objective quantification of the motion of soft tissues in the orbit

Orbital soft-tissue motion analysis aids in the localization and diagnosis of orbital disorders. A technique has been developed to objectively quantify and visualize motion in the orbit during gaze. T1-weighted MR volume sequences are acquired during gaze and soft-tissue motion is quantified using optical flow techniques. The flow field is visualized using color-coding: orientation of the flow vector is coded by hue and magnitude by saturation of the pixel. Current clinical circumstances limit MR image acquisition to short sequences and short acquisition times. The effect of these limitations on the performance of optical flow computation has been studied for four representative optical flow algorithms: on short (nine frames) and long (21 frames) simulated sequences of rotation of a magnetic resonance (MR) imaged object, on short measured MR sequences of controlled rotation of the same object and on short MR sequences of motion in the orbit. On the short simulated and motion-controlled sequences, the Lucas and Kanade algorithm showed the best performance with respect to both accuracy and robustness. These motion estimates were accurate to within 20%. Motion in the orbit ranged between 0.05 and 0.25 mm/degree gaze. Color-coding was found to be attractive as a visualization technique, because it shows both magnitude and orientation of all flow vectors without cluttering.     

Three-dimensional surface reconstruction using optical flow formedical imaging

The recovery of a three-dimensional (3-D) model from a sequence of two-dimensional (2-D) images is very useful in medical image analysis. Image sequences obtained from the relative motion between the object and the camera or the scanner contain more 3-D information than a single image. Methods to visualize the computed tomograms can be divided into two approaches: the surface rendering approach and the volume rendering approach. In this paper, a new surface rendering method using optical flow is proposed. Optical flow is the apparent motion in the image plane produced by the projection of real 3-D motion onto the 2-D image. The 3-D motion of an object can be recovered from the optical-flow field using additional constraints. By extracting the surface information from 3-D motion, it is possible to obtain an accurate 3-D model of the object. Both synthetic and real image sequences have been used to illustrate the feasibility of the proposed method. The experimental results suggest that the proposed method is suitable for the reconstruction of 3-D models from ultrasound medical images as well as other computed tomograms     

Real-time multimodal medical image processing: a dynamicvolume-rendering application

A system for medical image processing has been proposed, which allows multimodal dynamic three-dimensional (3-D) visualization interactively and in real time. The system has been conceived to support medical specialists in the diagnosis of moving organs, such as the heart during the cardiac cycle, allowing them to compare information on perfusion/contraction match as a basis for diagnosis of important cardiovascular diseases. The 3-D volume rendering algorithm runs on a SIMD machine because of the great amount of data to be manipulated by always using the same operations. One of the features of the algorithm is the possibility to change, interactively, image processing and visualization parameters at any step, and to perform simple and effective image manipulations. Performance studies have demonstrated a very high global efficiency in practical situations by using typical data-volume dimensions. The system has been tested in the medical environment, by using magnetic resonance (MR) and single-photon emission-computed tomographic (SPECT) images     

Two-dimensional mesh-based mosaic representation for manipulationof video objects with occlusion

We present a two-dimensional (2-D) mesh-based mosaic representation, consisting of an object mesh and a mosaic mesh for each frame and a final mosaic image, for video objects with mildly deformable motion in the presence of self and/or object-to-object (external) occlusion. Unlike classical mosaic representations where successive frames are registered using global motion models, we map the uncovered regions in the successive frames onto the mosaic reference frame using local affine models, i.e., those of the neighboring mesh patches. The proposed method to compute this mosaic representation is tightly coupled with an occlusion adaptive 2-D mesh tracking procedure, which consist of propagating the object mesh frame to frame, and updating of both object and mosaic meshes to optimize texture mapping from the mosaic to each instance of the object. The proposed representation has been applied to video object rendering and editing, including self transfiguration, synthetic transfiguration, and 2-D augmented reality in the presence of self and/or external occlusion. We also provide an algorithm to determine the minimum number of still views needed to reconstruct a replacement mosaic which is needed for synthetic transfiguration. Experimental results are provided to demonstrate both the 2-D mesh-based mosaic synthesis and two different video object editing applications on real video sequences.     

Model evaluation and calibration for prospective respiratory motion correction in coronary MR angiography based on 3-D image registration

Image processing was used as a fundamental tool to derive motion information from magnetic resonance (MR) images, which was fed back into prospective respiratory motion correction during subsequent data acquisition to improve image quality in coronary MR angiography (CMRA) scans. This reduces motion artifacts in the images and, in addition, enables the usage of a broader gating window than commonly used today to increase the scan efficiency. The aim of the study reported in this paper was to find a suitable motion model to be used for respiratory motion correction in cardiac imaging and to develop a calibration procedure to adapt the motion model to the individual patient. At first, the performance of three motion models [one-dimensional translation in feet-head (FH) direction, three-dimensional (3-D) translation, and 3-D affine transformation] was tested in a small volunteer study. An elastic image registration algorithm was applied to 3-D MR images of the coronary vessels obtained at different respiratory levels. A strong intersubject variability was observed. The 3-D translation and affine transformation model were found to be superior over the conventional FH translation model used today. Furthermore, a new approach is presented, which utilizes a fast model-based image registration to extract motion information from time series of low-resolution 3-D MR images, which reflects the respiratory motion of the heart. The registration is based on a selectable global 3-D motion model (translation, rigid, or affine transformation). All 3-D MR images were registered with respect to end expiration. The resulting time series of model parameters were analyzed in combination with additionally acquired motion information from a diaphragmatic MR pencil-beam navigator to calibrate the respiratory motion model. To demonstrate the potential of a calibrated motion model for prospective motion correction in coronary imaging, the approach was tested in CMRA examinations in five volunteers.     

Three-dimensional segmentation and interpolation of magnetic resonance brain images

The authors propose a method for the 3-D reconstruction of the brain from anisotropic magnetic resonance imaging (MRI) brain data. The method essentially consists in two original algorithms both for segmentation and for interpolation of the MRI data. The segmentation process is performed in three steps. A gray level thresholding of the white and gray matter tissue is performed on the brain MR raw data. A global white matter segmentation is automatically performed with a global 3-D connectivity algorithm which takes into account the anisotropy of the MRI voxel. The gray matter is segmented with a local 3-D connectivity algorithm. Mathematical morphology tools are used to interpolate slices. The whole process gives an isotropic binary representation of both gray and white matter which are available for 3-D surface rendering. The power and practicality of this method have been tested on four brain datasets. The segmentation algorithm favorably compares to a manual one. The interpolation algorithm was compared to the shaped-based method both quantitatively and qualitatively.     

Automated Motion Estimation for 2-D Cine DENSE MRI

Cine displacement encoding with stimulated echoes (DENSE) is a magnetic resonance (MR) method that directly encodes tissue displacement into MR phase images. This technique has successfully interrogated many forms of tissue motion, but is most commonly used to evaluate cardiac mechanics. Currently, motion analysis from cine DENSE images requires manually delineated anatomical structures. An automated analysis would improve measurement throughput, simplify data interpretation, and potentially access important physiological information during the MR exam. In this paper, we present the first fully automated solution for the estimation of tissue motion and strain from 2-D cine DENSE data. Results using both simulated and human cardiac cine DENSE data indicate good agreement between the automated algorithm and the standard semi-manual analysis method.     

A cone-beam reconstruction algorithm for circle-plus-arc data-acquisition geometry

In cone-beam computerized tomography (CT), projections acquired with the focal spot constrained on a planar orbit cannot provide a complete set of data to reconstruct the object function exactly. There are severe distortions in the reconstructed noncentral transverse planes when the cone angle is large. In this work, a new method is proposed which can obtain a complete set of data by acquiring cone-beam projections along a circle-plus-arc orbit. A reconstruction algorithm using this circle-plus-arc orbit is developed, based on the Radon transform and Grangeat's formula. This algorithm first transforms the cone-beam projection data of an object to the first derivative of the three-dimensional (3-D) Radon transform, using Grangeat's formula, and then reconstructs the object using the inverse Radon transform. In order to reduce interpolation errors, new rebinning equations have been derived accurately, which allows one-dimensional (1-D) interpolation to be used in the rebinning process instead of 3-D interpolation. A noise-free Defrise phantom and a Poisson noise-added Shepp-Logan phantom were simulated and reconstructed for algorithm validation. The results from the computer simulation indicate that the new cone-beam data-acquisition scheme can provide a complete set of projection data and the image reconstruction algorithm can achieve exact reconstruction. Potentially, the algorithm can be applied in practice for both a standard CT gantry-based volume tomographic imaging system and a C-arm-based cone-beam tomographic imaging system, with little mechanical modification required.     

Novel eye gaze tracking techniques under natural head movement

Most available remote eye gaze trackers have two characteristics that hinder them being widely used as the important computer input devices for human computer interaction. First, they have to be calibrated for each user individually; second, they have low tolerance for head movement and require the users to hold their heads unnaturally still. In this paper, by exploiting the eye anatomy, we propose two novel solutions to allow natural head movement and minimize the calibration procedure to only one time for a new individual. The first technique is proposed to estimate the 3-D eye gaze directly. In this technique, the cornea of the eyeball is modeled as a convex mirror. Via the properties of convex mirror, a simple method is proposed to estimate the 3-D optic axis of the eye. The visual axis, which is the true 3-D gaze direction of the user, can be determined subsequently after knowing the angle deviation between the visual axis and optic axis by a simple calibration procedure. Therefore, the gaze point on an object in the scene can be obtained by simply intersecting the estimated 3-D gaze direction with the object. Different from the first technique, our second technique does not need to estimate the 3-D eye gaze directly, and the gaze point on an object is estimated from a gaze mapping function implicitly. In addition, a dynamic computational head compensation model is developed to automatically update the gaze mapping function whenever the head moves. Hence, the eye gaze can be estimated under natural head movement. Furthermore, it minimizes the calibration procedure to only one time for a new individual. The advantage of the proposed techniques over the current state of the art eye gaze trackers is that it can estimate the eye gaze of the user accurately under natural head movement, without need to perform the gaze calibration every time before using it. Our proposed methods will improve the usability of the eye gaze tracking technology, and we believe that it represents an important step for the eye tracker to be accepted as a natural computer input device.     

Frequency domain volume rendering by the wavelet X-ray transform

We describe a wavelet based X-ray rendering method in the frequency domain with a smaller time complexity than wavelet splatting. Standard Fourier volume rendering is summarized and interpolation and accuracy issues are briefly discussed. We review the implementation of the fast wavelet transform in the frequency domain. The wavelet X-ray transform is derived, and the corresponding Fourier-wavelet volume rendering algorithm (FWVR) is introduced, FWVR uses Haar or B-spline wavelets and linear or cubic spline interpolation. Various combinations are tested and compared with wavelet splatting (WS). We use medical MR and CT scan data, as well as a 3-D analytical phantom to assess the accuracy, time complexity, and memory cost of both FWVR and WS. The differences between both methods are enumerated.     

Registration and tracking to integrate X-ray and MR images in an XMR facility

We describe a registration and tracking technique to integrate cardiac X-ray images and cardiac magnetic resonance (MR) images acquired from a combined X-ray and MR interventional suite (XMR). Optical tracking is used to determine the transformation matrices relating MR image coordinates and X-ray image coordinates. Calibration of X-ray projection geometry and tracking of the X-ray C-arm and table enable three-dimensional (3-D) reconstruction of vessel centerlines and catheters from bi-plane X-ray views. We can, therefore, combine single X-ray projection images with registered projection MR images from a volume acquisition, and we can also display 3-D reconstructions of catheters within a 3-D or multi-slice MR volume. Registration errors were assessed using phantom experiments. Errors in the combined projection images (two-dimensional target registration error--TRE) were found to be 2.4 to 4.2 mm, and the errors in the integrated volume representation (3-D TRE) were found to be 4.6 to 5.1 mm. These errors are clinically acceptable for alignment of images of the great vessels and the chambers of the heart. Results are shown for two patients. The first involves overlay of a catheter used for invasive pressure measurements on an MR volume that provides anatomical context. The second involves overlay of invasive electrode catheters (including a basket catheter) on a tagged MR volume in order to relate electrophysiology to myocardial motion in a patient with an arrhythmia. Visual assessment of these results suggests the errors were of a similar magnitude to those obtained in the phantom measurements.     

Nonlinear anisotropic filtering of MRI data

In contrast to acquisition-based noise reduction methods a postprocess based on anisotropic diffusion is proposed. Extensions of this technique support 3-D and multiecho magnetic resonance imaging (MRI), incorporating higher spatial and spectral dimensions. The procedure overcomes the major drawbacks of conventional filter methods, namely the blurring of object boundaries and the suppression of fine structural details. The simplicity of the filter algorithm permits an efficient implementation, even on small workstations. The efficient noise reduction and sharpening of object boundaries are demonstrated by applying this image processing technique to 2-D and 3-D spin echo and gradient echo MR data. The potential advantages for MRI, diagnosis, and computerized analysis are discussed in detail.     

Region Segmentation in the Frequency Domain Applied to Upper Airway Real-Time Magnetic Resonance Images

We describe a method for unsupervised region segmentation of an image using its spatial frequency domain representation. The algorithm was designed to process large sequences of real-time magnetic resonance (MR) images containing the 2-D midsagittal view of a human vocal tract airway. The segmentation algorithm uses an anatomically informed object model, whose fit to the observed image data is hierarchically optimized using a gradient descent procedure. The goal of the algorithm is to automatically extract the time-varying vocal tract outline and the position of the articulators to facilitate the study of the shaping of the vocal tract during speech production.      

Motion estimation from tagged MR image sequences

A method for reconstructing motion from sequences of tagged magnetic resonance (MR) images is presented. MR tagging is used to create a spatial pattern of varying magnetization so that objects which may otherwise have constant intensity are textured, which reduces the motion ambiguity associated with the aperture problem in computer vision. To compensate for the decay of the tag pattern, a new optical flow algorithm is developed and implemented. The resulting estimated velocity field is then used to recursively update the implied motion reference map over time, thereby tracking the motion of individual particles. If a segmentation of the object is known at the time the tag pattern is created, then an object may be selectively tracked, using the estimated reference map to update the object's position as time progresses. Results are shown for both simulated and actual MR phantom data.     

Stereopsis-guided brain shift compensation

Brain deformation models have proven to be a powerful tool in compensating for soft tissue deformation during image-guided neurosurgery. The accuracy of these models can be improved by incorporating intraoperative measurements of brain motion. We have designed and implemented a passive intraoperative stereo vision system capable of estimating the three-dimensional shape of the surgical scene in near real-time. This intraoperative shape is compared with the cortical surface in the co-registered preoperative magnetic resonance (MR) volume for the estimation of the cortical motion resulting from the open cranial surgery. The estimated cortical motion is then used to guide a full brain model, which updates a preoperative MR volume. We have found that the stereo vision system is accurate to within approximately 1 mm. Based on data from two representative clinical cases, we show that stereopsis guidance improves the accuracy of brain shift compensation both at and below the cortical surface.     

Estimating Motion From MRI Data

Magnetic resonance imaging (MRI) is an ideal imaging modality to measure blood flow and tissue motion. It provides excellent contrast between soft tissues, and images can be acquired at positions and orientations freely defined by the user. From a temporal sequence of MR images, boundaries and edges of tissues can be tracked by image processing techniques. Additionally, MRI permits the source of the image signal to be manipulated. For example, temporary magnetic tags displaying a pattern of variable brightness may be placed in the object using MR saturation techniques, giving the user a known pattern to detect for motion tracking. The MRI signal is a modulated complex quantity, being derived from a rotating magnetic field in the form of an induced current. Well-defined patterns can also be introduced into the phase of the magnetization, and could be thought of as generalized tags. If the phase of each pixel is preserved during image reconstruction, relative phase shifts can be used to directly encode displacement, velocity and acceleration. New methods for modeling motion fields from MRI have now found application in cardiovascular and other soft tissue imaging. In this review, we shall describe the methods used for encoding, imaging, and modeling motion fields with MRI.     

Magnetic resonance diffractive imaging

Magnetic resonance (MR) diffractive imaging is proposed as a new approach to MR angiography. The expression of the nuclear MR signal is similar to the equation for the Fresnel diffraction of a three-dimensional (3-D) object in light or sound waves. The proposed technique offers the possibility of fast angiographic imaging and the on-line reconstruction of 3-D volumetric images using the holographic technique. Static imaging experiments using an ultra-low-field MRI system are performed to verify the feasibility of the technique. It is shown that the images focused on an arbitrary plane can be reconstructed from data scanned in two dimensions, even though blurred image data is superimposed on the image. Moreover, the 3-D image can be observed in a coherent optical imaging system. This study demonstrates the possibility of the proposed method as a fast imaging technique for MR angiography.     

Background removal of multiview images by learning shape priors.

Image-based rendering has been successfully used to display 3-D objects for many applications. A well-known example is the object movie, which is an image-based 3-D object composed of a collection of 2-D images taken from many different viewpoints of a 3-D object. In order to integrate image-based 3-D objects into a chosen scene (e.g., a panorama), one has to meet a hard challenge--to efficiently and effectively remove the background from the foreground object. This problem is referred to as multiview images (MVIs) segmentation. Another task requires MVI segmentation is image-based 3-D reconstruction using multiview images. In this paper, we propose a new method for segmenting MVI, which integrates some useful algorithms, including the well-known graph-cut image segmentation and volumetric graph-cut. The main idea is to incorporate the shape prior into the image segmentation process. The shape prior introduced into every image of the MVI is extracted from the 3-D model reconstructed by using the volumetric graph cuts algorithm. Here, the constraint obtained from the discrete medial axis is adopted to improve the reconstruction algorithm. The proposed MVI segmentation process requires only a small amount of user intervention, which is to select a subset of acceptable segmentations of the MVI after the initial segmentation process. According to our experiments, the proposed method can provide not only good MVI segmentation, but also provide acceptable 3-D reconstructed models for certain less-demanding applications.     

Bias field inconsistency correction of motion-scattered multislice MRI for improved 3D image reconstruction

A common solution to clinical MR imaging in the presence of large anatomical motion is to use fast multislice 2D studies to reduce slice acquisition time and provide clinically usable slice data. Recently, techniques have been developed which retrospectively correct large scale 3D motion between individual slices allowing the formation of a geometrically correct 3D volume from the multiple slice stacks. One challenge, however, in the final reconstruction process is the possibility of varying intensity bias in the slice data, typically due to the motion of the anatomy relative to imaging coils. As a result, slices which cover the same region of anatomy at different times may exhibit different sensitivity. This bias field inconsistency can induce artifacts in the final 3D reconstruction that can impact both clinical interpretation of key tissue boundaries and the automated analysis of the data. Here we describe a framework to estimate and correct the bias field inconsistency in each slice collectively across all motion corrupted image slices. Experiments using synthetic and clinical data show that the proposed method reduces intensity variability in tissues and improves the distinction between key tissue types.     

Video denoising, deblocking, and enhancement through separable 4-d nonlocal spatiotemporal transforms

We propose a powerful video filtering algorithm that exploits temporal and spatial redundancy characterizing natural video sequences. The algorithm implements the paradigm of nonlocal grouping and collaborative filtering, where a higher dimensional transform-domain representation of the observations is leveraged to enforce sparsity, and thus regularize the data: 3-D spatiotemporal volumes are constructed by tracking blocks along trajectories defined by the motion vectors. Mutually similar volumes are then grouped together by stacking them along an additional fourth dimension, thus producing a 4-D structure, termed group, where different types of data correlation exist along the different dimensions: local correlation along the two dimensions of the blocks, temporal correlation along the motion trajectories, and nonlocal spatial correlation (i.e., self-similarity) along the fourth dimension of the group. Collaborative filtering is then realized by transforming each group through a decorrelating 4-D separable transform and then by shrinkage and inverse transformation. In this way, the collaborative filtering provides estimates for each volume stacked in the group, which are then returned and adaptively aggregated to their original positions in the video. The proposed filtering procedure addresses several video processing applications, such as denoising, deblocking, and enhancement of both grayscale and color data. Experimental results prove the effectiveness of our method in terms of both subjective and objective visual quality, and show that it outperforms the state of the art in video denoising.