Registration-assisted segmentation of real-time 3-D echocardiographic data using deformable models

Real-time three-dimensional (3-D) echocardiography is a new imaging modality that presents the unique opportunity to visualize the complex 3-D shape and motion of the left ventricle (LV) in vivo and to measure the associated global and local function parameters. To take advantage of this opportunity in routine clinical practice, automatic segmentation of the LV in the 3-D echocardiographic data, usually hundreds of megabytes large, is essential. We report a new segmentation algorithm for this task. Our algorithm has two distinct stages, initialization of a deformable model and its refinement, which are connected by a dual "voxel + wiremesh" template. In the first stage, mutual-information-based registration of the voxel template with the image to be segmented helps initialize the wiremesh template. In the second stage, the wiremesh is refined iteratively under the influence of external and internal forces. The internal forces have been customized to preserve the nonsymmetric shape of the wiremesh template in the absence of external forces, defined using the gradient vector flow approach. The algorithm was validated against expert-defined segmentation and demonstrated acceptable accuracy. Our segmentation algorithm is fully automatic and has the potential to be used clinically together with real-time 3-D echocardiography for improved cardiovascular disease diagnosis.     

Registration of real-time 3-D ultrasound images of the heart for novel 3-D stress echocardiography

Stress echocardiography is a routinely used clinical procedure to diagnose cardiac dysfunction by comparing wall motion information in prestress and poststress ultrasound images. Incomplete data, complicated imaging protocols and misaligned prestress and poststress views, however, are known limitations of conventional stress echocardiography. We discuss how the first two limitations are overcome via the use of real-time three-dimensional (3-D) ultrasound imaging, an emerging modality, and have called the new procedure "3-D stress echocardiography." We also show that the problem of misaligned views can be solved by registration of prestress and poststress 3-D image sequences. Such images are misaligned because of variations in placing the ultrasound transducer and stress-induced anatomical changes. We have developed a technique to temporally align 3-D images of the two sequences first and then to spatially register them to rectify probe placement error while preserving the stress-induced changes. The 3-D spatial registration is mutual information-based. Image registration used in conjunction with 3-D stress echocardiography can potentially improve the diagnostic accuracy of stress testing.     

Registration and real-time visualization of transcranial magnetic stimulation with 3-D MR images

This paper describes a method for registering and visualizing in real-time the results of transcranial magnetic stimulations (TMS) in physical space on the corresponding anatomical locations in MR images of the brain. The method proceeds in three main steps. First, the patient scalp is digitized in physical space with a magnetic-field digitizer, following a specific digitization pattern. Second, a registration process minimizes the mean square distance between those points and a segmented scalp surface extracted from the magnetic resonance image. Following this registration, the physician can follow the change in coil position in real-time through the visualization interface and adjust the coil position to the desired anatomical location. Third, amplitude of motor evoked potentials can be projected onto the segmented brain in order to create functional brain maps. The registration has subpixel accuracy in a study with simulated data, while we obtain a point to surface root-mean-square error of 1.17+/-0.38 mm in a 24 subject study.     

Reconstruction of time-varying 3-D left-ventricular shape from multiview X-ray cineangiocardiograms

This paper reports on the clinical application of a system for recovering the time-varying three-dimensional (3-D) left-ventricular (LV) shape from multiview X-ray cineangiocardiograms. Considering that X-ray cineangiocardiography is still commonly employed in clinical cardiology and computational costs for 3-D recovery and visualization are rapidly decreasing, it is meaningful to develop a clinically applicable system for 3-D LV shape recovery from X-ray cineangiocardiograms. The system is based on a previously reported closed-surface method of shape recovery from two-dimensional occluding contours with multiple views. To apply the method to "real" LV cineangiocardiograms, user-interactive systems were implemented for preprocessing, including detection of LV contours, calibration of the imaging geometry, and setting of the LV model coordinate system. The results for three real LV angiographic image sequences are presented, two with fixed multiple views (using supplementary angiography) and one with rotating views. 3-D reconstructions utilizing different numbers of views were compared and evaluated in terms of contours manually traced by an experienced radiologist. The performance of the preprocesses was also evaluated, and the effects of variations in user-specified parameters on the final 3-D reconstruction results were shown to be sufficiently small. These experimental results demonstrate the potential usefulness of combining multiple views for 3-D recovery from "real" LV cineangiocardiograms.     

Rigid and non-rigid 3D motion estimation from multiview image sequences

Multiview image sequence processing has been the focus of considerable attention in recent literature. This paper presents an efficient technique for object-based rigid and non-rigid 3D motion estimation, applicable to problems occurring in multiview image sequence coding applications. More specifically, a neural network is formed for the estimation of the rigid 3D motion of each object in the scene, using initially estimated 2D motion vectors corresponding to each camera view. Non-linear error minimization techniques are adopted for neural network weight update. Furthermore, a novel technique is also proposed for the estimation of the local non-rigid deformations, based on the multiview camera geometry. Experimental results using both stereoscopic and trinocular camera setups illustrate and evaluate the proposed scheme.     

Registration of 3-D head surfaces using multiple landmarks

A mathematical procedure based on Newton's method is described that enables surface measurements to be registered, or normalized, with respect to spatial position, orientation, and, optionally, scale in three dimensions. An operator is required to identify homologous landmarks on the computer graphics images of surfaces to be registered. In this application, where the method is used to measure changes in facial shape, these landmarks are restricted to parts of the surface that have remained unchanged between the surfaces to be registered. Error in the registration of landmarks is minimized in a least-squares sense; hence multiple landmarks are favored to minimize the effect of individual errors produced by the measuring system and the operator. Examples are presented using measurements of the head taken with an optical surface scanner and a conventional X-ray computed tomography scanner.     

Registration of 3-D images using weighted geometrical features

The authors present a weighted geometrical feature (WGF) registration algorithm. Its efficacy is demonstrated by combining points and a surface. The technique is an extension of Besl and McKay's (1992) iterative closest point (ICP) algorithm. The authors use the WGF algorithm to register X-ray computed tomography (CT) and T2-weighted magnetic resonance (MR) volume head images acquired from eleven patients that underwent craniotomies in a neurosurgical clinical trial. Each patient had five external markers attached to transcutaneous posts screwed into the outer table of the skull. The authors define registration error as the distance between positions of corresponding markers that are not used for registration. The CT and MR images are registered using fiducial paints (marker positions) only, a surface only, and various weighted combinations of points and a surface. The CT surface is derived from contours corresponding to the inner surface of the skull. The MR surface is derived from contours corresponding to the cerebrospinal fluid (CSF)-dura interface. Registration using points and a surface is found to be significantly more accurate then registration using only points or a surface     

Volume Registration Using the 3-D Pseudopolar Fourier Transform

This paper introduces an algorithm for the registration of rotated and translated volumes using the three-dimensional (3-D) pseudopolar Fourier transform, which accurately computes the Fourier transform of the registered volumes on a near-spherical 3-D domain without using interpolation. We propose a three-step procedure. The first step estimates the rotation axis. The second step computes the planar rotation relative to the rotation axis. The third step recovers the translational displacement. The rotation estimation is based on Euler's theorem, which allows one to represent a 3-D rotation as a planar rotation around a 3-D rotation axis. This axis is accurately recovered by the 3-D pseudopolar Fourier transform using radial integrations. The residual planar rotation is computed by an extension of the angular difference function to cylindrical motion. Experimental results show that the algorithm is accurate and robust to noise.     

Registration of 3-D CT and 2-D Flat Images of Mouse via Affine Transformation

It is difficult to directly coregister the 3-D fluorescence molecular tomography (FMT) image of a small tumor in a mouse whose maximal diameter is only a few millimeters with a larger CT image of the entire animal that spans about 10 cm. This paper proposes a new method to register 2-D flat and 3-D CT image first to facilitate the registration between small 3-D FMT images and large 3-D CT images. A novel algorithm combining differential evolution and improved simplex method for the registration between the 2-D flat and 3-D CT images is introduced and validated with simulated images and real images of mice. The visualization of the alignment of the 3-D FMT and CT image through 2-D registration shows promising results.      

Robust Gradient-Based 3-D/2-D Registration of CT and MR to X-Ray Images

One of the most important technical challenges in image-guided intervention is to obtain a precise transformation between the intrainterventional patient's anatomy and corresponding preinterventional 3-D image on which the intervention was planned. This goal can be achieved by acquiring intrainterventional 2-D images and matching them to the preinterventional 3-D image via 3-D/2-D image registration. A novel 3-D/2-D registration method is proposed in this paper. The method is based on robustly matching 3-D preinterventional image gradients and coarsely reconstructed 3-D gradients from the intrainterventional 2-D images. To improve the robustness of finding the correspondences between the two sets of gradients, hypothetical correspondences are searched for along normals to anatomical structures in 3-D images, while the final correspondences are established in an iterative process, combining the robust random sample consensus algorithm (RANSAC) and a special gradient matching criterion function. The proposed method was evaluated using the publicly available standardized evaluation methodology for 3-D/2-D registration, consisting of 3-D rotational X-ray, computed tomography, magnetic resonance (MR), and 2-D X-ray images of two spine segments, and standardized evaluation criteria. In this way, the proposed method could be objectively compared to the intensity, gradient, and reconstruction-based registration methods. The obtained results indicate that the proposed method performs favorably both in terms of registration accuracy and robustness. The method is especially superior when just a few X-ray images and when MR preinterventional images are used for registration, which are important advantages for many clinical applications.      

三维测量数据同步自动配准方法

提出一种三维(3-D)视觉测量数据同步自动配准方法。利用改进的3-Dshape context描述符建立当前测量数据与已获得数据间的匹配点对,得到候选的重叠数据,并用最近点迭代(ICP)算法进行验证和精确配准。将视觉传感器每次获得的测量数据与已获得的测量数据配准,最终将所有测量数据统一到同一坐标系下。实验时,采用视觉传感器对维纳斯石膏像进行多角度测量,对测量数据进行同步配准,获得石膏像表面完整的3-D模型。该配准方法无需任何外部辅助设备,便于操作者观察测量进度并及时调整视觉传感器路径规划,从而提高三维形貌测量效率。     

GPU Accelerated Generation of Digitally Reconstructed Radiographs for 2-D/3-D Image Registration

Recent advances in programming languages for graphics processing units (GPUs) provide developers with a convenient way of implementing applications which can be executed on the CPU and GPU interchangeably. GPUs are becoming relatively cheap, powerful, and widely available hardware components, which can be used to perform intensive calculations. The last decade of hardware performance developments shows that GPU-based computation is progressing significantly faster than CPU-based computation, particularly if one considers the execution of highly parallelisable algorithms. Future predictions illustrate that this trend is likely to continue. In this paper, we introduce a way of accelerating 2-D/3-D image registration by developing a hybrid system which executes on the CPU and utilizes the GPU for parallelizing the generation of digitally reconstructed radiographs (DRRs). Based on the advancements of the GPU over the CPU, it is timely to exploit the benefits of many-core GPU technology by developing algorithms for DRR generation. Although some previous work has investigated the rendering of DRRs using the GPU, this paper investigates approximations which reduce the computational overhead while still maintaining a quality consistent with that needed for 2-D/3-D registration with sufficient accuracy to be clinically acceptable in certain applications of radiation oncology. Furthermore, by comparing implementations of 2-D/3-D registration on the CPU and GPU, we investigate current performance and propose an optimal framework for PC implementations addressing the rigid registration problem. Using this framework, we are able to render DRR images from a 256×256×133 CT volume in ～?24 ms using an NVidia GeForce 8800 GTX and in ～?2 ms using NVidia GeForce GTX 580. In addition to applications requiring fast automatic patient setup, these levels of performance suggest image-guided radiation therapy at video frame rates is technically feasible using relatively low cost PC architecture.     

Concurrent 3-D motion segmentation and 3-D interpretation of temporal sequences of monocular images.

The purpose of this study is to investigate a variational method for joint multiregion three-dimensional (3-D) motion segmentation and 3-D interpretation of temporal sequences of monocular images. Interpretation consists of dense recovery of 3-D structure and motion from the image sequence spatiotemporal variations due to short-range image motion. The method is direct insomuch as it does not require prior computation of image motion. It allows movement of both viewing system and multiple independently moving objects. The problem is formulated following a variational statement with a functional containing three terms. One term measures the conformity of the interpretation within each region of 3-D motion segmentation to the image sequence spatiotemporal variations. The second term is of regularization of depth. The assumption that environmental objects are rigid accounts automatically for the regularity of 3-D motion within each region of segmentation. The third and last term is for the regularity of segmentation boundaries. Minimization of the functional follows the corresponding Euler-Lagrange equations. This results in iterated concurrent computation of 3-D motion segmentation by curve evolution, depth by gradient descent, and 3-D motion by least squares within each region of segmentation. Curve evolution is implemented via level sets for topology independence and numerical stability. This algorithm and its implementation are verified on synthetic and real image sequences. Viewers presented with anaglyphs of stereoscopic images constructed from the algorithm's output reported a strong perception of depth.     

光学三维形面分区域测量数据的拼接研究

根据空间坐标组合变换原理 ,提出在三维形面相邻子区域重合区人为引入空间非共线三点以实现测量对象全形面数字化的方法 ,分析了测量数据的合成精度 ,并以光切法三维测量系统作为应用对象进行了实验研究。结果表明 ,利用该方法实现三维形面分区域测量及后序多片数据拼接 ,具有速度快、精度高、可操作性强等特点。     

Recovery of the 3-D shape of the left ventricle from echocardiographic images

A computational method is reported which allows the fully automated recovery of the three-dimensional shape of the cardiac left ventricle from a reduced set of apical echo views. Two typically ill-posed problems have been faced: 1) the detection of the left ventricle contours in each view, and 2) the integration of the detected contour points (which form a sparse and partially inconsistent data set) into a single surface representation. The authors' solution to these problems is based on a careful integration of standard computer vision algorithms with neural networks. Boundary detection comprises three steps: edge detection, edge grouping, and edge classification. The first and second steps (which are typical early-vision tasks not involving specific domain-knowledge) have been performed through fast, well-established algorithms of computer vision. The higher level task of left ventricle-edge discrimination, which involves the exploitation of specific knowledge about the left ventricle silhouette, has been performed by feedforward neural networks. Following the most recent results in the field of computer vision, the first step in solving the problem of recovering the ventricle surface has been the adoption of a physically inspired model of it. Basically, the authors have modeled the left ventricle surface as a closed, thin, elastic surface and the data as a set of radial springs acting on it. The recovery process is equivalent to the settling of the surface-plus-springs system into a stable configuration of minimum potential energy. The finite element discretization of this model leads directly to an analog neural-network implementation. The efficiency of such an implementation has been remarkably enhanced through a learning algorithm which embeds specific knowledge about the shape of the left ventricle in the network. Experiments using clinical echographic sequences are described. Four apical views (each with a different rotation of the probe) have been acquired during a heartbeat from a set of seven normal subjects. These images have been utilized to set the various processing modules and test their capabilities.     

遗传算法和Powell法结合的多分辨率三维图像配准

基于互信息的配准方法具有精度高、鲁棒性强的特点,成为近年图像配准研究的热点,但基于互信息的目标函数存在许多局部极值,为配准的优化过程带来了很大的困难。本文以互信息作为相似性测度,采用基于小波变换的多分辨率策略,将遗传算法和Powell法结合起来对三维CT、MR图像进行了配准。实验结果表明,这种方法能够有效地克服信息函数的局部极值,配准精度达到亚像素级。     

LV volume quantification via spatiotemporal analysis of real-time3-D echocardiography

This paper presents a method of four-dimensional (4-D) (3-D + Time) space-frequency analysis for directional denoising and enhancement of real-time three-dimensional (RT3D) ultrasound and quantitative measures in diagnostic cardiac ultrasound. Expansion of echocardiographic volumes is performed with complex exponential wavelet-like basis functions called brushlets. These functions offer good localization in time and frequency and decompose a signal into distinct patterns of oriented harmonics, which are invariant to intensity and contrast range. Deformable-model segmentation is carried out on denoised data after thresholding of transform coefficients. This process attenuates speckle noise while preserving cardiac structure location. The superiority of 4-D over 3-D analysis for decorrelating additive white noise and multiplicative speckle noise on a 4-D phantom volume expanding in time is demonstrated. Quantitative validation, computed for contours and volumes, is performed on in vitro balloon phantoms. Clinical applications of this spaciotemporal analysis tool are reported for six patient cases providing measures of left ventricular volumes and ejection fraction.     

Postoperative Calculation of Acetabular Cup Position Using 2-D–3-D Registration

A method to accurately measure the position and orientation of an acetabular cup implant from postoperative X-rays has been designed and validated. The method uses 2-D-3-D registration to align both the prosthesis and the preoperative computed tomography (CT) volume to the X-ray image. This allows the position of the implant to be calculated with respect to a CT-based surgical plan. Experiments have been carried out using ten sets of patient data. A conventional plain-film measurement technique was also investigated. A gold standard implant position and orientation was calculated using postoperative CT. Results show our method to be significantly more accurate than the plain-film method for calculating cup anteversion. Cup orientation and position could be measured to within a mean absolute error of 1.4 mm or degrees.