FAIR: a hardware architecture for real-time 3-D image registration

Mutual information-based image registration, shown to be effective in registering a range of medical images, is a computationally expensive process, with a typical execution time on the order of minutes on a modern single-processor computer. Accelerated execution of this process promises to enhance efficiency and therefore promote routine use of image registration clinically. This paper presents details of a hardware architecture for real-time three-dimensional (3-D) image registration. Real-time performance can be achieved by setting up a network of processing units, each with three independent memory buses: one each for the two image memories and one for the mutual histogram memory. Memory access parallelization and pipelining, by design, allow each processing unit to be 25 times faster than a processor with the same bus speed, when calculating mutual information using partial volume interpolation. Our architecture provides superior per-processor performance at a lower cost compared to a parallel supercomputer.     

一种基于SIFT的仿射不变特征提取新方法

图像局部特征提取是图像理解及机器视觉领域一个非常关键的问题,其中SIFT特征因具有良好的显著性和鲁棒性而得到广泛应用。但是,SIFT采用DOG检测子,定位的特征区域为各向同尺度变化的圆形区域,故其只具有尺度不变性,并不具备仿射不变性。此外,SIFT采用128维特征向量表示,当在图像特征点较多情况下进行匹配实验时,存在存储空间大、匹配耗时多等缺点。针对这两个问题,本文提出一种新的仿射不变特征提取方法,即HA-DR-SIFT(Hessian Affine-Dimensionality Reduction-SIFT)。首先,用Hessian-Affine 检测子代替DOG检测子,使提取的椭圆图像区域满足仿射不变性需求;其次,用PCA或NLPCA方法对128维特征向量进行降维处理,提高后续运算效率。实验表明,新方法不仅具有良好的仿射不变性,而且在匹配时间和存储空间上优于SIFT算子。      

Robust Gradient-Based 3-D/2-D Registration of CT and MR to X-Ray Images

One of the most important technical challenges in image-guided intervention is to obtain a precise transformation between the intrainterventional patient's anatomy and corresponding preinterventional 3-D image on which the intervention was planned. This goal can be achieved by acquiring intrainterventional 2-D images and matching them to the preinterventional 3-D image via 3-D/2-D image registration. A novel 3-D/2-D registration method is proposed in this paper. The method is based on robustly matching 3-D preinterventional image gradients and coarsely reconstructed 3-D gradients from the intrainterventional 2-D images. To improve the robustness of finding the correspondences between the two sets of gradients, hypothetical correspondences are searched for along normals to anatomical structures in 3-D images, while the final correspondences are established in an iterative process, combining the robust random sample consensus algorithm (RANSAC) and a special gradient matching criterion function. The proposed method was evaluated using the publicly available standardized evaluation methodology for 3-D/2-D registration, consisting of 3-D rotational X-ray, computed tomography, magnetic resonance (MR), and 2-D X-ray images of two spine segments, and standardized evaluation criteria. In this way, the proposed method could be objectively compared to the intensity, gradient, and reconstruction-based registration methods. The obtained results indicate that the proposed method performs favorably both in terms of registration accuracy and robustness. The method is especially superior when just a few X-ray images and when MR preinterventional images are used for registration, which are important advantages for many clinical applications.      

Projection-based image registration in the presence offixed-pattern noise

A computationally efficient method for image registration is investigated that can achieve an improved performance over the traditional two-dimensional (2-D) cross-correlation-based techniques in the presence of both fixed-pattern and temporal noise. The method relies on transforming each image in the sequence of frames into two vector projections formed by accumulating pixel values along the rows and columns of the image. The vector projections corresponding to successive frames are in turn used to estimate the individual horizontal and vertical components of the shift by means of a one-dimensional (1-D) cross-correlation-based estimator. While gradient-based shift estimation techniques are computationally efficient, they often exhibit degraded performance under noisy conditions in comparison to cross-correlators due to the fact that the gradient operation amplifies noise. The projection-based estimator, on the other hand, significantly reduces the computational complexity associated with the 2-D operations involved in traditional correlation-based shift estimators while improving the performance in the presence of temporal and spatial noise. To show the noise rejection capability of the projection-based shift estimator relative to the 2-D cross correlator, a figure-of-merit is developed and computed reflecting the signal-to-noise ratio (SNR) associated with each estimator. The two methods are also compared by means of computer simulation and tests using real image sequences.     

Vision processing for realtime 3-D data acquisition based on coded structured light

Structured light vision systems have been successfully used for accurate measurement of 3-D surfaces in computer vision. However, their applications are mainly limited to scanning stationary objects so far since tens of images have to be captured for recovering one 3-D scene. This paper presents an idea for real-time acquisition of 3-D surface data by a specially coded vision system. To achieve 3-D measurement for a dynamic scene, the data acquisition must be performed with only a single image. A principle of uniquely color-encoded pattern projection is proposed to design a color matrix for improving the reconstruction efficiency. The matrix is produced by a special code sequence and a number of state transitions. A color projector is controlled by a computer to generate the desired color patterns in the scene. The unique indexing of the light codes is crucial here for color projection since it is essential that each light grid be uniquely identified by incorporating local neighborhoods so that 3-D reconstruction can be performed with only local analysis of a single image. A scheme is presented to describe such a vision processing method for fast 3-D data acquisition. Practical experimental performance is provided to analyze the efficiency of the proposed methods.     

稳健的图像匹配方法

景象匹配是计算机视觉研究的一个重要方向,如何在复杂的环境下提高匹配概率与定位精度是其亟待解决的难点之一。在分析畸变校正的量化噪声为高斯噪声的基础上,提出了基于特征和时空关联的积相关图像匹配算法。采用基于噪声抑制小波边缘检测方法,提取实时图和基准图的边缘特征;基于归一化积相关实现序列实时图(3帧)与基准图的匹配,再根据位置信息实现相关峰值的数据融合得到匹配点。此方法克服了因面积增大由几何失真导致的匹配概率下降的缺点,提高了匹配概率,具有较好的匹配稳健性。     

DETERMINING POSE PARAMETERS WITHOUT PRIMITIVE-TO-PRIMITIVE CORRESPONDENCE

The problem of determining the pose of an object in 3-D space is essential in many computer vision applications. In this paper, a model-based approach for solving this problem is proposed. This approach does not require the knowledge of point-to-point correspondences between 3-D points on the model and 2-D points in the observed image. The spatial location of the object is iteratively estimated and updated from the values globally defined over the model image and the observed image.     

A 3-D reconstruction system for the human jaw using a sequence of optical images

This paper presents a model-based vision system for dentistry that will assist in diagnosis, treatment planning, and surgical simulation. Dentistry requires an accurate three-dimensional (3-D) representation of the teeth and jaws for diagnostic and treatment purposes. The proposed integrated computer vision system constructs a 3-D model of the patient's dental occlusion using an intraoral video camera. A modified shape from shading (SFS) technique, using perspective projection and camera calibration, extracts the 3-D information from a sequence of two-dimensional (2-D) images of the jaw. Data fusion of range data and 3-D registration techniques develop the complete jaw model. Triangulation is then performed, and a solid 3-D model is reconstructed. The system performance is investigated using ground truth data, and the results show acceptable reconstruction accuracy.     

Markerless real-time 3-D target region tracking by motion backprojection from projection images

Accurate and fast localization of a predefined target region inside the patient is an important component of many image-guided therapy procedures. This problem is commonly solved by registration of intraoperative 2-D projection images to 3-D preoperative images. If the patient is not fixed during the intervention, the 2-D image acquisition is repeated several times during the procedure, and the registration problem can be cast instead as a 3-D tracking problem. To solve the 3-D problem, we propose in this paper to apply 2-D region tracking to first recover the components of the transformation that are in-plane to the projections. The 2-D motion estimates of all projections are backprojected into 3-D space, where they are then combined into a consistent estimate of the 3-D motion. We compare this method to intensity-based 2-D to 3-D registration and a combination of 2-D motion backprojection followed by a 2-D to 3-D registration stage. Using clinical data with a fiducial marker-based gold-standard transformation, we show that our method is capable of accurately tracking vertebral targets in 3-D from 2-D motion measured in X-ray projection images. Using a standard tracking algorithm (hyperplane tracking), tracking is achieved at video frame rates but fails relatively often (32% of all frames tracked with target registration error (TRE) better than 1.2 mm, 82% of all frames tracked with TRE better than 2.4 mm). With intensity-based 2-D to 2-D image registration using normalized mutual information (NMI) and pattern intensity (PI), accuracy and robustness are substantially improved. NMI tracked 82% of all frames in our data with TRE better than 1.2 mm and 96% of all frames with TRE better than 2.4 mm. This comes at the cost of a reduced frame rate, 1.7 s average processing time per frame and projection device. Results using PI were slightly more accurate, but required on average 5.4 s time per frame. These results are still substantially faster than 2-D to 3-D registration. We conclude that motion backprojection from 2-D motion tracking is an accurate and efficient method for tracking 3-D target motion, but tracking 2-D motion accurately and robustly remains a challenge.     

3-D/2-D registration by integrating 2-D information in 3-D

In image-guided therapy, high-quality preoperative images serve for planning and simulation, and intraoperatively as "background", onto which models of surgical instruments or radiation beams are projected. The link between a preoperative image and intraoperative physical space of the patient is established by image-to-patient registration. In this paper, we present a novel 3-D/2-D registration method. First, a 3-D image is reconstructed from a few 2-D X-ray images and next, the preoperative 3-D image is brought into the best possible spatial correspondence with the reconstructed image by optimizing a similarity measure (SM). Because the quality of the reconstructed image is generally low, we introduce a novel SM, which is able to cope with low image quality as well as with different imaging modalities. The novel 3-D/2-D registration method has been evaluated and compared to the gradient-based method (GBM) using standardized evaluation methodology and publicly available 3-D computed tomography (CT), 3-D rotational X-ray (3DRX), and magnetic resonance (MR) and 2-D X-ray images of two spine phantoms, for which gold standard registrations were known. For each of the 3DRX, CT, or MR images and each set of X-ray images, 1600 registrations were performed from starting positions, defined as the mean target registration error (mTRE), randomly generated and uniformly distributed in the interval of 0-20 mm around the gold standard. The capture range was defined as the distance from gold standard for which the final TRE was less than 2 mm in at least 95% of all cases. In terms of success rate, as the function of initial misalignment and capture range the proposed method outperformed the GBM. TREs of the novel method and the GBM were approximately the same. For the registration of 3DRX and CT images to X-ray images as few as 2-3 X-ray views were sufficient to obtain approximately 0.4 mm TREs, 7-9 mm capture range, and 80%-90% of successful registrations. To obtain similar results for MR to X-ray registrations, an image, reconstructed from at least 11 X-ray images was required. Reconstructions from more than 11 images had no effect on the registration results.     

图像配准技术研究进展

图像配准是解决图像融合、图像镶嵌和变化检测等问题的必要前提,其应用遍及军事、遥感、医学和计算机视觉等多个领域.简要回顾了图像配准技术的发展史和研究现状,重点阐述了当前的技术热点和应用趋势,最后展望了进一步的研究方向.     

Snakes, shapes, and gradient vector flow

Snakes, or active contours, are used extensively in computer vision and image processing applications, particularly to locate object boundaries. Problems associated with initialization and poor convergence to boundary concavities, however, have limited their utility. This paper presents a new external force for active contours, largely solving both problems. This external force, which we call gradient vector flow (GVF), is computed as a diffusion of the gradient vectors of a gray-level or binary edge map derived from the image. It differs fundamentally from traditional snake external forces in that it cannot be written as the negative gradient of a potential function, and the corresponding snake is formulated directly from a force balance condition rather than a variational formulation. Using several two-dimensional (2-D) examples and one three-dimensional (3-D) example, we show that GVF has a large capture range and is able to move snakes into boundary concavities.     

Computer vision applied to super resolution

Super-resolution (SR) restoration aims to solve the following problem: given a set of observed images, estimate an image at a higher resolution than is present in any of the individual images. Where the application of this technique differs in computer vision from other fields is in the variety and severity of the registration transformation between the images. In particular this transformation is generally unknown, and a significant component of solving the SR problem in computer vision is the estimation of the transformation. The transformation may have a simple parametric form, or it may be scene dependent and have to be estimated for every point. In either case the transformation is estimated directly and automatically from the images. We describe the two key components that are necessary for successful SR restoration: the accurate alignment or registration of the LR images and the formulation of an SR estimator that uses a generative image model together with a prior model of the super-resolved image itself. As with many other problems in computer vision, these different aspects are tackled in a robust, statistical framework.     

Improvement of depth position in 2-D/3-D registration of knee implants using single-plane fluoroscopy

Two-dimensional (2-D)/three-dimensional (3-D) registration techniques using single-plane fluoroscopy are highly important for analyzing 3-D kinematics in applications such as total knee arthroplasty (TKA) implants. The accuracy of single-plane fluoroscopy-based techniques in the determination of translation perpendicular to the image plane (depth position), however, is relatively poor because a change in the depth position causes only small changes in the 2-D silhouette. Accuracies achieved in depth position using conventional 2-D/3-D registration techniques are insufficient for clinical applications. Therefore, we propose a technique for improving the accuracy of depth position determination in order to develop a system for analyzing knee kinematics over the full six degrees of freedom (6 DOF) using single-plane fluoroscopy. In preliminary experiments, the behaviors of errors for each free variable were quantified as evaluation curves by examining changes in cost function with variations in the free variable. The evaluation curve for depth position was more jagged, and the curve peak less pointy, compared to the evaluation curves of the other five variables, and the curve was found to behave differently. Depth position is therefore optimized independently of the other variables, using an approximate evaluation curve of depth position prepared after initial registration. Accuracy of the proposed technique was evaluated by computer simulation and in vitro tests, with validation of absolute position and orientation performed for each knee component. In computer simulation tests, root-mean-square error (RMSE) in depth position was improved from 2.6 mm (conventional) to 0.9 mm (proposed), whereas for in vitro tests, RMSE improved from 3.2 mm to 1.4 mm. Accuracy of the estimation of the remaining two translational and three rotational variables was found to be almost the same as that obtained by conventional techniques. Results of in vivo tests are also described in which the possibility of full 6 DOF kinematic analysis of TKA implants is shown.     

图像配准技术性能评估及实现概况

图像配准是解决图像融合、图像镶嵌和变化检测等问题的必要前提,其应用遍及军事、遥感、医学和计算机视觉等多个领域.首先概括了图像配准技术的研究内容,然后全面论述了配准技术的性能评估及其系统实现问题,最后指出了该领域存在的主要问题,并展望了进一步的发展方向.     

遥感数字图像配准技术综述

图像配准技术是近年来迅速发展的图像处理技术之一,也是解决图像融合、图像镶嵌和变化检测等问题的必要前提,并在遥感、军事、医学、计算机视觉等众多领域都有广泛的应用.本文首先简要介绍了图像配准的步骤和过程,接着具体分析了目前几种常用的配准方法,并归纳总结了其在遥感领域应用中的技术特点,最后讨论了图像配准在遥感领域中的发展趋势.     

A maximum likelihood approach for image registration using control point and intensity

Registration of multidate or multisensor images is an essential process in many image processing applications including remote sensing, medical image analysis, and computer vision. Control point (CP) and intensity are the two basic features used separately for image registration in the literature. In this paper, an exact maximum likelihood (EML) registration method, which combines both CP and intensity, is proposed for image alignment. The EML registration method maximizes the likelihood function based CP and intensity to estimate the registration parameters, including affine transformation and CP coordinates. The explicit formulas of the Cramer-Rao bound (CRB) are also derived for the proposed EML and conventional image registration algorithms. The performances of these image registration techniques are evaluated with the CRBs.     

Registration of 3-D CT and 2-D Flat Images of Mouse via Affine Transformation

It is difficult to directly coregister the 3-D fluorescence molecular tomography (FMT) image of a small tumor in a mouse whose maximal diameter is only a few millimeters with a larger CT image of the entire animal that spans about 10 cm. This paper proposes a new method to register 2-D flat and 3-D CT image first to facilitate the registration between small 3-D FMT images and large 3-D CT images. A novel algorithm combining differential evolution and improved simplex method for the registration between the 2-D flat and 3-D CT images is introduced and validated with simulated images and real images of mice. The visualization of the alignment of the 3-D FMT and CT image through 2-D registration shows promising results.      

基于机器视觉的香烟小包装外观质量检测系统

讨论了基于机器视觉的香烟小包装外观质量在线检测系统总体方案的设计,并针对香烟小包装流水线的具体情况,分别对香烟小包的检测位置以及图像采集方案进行了探讨.文中采用了一种用于香烟包装质量检测的快速图像匹配算法,并利用开源计算机视觉库OpenCV进行算法实现,仿真实验结果表明,该图像匹配算法计算速度快、检测精度高,满足香烟包装质量检测的需要.     

两阶段平移、旋转图像高精度配准算法

图像配准在图像处理中是十分重要的,通常是许多现代图像处理和计算机视觉任务一个关键的预处理步骤,许多算法和技术已经被提出用来解决配准问题.本文提出一种稳健的两阶段层次配准算法,在频域组合相位相关和谱对消技术对平移和旋转图像获得亚像素精度的配准.算法第一阶段首先利用一维FFT技术实现图像的旋转以确定旋转角度,并利用这个旋转角度将两幅图像之间的运动简化为平移运动,然后利用相位相关法确定整数像素平移参数.算法第二阶段运用谱对消技术确定亚像素平移参数.提出的算法甚至在图像因为下采样而包含混淆误差情况下仍然能够获得亚像素精度配准.