Robust Gradient-Based 3-D/2-D Registration of CT and MR to X-Ray Images

One of the most important technical challenges in image-guided intervention is to obtain a precise transformation between the intrainterventional patient's anatomy and corresponding preinterventional 3-D image on which the intervention was planned. This goal can be achieved by acquiring intrainterventional 2-D images and matching them to the preinterventional 3-D image via 3-D/2-D image registration. A novel 3-D/2-D registration method is proposed in this paper. The method is based on robustly matching 3-D preinterventional image gradients and coarsely reconstructed 3-D gradients from the intrainterventional 2-D images. To improve the robustness of finding the correspondences between the two sets of gradients, hypothetical correspondences are searched for along normals to anatomical structures in 3-D images, while the final correspondences are established in an iterative process, combining the robust random sample consensus algorithm (RANSAC) and a special gradient matching criterion function. The proposed method was evaluated using the publicly available standardized evaluation methodology for 3-D/2-D registration, consisting of 3-D rotational X-ray, computed tomography, magnetic resonance (MR), and 2-D X-ray images of two spine segments, and standardized evaluation criteria. In this way, the proposed method could be objectively compared to the intensity, gradient, and reconstruction-based registration methods. The obtained results indicate that the proposed method performs favorably both in terms of registration accuracy and robustness. The method is especially superior when just a few X-ray images and when MR preinterventional images are used for registration, which are important advantages for many clinical applications.      

Nonrigid registration of 3-d multichannel microscopy images of cell nuclei.

We present an intensity-based nonrigid registration approach for the normalization of 3-D multichannel microscopy images of cell nuclei. A main problem with cell nuclei images is that the intensity structure of different nuclei differs very much; thus, an intensity-based registration scheme cannot be used directly. Instead, we first perform a segmentation of the images from the cell nucleus channel, smooth the resulting images by a Gaussian filter, and then apply an intensity-based registration algorithm. The obtained transformation is applied to the images from the nucleus channel as well as to the images from the other channels. To improve the convergence rate of the algorithm, we propose an adaptive step length optimization scheme and also employ a multiresolution scheme. Our approach has been successfully applied using 2-D cell-like synthetic images, 3-D phantom images as well as 3-D multichannel microscopy images representing different chromosome territories and gene regions. We also describe an extension of our approach, which is applied for the registration of 3D + t (4-D) image series of moving cell nuclei.     

Nonrigid registration of 3-D free-hand ultrasound images of the breast

Three-dimensional (3-D) ultrasound imaging of the breast enables better assessment of diseases than conventional two-dimensional (2-D) imaging. Free-hand techniques are often used for generating 3-D data from a sequence of 2-D slice images. However, the breast deforms substantially during scanning because it is composed primarily of soft tissue. This often causes tissue mis-registration in spatial compounding of multiple scan sweeps. To overcome this problem, in this paper, instead of introducing additional constraints on scanning conditions, we use image processing techniques. We present a fully automatic algorithm for 3-D nonlinear registration of free-hand ultrasound data. It uses a block matching scheme and local statistics to estimate local tissue deformation. A Bayesian regularization method is applied to the sample displacement field. The final deformation field is obtained by fitting a B-spline approximating mesh to the sample displacement field. Registration accuracy is evaluated using phantom data and similar registration errors are achieved with (0.19 mm) and without (0.16 mm) gaps in the data. Experimental results show that registration is crucial in spatial compounding of different sweeps. The execution time of the method on moderate hardware is sufficiently fast for fairly large research studies.     

Surgical navigation by autostereoscopic image overlay of integral videography

This paper describes an autostereoscopic image overlay technique that is integrated into a surgical navigation system to superimpose a real three-dimensional (3-D) image onto the patient via a half-silvered mirror. The images are created by employing a modified version of integral videography (IV), which is an animated extension of integral photography. IV records and reproduces 3-D images using a microconvex lens array and flat display; it can display geometrically accurate 3-D autostereoscopic images and reproduce motion parallax without the need for special devices. The use of semitransparent display devices makes it appear that the 3-D image is inside the patient's body. This is the first report of applying an autostereoscopic display with an image overlay system in surgical navigation. Experiments demonstrated that the fast IV rendering technique and patient-image registration method produce an average registration accuracy of 1.13 mm. Experiments using a target in phantom agar showed that the system can guide a needle toward a target with an average error of 2.6 mm. Improvement in the quality of the IV display will make this system practical and its use will increase surgical accuracy and reduce invasiveness.     

Three-Dimensional Assessment of Skin Wounds Using a Standard Digital Camera

In this paper, after an overview of the literature concerning the imaging technologies applied to skin wounds assessment, we present an original approach to build 3-D models of skin wounds from color images. The method can deal with uncalibrated images acquired with a handheld digital camera with free zooming. Compared with the cumbersome imaging systems already proposed, this novel solution uses a low-cost and user-friendly image acquisition device suitable for widespread application in health care centers. However, this method entails the development of a robust image processing chain. An original iterative matching scheme is used to generate a dense estimation of the surface geometry from two widely separated views. The best configuration for taking photographs lies between 15^deg and 30^deg for the vergency angle. The metric reconstruction of the skin wound is fully automated through self-calibration. From the 3-D model of the skin wound, accurate volumetric measurements are achieved. The accuracy of the inferred 3-D surface is validated by registration to a ground truth and repetitive tests on volume. The global precision around 3% is in accordance with the clinical requirement of 5% for assessing the healing process.     

Consistent landmark and intensity-based image registration

Two new consistent image registration algorithms are presented: one is based on matching corresponding landmarks and the other is based on matching both landmark and intensity information. The consistent landmark and intensity registration algorithm produces good correspondences between images near landmark locations by matching corresponding landmarks and away from landmark locations by matching the image intensities. In contrast to similar unidirectional algorithms, these new consistent algorithms jointly estimate the forward and reverse transformation between two images while minimizing the inverse consistency error-the error between the forward (reverse) transformation and the inverse of the the reverse (forward) transformation. This reduces the ambiguous correspondence between the forward and reverse transformations associated with large inverse consistency errors. In both algorithms a thin-plate spline (TPS) model is used to regularize the estimated transformations. Two-dimensional (2-D) examples are presented that show the inverse consistency error produced by the traditional unidirectional landmark TPS algorithm can be relatively large and that this error is minimized using the consistent landmark algorithm. Results using 2-D magnetic resonance imaging data are presented that demonstrate that using landmark and intensity information together produce better correspondence between medical images than using either landmarks or intensity information alone.     

3-D/2-D registration by integrating 2-D information in 3-D

In image-guided therapy, high-quality preoperative images serve for planning and simulation, and intraoperatively as "background", onto which models of surgical instruments or radiation beams are projected. The link between a preoperative image and intraoperative physical space of the patient is established by image-to-patient registration. In this paper, we present a novel 3-D/2-D registration method. First, a 3-D image is reconstructed from a few 2-D X-ray images and next, the preoperative 3-D image is brought into the best possible spatial correspondence with the reconstructed image by optimizing a similarity measure (SM). Because the quality of the reconstructed image is generally low, we introduce a novel SM, which is able to cope with low image quality as well as with different imaging modalities. The novel 3-D/2-D registration method has been evaluated and compared to the gradient-based method (GBM) using standardized evaluation methodology and publicly available 3-D computed tomography (CT), 3-D rotational X-ray (3DRX), and magnetic resonance (MR) and 2-D X-ray images of two spine phantoms, for which gold standard registrations were known. For each of the 3DRX, CT, or MR images and each set of X-ray images, 1600 registrations were performed from starting positions, defined as the mean target registration error (mTRE), randomly generated and uniformly distributed in the interval of 0-20 mm around the gold standard. The capture range was defined as the distance from gold standard for which the final TRE was less than 2 mm in at least 95% of all cases. In terms of success rate, as the function of initial misalignment and capture range the proposed method outperformed the GBM. TREs of the novel method and the GBM were approximately the same. For the registration of 3DRX and CT images to X-ray images as few as 2-3 X-ray views were sufficient to obtain approximately 0.4 mm TREs, 7-9 mm capture range, and 80%-90% of successful registrations. To obtain similar results for MR to X-ray registrations, an image, reconstructed from at least 11 X-ray images was required. Reconstructions from more than 11 images had no effect on the registration results.     

Registration of 3-D CT and 2-D Flat Images of Mouse via Affine Transformation

It is difficult to directly coregister the 3-D fluorescence molecular tomography (FMT) image of a small tumor in a mouse whose maximal diameter is only a few millimeters with a larger CT image of the entire animal that spans about 10 cm. This paper proposes a new method to register 2-D flat and 3-D CT image first to facilitate the registration between small 3-D FMT images and large 3-D CT images. A novel algorithm combining differential evolution and improved simplex method for the registration between the 2-D flat and 3-D CT images is introduced and validated with simulated images and real images of mice. The visualization of the alignment of the 3-D FMT and CT image through 2-D registration shows promising results.      

改进配准测度的SUFT红外图像快速配准算法

为方便电路板卡故障诊断,实现红外图像快速、有效配准,提出一种基于SUFT(Speeded-Up Robust Features)和相似四边形的红外图像快速配准算法。该算法首先对红外图像进行特征点检测,生成SUFT特征点描述子;然后采用欧氏距离进行相似性度量,提取粗匹配特征点对,再利用相似四边形进行精匹配,去除误配准点对;最后依据精匹配点对求解变换模型参数,实现红外图像的配准。实验表明,改进后的算法能有效剔除误匹配点对,提高配准精度,配准结果较理想,与同类算法相比耗时较短。因此该算法具有快速性、稳定性等优点,且配准精度较高,有很好的实用价值。     

基于邻域一致性的极化SAR图像仿射配准

极化SAR图像的配准是极化SAR图像处理的基础,需要具备较高的精度与速度.基于深度学习的极化SAR图像配准大多数是结合图像块特征的匹配与基于随机抽样一致性的参数迭代估计来实现的.目前尚未实现端到端的基于深度卷积神经网络的一步仿射配准.该文提出了一种基于弱监督学习的端到端极化SAR图像配准框架,无需图像切块处理或迭代参数...     

An automatic technique for finding and localizing externallyattached markers in CT and MR volume images of the head

An image processing technique is presented for finding and localizing the centroids of cylindrical markers externally attached to the human head in computed tomography (CT) and magnetic resonance (MR) image volumes. The centroids can be used as control points for image registration. The technique, which is fast, automatic, and knowledge-based, has two major steps. First, it searches the entire image volume to find one voxel inside each marker-like object. The authors call this voxel a “candidate” voxel, and they call the object a candidate marker. Second, it classifies the voxels in a region surrounding the candidate voxel as marker or nonmarker voxels using knowledge-based rules and calculates an intensity-weighted centroid for each true marker. The authors call this final centroid the “fiducial” point of the marker. The technique was developed on 42 scans of six patients-one CT and six MR scans per patient. There are four markers attached to each patient for a total of 168 marker images. For the CT images the false marker rate was zero. For MR the false marker rate was 1.4% (Two out of 144 markers). To evaluate the accuracy of the fiducial points, CT-MR registration was performed after correcting the MR images for geometrical distortion. The fiducial registration accuracy averaged 0.4 mm and was better than 0.6 mm for each of the eighteen image pairs     

An ultra-fast user-steered image segmentation paradigm: live wire on the fly

We have been developing general user steered image segmentation strategies for routine use in applications involving a large number of data sets. In the past, we have presented three segmentation paradigms: live wire, live lane, and a three-dimensional (3-D) extension of the live-wire method. In this paper, we introduce an ultra-fast live-wire method, referred to as live wire on the fly, for further reducing user's time compared to the basic live-wire method. In live wire, 3-D/four-dimensional (4-D) object boundaries are segmented in a slice-by-slice fashion. To segment a two-dimensional (2-D) boundary, the user initially picks a point on the boundary and all possible minimum-cost paths from this point to all other points in the image are computed via Dijkstra's algorithm. Subsequently, a live wire is displayed in real time from the initial point to any subsequent position taken by the cursor. If the cursor is close to the desired boundary, the live wire snaps on to the boundary. The cursor is then deposited and a new live-wire segment is found next. The entire 2-D boundary is specified via a set of live-wire segments in this fashion. A drawback of this method is that the speed of optimal path computation depends on image size. On modestly powered computers, for images of even modest size, some sluggishness appears in user interaction, which reduces the overall segmentation efficiency. In this work, we solve this problem by exploiting some known properties of graphs to avoid unnecessary minimum-cost path computation during segmentation. In live wire on the fly, when the user selects a point on the boundary the live-wire segment is computed and displayed in real time from the selected point to any subsequent position of the cursor in the image, even for large images and even on low-powered computers. Based on 492 tracing experiments from an actual medical application, we demonstrate that live wire on the fly is 1.3-31 times faster than live wire for actual segmentation for varying image sizes, although the pure computational part alone is found to be about 120 times faster.     

Magnetic resonance diffractive imaging

Magnetic resonance (MR) diffractive imaging is proposed as a new approach to MR angiography. The expression of the nuclear MR signal is similar to the equation for the Fresnel diffraction of a three-dimensional (3-D) object in light or sound waves. The proposed technique offers the possibility of fast angiographic imaging and the on-line reconstruction of 3-D volumetric images using the holographic technique. Static imaging experiments using an ultra-low-field MRI system are performed to verify the feasibility of the technique. It is shown that the images focused on an arbitrary plane can be reconstructed from data scanned in two dimensions, even though blurred image data is superimposed on the image. Moreover, the 3-D image can be observed in a coherent optical imaging system. This study demonstrates the possibility of the proposed method as a fast imaging technique for MR angiography.     

智能三坐标测量机零件位姿识别中的立体匹配

为提高三坐标测量机零件位姿识别的效率和准确率,提出了一种基于图像质心偏移的立体匹配新方法。将CCD摄像机安装在三坐标测量机的Z轴上,利用三坐标测量机的精确平移特性,构成单摄像机立体视觉系统,在两个不同位置获取被测零件的实时图像,对其进行处理得到图像质心,将图像质心的偏移距离作为约束条件完成立体匹配。该方法避开了极线约束、灰度约束等复杂约束条件,实验表明该方法可以达到较高的匹配精度,实验件的匹配时间为1.818s。     

Model evaluation and calibration for prospective respiratory motion correction in coronary MR angiography based on 3-D image registration

Image processing was used as a fundamental tool to derive motion information from magnetic resonance (MR) images, which was fed back into prospective respiratory motion correction during subsequent data acquisition to improve image quality in coronary MR angiography (CMRA) scans. This reduces motion artifacts in the images and, in addition, enables the usage of a broader gating window than commonly used today to increase the scan efficiency. The aim of the study reported in this paper was to find a suitable motion model to be used for respiratory motion correction in cardiac imaging and to develop a calibration procedure to adapt the motion model to the individual patient. At first, the performance of three motion models [one-dimensional translation in feet-head (FH) direction, three-dimensional (3-D) translation, and 3-D affine transformation] was tested in a small volunteer study. An elastic image registration algorithm was applied to 3-D MR images of the coronary vessels obtained at different respiratory levels. A strong intersubject variability was observed. The 3-D translation and affine transformation model were found to be superior over the conventional FH translation model used today. Furthermore, a new approach is presented, which utilizes a fast model-based image registration to extract motion information from time series of low-resolution 3-D MR images, which reflects the respiratory motion of the heart. The registration is based on a selectable global 3-D motion model (translation, rigid, or affine transformation). All 3-D MR images were registered with respect to end expiration. The resulting time series of model parameters were analyzed in combination with additionally acquired motion information from a diaphragmatic MR pencil-beam navigator to calibrate the respiratory motion model. To demonstrate the potential of a calibrated motion model for prospective motion correction in coronary imaging, the approach was tested in CMRA examinations in five volunteers.     

基于多快拍图像联合的MIMO雷达三维成像方法

为了提高多输入多输出(MIMO)雷达三维成像沿运动方向的方位分辨率,该文从多快拍图像联合利用的角度入手,提出一种新的多输入多输出-逆合成孔径雷达(MIMO-ISAR)三维成像方法。其基本思路是通过对一段时间观测下二维平面阵列获取的多个单快拍三维图像进行相干处理,沿着散射点线性拟合的方向提取峰值并重构出新的三维图像。仿真实验结果表明,与单快拍三维成像方法相比,该方法可以显著提高成像结果沿运动方向的方位分辨率;与现有基于重排和插值的经典MIMO-ISAR方法相比,该方法对慢速和快速运动目标均适用,得到的成像结果聚焦良好并能够有效抑制沿运动方向的旁瓣。      

高精度机载可见光图像定位方法

针对现有技术不能解决机载可见光图像快速自动定位的问题,提出了一种定位精度高、处理速度快、可扩展性好的机载可见光图像定位方法.该方法以传感器共线成像模型为中心,将实时图像和传感器成像参数送入共线模型进行正射校正,校正后的图像与相应的基准图像进行图像配准,使用配准同名点信息更新传感器成像参数,最后再次通过共线模型获取实时图地理坐标定位信息.这种通过图像配准再计算成像模型的间接定位方法,不仅减少了计算量,而且使自动配准算法稳定,较传统的直接定位方法或图像配准方法有突出的优势.此外,该方法具有严格的误差传递计算公式,能计算定位结果的误差.仿真实验表明即使在大倾斜角条件下,该方法也能获得优于100 m的定位精度.     

基于SURF的图像匹配算法改进

在图像匹配中,SURF算法采用最邻近与次邻近欧式距离比查寻匹配点,当阈值过大时,这种方法会产生大量误匹配点,从而严重影响图像配准的精度。RANSAC算法是一种有效剔除误匹配点的方法,但需要人工确定部分参数值。提出一种多层次图像匹配方法,不仅能够剔除误匹配点还可以无需人工参与完成图像匹配。实验表明,该匹配方法能够精确提取匹配点和实现自动匹配,很好地满足图像配准的要求,具有一定的应用价值。     

Registration of real-time 3-D ultrasound images of the heart for novel 3-D stress echocardiography

Stress echocardiography is a routinely used clinical procedure to diagnose cardiac dysfunction by comparing wall motion information in prestress and poststress ultrasound images. Incomplete data, complicated imaging protocols and misaligned prestress and poststress views, however, are known limitations of conventional stress echocardiography. We discuss how the first two limitations are overcome via the use of real-time three-dimensional (3-D) ultrasound imaging, an emerging modality, and have called the new procedure "3-D stress echocardiography." We also show that the problem of misaligned views can be solved by registration of prestress and poststress 3-D image sequences. Such images are misaligned because of variations in placing the ultrasound transducer and stress-induced anatomical changes. We have developed a technique to temporally align 3-D images of the two sequences first and then to spatially register them to rectify probe placement error while preserving the stress-induced changes. The 3-D spatial registration is mutual information-based. Image registration used in conjunction with 3-D stress echocardiography can potentially improve the diagnostic accuracy of stress testing.     

Robust and accurate registration of 2-D electrophoresis gels using point-matching.

Point-matching is a widely applied image registration method and many algorithms have been developed. Registration of 2-D electrophoresis gels is an important problem in biological research that presents many of the technical difficulties that beset point-matching: large numbers of points with variable densities, large nonrigid transformations between point sets, paucity of structural information and large numbers of unmatchable points (outliers) in either set. In seeking the most suitable algorithm for gel registration we have evaluated a number of approaches for accuracy and robustness in the face of these difficulties. Using synthetic images we test combinations of three algorithm components: correspondence assignment, distance metrics and image transformation. We show that a version of the iterated closest point (ICP) algorithm using a non-Euclidean distance metric and a robust estimation of transform parameters provides best performance, equalling SoftAssign in the presence of moderate image distortion, and providing superior robustness against large distortions and high outlier proportions. From this evaluation we develop a gel registration algorithm based on robust ICP and a novel distance metric combining Euclidean, shape context and image-related features. We demonstrate the accuracy of gel matching using synthetic distortions of real gels and show that robust estimation of transform parameters using M-estimators can enforce inverse consistency, ensuring that matching results are independent of the order of the images.