Multimodality image registration by maximization of quantitative–qualitative measure of mutual information

This paper presents a novel image similarity measure, referred to as quantitative–qualitative measure of mutual information (Q-MI), for multimodality image registration. Conventional information measures, e.g., Shannon's entropy and mutual information (MI), reflect quantitative aspects of information because they only consider probabilities of events. In fact, each event has its own utility to the fulfillment of the underlying goal, which can be independent of its probability of occurrence. Thus, it is important to consider both quantitative (i.e., probability) and qualitative (i.e., utility) measures of information in order to fully capture the characteristics of events. Accordingly, in multimodality image registration, Q-MI should be used to integrate the information obtained from both the image intensity distributions and the utilities of voxels in the images. Different voxels can have different utilities, for example, in brain images, two voxels can have the same intensity value, but their utilities can be different, e.g., a white matter (WM) voxel near the cortex can have higher utility than a WM voxel inside a large uniform WM region. In Q-MI, the utility of each voxel in an image can be determined according to the regional saliency value calculated from the scale-space map of this image. Since the voxels with higher utility values (or saliency values) contribute more in measuring Q-MI of the two images, the Q-MI-based registration method is much more robust, compared to conventional MI-based registration methods. Also, the Q-MI-based registration method can provide a smoother registration function with a relatively larger capture range. In this paper, the proposed Q-MI has been validated and applied to the rigid registrations of clinical brain images, such as MR, CT and PET images.     

Phase correlation pixel‐to‐pixel image co‐registration based on optical flow and median shift propagation

With singular value decomposition (SVD) and robust 2‐dimensional fitting phase correlation algorithms, it is possible to achieve pixel‐to‐pixel image co‐registration at sub‐pixel accuracy via local feature matching. However, the method often fails in featureless and low correlation areas making it not robust for co‐registration of images with considerable spectral differences and large featureless ground objects. A median shift propagation (MSP) technique is proposed to eliminate the problem, in a phase correlation and Normalized Cross‐Correlation (NCC) combined approach. The experiment results using images from different sensor platforms and spectral bands indicate that the new method is very robust to featureless and low correlation areas and can achieve very accurate pixel‐to‐pixel image co‐registration with good tolerance of spectral and spatial differences between images. The method will significantly improve change detection in various remote sensing applications.     

针对大视差图像拼接的显性子平面配准

目的 针对图像拼接中大视差图像难以配准的问题,提出一种显性子平面自动配准算法。方法 假设大视差图像包含多个显性子平面且每个平面内所含特征点密集分布。对该假设进行了验证性实验。所提算法以特征点分布为依据,通过聚类算法实现子平面分割,进而对子平面进行局部配准。首先,使用层次聚类算法对已匹配的特征点聚类,通过一种本文设计的拼接误差确定分组数目,并以各组特征点的聚类中心为新的聚类中心对重叠区域再聚类,分割出目标图像的显性子平面。然后,求解每个显性子平面的投影参数,并采用就近原则分配非重叠区域的单应性矩阵。结果 采用公共数据集对本文算法进行测试,并与Auto-Stitching、微软Image Composite Editor两种软件及全局投影拼接方法（Baseline）、尽可能投影算法（APAP）进行对比,采用均方根误差作为配准精度的客观评判标准。实验结果表明,该算法在拼接大视差图像时,能有效地配准局部区域,解决软件和传统方法由误配准引起的鬼影、错位等问题。其均方根误差比Baseline方法平均减小55%左右。与APAP算法相比,均方根误差平均相差10%左右,但可视化配准效果相同且无需调节复杂参数,可实现自动配准。结论 提出的显性子平面自动配准算法,通过分割图像所含子平面进而实现局部配准。该方法具有较高的配准精度,在大视差图像配准方面,优于部分软件及算法,可应用于图像拼接中大视差图像的自动配准。     

基于边缘几何特征的图像精确匹配方法

提出一组快速高精度计算切线斜率的五点公式，用以估计图像边缘曲线的角度特征，并利用角度直方图估计图像几何变换的旋转参数，实现具有大旋转差异图像间的粗匹配．在进行角度补偿后，利用灰度互相关判据搜索匹配点对，计算出几何变换参数，实现较高精度的旋转和平移校正，最后用松弛迭代法完成图像的精确匹配．与基于小波方向角特征的匹配方法相比，文中方法利用图像中主要的边缘信息实施匹配，具有较好的鲁捧性，可成功实现对各类具有较大相关程度图像间的精确匹配，对图形匹配也具有重要意义．     

基于SURF描述子的遥感影像配准

针对多源遥感影像的配准问题,提出一种基于SURF描述子的匹配算法,该算法具有稳健及高精度特性,采用SURF匹配算法得到同名点,利用一次多项式、Forstner算子和最小二乘原理由粗及精地剔除残差较大的同名点,利用提取的密集同名点构建三角网,对影像进行分块精确纠正。实验结果表明,该算法能够实现多源遥感影像的高精度配准。     

Jointly registering and fusing images from multiple sensors

In this paper, a novel approach is proposed for jointly registering and fusing a multisensor ensemble of images. Based on the idea that both groupwise registration and fusion can be treated as estimation problems, the proposed approach simultaneously models the mapping from the fused image to the source images and the joint intensity of all images with motion parameters at first, and then combines these models into a maximum likelihood function. The relevant parameters are determined through employing an expectation maximization algorithm. To evaluate the performance of the proposed approach, some representative image registration and fusion approaches are compared on different multimodal image datasets. The criterion, which is defined as the average pixel displacement from its true registered position, is used to compare the performances of registration approaches. As for evaluating the fusion performance, three fusion quality measures which are metric Q^ab/f, mutual information and average gradient are employed. The experimental results show that the proposed approach has improved performance compared to conventional approaches.     

Using selective correlation coefficient for robust image registration

A new method is proposed for robust image registration named Selective Correlation Coefficient in order to search images under ill-conditioned illumination or partial occlusion. A correlation mask-image is generated for selecting pixels of an image before matching. The mask-image can be derived from a binary-coded increment sign-image defined from any object-image and the template. The mask-rate of occluded regions is theoretically expected to be 0.5, while unoccluded regions have much lower rate than 0.5. Robustness for ill-conditioned environment can be realized since inconsistent brightness of occluded regions can be omitted from the mask operation. Furthermore, the mask enhancement procedure is proposed to get more stable robustness. The effectiveness of masking increases by the procedure, resulting in the rate around 0.7 for masking of occluded regions. This paper includes a theoretical modeling and analysis of the proposed method and some experimental results with real images.     

Wavelet-based image registration technique for high-resolution remote sensing images

Image registration is the process of geometrically aligning one image to another image of the same scene taken from different viewpoints at different times or by different sensors. It is an important image processing procedure in remote sensing and has been studied by remote sensing image processing professionals for several decades. Nevertheless, it is still difficult to find an accurate, robust, and automatic image registration method, and most existing image registration methods are designed for a particular application. High-resolution remote sensing images have made it more convenient for professionals to study the Earth; however, they also create new challenges when traditional processing methods are used. In terms of image registration, a number of problems exist in the registration of high-resolution images: (1) the increased relief displacements, introduced by increasing the spatial resolution and lowering the altitude of the sensors, cause obvious geometric distortion in local areas where elevation variation exists; (2) precisely locating control points in high-resolution images is not as simple as in moderate-resolution images; (3) a large number of control points are required for a precise registration, which is a tedious and time-consuming process; and (4) high data volume often affects the processing speed in the image registration. Thus, the demand for an image registration approach that can reduce the above problems is growing. This study proposes a new image registration technique, which is based on the combination of feature-based matching (FBM) and area-based matching (ABM). A wavelet-based feature extraction technique and a normalized cross-correlation matching and relaxation-based image matching techniques are employed in this new method. Two pairs of data sets, one pair of IKONOS panchromatic images from different times and the other pair of images consisting of an IKONOS panchromatic image and a QuickBird multispectral image, are used to evaluate the proposed image registration algorithm. The experimental results show that the proposed algorithm can select sufficient control points semi-automatically to reduce the local distortions caused by local height variation, resulting in improved image registration results.     

An efficient incremental strategy for constrained groupwise registration based on symmetric pairwise registration

Neuroimaging studies carried out on healthy or pathological cohorts generally require to map the set of all subject images in a common coordinate system thanks to some registration techniques. This is usually done by considering an arbitrarily chosen reference image (also called template) on which all other images are registered. However, the choice of the template can significantly impact the results and the interpretation of statistical comparisons between cohorts for both functional and morphometric studies. This is why we propose an efficient strategy for the automatic building of study-specific templates. The main contribution of this work is to propose a method, based on a symmetric formulation of the pairwise registration problem, that enables to enforce the template image to be at the geometric center of the population with little computational overhead. Moreover, the template image is estimated in an incremental way, thus being conveniently updatable when considering additional images. This property is of major interest for current clinical studies which involve very large databases that are constantly growing. Experiments on both synthetic and real data highlight the good convergence properties of the approach compared to a standard strategy based on pairwise registration. The benefit of using the proposed symmetric formulation in the standard template construction strategy is also pointed out.     

星载InSAR非相干图像对精确配准

为了解决星载InSAR非相干图像配准中不同区域对应不同形变参数的问题,将几种传统的SAR图像配准和光学图像配准的方法有机结合,提出了一种基于概略配准、Harris角点检测、相关系数匹配、Delaunay三角网剖分插值映射的InSAR复图像精确配准方法。通过仿真验证,该方法能够适用相干和非相干的InSAR复图像。配准精度达到0.1像素级。     

利用Kolmogorov-Smirnov统计的区域化图像分割

目的 为了在未知或无法建立图像模型的情况下,实现统计图像分割,提出一种结合Voronoi几何划分、K-S(Kolmogorov-Smirnov)统计以及M-H(Metropolis-Hastings)算法的图像分割方法.方法 首先利用Voronoi划分将图像域划分成不同的子区域,而每个子区域为待分割同质区域的一个组成部分,并利用K-S统计定义类属异质性势能函数,然后应用非约束吉布斯表达式构建概率分布函数,最后采用M-H算法进行采样,从而实现图像分割.结果 采用本文算法,分别对模拟图像、合成图像、真实光学和SAR图像进行分割实验,针对模拟图像和合成图像,分割结果精度均达到98%以上,取得较好的分割结果.结论 提出基于区域的图像分割算法,由于该算法中图像分割模型的建立无需原先假设同质区域内像素光谱测度的概率分布,因此提出算法具有广泛的适用性.为未知或无法建立图像模型的统计图像分割提供了一种新思路.     

一种应用于图像配准中大尺度位移估计的改进光流法

本文介绍了一种改进的光流法来对图像进行配准. 其新颖之处在于, 引入了一种初始运动估计器 (扩展相位相关法) 来改善光流法的性能. 光流法配准可达到亚像素级精度, 并能计算某些复杂的运动模式, 如 chirping 和 tilting, 但它处理大尺度位移的能力较弱. 相对而言, 因为扩展相位相关法可在像素级精度上对大尺度的旋转和平移进行估计, 且计算效率较高, 所以, 它可以作为光流法一种较好的运动预估器. 实验表明, 这种改进的光流法可显著提高配准精度, 特别是对存在大尺度位移的图像, 并且对随机噪声不敏感.     

基于改进CNN-RANSAC的水下图像特征配准方法

水对光的吸收和散射效应降低了水下图像的质量,水下图像的可视范围受到限制,复杂水下场景下的鲁棒性和精确性问题使得特征提取与匹配成为一项具有挑战性的任务。为了更好地配准水下图像,提出了一种改进CNN-RANSAC的水下图像特征配准方法,首先通过基于深度卷积神经网络的水下图像增强方法对水下图像进行增强预处理,通过水下图像分类数据集迁移学习训练VGGNet-16网络框架,利用修改后的网络框架进行特征提取,生成鲁棒的多尺度特征描述符与特征点,经过特征粗匹配与动态内点选择,使用改进的RANSAC方法剔除误匹配点。在大量水下图像数据集上进行了充分的特征提取和特征匹配实验,与基于SIFT和SURF的配准方法相比,该方法能够检测到更多的特征点,实现了匹配正确率的大幅度提高。     

小物体3维重建中影像与几何模型的配准

目的 近景摄影测量中的目标几何形状复杂,且拍摄影像的角度变化大,给影像与几何模型的配准带来了困难。传统单幅影像与几何模型配准的做法,由于缺乏自动粗配准的方法,效率相对较低。将多视影像首先统一坐标系的做法,在近景目标的复杂背景下也难以实现。方法 为此,将近景目标置于平面标定板上,利用相机标定的方法同时解算出影像的内外方位元素,实现多视影像坐标系的统一。然后人工选取3组以上同名点,做多视影像与几何模型的绝对定向,得到初始配准参数。最后使用多视影像与几何模型漫反射渲染图之间的归一化互信息作为相似性测度,用Powell非线性优化方法得到配准参数的精确值。结果 实验结果表明,使用标定板可以稳定地获取多视影像的内外方位元素,用绝对定向得到的配准参数进行影像和几何模型的交替显示仍然可以看到明显的裂缝,在经过互信息优化后裂缝现象得到明显改善。结论 多视影像与几何模型配准相比传统单幅影像与几何模型配准,人工选取同名点的工作量大大减少,由于人工选点存在误差,影响绝对定向的精度,使用归一化互信息作为测度进行非线性优化,可以获得更高的精度。     

基于角点特征和最大互信息的图像配准

基于互信息的配准方法是图像配准领域的重要方法。互信息是图像配准中常用的相似性度量,具有鲁棒、精度高等优点,但基于互信息的配准计算量大,制约了它的实际应用。文章提出一种基于角点和最大互信息配准方法:首先采用间接算法来计算曲率的极大值点,从而能快速准确的提取角点集;接着计算两幅图像角点集间的互信息,最后通过POWELL算法搜索使互信息最大以实现配准。实验表明,该算法计算简单,配准速度快,具有更好的精确性和鲁棒性。     

Fusion and registration of THEOS multispectral and panchromatic images

This article presents a new method for the fusion and registration of THEOS (Thailand Earth Observation Satellite) multispectral and panchromatic images in a single step. In the usual procedure, fusion is an independent process separated from the registration process. However, both image registration and fusion can be formulated as estimation problems. Hence, the registration parameters can be automatically tuned so that both fusion and registration can be optimized simultaneously. Here, we concentrate on the relationship between low-resolution multispectral and high-resolution panchromatic imagery. The proposed technique is based on a statistical framework. It employs the maximum a posteriori (MAP) criterion to jointly solve the fusion and registration problem. Here, the MAP criterion selects the most likely fine resolution multispectral and mapping parameter based on observed coarse resolution multispectral and fine resolution panchromatic images. The Metropolis algorithm was employed as the optimization algorithm to jointly determine the optimum fine resolution multispectral image and mapping parameters. In this work, a closed-form solution that can find the fused multispectral image with correcting registration is also derived. In our experiment, a THEOS multispectral image with high spectral resolution and a THEOS panchromatic image with high spatial resolution are combined to produce a multispectral image with high spectral and spatial resolution. The results of our experiment show that the quality of fused images derived directly from misaligned image pairs without registration error correction can be very poor (blurred and containing few sharp edges). However, with the ability to jointly fuse and register an image pair, the quality of the resulting fused images derived from our proposed algorithm is significantly improved, and, in the simulated cases, the fused images are very similar to the original high resolution multispectral images, regardless of the initial registration errors.     

基于SIFT特征的多帧图像超分辨重建

精确的亚像素级图像配准是图像超分辨重建中的关键问题.在图像超分辨重建中广泛使用的基于像素特征的光流法,对于大幅度运动场的计算很难做到精确的亚像素级配准.本文考虑了一种基于SIFT(scale invariant feature transform)特征的鲁棒性多帧图像超分辨重建算法.首先提取输入的低分辨待匹配图像对的SIFT关键点及其特征矢量.随后选取候选匹配关键点对,通过RANSAC(random sample consensus)鲁棒方法去除奇异值,并根据假设的平移性几何约束模型,获得图像对的平移运动配准参数,然后选取视场中心对应的或指定的图像帧为初始参考帧,再使用传统的超分辨重建框架获得最终的重建结果.仿真实验结果表明,提出的基于SIFT特征的图像超分辨重建方案是有效的,超分辨重建的图像质量在主观评价和客观指标上都获得了优于经典算法的效果.     

Pedestrian registration in static images with unconstrained background

This paper introduces a human body contour registration method for static pedestrian images with unconstrained backgrounds. By using a statistical compound model to impose structural and textural constraints on valid pedestrian appearances, the matching process is robust to image clutter. Experimental results show that the proposed method register pedestrian contours in complex backgrounds effectively.     

融合密集残差块和GAN变体的医学图像非刚性配准

目的 现有的医学图像配准算法在处理较大非刚性形变的医学图像时,存在配准精度低和泛化能力不足的问题。针对该问题,提出结合密集残差块和生成对抗网络（generative adversarial network,GAN）的图像配准方法,用于多模态医学图像的非刚性配准。方法 将密集残差块引入到生成器,提取待配准图像对的更多高层特征,从而提高配准精度;在GAN中采用最小二乘损失代替逻辑回归构造的交叉熵损失,由于最小二乘损失的收敛条件更严格,同时能缓解梯度消失和过拟合,从而提高配准模型的稳定性;在判别器网络中引入相对平均GAN（relative average GAN,RaGAN）,即在标准判别器的基础上增加一项梯度惩罚因子,降低判别器的判别误差,从而相对减少配准模型的配准误差,有助于稳定配准精度。结果 在DRIVE（digital retinal images for vessel extraction）数据集上进行训练和验证,在Sunybrook Cardiac数据集和Brain MRI数据集上进行泛化性能测试,并与几种主流医学图像非刚性配准算法进行对比实验,表明,本文配准模型在精度和泛化能力上均有一定程度的提升,相比其他方法,视网膜图像、心脏图像和脑部图像的配准Dice值分别提升了3.3%、3.0%、1.5%。结论 提出的配准方法能够获取更多高层特征信息,从而提升配准精度;同时基于最小二乘法构建损失函数并对判别器进行加强,能够使得配准模型在训练阶段快速收敛,提高了配准模型的稳定性和泛化能力,适合存在较大非刚性形变的医学图像的配准。