Robust image registration by increment sign correlation

A novel and robust statistic as a similarity measure for robust image registration is proposed. The statistic is named as increment sign correlation because it is based on the average evaluation of incremental tendency of brightness in adjacent pixels. It is formalized to be a binary distribution or a Gaussian distribution for a large image size through statistical analysis and modeling. By utilizing the proposed statistical model, for example, we can theoretically determine a reasonable value of threshold for verification of matching. This sign correlation can also be proved to expectedly have the constant value 0.5 for any uncorrelated images to a template image, and then the property of the constancy can be utilized to analyze the high robustness for occlusion. The good performance for the case of saturation or highlight can also be proved through theoretical analysis and fundamental experiments. A basic algorithm for image scanning, search and registration over a large scene is represented with a technique for a fast version by the branch-and-bound approach. Many experimental evidences with real images are provided and discussed.     

一种多聚焦图像融合方法

提出一种基于小波变换的多聚焦图像融合方法．该方法首先通过小波变换得到源图像的小波系数，然后对低频和高频分量采用不同的融合规则和融合算子构造融合图像的小波系数，最后基于一致性检测由高低频融合系数得到最终的融合结果．用两组源图像数据验证所提出的融合算法，并与其他融合算法进行比较，仿真结果表明该算法在多聚焦图像融合中优于其他方法．     

Maximum Margin Multisurface Support Tensor Machines with application to image classification and segmentation

Virtually all previous classifier models take vectors as inputs, performing directly based on the vector patterns. But it is highly necessary to consider images as matrices in real applications. In this paper, we represent images as second order tensors or matrices. We then propose two novel tensor algorithms, which are referred to as Maximum Margin Multisurface Proximal Support Tensor Machine (M³PSTM) and Maximum Margin Multi-weight Vector Projection Support Tensor Machine (M³VSTM), for classifying and segmenting the images. M³PSTM and M³VSTM operate in tensor space and aim at computing two proximal tensor planes for multisurface learning. To avoid the singularity problem, maximum margin criterion is used for formulating the optimization problems. Thus the proposed tensor classifiers have an analytic form of projection axes and can achieve the maximum margin representations for classification. With tensor representation, the number of estimated parameters is significantly reduced, which makes M³PSTM and M³VSTM more computationally efficient when handing the high-dimensional datasets than applying the vector representations based methods. Thorough image classification and segmentation simulations on the benchmark UCI and real datasets verify the efficiency and validity of our approaches. The visual and numerical results show M³PSTM and M³VSTM deliver comparable or even better performance than some state-of-the-art classification algorithms.     

Multispectral and multiresolution image fusion using particle swarm optimization

Multispectral and multiresolution image fusion is important for many multimedia and remote sensing applications, such as video surveillance, medical imaging, and satellite imaging. For the commercial satellite ??IKONOS,?? spatial resolutions of high-resolution panchromatic (PAN) and low-resolution multispectral (MS) satellite images are 1?m and 4?m, respectively. To cope with color distortion and blocking artifacts in fused images, in this study, a multispectral and multiresolution image fusion approach using PSO is proposed. The pixels of fused images in the training set are classified into several categories based on the characteristics of low-resolution MS images. Then, the smooth parameters of spatial and spectral responses between the high-resolution PAN and low-resolution MS images are determined by PSO. All the pixels within each category are normalized by its own smooth parameter so that color distortion and blocking artifacts can be greatly reduced. Based on the experimental results obtained in this study, the overall visual quality of the fused images by the proposed approach is better than that by three comparison approaches, whereas the correlation coefficients, ?? _PAN , for the fused images by the proposed approach are greater than that by three comparison approaches.     

基于拉普拉高斯金字塔的数字图像融合研究

阐述了基于拉普拉斯金字塔图像融合的原理和方法,首先对原图像分别进行拉普拉斯金字塔分解,接着在对应的分层子图像上采用变换区域特征量测法进行融合,然后对得到的拉普拉斯金字塔反变换得到融合图像。通过使用MATLAB软件对本文研究的融合算法进行仿真实验,实验结果证明基于拉普拉斯金字塔图像融合方法得到的融合图像视觉效果良好,可视化清晰,图像质量得到了较大提高,该技术可广泛应用于多种研究领域。     

Pixel-based and region-based image fusion schemes using ICA bases

The task of enhancing the perception of a scene by combining information captured by different sensors is usually known as image fusion. The pyramid decomposition and the Dual-Tree Wavelet Transform have been thoroughly applied in image fusion as analysis and synthesis tools. Using a number of pixel-based and region-based fusion rules, one can combine the important features of the input images in the transform domain to compose an enhanced image. In this paper, the authors test the efficiency of a transform constructed using Independent Component Analysis (ICA) and Topographic Independent Component Analysis bases in image fusion. The bases are obtained by offline training with images of similar context to the observed scene. The images are fused in the transform domain using novel pixel-based or region-based rules. The proposed schemes feature improved performance compared to traditional wavelet approaches with slightly increased computational complexity.     

A fast recursive algorithm based on fuzzy 2-partition entropy approach for threshold selection

The fuzzy c-partition entropy approach for threshold selection is an effective approach for image segmentation. The approach models the image with a fuzzy c-partition, which is obtained using parameterized membership functions. The ideal threshold is determined by searching an optimal parameter combination of the membership functions such that the entropy of the fuzzy c-partition is maximized. It involves large computation when the number of parameters needed to determine the membership function increases. In this paper, a recursive algorithm is proposed for fuzzy 2-partition entropy method, where the membership function is selected as S-function and Z-function with three parameters. The proposed recursive algorithm eliminates many repeated computations, thereby reducing the computation complexity significantly. The proposed method is tested using several real images, and its processing time is compared with those of basic exhaustive algorithm, genetic algorithm (GA), particle swarm optimization (PSO), ant colony optimization (ACO) and simulated annealing (SA). Experimental results show that the proposed method is more effective than basic exhaustive search algorithm, GA, PSO, ACO and SA.     

Fusion of color and infrared video for moving human detection

We approach the task of human silhouette extraction from color and thermal image sequences using automatic image registration. Image registration between color and thermal images is a challenging problem due to the difficulties associated with finding correspondence. However, the moving people in a static scene provide cues to address this problem. In this paper, we propose a hierarchical scheme to automatically find the correspondence between the preliminary human silhouettes extracted from synchronous color and thermal image sequences for image registration. Next, we discuss strategies for probabilistically combining cues from registered color and thermal images for improved human silhouette detection. It is shown that the proposed approach achieves good results for image registration and human silhouette extraction. Experimental results also show a comparison of various sensor fusion strategies and demonstrate the improvement in performance over non-fused cases for human silhouette extraction.     

Progressive sharing for a secret image

Based on the wavelet transform, a new progressive sharing scheme is proposed to share a secret image into several shadow images using SPIHT encoding processes and Shamir’s threshold scheme. Quality refinement of the recovered image is achieved by the data consumed from the threshold number (r) of shadow images and each single shadow image reveals no information about the secret image. The size of each shadow image is smaller than 1/r of the secret image and any number of shadow images that is less than r reveals no information about the secret image. The proposed approach is secure for image sharing and provides excellent peak signal-to-noise ratio (PSNR) versus rate performance. Experimental results have demonstrated the promising performance of this method in progressive sharing.     

Fusing continuous spectral images for face recognition under indoor and outdoor illuminants

Novel image fusion approaches, including physics-based weighted fusion, illumination adjustment and rank-based decision level fusion, for spectral face images are proposed for improving face recognition performance compared to conventional images. A new multispectral imaging system is briefly presented which can acquire continuous spectral face images for our concept proof with fine spectral resolution in the visible spectrum. Several experiments are designed and validated by calculating the cumulative match characteristics of probe sets via the well-known recognition engine-FaceIt^®. Experimental results demonstrate that proposed fusion methods outperform conventional images when gallery and probes are acquired under different illuminations and with different time lapses. In the case where probe images are acquired outdoors under different daylight situations, the fused images outperform conventional images by up to 78%.     

可见光与红外图像区域级反馈融合算法

目的 可见光图像具有丰富的纹理信息,红外图像具有较强的目标指示信息,进行融合时只有合理地设计融合规则才能充分利用两者的互补信息,为此,提出一种基于效果评估的可见光与红外图像区域级反馈融合算法.方法 首先对待融合图像进行非下采样轮廓波变换(NSCT),将其分解为低频和高频部分.同时采用分形特征对红外图像进行人造目标增强,通过阈值分割得到目标区域与背景区域.在设计低频融合规则时,选取目标区域与背景区域的加权融合系数作为参数,根据图像融合效果评估的量化指标,运用遗传算法进行参数的优化求解.对高频部分采用基于区域的加权平均融合规则.最后,利用优化后的融合系数进行NSCT逆变换得到融合图像.结果 采用3组图像,结合主观评价和客观评价指标对4种融合算法的结果进行了比较分析,实验结果表明,本文算法融合后图像更自然,目标更显著,客观评价结果总体上最优.结论 本文算法有效结合了红外图像的目标信息与可见光图像的背景信息,融合图像具有更强的对比度,有利于进行战场态势显示和目标识别任务.     

Fusion and registration of THEOS multispectral and panchromatic images

This article presents a new method for the fusion and registration of THEOS (Thailand Earth Observation Satellite) multispectral and panchromatic images in a single step. In the usual procedure, fusion is an independent process separated from the registration process. However, both image registration and fusion can be formulated as estimation problems. Hence, the registration parameters can be automatically tuned so that both fusion and registration can be optimized simultaneously. Here, we concentrate on the relationship between low-resolution multispectral and high-resolution panchromatic imagery. The proposed technique is based on a statistical framework. It employs the maximum a posteriori (MAP) criterion to jointly solve the fusion and registration problem. Here, the MAP criterion selects the most likely fine resolution multispectral and mapping parameter based on observed coarse resolution multispectral and fine resolution panchromatic images. The Metropolis algorithm was employed as the optimization algorithm to jointly determine the optimum fine resolution multispectral image and mapping parameters. In this work, a closed-form solution that can find the fused multispectral image with correcting registration is also derived. In our experiment, a THEOS multispectral image with high spectral resolution and a THEOS panchromatic image with high spatial resolution are combined to produce a multispectral image with high spectral and spatial resolution. The results of our experiment show that the quality of fused images derived directly from misaligned image pairs without registration error correction can be very poor (blurred and containing few sharp edges). However, with the ability to jointly fuse and register an image pair, the quality of the resulting fused images derived from our proposed algorithm is significantly improved, and, in the simulated cases, the fused images are very similar to the original high resolution multispectral images, regardless of the initial registration errors.     

基于提升小波变换的医学图像融合

目的 将不同模态的医学图像(如CT/MRI图像)进行科学融合,可以有效地丰富图像的信息,提高信息的利用效能,这对于医学临床诊断具有重要的理论研究意义和应用价值。方法 基于提升小波变换的特性,对多模态医学图像的融合算法进行研究。首先,对已配准的源图像进行多尺度分解,得到低频子带和多层高频子带;进而,根据低频子带的特点和各层高频子带的噪声含量不同,提出了低频子带系数采用基于区域平均能量的加权融合规则;对噪声含量较低的低层高频子带采用基于计盒分维法获取分维数,而对噪声含量较高的高层高频子带提出了基于区域梯度能量加权融合规则。结果 分别对灰度图像和彩色图像进行了大量融合实验,并分别在主观视觉特性及客观评价指标下对不同融合算法产生的融合图像的质量进行了分析对比,表明本文算法具有较好的边缘保持度。结论 实验结果表明,较现有算法产生的融合图像,应用本文融合算法得到的图像具有更丰富的信息,更能使图像灰度级分散,具有更良好的视觉特性和评价指标。     

A new image fusion performance metric based on visual information fidelity

Because subjective evaluation is not adequate for assessing work in an automatic system, using an objective image fusion performance metric is a common approach to evaluate the quality of different fusion schemes. In this paper, a multi-resolution image fusion metric using visual information fidelity (VIF) is presented to assess fusion performance objectively. This method has four stages: (1) Source and fused images are filtered and divided into blocks. (2) Visual information is evaluated with and without distortion information in each block. (3) The visual information fidelity for fusion (VIFF) of each sub-band is calculated. (4) The overall quality measure is determined by weighting the VIFF of each sub-band. In our experiment, the proposed fusion assessment method is compared with several existing fusion metrics using the subjective test dataset provided by Petrovic. We found that VIFF performs better in terms of both human perception matching and computational complexity.     

Research on fundus image registration and fusion method based on nonsubsampled contourlet and adaptive pulse coupled neural network

We present a registration and fusion method of fluorescein fundus angiography image and color fundus image which combines Nonsubsampled Contourlet (NSCT) and adaptive Pulse Coupled Neural Network (PCNN). Firstly, we register two images by Speeded Up Robust Features (SURF) feature points, the nearest neighbor and the next nearest neighbor distance ratio method to eliminate the spatial difference between the source images. Secondly, we use Random Sample Consensus (RANSAC) algorithm to achieve precise matching of feature points. Then, according to the transformation parameters obtained by RANSAC algorithm, we perform spatial transformation on the floating image to complete the registration. Finally, we obtain the low-frequency sub-band and high-frequency sub-band of the image to be fused by NSCT decomposition. The low-frequency sub-band is fused by the regional energy. The high-frequency sub-bands are studied using a simplified-PCNN model and the Particle Swarm Optimization algorithm. The link strength of the simplified-PCNN is an improved Laplacian energy and the images are fused based on the number of times the pixels are ignited. The proposed method has higher average gradient (AG) value and information entropy (IE) value and lower relative global dimensional synthesis error (ERGAS) than the existing fusion methods of the fundus image. The fusion image can accurately synthesize the image information, clarify the performance of the details, and has better spectral quality in the spectral range. The image of fused provides an effective reference for the clinical diagnosis of fundus diseases.

     

一种混杂的图像配准方法

提出一种基于特征和亮度的混杂图像配准方法．首先对源图像进行小波分解,在小波金字塔最顶层的低频近似子图像上进行角点检测,通过仿射变换模型得到初始配准参数,然后将这一参数逐层进行迭代精炼,直至达到金字塔最底层,得到最终精确的配准参数．将这些不同配准方法结合起来,可以克服单独使用任一方法存在的不足,理论分析和实验结果表明,这种混杂图像配准算法精度较高．     

Image registration by compression

Image registration consists in finding the transformation that brings one image into the best possible spatial correspondence with another image. In this paper, we present a new framework for image registration based on compression. The basic idea underlying our approach is the conjecture that two images are correctly registered when we can maximally compress one image given the information in the other. The contribution of this paper is twofold. First, we show that image registration can be formulated as a compression problem. Second, we demonstrate the good performance of the similarity metric, introduced by Li et al., in image registration. Two different approaches for the computation of this similarity metric are described: the Kolmogorov version, computed using standard real-world compressors, and the Shannon version, calculated from an estimation of the entropy rate of the images.     

融合密集残差块和GAN变体的医学图像非刚性配准

目的 现有的医学图像配准算法在处理较大非刚性形变的医学图像时,存在配准精度低和泛化能力不足的问题。针对该问题,提出结合密集残差块和生成对抗网络（generative adversarial network,GAN）的图像配准方法,用于多模态医学图像的非刚性配准。方法 将密集残差块引入到生成器,提取待配准图像对的更多高层特征,从而提高配准精度;在GAN中采用最小二乘损失代替逻辑回归构造的交叉熵损失,由于最小二乘损失的收敛条件更严格,同时能缓解梯度消失和过拟合,从而提高配准模型的稳定性;在判别器网络中引入相对平均GAN（relative average GAN,RaGAN）,即在标准判别器的基础上增加一项梯度惩罚因子,降低判别器的判别误差,从而相对减少配准模型的配准误差,有助于稳定配准精度。结果 在DRIVE（digital retinal images for vessel extraction）数据集上进行训练和验证,在Sunybrook Cardiac数据集和Brain MRI数据集上进行泛化性能测试,并与几种主流医学图像非刚性配准算法进行对比实验,表明,本文配准模型在精度和泛化能力上均有一定程度的提升,相比其他方法,视网膜图像、心脏图像和脑部图像的配准Dice值分别提升了3.3%、3.0%、1.5%。结论 提出的配准方法能够获取更多高层特征信息,从而提升配准精度;同时基于最小二乘法构建损失函数并对判别器进行加强,能够使得配准模型在训练阶段快速收敛,提高了配准模型的稳定性和泛化能力,适合存在较大非刚性形变的医学图像的配准。     

结合最佳缝合线和多分辨率融合的图像拼接

目的 针对图像拼接过程中,缝合线通过运动物体或配准不准确区域等情况导致融合图像出现鬼影、重影的问题,提出了一种基于差异图像加权的改进最佳缝合线算法,采用基于多分辨率和加权平均的分区图像融合算法解决了拼接线问题。方法 首先将两幅图像的重叠区域划分为缝合线区域和过渡区域;在缝合线区域内,使用差异图像加权的最佳缝合线搜索准则构建准则值图像,基于动态规划思想来搜索得到最佳缝合线;基于缝合线生成掩码图像,并对重叠区域图像进行扩展,采用多分辨率融合算法实现了非严格重叠区域的融合;在过渡区域采用加权平均算法来消除拼接线。结果 采用含有大量运动物体的图像序列对算法进行测试,实验结果表明,基于差分图像加权的最佳缝合线有效避开了大部分运动物体,当缝合线难以绕开运动物体时,能够尽量少地穿过运动物体;通过多分辨率和加权平均融合算法消除了拼缝等问题。结论 提出的最佳缝合线算法能够有效地避免缝合线通过运动物体、配准不准确的区域,将多分辨率图像融合算法应用于非严格重叠图像融合,能够合成高质量的全景图像。     

Indoor Scene Reconstruction from Sets of Noisy Range Images

This paper describes a system for building 3D models of indoor scenes from sets of noisy laser range images. It addresses several important aspects of this problem, namely, preprocessing, which includes image segmentation and planar model fitting; view registration, which is the method of determining the rigid transformation that describes the relative pose of the camera platform between views; and reconstruction, which is the subsequent integration or fusion of separate range images into a single 3D model. Our proposed strategy is to use a statistical sensor model. We thus account for noise properties of the data at each stage in the reconstruction process, which produces reliable results even in the presence of significant measurement noise. We give an empirical analysis of a plane-based registration method and present results using real range data that demonstrate the performance of the entire reconstruction system.