Steerable local frequency based multispectral multifocus image fusion

Design of a focus measure and a fusion algorithm which will perform well across different spectra remains an extremely challenging task. In this work, the problem of multispectral multifocus image fusion is addressed using the phase information of the source image pixels at different orientations. We make the local frequency, the spatial derivative of the local phase of the pixels, steerable to obtain a good novel focus measure. Oriented analytic image based on the theory of steerable filters is constructed for that purpose. A multifocus fusion algorithm is proposed next using this focus measure. Comprehensive experimentations clearly demonstrate that our focus measure as well as the multifocus fusion algorithm yield promising results across the visual (VIS), the near-infrared (NIR) and the thermal (TH) spectra.     

A region-based multi-sensor image fusion scheme using pulse-coupled neural network

For most image fusion algorithms split relationship among pixels and treat them more or less independently, this paper proposes a region-based image fusion scheme using pulse-coupled neural network (PCNN), which combines aspects of feature and pixel-level fusion. The basic idea is to segment all different input images by PCNN and to use this segmentation to guide the fusion process. In order to determine PCNN parameters adaptively, this paper brings forward an adaptive segmentation algorithm based on a modified PCNN with the multi-thresholds determined by a novel water region area method. Experimental results demonstrate that the proposed fusion scheme has extensive application scope and it outperforms the multi-scale decomposition based fusion approaches, both in visual effect and objective evaluation criteria, particularly when there is movement in the objects or mis-registration of the source images.     

Fast cross-spectral image registration using new robust correlation

In this paper, we explore a new correlation technique for cross-spectral image registration. The proposed technique matches the orientation feature of the second derivatives while making use of a statistical robust M estimator. Furthermore, it takes advantage of Fourier and multi-resolution techniques to reduce the complexity of spatial correlation. Simulation results show that our proposed approach gives more accurate results than the mutual information, and the normalized cross-correlation with prefiltering in terms of speed and accuracy.     

Object's contour detection scheme based on image fusion

Three novel object's contour detection schemes based on image fusion are proposed in this paper. In these schemes an active contour model is applied to detect the object's contour edge. Since an object's contour in an infrared (IR) image is usually clearer than that in a visible image, the convergent active contour in a visible image is improved with that in an IR image. The first contour detection scheme is realized by revising the shape-preserving active contour model. The second scheme minimizes the B-spline L ₂ norm's square of the difference of the B-spline control point vectors in two modal images. Contour tracking and extraction experiments indicate that the first scheme outperforms the second one. Moreover, a third scheme based on the active contour and pixel-level image fusion is proposed for images with incomplete but complementary scene information. An example using contour extraction of a partially hidden tank proves its efficacy.     

Combine crossing matching scores with conventional matching scores for bimodal biometrics and face and palmprint recognition experiments

In this paper, we define for the first time the crossing matching score of two biometrics traits and combine it with the conventional matching scores to perform personal authentication. The proposed method is very suitable for the bimodal biometrics systems with two similar biometrics traits such as the system with visible light and infrared face images and the system with palm images captured at two bands. The proposed method first runs for the first and biometrics traits, respectively. For each of these two biometrics traits, the proposed method calculates the matching scores between the testing sample and each training sample. The matching scores generated from the first and second traits are referred to as the first and second matching scores, respectively. Second, the proposed method calculates the crossing matching scores, i.e. the matching scores between the testing sample of the second biometrics trait and the training samples of the first biometrics trait. Finally, we use a weighted fusion scheme to combine the first, second and crossing matching scores for personal authentication.     

Combining multiple classifiers based on third-order dependency for handwritten numeral recognition

Storing and estimating high order probability distribution of classifiers and class labels is exponentially complex and unmanageable, so we rely on an approximation scheme using the dependency. As an extension of the second-order dependency approach, the probability distribution is optimally approximated by the third-order dependency and then multiple classifiers are combined by such third-order dependency approximation.     

基于小波变换多聚焦图像融合算法改进

传统于小波变换多聚焦图像融合算法忽略了低频分量融合处理,这样会导致边缘失真问题.为了解决这个问题,文章提出了一种改进小波变换多聚焦图像融合算法,该算法的核心思想是对待融合图像进行多尺度分解,这样得到的图像就包含了低频图像,在对高频图像进行处理的同时,对低频图像使用局部区域梯度信息的方法进行处理.测试结果表明,和传统的融合方法相比本方法的融合结果效果更好.     

Gradient entropy metric and p-Laplace diffusion constraint-based algorithm for noisy multispectral image fusion

Noise is easily mistaken as useful features of input images, and therefore, significantly reducing image fusion quality. In this paper, we propose a novel gradient entropy metric and p-Laplace diffusion constraint-based method. Specifically, the method is based on the matrix of structure tensor to fuse the gradient information. To minimize the negative effects of noise on the selections of image features, the gradient entropy metric is proposed to construct the weight for each gradient of input images. Particularly, the local adaptive p-Laplace diffusion constraint is constructed to further suppress noise when rebuilding the fused image from the fused gradient field. Experimental results show that the proposed method effectively preserves edge detail features of multispectral images while suppressing noise, achieving an optimal visual effect and more comprehensive quantitative assessments compared to other existing methods.     

基于小波变换的红外显微图像融合

图像融合是指把来自多传感器数据的互补信息合并形成一幅新的图像,以便达到进行目标监视与识别等目的。本文提出了基于小波变换的图像融合方法,并分析了可见光与红外的显微图像的融合结果。结果融合效果不错,结果图像比较清晰,在一图像中即可得到比较完整的信息。因而,这一方法具有广阔的应用前景。     

Application of image correction and bit-plane fusion in generalized PCA based face recognition

A novel generalized PCA based face recognition algorithm is proposed in this paper. Two approaches to improve the illumination robustness of the algorithm are presented, symmetrical image correction (SIC) and bit-plane feature fusion (BPFF). Specifically, for an assumed eudipleural face image, SIC first compares a pixel with the mean of this pixel and its symmetrical one and constructs a weight using the difference, then performs correction of the face image by adding the weight image to it to reduce bright speckles and shadows caused by over lighting. BPFF decomposes a face image into its eight bit-planes and extracts outline features and texture features respectively from them, then it constructs a new virtual face by combining those two features. Finally, Generalized PCA is applied to the virtual faces to achieve face recognition. Experimental results show that, the proposed combined approach can effectively reduce the sensitivity of face recognition algorithm to illumination variances and thus fewer projection vectors are required to achieve the same recognition rate than the comparing approaches.     

A general framework for image fusion based on multi-scale transform and sparse representation

In image fusion literature, multi-scale transform (MST) and sparse representation (SR) are two most widely used signal/image representation theories. This paper presents a general image fusion framework by combining MST and SR to simultaneously overcome the inherent defects of both the MST- and SR-based fusion methods. In our fusion framework, the MST is firstly performed on each of the pre-registered source images to obtain their low-pass and high-pass coefficients. Then, the low-pass bands are merged with a SR-based fusion approach while the high-pass bands are fused using the absolute values of coefficients as activity level measurement. The fused image is finally obtained by performing the inverse MST on the merged coefficients. The advantages of the proposed fusion framework over individual MST- or SR-based method are first exhibited in detail from a theoretical point of view, and then experimentally verified with multi-focus, visible-infrared and medical image fusion. In particular, six popular multi-scale transforms, which are Laplacian pyramid (LP), ratio of low-pass pyramid (RP), discrete wavelet transform (DWT), dual-tree complex wavelet transform (DTCWT), curvelet transform (CVT) and nonsubsampled contourlet transform (NSCT), with different decomposition levels ranging from one to four are tested in our experiments. By comparing the fused results subjectively and objectively, we give the best-performed fusion method under the proposed framework for each category of image fusion. The effect of the sliding window’s step length is also investigated. Furthermore, experimental results demonstrate that the proposed fusion framework can obtain state-of-the-art performance, especially for the fusion of multimodal images.     

Locality guided cross-modal feature aggregation and pixel-level fusion for multispectral pedestrian detection

Multispectral pedestrian detection has received much attention in recent years due to its superiority in detecting targets under adverse lighting/weather conditions. In this paper, we aim to generate highly discriminative multi-modal features by aggregating the human-related clues based on all available samples presented in multispectral images. To this end, we present a novel multispectral pedestrian detector performing locality guided cross-modal feature aggregation and pixel-level detection fusion. Given a number of single bounding boxes covering pedestrians in both modalities, we deploy two segmentation sub-branches to predict the existence of pedestrians on visible and thermal channels. By referring to the important locality information in the reference modality, we perform locality guided cross-modal feature aggregation to learn highly discriminative human-related features in the complementary modality by exploring the clues of all available pedestrians. Moreover, we utilize the obtained spatial locality maps to provide prediction confidence scores in visible and thermal channels and conduct pixel-wise adaptive fusion of detection results in complementary modalities. Extensive experiments demonstrate the effectiveness of our proposed method, outperforming the current state-of-the-art detectors on both KAIST and CVC-14 multispectral pedestrian detection datasets.     

一种基于小波变换的图像融合新算法

给出了一种基于小波变换的图像融合方法,对小波分解后的低频分量通过度量其图像块空间频率和对比度来确定融合图像的低频分量,对分解后得到的高频分量,选择高频系数时,基于绝对值最大的原则,并对选择结果进行一致性验证,最后重构得到融合图像。从仿真结果可以看出,给出的方法很好地保留了多幅原图像的有用信息,融合图像清晰度和对比度都较好,是一种有效的图像融合算法。     

单幅近红外手掌图像掌静脉和掌纹多特征识别

传统的掌静脉和掌纹图像融合识别一般需分别采集掌静脉和掌纹两类图像,而单幅近红外手掌图像中实际上同时包含了掌静脉和掌纹结构信息。由于二者局部纹理细节差异较大,且像素值分布范围不同,因此,可以先分离再分别增强处理。首先,提出了改进的引导滤波算法以便去除掌纹结构,并设计了反模糊细节增强模型增强掌静脉结构图像;然后,提出了一种改进的分块增强算法,可以在增强掌纹结构图像的同时滤除掌静脉结构信息,再利用基于Sobel算子的反锐化掩模算法以便突出掌纹主线条结构信息;最后,对单幅近红外手掌图像中获取的掌静脉和掌纹图像进行融合识别。在香港理工大学近红外手掌数据库上进行了实验,结果表明：所提出的算法识别率达到了99.63%,与其他已有算法相比等误率平均降低了0.66%,验证了所提出算法的有效性。     

多源遥感图像像素级融合相关技术研究

多源遥感图像融合作为图像融合领域的一个重要分支,已成为遥感技术领域的研究热点。介绍了多源遥感图像融合技术的发展现状以及图像融合技术的基本理论,并分析了图像融合前的预处理工作。基于多分辨率分析的图像融合方法在现有的普遍的图像融合方法中应用极为广泛,是非常重要的一类算法。通过对融合图像的主观和客观评价标准的详细介绍,并采用定性定量相结合的评价标准对融合效果进行系统的评价。     

Attribute filter based infrared and visible image fusion

Infrared and visible image fusion is an effective image processing technique to obtain more comprehensive information, which can help people better understand various scenarios. In this paper, a novel infrared and visible image fusion method is proposed which fully considers the attributes of objects in source images. Benefiting from the attribute and the edge-preserving filters, the prominent objects in the infrared source image are effectively extracted. Then, the weight-based Laplacian pyramid fusion strategy is adopted to get more natural fusion results. The experimental results on the public image fusion datasets and a new infrared–visible video fusion dataset show that the proposed method achieves state-of-the-art fusion performance in terms of both visual and objective evaluations. The proposed algorithm is also implemented in an infrared–visible dual sensor system, which demonstrated the practicability of our fusion method.     

基于脊波的多光谱和全色图像融合方法研究

应用了双线性插值的矩形阵列到径向阵列的变换算法,给出了一个离散脊波变换的实现方法,将其应用于多光谱图像与全色图像的融合算法中,通过清晰度、灰度方差、信息熵三个方面,将算法结果与小波变换的结果进行了对比,实验结果表明,相对于小波变换而言,脊波变换能更好地处理线和面的奇异性,而且由融合的结果来看,脊波变换得到的结果在清晰度等方面要高于小波变换。     

基于多特征融合的医学图像识别研究*

以医学图像为研究对象,针对任何一类特征都不能很好地表达医学图像的缺点以及进一步提高医学图像的识别率,提出了一种基于特征级数据融合与决策级数据融合相结合的分类方法。实验结果表明,采用特征级数据融合,融合后的特征可以较好地表达医学图像,且减少了后期分类的计算量;采用决策级数据融合,取得了比单个分类器更高的识别率。     

一种基于小波变换的图像融合新算法*

在阐述小波图像融合算法的基础上,针对小波分解后各频域融合算子和融合规则的选择,提出一种新的基于小波分解的图像融合算法。对小波分解后的低频分量通过度量其图像块之间的相关系数和空间频率来确定融合图像的尺度系数;对高频分量,以方向对比度为判据确定融合图像的小波系数;最后,通过小波逆变换得到融合图像。对多组图像进行实验,实验结果表明,该方法是有效的。     

利用局部极值点特性的多聚焦图像快速融合

目的：多聚焦图像融合技术一个关键问题是如何准确地判断待融合图像的清晰度。本文提出了基于归一化结构极值点数目的清晰度判断准则。方法：本文基于图像的局部极值点特性,定义了归一化结构极值点数目这个指标作为清晰度判断准则,同时还给出了利用该准则和融合决策矩阵快速估计技术的多聚焦图像快速融合方法。结果：利用本文提出的清晰度判断准则和融合方法,实验表明上述问题得到了很好的解决。结论：本文提出了一个新的图像清晰度判断准则,该准则判断准确率高,且对脉冲噪声有好的鲁棒性。通过与传统融合方法对两组实验图像融合结果的主客观比较表明,该方法的融合速度和效果比现有多聚焦图像融合方法有明显提高。