基于ResNet的红外与低照度可见光图像融合方法

针对传统红外与低照度可见光图像融合后，容易造成目标模糊不清、细节信息缺失等问题，本文提出一种低照度可见光图像预增强与残差网络(Residual Network, ResNet)相结合的图像融合方法。该方法首先利用单尺度Retinex(Single Scale Retinex, SSR)算法对低照度可见光图像进行增强预处理，得到增强的可见光图像。其次，利用ResNet-50分别从增强后的可见光图像和红外图像中提取深度特征。然后，采用L1范数对生成的深度特征进行正则化处理，并通过上采样操作将其分辨率恢复至输入图像大小，得到权重图。最后，使用加权平均策略获取融合图像。实验结果表明，本文算法能更好地保留输入图像的纹理细节和结构信息；使用TNO数据集与现有的三种典型算法对比，该算法融合结果的离散余弦特征互信息(FMI_dct)、小波特征互信息(FMI_w)、基于噪声评估的融合性能(Nabf)、结构相似度测量(SSIM)四种客观指标总体优于对比算法。

Infrared and low illumination visible image fusion method based on ResNet

Aiming at the problems of blurred targets and missing details after fusion of traditional infrared and low illumination visible images,an image fusion method combining low illumination visible image pre enhancement and residual network(ResNet)is proposed in this paper.Firstly,the Single Scale Retinex(SSR)algorithm is used to enhance the low illumination visible image to obtain the enhanced visible image.Secondly,ResNet 50 is used to extract the depth features from the enhanced visible image and infrared image respectively.Then,L1 norm is used to regularize the generated depth features,and its resolution is restored to the size of the input image through up sampling operation to obtain the weight map.Finally,the weighted average strategy is used to obtain the fusion image.The experimental results show that the proposed algorithm can better preserve the texture details and structural information of the input image.Compared with the existing three typical algorithms using TNO data set,the four objective indexes of the fusion results of the algorithms,namely discrete cosine feature mutual information(FMIdct),wavelet feature mutual information(FMIw),fusion performance based on noise evaluation(Nabf)and structural similarity measurement(SSIM),are generally better than the comparison algorithms.