基于无监督深度学习的红外图像与可见光图像融合算法

红外和可见光图像表征了互补的场景信息. 现有的基于深度学习的融合方法大多通过独立提取网络分别提取两个源图像特征,从而丢失了源图像之间的深度特征联系. 基于此,提出了一种新的基于无监督深度学习的红外图像与可见光图像融合算法,针对不同模态的特点采用不同的编码方式提取图像特征,利用一个模态的信息补充另一个模态的信息,并对提取到的特征进行融合,最后根据融合特征重建融合图像. 该算法可在两个模态的特征提取路径之间建立交互,不仅可预融合梯度信息和强度信息,且能增强后续处理的信息. 同时设计了损失函数,引导模型保留可见光的细节纹理,并保持红外的强度分布. 将所提算法与多种融合算法在公开数据集上进行对比实验,结果表明,所提算法获得了良好的视觉效果,客观指标评价方面对比现有的优秀算法也有一定的提升.

Infrared Image and Visible Image Fusion Algorithm Based on Unsupervised Deep Learning

Infrared and visible images represent complementary scene information. Most of the existing deep learning-based fusion methods extract the feature of the two source images through independent extraction networks, which leads to the loss of deep feature relationships between source images. To solve this problem, a new infrared and visible image fusion algorithm based on unsupervised deep learning is proposed. Specifically, the proposed algorithm adopts different encoding approaches to extract image features according to the characteristics of different modalities, and uses the information of one modality to supplement that of another one. Then, the extracted features are fused, and finally the fused image is reconstructed according to the fused features. The algorithm can establish an interaction between the feature extraction paths of the two modalities, which can not only pre-fuse gradient information and intensity information, but also enhance the information for subsequent processing. A loss function is designed to guide the model to preserve the detailed texture of visible image and retain the intensity distribution of infrared image. The proposed algorithm is compared with a variety of fusion algorithms on the public dataset. The experimental results show that the proposed algorithm has achieved good visual effects, and that the objective evaluation is also improved compared with the existing excellent algorithms.