增强细节的RGB‐IR多通道特征融合语义分割网络

现有基于深度学习的语义分割方法对于遥感图像的地物边缘分割不准确，小地物分割效果较差，并且RGB图像质量也会严重影响分割效果。提出一种增强细节的RGB-IR多通道特征融合语义分割网络MFFNet。利用细节特征抽取模块获取RGB和红外图像的细节特征并进行融合，生成更具区分性的特征表示并弥补RGB图像相对于红外图像所缺失的信息。在融合细节特征和高层语义特征的同时，利用特征融合注意力模块自适应地为每个特征图生成不同的注意力权重，得到具有准确语义信息和突出细节信息的优化特征图。将细节特征抽取模块和特征融合注意力模块结构在同一层级上设计为相互对应，从而与高层语义特征进行融合时抑制干扰或者无关细节信息的影响，突出重要关键细节特征，并在特征融合注意力模块中嵌入通道注意力模块，进一步加强高低层特征有效融合，产生更具分辨性的特征表示，提升网络的特征表达能力。在公开的Postdam数据集上的实验结果表明，MFFNet的平均交并比为70.54%，较MFNet和RTFNet分别提升3.95和4.85个百分点，并且对于边缘和小地物的分割效果提升显著。

Detail-Enhanced RGB-IR Multichannel Feature Fusion Network for Semantic Segmentation

Existing semantic segmentation methods based on deep learning are inaccurate for the edge segmentation of remote sensing images.The segmentation effect of small ground objects is poor, and the quality of RGB images seriously affects the segmentation effect.This study proposes a detail-enhanced RGB-IR multichannel feature fusion network for semantic segmentation, named MFFNet.The detail feature extraction module is used to obtain detailed features of RGB and infrared images.These are fused to generate a more distinctive feature representation and offset the missing information of RGB images relative to infrared images.While fusing the detail and high-level semantic features, the feature fusion attention module is used to adaptively generate different attention weights for each feature map to obtain an optimized feature map with more accurate semantic information and prominent detail information.The detail feature extraction and feature fusion attention module structures are designed to correspond to each other at the same level and suppress the influence of interference or irrelevant detail information when fusing high-level semantic features, highlight key detail features, and embed the channel attention module in the feature fusion attention module.This strengthens the effective fusion of high- and low-level features and generates a more discriminative feature representation, thereby improving the feature expression ability of the network.Experiments on the public Postdam dataset show that the mean intersection over union of MFFNet is 70.54%, which is an improvement of 3.95 and 4.85 percentage points compared with that of MFNet and RTFNet, respectively, and the segmentation effect for edges and small ground objects is significantly improved.