基于多尺度轮廓增强的RGB-IR双波段图像语义分割算法北大核心CSCD

语义分割被广泛应用于机器人、医学成像和自动驾驶等领域,但当前语义分割主要针对可见光图像。可见光图像在光照不足或天气差的情况下成像效果较差,而红外图像受光照影响较小。因此,将可见光图像和红外图像联合使用可以有效提升模型的鲁棒性。本文针对可见光/红外(RGB-IR)双波段图像语义分割任务中目标轮廓预测不准确的问题,提出一种基于多尺度轮廓增强的双波段语义分割算法。首先,本文提出一种新的位置和通道注意力模块EEFM,基于该模块可以高效地对多个尺度的融合特征分别进行轮廓预测。其次,本文将多尺度的预测结果用于对轮廓特征进行由高分辨率至低分辨率的逐步增强。最后,本文还提出了一种新的位置和通道注意力模块SAC对融合图像特征进行增强,以最终获得更准确的分割结果。实验在一个公开RGB-IR数据集以及一个自建数据集上进行,本文所提出的模型使用较小的参数量在公开数据库上取得了57.2的分割精度,综合性能达到了最高水平。同时,消融实验也验证了所提出的各模块的有效性。

RGB IR semantic segmentation based onmulti scale edge enhancement

Semantic segmentation is widely used in robotics,medical imaging,autonomous driving,and other fields.However,the existing methods mainly focus on visible (RGB) images.Visible (RGB) images are poorly imaged with insufficient light or poor weather,while infrared (IR) images are less affected by lights.Therefore,the combination of RGB and IR images can improve the robustness of the model.In this paper,a dual band semantic segmentation algorithm based on the so called multi scale edge enhancement mechanism is proposed to improve the ambiguity of the edge prediction.Firstly,a new edge detection module EEFM is introduced,which can efficiently predict the edges of multi scale fusion features.Secondly,multi scale edge predictions are used to enhance features from high resolution to low resolution progressively.Finally,a new spatial and channel attention module SAC is proposed to improve multi scale fusion features for more accurate segmentation results.The experiments are conducted on a public RGB IR dataset and a self built dataset.In this paper,the proposed model achieved 57.2 on the public dataset with a smaller size,and the overall performance reaches the highest level.Also,the ablation experiment verified the effectiveness of the proposed modules.