RAFNet: RGB-D attention feature fusion network for indoor semantic segmentation |
| |
| Affiliation: | 1. School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China;2. School of Engineering, Beijing University of Technology, Beijing 101303, China;1. School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China;2. School of Engineering, Beijing University of Technology, Beijing 101303, China;1. School of Journalism and New Media, Xi’an Jiaotong University, No.28, Xianning West Road, Xi''an, Shaanxi 710049, PR China;2. Departement of Information Management, Xidian University, 266 Xinglong Section of Xifeng Road, 710126 Xi’an, Shaanxi, China;3. UTT (Université de Technologie de Troyes) ICD/Tech-CICO Lab, BP 2060, 10010 Troyes, France;1. University of Southern Mississippi, Hattiesburg, MS 39406, USA;2. Department of Electrical Engineering, Indian Institute of Technology Jammu, Nagrota, Jammu 181221, India;3. Department of Electronics and Communication Engineering, National Institute of Technology Goa, Ponda, Goa 403401, India;4. Machine Intelligence Unit, Indian Statistical Institute, Kolkata 700108, India |
| |
| Abstract: | Semantic segmentation based on the complementary information from RGB and depth images has recently gained great popularity, but due to the difference between RGB and depth maps, how to effectively use RGB-D information is still a problem. In this paper, we propose a novel RGB-D semantic segmentation network named RAFNet, which can selectively gather features from the RGB and depth information. Specifically, we construct an architecture with three parallel branches and propose several complementary attention modules. This structure enables a fusion branch and we add the Bi-directional Multi-step Propagation (BMP) strategy to it, which can not only retain the feature streams of the original RGB and depth branches but also fully utilize the feature flow of the fusion branch. There are three kinds of complementary attention modules that we have constructed. The RGB-D fusion module can effectively extract important features from the RGB and depth branch streams. The refinement module can reduce the loss of semantic information and the context aggregation module can help propagate and integrate information better. We train and evaluate our model on NYUDv2 and SUN-RGBD datasets, and prove that our model achieves state-of-the-art performances. |
| |
| Keywords: | RGB-D semantic segmentation Three parallel branches Attention modules |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
