基于双金字塔特征融合网络的RGB-D多类实例分割

针对RGB图像的实例分割任务在图像目标物体纹理相近但类别不同的区域可能出现分割错误的问题,引入Depth信息,结合RGB-D图像的三维几何结构特点,提出一种以双金字塔特征融合网络为框架的RGB-D实例分割方法.所提出的方法通过构建两种不同复杂度的金字塔深度卷积神经网络分别提取不同梯度分辨率大小的RGB特征及Depth特征,将对应分辨率大小的两种特征相加输入区域候选网络,以此改变输入区域候选网络层的共享特征,共享特征再经过分类、回归与掩码网络分支输出定位与分类结果,从而实现RGB-D图像的实例分割.实验结果表明,所提出的双金字塔特征融合网络模型能够完成RGB-D图像的实例分割任务,有效学习到深度图像与彩色图像之间的互补信息,与不包含Depth信息的Mask R-CNN相比,平均精度提高7.4%.

RGB-D multi-class instance segmentation based on double pyramid feature fusion model

For RGB images instance segmentation, some segmentation errors may occur in areas with similar textures but different categories. This paper introduces depth information and makes use of three-dimensional geometric features of RGB-D images, proposing the double pyramid feature fusion model. The method constructs two pyramid depth networks with different complexity to extract RGB and Depth features of different resolutions, then add two features of corresponding resolution. In this way, we change the input features of region Proposal network, then the classification network, regression network and mask network output positioning and classification results to get RGB-D images instance segmentation results. The experimental results show that the proposed model can learn the complementary information between depth images and RGB images, and get satisfactory RGB-D instance segmentation results. Compared to the mask R-CNN model that does not contain depth information, the average precision of the proposed model is increased by 7.4%.