基于多引导结构感知网络的深度补全

针对三维场景深度信息观测稀疏问题,提出一种融合彩色图像的多引导结构感知网络模型以补全稀疏深度.首先,利用三维平面法向量与场景梯度信息之间的映射关系,设计一种两分支主干网络框架,结合图像特征和几何特征进行深度预测,以充分提取空间位置信息的特征表示;然后,考虑到大范围场景下不同物体的结构差异性,基于网络通道注意力机制设计一种自适应感受野的结构感知模块,以对不同尺度的信息进行表征;最后,在网络采样的过程中,以不同尺寸图像为指导对预测子深度图进行滤波并修复物体的边缘细节.公开数据集上的实验结果表明,所设计的深度补全算法可以获得准确的稠密深度,同时通过两个下游感知任务进行深入评估,表明利用所提出方法能够有效提升其他感知任务的效果.

Depth completion method based on multi-guided structure-aware networks

Aiming at the problem of sparse depth information observation in 3D scenes, this paper proposes a multi-guided structure-aware network model fused with color images to complement the sparse depth. Using the mapping relationship between the 3D plane normal vector and the scene gradient information, we design a two-branch backbone network framework and combine image features and geometric features for depth prediction to fully extract the feature representation of spatial location information. Secondly, considering the structural differences of different objects in large-scale scenes, a network channel attention mechanism is designed. A structure-aware module with an adaptive receptive field is used to characterize information at different scales. Finally, in the process of network upsampling, the predicted sub-depth maps are filtered and the edge details of objects are repaired with the guidance of images of different sizes. The experimental results on public datasets show that the designed depth completion algorithm can obtain accurate dense depth. At the same time, through the in-depth evaluation of two downstream sensing tasks, the results show that the propsed method can effectively improve the effect of other sensing tasks.