门控多层融合的实时语义分割

针对语义分割任务中因模型下采样过程中的像素损失而导致的上采样像素难以精确还原的问题,提出一种基于门控多层融合的实时语义分割方法.考虑分割的实时性,采用轻量级模型作为基础网络进行特征信息的提取.为解决像素难以精确还原问题,设计了一种横向连接的门控注意力结构,此结构可以对目标特征进行筛选,并通过横向传递增强上采样特征图信息的多样性,从而提高特征图的还原精度.此外,还提出采用多层融合结构来整合不同网络层的语义信息,利用不同网络层间的语义表达差异对缺失像素进行补充.实验以CamVid和VOC为数据集,以512×512大小的图像为输入,测试结果表明,方法的图像语义分割精度达到72.9%,平均分割速度为43.1帧/s.

Gated Multi-Layer Fusion for Real-Time Semantic Segmentation

In order to solve the problem that it is difficult to recover the pixels precisely in upsampling because of the loss of pixels caused by downsampling,a real-time semantic segmentation method based on gated multi-layer fusion is proposed.Considering the real-time of segmentation,we use the light model as the basic network to extract feature information.To improve the recovering accuracy of pixels,a gated attention structure is designed to horizontally connect the downsample lay to upsample lay,which can filter the features of object,and enhance the information diversity of the upsampling feature map.In addition,a multi-layer fusion structure is proposed to integrate the semantic information of different network layers,and is used to supplement the missing pixels by utilizing the semantic differences between different layers.The experimental results on CamVid and VOC datasets show that the segmentation accuracy of the proposed model is 72.9%,and the average segmentation speed is 43.1 frames per second,where the input image size of the network is 512×512.