基于边缘PUnet网络的虚拟视图空洞修复

基于深度图的绘制(DIBR)是虚拟视点合成的关键技术, 但其生成的虚拟视图存在大面积连续的空洞. 传统的图像修复算法修复后的空洞缺乏语义感, 现有的部分卷积神经网络修复后的空洞边缘失真, 因此本文提出基于边缘信息的部分卷积神经网络修复算法. 首先本文利用视差移位来生成虚拟视图, 并对虚拟视图进行赋值和膨胀预处理操作, 以消除裂缝和伪影对后期空洞修复的影响, 然后在部分卷积神经网络中加入设计的边缘检测器, 使部分卷积神经网络重点学习图片的边缘部分, 最后利用学习好的网络模型修复虚拟视图中的大面积空洞. 实验结果表明本文方法可以对大面积连续的空洞进行修复, 修复后的空洞区域不仅具有语义感, 且边缘细节也更精细.

Virtual View Holes Inpainting Based on Edge PUnet Network

Depth Image Based Rendering (DIBR) is the key technology of virtual view synthesis, but the generated virtual views have large areas of continuous holes. The holes repaired by traditional image repair algorithms lack semantic sense, and the existing partial convolutional neural network distort the edge of the holes area, so this study proposes a partial convolution neural network inpainting algorithm based on edge information. First, the disparity shift is used to generate the virtual view, then the virtual view is assigned and expanded to eliminate the effects of cracks and artifacts on the later holes inpainting, and the edge detector is designed to the partial convolutional neural network which make the network focus on the edge part of the pictures. Finally we use the well learned network model to inpaint large area holes in the virtual view. The experimental results show that the method presented in this paper can repair large areas of continuous holes. The repaired holes area not only has a sense of semantics, but also has finer edge details.