基于运动估计与时空结合的多帧融合去雨网络

降雨天气会导致视觉质量下降,从而影响目标识别和追踪等视觉任务的处理效果.为了减小雨的影响,完成对运动视频背景细节的有效恢复,近年来相关研究者在视频去雨方向提出了很多方法.其中基于卷积神经网络的视频去雨方法使用最为广泛,它们大多采用单帧增强后多帧融合去雨的方式.但由于直接单帧增强使相邻帧之间部分像素的移动无法完成时间维度...

Motion-estimation Based Space-temporal Feature Aggregation Network for Multi-frames Rain Removal

Outdoor videos obtained under rainy weather cause visual quality degradation,which affects the processing effects of visual tasks such as object recognition and tracking.In order to enhance the quality of video and complete the effective recovery of the details in the motion video,many methods have been proposed in video rain removal.At this stage,most of the video rain removal methods based on convolutional neural networks employ single-frame enhancement and multi-frame fusion to remove rain.But the movement of some pixels between adjacent frames with direct enhancement is difficult to be completed in the temporal dimension.And the manner cannot effectively achieve end-to-end training,making the final result still relatively blurry and many detailed information losses.In order to effectively solve the above problems,this paper proposes a multi-frame fusion rain removal network based on the combination of motion estimation and space-temporal feature aggregation,ME-Derain for short.First,the optical flow estimation method is used to establish a reference frame to complete the alignment of adjacent frames,and then an encoder-decoder structure is introduced.The convolutional neural network connected by the residual connection and the time-related attention enhancement mechanism together form a multi-frame fusion network.Finally,the enhancement module related to the spatial sequence is used to obtain the rain removal video.A large number of experiments on different data sets show that the proposed method is better than most common methods at this stage and can obtain better rain removal effect.