基于多尺度特征残差学习卷积神经网络的视频超分辨率方法 |
| |
引用本文: | 林琦,陈婧,曾焕强,朱建清,蔡灿辉. 基于多尺度特征残差学习卷积神经网络的视频超分辨率方法[J]. 信号处理, 2020, 36(1): 50-57. DOI: 10.16798/j.issn.1003-0530.2020.01.007 |
| |
作者姓名: | 林琦 陈婧 曾焕强 朱建清 蔡灿辉 |
| |
作者单位: | 华侨大学信息科学与工程学院,厦门市移动多媒体通信重点实验室 |
| |
基金项目: | 国家自然科学基金(61802136,61871434);福建省自然科学基金(2017J05103);福建省杰出青年科学基金项目(2019J06017);福建省教改项目(FBJG20180038);泉州市高层次人才项目(2017G027);华侨大学中青年教师科研资助项目(ZQN YX403,ZQN PY418);华侨大学高层次科研人才资助项目(16BS709);华侨大学研究生科研创新能力培育计划资助项目(18014082026)
|
| |
摘 要: | 本文提出了一种基于多尺度特征残差学习卷积神经网络的视频超分辨率方法,考虑到视频帧间的时空相关性,所提的方法采用由双三次插值预处理后的连续五帧视频作为卷积神经网络的输入,经由网络重建中间帧作为输出,依次按顺序重建直至获得整个高分辨率视频。本文所提出的卷积神经网络主要由多尺度特征提取、残差学习、亚像素卷积层、残差连接(skip-connection)四大部分组成,通过对视频的多尺度特征的提取获得更丰富的不同尺度特征和残差学习达到较好地恢复高频信息的目的。本文采用峰值信噪比(PSNR)和结构相似性指数(SSIM)作为损失函数优化网络。实验结果表明,本方法在平均评价指标上较其他方法均有一定的提升(PSNR +3.151dB,SSIM +0.102),从主观评价上看可以有效地减少视频边缘模糊的现象。
|
关 键 词: | 视频超分辨率 卷积神经网络 多尺度特征 残差学习 |
收稿时间: | 2019-08-15 |
Video Super-Resolution Method Based on Multi-Scale Characteristics Residual Learning Convolutional Neural Network |
| |
Affiliation: | School of Information Science and Engineering, Huaqiao University, Xiamen Key Laboratory of Mobile Multimedia Communications |
| |
Abstract: | A video super-resolution method based on multi-scale characteristics residual learning convolutional neural network is pro-posed in this paper. Considering the spatial-temporal relationships between frames of video, five adjacent frames of video is preprocessed by bicubic interpolation as the input of the proposed convolutional neural network; After the network, the re-constructed intermediate frame is output, reconstructed in order till the whole high-resolution video is obtained. The pro-posed convolutional neural network is composed of four parts, including multi-scale feature extraction, residual learning, sub-pixel convolutional layer, and skip-connection. With multi-scale characteristics extraction and the residual learning, the proposed method provides abundant features and restoring more high-frequency information. Peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are used as the loss function to optimize the network. Experimental results have shown that compared with latest algorithms, the proposed method has a significant improvement on the average evalu-ation indicator (PSNR +3.151dB, SSIM +0.102) and effectively reduces the edge blurring phenomenon of the subjective visual effect. |
| |
Keywords: | |
|
| 点击此处可从《信号处理》浏览原始摘要信息 |
|
点击此处可从《信号处理》下载免费的PDF全文 |
|