基于深层特征学习的高效率视频编码中帧内快速预测算法

高效视频编码(HEVC)标准相对于H.264/AVC标准提升了压缩效率,但由于引入的编码单元四叉树划分结构也使得编码复杂度大幅度提升。对此,该文提出一种针对HEVC帧内编码模式下编码单元(CU)划分表征矢量预测的多层特征传递卷积神经网络(MLFT-CNN),大幅度降低了视频编码复杂度。首先,提出融合CU划分结构信息的降分辨率特征提取模块;其次,改进通道注意力机制以提升特征的纹理表达性能;再次,设计特征传递机制,用高深度编码单元划分特征指导低深度编码单元的划分;最后建立分段特征表示的目标损失函数,训练端到端的CU划分表征矢量预测网络。实验结果表明,在不影响视频编码质量的前提下,该文所提算法有效地降低了HEVC的编码复杂度,与标准方法相比,编码复杂度平均下降了70.96%。     

基于深层特征学习的高效率视频编码中帧内快速预测算法

高效视频编码(HEVC)标准相对于H.264/AVC标准提升了压缩效率,但由于引入的编码单元四叉树划分结构也使得编码复杂度大幅度提升.对此,该文提出一种针对HEVC帧内编码模式下编码单元(CU)划分表征矢量预测的多层特征传递卷积神经网络(MLFT-CNN),大幅度降低了视频编码复杂度.首先,提出融合CU划分结构信息的降分辨率特征提取模块;其次,改进通道注意力机制以提升特征的纹理表达性能;再次,设计特征传递机制,用高深度编码单元划分特征指导低深度编码单元的划分;最后建立分段特征表示的目标损失函数,训练端到端的CU划分表征矢量预测网络.实验结果表明,在不影响视频编码质量的前提下,该文所提算法有效地降低了HEVC的编码复杂度,与标准方法相比,编码复杂度平均下降了70.96％.     

采用光流特征对齐的压缩视频超分辨率重建

针对在有限带宽条件下,传输视频时,往往需要先将视频下采样为低分辨率视频再进一步压缩,从而导致视频质量严重降低的问题,提出了一种采用光流特征对齐的压缩视频超分辨率重建方法,能够有效恢复下采样和压缩过程造成的信息损失,最终获得高质量的重建视频。首先将压缩后的低分辨率视频通过质量增强网络进行去压缩处理,其次通过超分辨率重建网络将质量增强的低分辨率视频恢复至原始分辨率。上述网络均采用基于光流的特征对齐,并且特征以双向传播的方式在网络中进行传播。实验结果表明,相比于高效视频编码（High Efficiency Video Coding,HEVC）标准和传统的超分辨率重建算法,当量化参数（Quantization Parameter,QP）为32时,所提方法在随机接入配置下,峰值信噪比（Peak Signal-to-Noise Ratio,PSNR）平均可提升0.92 dB,有效提升了压缩视频的质量。     

超分辨率重建在低码率视频压缩编码中的应用

在低码率段,HEVC压缩过程会造成大量视频信息的丢失,进而导致压缩视频中存在比较严重的失真.针对这一问题,我们提出了一种基于HEVC的低码率视频压缩编码框架,该框架主要包括以下步骤:在编码端,首先对视频进行两倍下采样,然后进行HEVC编码;在解码端,采用LSM-MNLR(Laplacian scale mixture modeling and Multi-non-local low-rank regularization)对解码后的视频进行去噪以及利用超分辨率重建方法将视频重建到原始尺寸.实验证明,相比HEVC,我们的低码率视频压缩编码框架在低码率段能取得更好的主客观效果.     

一种基于组合神经网络的HEVC帧间预测方法

高效视频编码(HEVC)作为最新视频编码标准,有着非常高的压缩效率,但是由于各种新技术的提出,其编码复杂度也大大提高。复杂度对视频编码有着重要意义,低复杂度编码的研究非常必要。利用神经网络进行HEVC的分区预测为低复杂度编码提供了有效的解决方案。文中提出了一种基于卷积神经网络(CNN)和长短期记忆网络(LSTM)的组合网络架构来对帧间分区进行预测的方法,利用自建数据库对网络进行训练;文中设计了一种预搜索模块来建立训练数据库,仿真结果表明,神经网络的精度可达87%,利用该网络架构进行帧间预测可以实现52%~71%的复杂度节省。     

浅析超高清数字电视视频压缩编码技术

近期超高清数字电视成为各国研究的热点,由于超高清电视具有超高分辨率的特点,造成了其信号数据量庞大,原视频编码标准H.264/AVC已经无法满足高压缩比的要求,必须采用新的视频压缩编码方案。新一代视频压缩标准HEVC正是面向高清和超高清视频图像而被提出,它可以保证在与H.264相同视频质量的前提下,使得视频流的码率减少50%。重点介绍了超高清电视的视频压缩编码技术HEVC,包括HEVC的四叉树块分割结构、帧内预测编码技术和滤波技术。     

测控通信系统中低延迟视频编码传输方法研究

为降低测控通信系统延迟对无人飞行器性能影响,在深入分析影响测控通信视频编码传输延迟基础上,提出了一种适合于测控通信传输的低时延视频编码传输帧结构。算法引入帧内刷新编码方式将I帧码流平均到其他P帧中,使得压缩后各帧编码码流尽可能得到平均,从而减少编码发送缓冲区延迟;根据slice编码方式的独立性将一帧图像分成多个slice并行编码模式以减少编码处理延迟;同时去除双向预测编码B帧模式以减少帧排序延迟。实验采用新一代HEVC标准进行仿真分析,结果表明算法在不明显降低视频编码质量前提下,使得压缩后各帧码流大小尽可能平均,有效地减少了测控通信系统中视频压缩传输延迟。     

全新视频压缩标准将于明年出台

《Via Satellite》,2012年9月刊下一代视频压缩编码技术——高效视频编码(HEVC,或H.265)的国际标准将于2013年1月正式出台,兼容HEVC标准的接收机和机顶盒最快将于2014年上半年面世。与MPEG-4(H.264)标准相比,HEVC技术的压缩效率将提高30%～50%,意味着传输同等画质视频的带宽需求可能将降低到现在的一半。由于卫星系统的带宽相对有限且成本较高,卫星运营商一直乐于尝试最新的视频压缩技术。目前,美国卫星电视运营商"回声星"(EchoStar)公司的全部高清电视频道均已采用MPEG-4编码,使其成为下一代视频压缩编码技术——高效视频编码(HEVC,或H.265)的国际标     

采用双网络结构的压缩视频超分辨率重建

在实际应用中,为了节省带宽和方便存储,图像和视频通常被下采样和压缩,而降质的图像与视频无法满足人们的实际需求。针对这一问题,采用了一种双网络结构的超分辨率重建方法,首先建立下采视频与压缩后的低分辨率视频的映射关系,然后建立质量增强的压缩视频与原始视频的映射关系,最终在输出端可以得到质量提升的视频帧。在网络中,采用密集残差块来提取压缩视频中丰富的局部分层特征,并结合全局残差学习恢复视频中的高频信息。在压缩环节,采用高性能视频编码来验证所提算法的有效性。实验结果表明,相比于主流的视频编码标准和先进的超分辨率重建算法,所提方法能有效提升编码视频的率失真性能。     

HEVC帧内预测模式和分组码的视频信息隐藏

为了在最新的视频压缩标准－高效视频编码(HEVC) 下实现信息隐藏并在保证隐秘信息嵌入容量的前提下减少对宿 主信息的修改,提出基于帧内预测模式和分组码的HEVC视频信息隐藏方法。首先,利用分组 码的标准阵 列译码方法,建立(4,3)码标准阵列译表;然后,根据译码表与预测模式的映射关系,调 制帧内4×4亮 度块的预测模式嵌入隐秘信息;最后,对调制后的4×4亮度块重新编码,使得在连续4个4× 4帧内亮度 块嵌入3bit隐秘信息,平均修改1.25位预测模式,减小因调制预测模式对视频造成的影响 。实验结果表 明:所提算法的PSNR值下降在0.05 dB以内,码率增长在1%左右 ,算法能很好的保证视频主客观质量,对视频的编码比特率影响很小。     

Improving video quality by predicting inter-frame residuals based on an additive 3D-CNN model

Video compression is essential for uploading videos to online platforms which usually have bandwidth limitations. However, the compression reduces the visual quality. To overcome this problem, the visual quality of the low bitrate compressed videos for various standards, including H.264 and HEVC in decoders, needs to be improved. Accordingly, this paper proposes a novel method for improving video quality based on 3D convolutional neural networks (CNNs). This method is totally compatible with the encoders of video compression standards, i.e., H.264, VVC, and HEVC, and can be implemented easily. In particular, the proposed neural network model receives five frames of the low bitrate compressed video as input and subsequently predicts the compression error of frames using the first and fifth frames. Finally, it reconstructs an improved version of the frame with high quality. The CNN is an Additive (3D) model that can predict the eliminated inter-frame redundancies resulting from compression. Our goal is to increase the peak signal to noise ratio (PSNR) and structural index similarity (SSIM) of the luminance (Y) and chrominance (U, V) frames in the video. Additive 3D-CNN achieves an average of 12.4%, 9.9% and 5% BD-rate increases for LP, LB and RA for the Y component. The results indicate that the new proposed algorithm outperforms the previous methods in terms of PSNR, SSIM, and BD-rate.     

基于模式对应与机器学习的HEVC降分辨率转码算法

HEVC是ITU-T VCEG 继H.264之后所制定的新的视频编码标准,它提高了视频的编码效率,在相同视频质量的前提下,压缩比与H.264相比提高了一倍。另外,随着4G网络的兴起和智能手机的普及,移动终端成为人们观看网上视频的一大主流平台。但是,网络中存储的视频分辨率普遍要大于移动终端屏幕分辨率,为解决这个问题,本文开展了针对HEVC的降分辨率转码研究工作,利用高分辨率视频的编码信息,通过模式对应来简化低分辨率视频的编码模式的计算过程,并采用机器学习的方法来确定降分辨率时的组块阈值,以提高模式对应的准确性。实验结果表明,提出的算法与Trivial transcoder相比,在保持PSNR和比特率几乎不变的同时,编码时间平均节省了60%左右。     

Deep learning based HEVC in-loop filter and noise reduction

The lossy compression techniques at low bit rate often create ringing and contouring effects on the output images and introduce various blurring and distortion at block bounders. To overcome those compression artifacts different neural network based post-processing techniques have been experimented with over the last few years. The traditional loop-filter methods in the HEVC frame-work support two post-processing operations namely a de-blocking filter followed by a sample adaptive offset (SAO) filter. These operations usually introduce extra signaling bits and become overhead to the network with high-resolution video processing. In this study, we came up with a new deep learning-based algorithm for SAO filtering operations and substantiated the merits of the proposed method. We introduced a variable filter size sub-layered dense CNN (SDCNN) to improve the denoising operation and incorporated large stride deconvolution layers for further computation improvement. We demonstrate that our deconvolution model can effectively be trained by leveraging the high-frequency edge features learned in a shallow network using residual learning and data augmentation techniques. Extensive experiments show that our approach outperformed other state-of-the-art approaches in terms of SSIM, Bjøntegaard delta bit-rate (BD-BR), BD-PSNR measurements on the standard video test set and achieves an average of 8.73 % bit rate saving compared to HEVC baseline.     

Intra mode prediction for H.266/FVC video coding based on convolutional neural network

The next-generation video compression standard H.266/Future Video Coding (FVC) provides high compression efficiency in terms of the cost of computing the optimal intra mode from 67 modes. We propose an intra mode prediction method based on a convolutional neural network (CNN). An input image set of 20 × 20 blocks is used to train the CNN; the CNN is used to predict the best classes of intra mode direction. The CNN architecture comprises two convolutional layers and a fully connected layer. Compared with the default fast search method in FVC, the proposed method can achieve a 0.033% decrease in Bjøntegaard delta bit rate (BDBR) with only a slight increase in time.     

ViSTRA2: Video coding using spatial resolution and effective bit depth adaptation

We present a new video compression framework (ViSTRA2) which exploits adaptation of spatial resolution and effective bit depth, down-sampling these parameters at the encoder based on perceptual criteria, and up-sampling at the decoder using a deep convolution neural network. ViSTRA2 has been integrated with the reference software of both the HEVC (HM 16.20) and VVC (VTM 4.0.1), and evaluated under the Joint Video Exploration Team Common Test Conditions using the Random Access configuration. Our results show consistent and significant compression gains against HM and VVC based on Bjønegaard Delta measurements, with average BD-rate savings of 12.6% (PSNR) and 19.5% (VMAF) over HM and 5.5% (PSNR) and 8.6% (VMAF) over VTM.     

视频质量增强模型加速算法

提出了一种应用于视频质量增强算法的动态结构性剪裁算法Maskcut,它可以有效提高基于深度学习的视频质量增强算法的运行速度。Maskcut是一种通用的剪裁思路,支持绝大多数的基于卷积神经网络(CNN)深度学习网络模型的剪裁加速。基于原模型中已经训练好的参数数据,Maskcut使用一种针对剪裁加速的二次训练策略来进一步微调参数,从而在保证模型有效性损失不大的同时,缩短模型运行时间。以一种先进的视频质量增强算法——多帧质量增强2.0(MFQE 2.0)为目标,Maskcut剪裁后可以快速达到峰值信噪比(PSNR)指标损失低于1%、时间缩短10%以上的加速指标。     

Early Coding Unit–Splitting Termination Algorithm for High Efficiency Video Coding (HEVC)

A new‐generation video coding standard, named High Efficiency Video Coding (HEVC), has recently been developed by JCT‐VC. This new standard provides a significant improvement in picture quality, especially for high‐resolution videos. However, one the most important challenges in HEVC is time complexity. A quadtree‐based structure is created for the encoding and decoding processes and the rate‐distortion (RD) cost is calculated for all possible dimensions of coding units in the quadtree. This provides a high encoding quality, but also causes computational complexity. We focus on a reduction scheme of the computational complexity and propose a new approach that can terminate the quadtree‐based structure early, based on the RD costs of the parent and current levels. Our proposed algorithm is compared with HEVC Test Model version 10.0 software and a previously proposed algorithm. Experimental results show that our algorithm provides a significant time reduction for encoding, with only a small loss in video quality.     

Video frame interpolation using deep cascaded network structure

Video frame interpolation is a technology that generates high frame rate videos from low frame rate videos by using the correlation between consecutive frames. Presently, convolutional neural networks (CNN) exhibit outstanding performance in image processing and computer vision. Many variant methods of CNN have been proposed for video frame interpolation by estimating either dense motion flows or kernels for moving objects. However, most methods focus on estimating accurate motion. In this study, we exhaustively analyze the advantages of both motion estimation schemes and propose a cascaded system to maximize the advantages of both the schemes. The proposed cascaded network consists of three autoencoder networks, that process the initial frame interpolation and its refinement. The quantitative and qualitative evaluations demonstrate that the proposed cascaded structure exhibits a promising performance compared to currently existing state-of-the-art-methods.