基于深度学习的视频编码发展现状与未来展望

近年来,深度学习技术不仅在人工智能领域取得了巨大成功,也为视频编码领域带来了新的发展机遇。文章从两个方面介绍了深度学习技术在视频编码领域的发展现状,即传统编码框架下深度学习视频编码工具和以深度学习模型为基础的视频编码新框架,并对相关代表性工作进行了详细介绍和性能分析。最后,对深度学习视频编码技术面临的挑战和未来发展方向做了分析和展望。     

基于HEVC帧内预测的复杂度控制

帧内预测在视频编码中是非常重要的模块.在视频实时编码与传输过程中,场景切换会经常出现.此时,一般会采用全Ⅰ帧编码.研究发现,即使是全Ⅰ帧编码,也往往会非常耗时.基于编码单元深度范围和帧内预测中候选预测方向个数研究了HEVC编码器的复杂度控制问题.针对不同的目标编码复杂度,算法自适应地选择不同的方法来优化编码过程.实验结果表明,该算法在保证视频质量的前提下实现了对不同复杂度目标的控制.     

云平台上AVS视频转码系统的设计与实现

随着网络与宽带技术的飞速发展,数字视频呈现出海量化与多样化的特征.AVS作为我国自主音视频标准,编码效率优于同期国际标准,在保证图像质量的同时,便于视频数据的存储与传输.为了将数字视频进行高效的AVS转码,提出并实现了一种云平台上的AVS转码系统,该系统采用音视频分离方法将其他格式视频文件快速转码成AVS格式,并避免了转码文件中音视频内容间不同步问题,实验结果证明了方案的有效性.     

基于可变块运动矢量的边信息生成算法*

在分布式视频编码系统中,针对图像中细节丰富的区域易造成严重的块效应,提出了一种基于可变块运动矢量的边信息生成算法。根据前后相邻关键帧对应块的相关性,将像素块分为保留块和运动块。对保留块直接作保留处理,对运动块中的像素块继续进行分割并计算子块的初始运动矢量,最后将所有对应块的运动矢量进行加权自适应运动补偿得到改进的边信息。实验结果表明,对于运动较剧烈复杂的视频序列,该算法能够提高边信息生成质量, 并且使得改进后的边信息PSNR值提高了0.98~1.33 dB。     

A novel adaptive logic for dynamic adaptive streaming over HTTP

In this paper, we propose an estimation method that estimates the throughput of upcoming video segments based on variations in the network throughput observed during the download of previous video segments. Then, we propose a rate-adaptive algorithm for Hypertext Transfer Protocol (HTTP) streaming. The proposed algorithm selects the quality of the video based on the estimated throughput and playback buffer occupancy. The proposed method selects high-quality video segments, while minimizing video quality changes and the risk of playback interruption, improving user’s experience. We evaluate the algorithm for single- and multi-user environments and demonstrate that it performs remarkably well under varying network conditions. Furthermore, we determine that it efficiently utilizes network resources to achieve a high video rate; competing HTTP clients achieve equitable video rates. We also confirm that variations in the playback buffer size and segment duration do not affect the performance of the proposed algorithm.     

基于分块三维小波变换的视频图像序列编码方法的研究

该文给出了一种基于分块三维小波变换的视频图像序列编码方法。将视频图像序列中表示帧序的t坐标代换成z坐标后,可把一视频图像序列看成是三维空间中的体。将视频图像序列分成子块后,仿照二维图像小波变换的方法,将它作三维小波变换。变换后的图像能量主要集中于低频波段,这些波段对该视频图像序列的视觉效果影响最大。将不同波段按不同的精度量化并进行熵编码,可以达到去除帧内和帧间冗余、压缩数据的目的。试验表明,使用这种方法可以达到较好的压缩效果。此方法直观,速度也比较快。     

SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming

The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of the video streaming application over the error-prone network, because it does not consider the transmission induced distortion in the encoding process. In this paper, a SSIM-based error-resilient RDO scheme for H.264/AVC is proposed to improve the wireless video streaming performance. Firstly, with the help of the SSE-based RDO, we present a low-complexity Lagrange multiplier decision method for the SSIM-based RDO video coding in the error-free environment. Then, the SSIM-based decoding distortion of the user end is estimated at the encoder and is correspondingly introduced into the RDO to involve the transmission induced distortion into the encoding process. Further, the Lagrange multiplier is theoretically derived to optimize the encoding mode selection in the error-resilient RDO process. Experimental results show that the proposed SSIM-based error-resilient RDO can obtain superior perceptual video quality (more structural information) to the traditional SSE-based error-resilient RDO for wireless video streaming at the same bit rate condition.     

Cross-band noise model refinement for transform domain Wyner-Ziv video coding

Distributed Video Coding (DVC) is a new video coding paradigm, which mainly exploits the source statistics at the decoder based on the availability of decoder side information. One approach to DVC is feedback channel based Transform Domain Wyner-Ziv (TDWZ) video coding. The efficiency of current TDWZ video coding trails that of conventional video coding solutions, mainly due to the quality of side information, inaccurate noise modeling and loss in the final coding step. The major goal of this paper is to enhance the accuracy of the noise modeling, which is one of the most important aspects influencing the coding performance of DVC. A TDWZ video decoder with a novel cross-band based adaptive noise model is proposed, and a noise residue refinement scheme is introduced to successively update the estimated noise residue for noise modeling after each bit-plane. Experimental results show that the proposed noise model and noise residue refinement scheme can improve the rate-distortion (RD) performance of TDWZ video coding significantly. The quality of the side information modeling is also evaluated by a measure of the ideal code length.     

客观视频质量评估方法研究

介绍了三类客观视频质量评估方法的基本概念和实现方法;在此基础上,提出了一种基于H.264视频压缩算法的无参考评估的特征提取方法,并进行了实验论证,证明该方法能够有效结合编码算法的特性进行无参考视频质量评估;最后对视频质量评估方法的研究进行了总结和展望。     

Quality perception when streaming video on tablet devices

The proposed work aims at analyzing the quality perceived by the user when streaming video on tablet devices. The contributions of this paper are: (i) to analyze the results of subjective quality assessments to determine which Quality of Service (QoS) parameters mainly affect the users’ Quality of Experience (QoE) in video streaming over tablet devices; (ii) to define a parametric quality model useful in system control and optimization for the considered scenarios; (iii) to compare the performance of the proposed model with subjective quality results obtained in alternative state-of-the-art studies and investigate whether other models could be applied to our case and vice versa.     

3D Color Set Partitioning in Hierarchical Trees

This paper introduces the 3D color set partitioning in hierarchical trees (3D-CSPIHT) low bit rate embedded video coding scheme. The codec exploits the correlation between temporal and spatial wavelet coefficients and the interdependency between luminance and chrominance components to code color video sequences without the need for explicit bit allocation. Besides offering rate scalability, the new codec also produces multi-resolution scalable code streams. The hierarchical variable size block matching motion estimation technique is also integrated to demonstrate the motion estimation option with 3D-CSPIHT. The coding results show that 3D-CSPIHT produces better performance and visual quality compared to 3D-SPIHT.     

使用IPP函数提高手机视频编码性能

当前,如何提高居于LINUX平台手机的编码性能已经成为手机视频开发的重要一环.文章首先对手机开发的编码优化原则和优化步骤进行了介绍,然后提出了如何使用IPP函数提高居于LINUX平台手机视频编码性能的方法,通过测试表明编码性能得到了较大的提高,对于推进居于LINUX平台上的视频开发具有实际的参考价值.     

Experimental Analysis and Evaluation of RaptorQ Codes for Video Multicasting Over Wi-Fi

This paper presents a reliable and efficient high quality video streaming solution for use in challenging outdoor environments over Wi-Fi. An application layer forward error correction based on RaptorQ codes was implemented in a practical Wi-Fi based server and client system to enhance reliability. Thus, this paper presents the first detailed analysis on the implementation of RaptorQ codes for streaming high definition video over Wi-Fi. The measurements were performed in central Bristol with parameters such as RaptorQ symbol size, code rate, buffering time and modulation and coding scheme, and user quality of experience based on these parameters was evaluated. For multicast live video streaming it is demonstrated that system performance is mostly dominated by hardware and software limitations on constrained host platforms where the incoming packet rate exceeds the device`s ability to consume the traffic, i.e., Wi-Fi clients are a major source of packet loss, even in ideal channel conditions. Client limitations were found to be a function of modulation and coding schemes and RaptorQ coding parameters. Therefore, the optimum system design parameters such as RaptorQ symbol size, code rate and buffering time with respect to modulation and coding schemes were suggested considering practical limitations from real-world measurements.

     

Layered coding for ATM based video distribution systems

This paper discusses packet loss and its protection in an asynchronous transfer mode (ATM) based video distribution system. Packet losses in ATM based networks have such a great impact on the design of coding algorithms and network architectures that they should be exhaustively discussed and resolved. In this paper, first basic configuration of the ATM based video transmission system and its packet-loss protection schemes are discussed. The DCT based layered coding scheme with packet priority classification is proposed as an effective packet-loss protection scheme. Burstiness characteristics of the broadcast video sources are evaluated and modeled to clarify statistical multiplexing performance and packet-loss properties. The quality degradation caused by the packet losses is also evaluated by the SNR, and the superior performance of the proposed layered coding scheme is verified.     

End-to-end optimized TCP-friendly rate control for real-time video streaming over wireless multi-hop networks

Rate control is an important issue in video streaming applications. The most popular rate control scheme over wired networks is TCP-Friendly Rate Control (TFRC), which is designed to provide optimal transport service for unicast multimedia delivery based on the TCP Reno’s throughput equation. It assumes perfect link quality, treating network congestion as the only reason for packet losses. Therefore, when used in wireless environment, it suffers significant performance degradation because of packet losses arising from time-varying link quality. Most current research focuses on enhancing the TFRC protocol itself, ignoring the tightly coupled relation between the transport layer and other network layers. In this paper, we propose a new approach to address this problem, integrating TFRC with the application layer and the physical layer to form a holistic design for real-time video streaming over wireless multi-hop networks. The proposed approach can achieve the best user-perceived video quality by jointly optimizing system parameters residing in different network layers, including real-time video coding parameters at the application layer, packet sending rate at the transport layer, and modulation and coding scheme at the physical layer. The problem is formulated and solved as to find the optimal combination of parameters to minimize the end-to-end expected video distortion constrained by a given video playback delay, or to minimize the video playback delay constrained by a given end-to-end video distortion. Experimental results have validated 2–4 dB PSNR performance gain of the proposed approach in wireless multi-hop networks by using H.264/AVC and NS-2.     

Direction-adaptive fixed length discrete cosine transform framework for efficient H.264/AVC video coding

The 2D-discrete cosine transform (2D-DCT) is one of the popular transformation for video coding. Yet, 2D-DCT may not be able to efficiently represent video data with fewer coefficients for oblique featured blocks. To further improve the compression gain for such oblique featured video data, this paper presents a directional transform framework based on direction-adaptive fixed length discrete cosine transform (DAFL-DCT) for intra-, and inter-frame. The proposed framework selects the best suitable transform mode from eight proposed directional transform modes for each block, and modified zigzag scanning pattern rearranges these transformed coefficients into a 1D-array, suitable for entropy encoding. The proposed scheme is analysed on JM 18.6 of H.264/AVC platform. Performance comparisons have been made with respect to rate-distortion (RD), Bjontegaard metrics, encoding time etc. The proposed transform scheme outperforms the conventional 2D-DCT and other state-of-art techniques in terms of compression gain and subjective quality.     

Real-time 16K video coding on a GPU with complexity scalable BPC-PaCo

The advent of new technologies such as high dynamic range or 8K screens has enhanced the quality of digital images but it has also increased the codecs’ computational demands to process such data. This paper presents a video codec that, while providing the same coding features and performance as those of JPEG2000, can process 16K video in real time using a consumer-grade GPU. This high throughput is achieved with a technique that introduces complexity scalability to a bitplane coding engine, which is the most computationally complex stage of the coding pipeline. The resulting codec can trade throughput for coding performance depending on the user’s needs. Experimental results suggest that our method can double the throughput achieved by CPU implementations of the recently approved High-Throughput JPEG2000 and by hardwired implementations of HEVC in a GPU.     

Distributed and Adaptive Analog Coding for Video Broadcast in Wireless Cooperative System

Recently, some analog joint source-channel coding (AJSCC) schemes have been proposed to deal with cliff effect in wireless video broadcasting system. And wireless video broadcast with user cooperation tends to be a promising way to improve broadcast video quality in the near future. In this paper, we introduce a distributed and adaptive analog coding scheme called ACVC (adaptive cooperative video coding) based on AJSCC and with the concept of coset coding in distributed source coding, to improve the overall video broadcast quality in wireless cooperative system. Particularly, an adaptive packet discarding module is introduced to the framework to avoid video quality deterioration under severe channel conditions. And a model for quantization step selection of coset coding is built to minimize the redundancy in the cooperative signal and improve the anti-noise ability of the video. The experimental results show that, ACVC has stronger adaptability and thus obtains higher quality of broadcasted video than existing wireless cooperative schemes in the literature under different channel conditions.     

A variable bit rate video codec for asynchronous transfer modenetworks

The bandwidth flexibility offered by the asynchronous transfer mode (ATM) technique makes it possible to select picture quality and bandwidth over a wide range in a simple and straightforward manner. A prototype model of a video codec was developed that demonstrates the feasibility of both variable bit rate (VBR) coding and user-selectable picture quality. The VBR coding algorithm is discussed and it is shown how a stabilized quality is achieved and how this quality and associated bandwidth can be selected by the user. How error propagation is limited to reduce the visibility of cell losses is also discussed. Interfaces with the ATM network are analyzed, with emphasis on decoder synchronization and absorption of cell delay jitter. The VBR codec offers very good picture quality for videophony applications at an equivalent load of 5.9 Mb/s. Picture quality remains relatively constant, even for heavy motion