Motion vector refinement in a Wyner--Ziv to H.264 transcoder for mobile telephony

The authors develop a decoder/encoder system (transcoder) to solve the consumption constraint in the communications between end-user devices, when a new Wyner?Ziv (WZ)/H.264 framework is defined for being used in mobile-to-mobile environments. This approach is based on leaving to the devices only WZ video encoding and traditional video decoding; the lowest complexity algorithms in both paradigms. The system shifts the burden of complexity to the network, where an improved transcoder that reuses information between both paradigms is allocated. The WZ decoding motion vectors are used to reduce the H.264 motion estimation process. The proposed transcoder offers a complexity reduction up to 60% on average, without any rate distortion drop.     

多视点视差估计中基于图像块特征矢量和相似尺度的参考视点选择方法

在多视点视频处理中，采用多参考视点的视差估计方法能够在很大程度上消除各个视点间的冗余性，达到对多视点视频数据有效编码压缩的目的。研究了多视点信号处理中视差估计的多参考视点选择问题，采用快速Walsh变换谱构造图像块特征矢量，并提出了基于图像块特征和相似尺度的多参考视点选择方法。实验结果证明，本文方法能在保证编码质量的前提下，减少视差估计的计算复杂度。     

Distributed video coding with adaptive two-step side information generation for smart and interactive media

Conventional video coding standards, such as MPEGx and H.26x, use a hybrid architecture of block-based motion compensation and discrete cosine transform (DCT) within the structure of a complex encoder and a simple decoder. Contrary to conventional video codecs, Wyner-Ziv (WZ) video coding, which is a practical application of distributed video coding (DVC) based on the Slepian-Wolf and WZ theorems, exploits the source correlation at the decoder, thereby allowing for the development of simpler encoders. However, the current WZ video coding algorithms cannot match the coding performance of conventional video coding. In order to improve the coding performance of transform-domain WZ video coding, an adaptive two-step side-information generation method is thus proposed for evaluation and analysis in this study. The proposed method uses decoded WZ frames in the down-sampled spatial resolution along with already decoded coefficients to successively improve rate-distortion (RD) performance as the decoding progresses. The experimental results show that the proposed method, compared to previous WZ video coding method, shows improved coding performance, particularly under critical conditions, such as cases with high motion content.     

基于H.264的复杂度-失真最优的运动估计算法

为了满足嵌入式、移动设备的实时视频编码应用，提出一种可分级的复杂度-失真最优的运动估计算法，通过调整复杂度控制参数来实现视频编码器计算复杂度的可分级性，确保视频质量达到最优，在编码效率最高和失真最小间取得相对最佳平衡。实验结果表明，随着处理器计算能力的变化，该算法能自动调整编码器的计算复杂度，大大减少运动估计的运算量，同时图像质量和码率变化不大。     

A frame-level encoder rate control scheme for transform domain Wyner-Ziv video coding

Available distributed video coding codecs are mostly based on decoder rate control scheme where the parity bits for decoding can be achieved over a feedback channel. Meanwhile, the frequent requests over feedback channel increase the transmission delay. The feedback-free distributed video coding, relying on encoder rate control in literatures, has overcome the aforementioned shortcoming. However, when performing parity bitrate estimation and other operations, the feedback-free distributed video coding systems based on bit-plane usually require high precision of bitrate estimation and high quality of side information at the encoder. In this paper, we propose a frame-level distributed video coding system based on encoder rate control. The innovations include three parts: 1) an adaptive coding mode selection algorithm is proposed, which utilizes both temporal and spatial correlation and reduces the complexity of encoder; 2) a bit-plane rearrangement method is adopted, which makes the coding rate on each bit-plane homogeneous and effectively reduces the accuracy requirement of the parity bitrate prediction and improves the efficiency of rate estimation; 3) a frame-level parity bitrate estimation scheme is presented to enhance the efficiency of rate estimation on the basis of a look-up table. Numerical results verify that the proposed scheme remarkably improves the rate distortion performance of distributed video coding at low bitrate.     

使用对极几何的多视角分布式视频编码

为了降低多视角分布式视频编码中解码器复杂度,解决视角间边信息的获取精度等问题,将一种新的基于对极几何的视差估计算法用于多视角分布式视频编码系统。提出的系统通过对视角间的冗余信息使用对极几何约束,采用新的视差搜索开始点,限制垂直方向的搜索范围以加快搜索速度。实验结果表明,提出的系统相对于运动JPEG（MJPEG）编码的主要运算绝对差值和（SAD）计算次数减少10倍,相对H.263+减少7倍。同时提出系统相对MJPEG编码有6～7 dB增益,相对H.263+有3 dB增益。     

Statistical motion learning for improved transform domain Wyner-Ziv video coding

Wyner-Ziv (WZ) video coding is a particular case of distributed video coding (DVC), the recent video coding paradigm based on the Slepian-Wolf and Wyner-Ziv theorems which exploits the source temporal correlation at the decoder and not at the encoder as in predictive video coding. Although some progress has been made in the last years, WZ video coding is still far from the compression performance of predictive video coding, especially for high and complex motion contents. The WZ video codec adopted in this study is based on a transform domain WZ video coding architecture with feedback channel-driven rate control, whose modules have been improved with some recent coding tools. This study proposes a novel motion learning approach to successively improve the rate-distortion (RD) performance of the WZ video codec as the decoding proceeds, making use of the already decoded transform bands to improve the decoding process for the remaining transform bands. The results obtained reveal gains up to 2.3 dB in the RD curves against the performance for the same codec without the proposed motion learning approach for high motion sequences and long group of pictures (GOP) sizes.     

H.266/VVC中一种改进的仿射运动估计算法

通用视频编码标准H.266/VVC通过引入多种新的编码技术,如仿射运动补偿预测、自适应运动矢量精度、多核变换等,以支持360°视频和HDR视频的编解码,从而为用户提供最优的视频质量,但是在H.266/VVC帧间预测过程中,仿射运动估计计算复杂度高导致编码时间显著增加。针对该问题,提出一种改进的仿射运动估计算法。通过对仿射高级矢量预测（AAMVP）候选列表的构建过程进行改进,并构建一种AAMVP候选列表候选项筛选准则,使得列表的候选项更接近编码块真实的运动矢量,从而缩短编码时间。同时对仿射运动估计中迭代搜索最优仿射运动矢量的迭代过程进行优化,以加快迭代搜索速度。实验结果表明,在低时延的编码器配置下,相比VVC原始算法,当BD-BR增加了0.023%时,该算法的总体编码时间平均缩短13%,在保证编码质量的前提下能够有效降低编码的计算复杂度。     

一种基于H.264的快速运动估计算法

H.264标准采用了新的编码模式,使H.264的编码压缩效率比先前的标准提高了50%以上,但这些新的编码模式也增加编码器的复杂度和编码运算时间.一种基于H.264的全方向十字型快速运动估计搜索算法,能够在视频质量最少损失的情况下,显著提高运动估计的搜索速度,降低编码器的设计复杂度并可以节省编码时间.     

面向CPU-GPU平台的HEVC编码器优化

针对CPU-GPU平台提供了一种能显著降低高效视频编码(high efficiency video coding,简称HEVC)复杂度的优化方案.根据编码器的复杂度分布及不同模块的特点,针对帧内预测、帧间预测以及环路滤波分别进行了优化.在帧内预测中,基于相邻编码单元(coding unit,简称CU)之间的相关性,提出了一种CU的深度决策方法以及一种减少率失真优化(RDO)的模式数量的方法,降低了帧内编码的复杂度.在帧间预测中,提出将耗时最大的运动估计模块完善在图形处理单元(GPU)上,通过中央处理单元(CPU)和GPU的流水线工作获得了明显的加速,并基于预测残差的能量提出了一种编码单元提前终止划分的方法,有效降低了帧间编码复杂度.在环路滤波中,提出了一种GPU端的自适应样本点补偿(sample adaptive offset,简称SAO)参数决策方法及去块滤波方法,有效分担了CPU端的复杂度.上述优化实现在HM16.2上,实验结果表明,提出的优化方案可以获得高达68%的编码复杂度节省,而平均性能损失仅为0.5%.     

分布式视频编码中关键帧丢失错误保护

目的 分布式视频编码较其传统视频编码具有编码简单、误码鲁棒性高等特点,可以很好地满足如无人机航拍、无线监控等新型视频业务的需求。在分布式视频编码中,视频图像被交替分为关键帧和Wyner-Ziv帧,由于受到信道衰落和干扰等因素的影响,采用传统帧内编码方式的关键帧的误码鲁棒性远不如基于信道编码的Wyner-Ziv帧。关键帧能否正确传输和解码对于Wyner-Ziv帧能否正确解码起着决定性的作用,进而影响着整个系统的压缩效率和率失真性能。为此针对关键帧在异构网络中的鲁棒性传输问题,提出一种基于小波域的关键帧质量可分级保护传输方案。方法 在编码端对关键帧同时进行传统的帧内视频编码和基于小波域的Wyner-Ziv编码,解码端将经过错误隐藏后的误码关键帧作为基本层,Wyner-Ziv编码产生的校验信息码流作为增强层。为了提高系统的分层特性以便使系统的码率适应不同的网络条件,进一步将小波分解后图像的各个不同层的低频带和高频带组合成不同的增强层,根据不同信道环境,传输不同层的Wyner-Ziv校验数据。同时对误码情况下关键帧的虚拟噪声模型进行了改进,利用第1个增强层已解码重建的频带与其对应边信息来获得第2个和第3个增强层对应频带的更加符合实际的虚拟信道模型的估计。结果 针对不同的视频序列在关键帧误码率为1%20%时,相比较于传统的帧内错误隐藏算法,所提方案可以提高视频重建图像的主观质量和整体系统的率失真性能。例如在关键帧误码率为5%时,通过传输第1个增强层,不同的视频序列峰值信噪比（PSNR）提升可达25 dB左右;如果继续传输第2个增强层的校验信息,视频图像的PSNR也可以提升0.51.6 dB左右;如果3个增强层的校验信息都传输的话,基本上可以达到无误码情况下关键帧的PSNR。结论 本文所提方案可以很好地解决分布式视频编码系统中的关键帧在实际信道传输过程中可能出现的误码问题,同时采用的分层传输方案可以适应不同网络的信道情况。     

Adaptive deblocking filter for transform domain Wyner?Ziv video coding

Wyner-Ziv (WZ) video coding is a particular case of distributed video coding, the recent video coding paradigm based on the Slepian-Wolf and Wyner-Ziv theorems that exploits the source correlation at the decoder and not at the encoder as in predictive video coding. Although many improvements have been done over the last years, the performance of the state-of-the-art WZ video codecs still did not reach the performance of state-of-the-art predictive video codecs, especially for high and complex motion video content. This is also true in terms of subjective image quality mainly because of a considerable amount of blocking artefacts present in the decoded WZ video frames. This paper proposes an adaptive deblocking filter to improve both the subjective and objective qualities of the WZ frames in a transform domain WZ video codec. The proposed filter is an adaptation of the advanced deblocking filter defined in the H.264/AVC (advanced video coding) standard to a WZ video codec. The results obtained confirm the subjective quality improvement and objective quality gains that can go up to 0.63 dB in the overall for sequences with high motion content when large group of pictures are used.     

DCT-domain coder for digital video applications

In this paper, we present an effective DCT-domain video encoder architecture that decreases the computational complexity of conventional hybrid video encoders by reducing the number of transform operations between the pixel and the DCT domains. The fixed video encoder architecture (such as a fixed DCT block of 8 × 8 size) and a huge number of DCT/IDCT transforms performed during the video encoding process limit the minimum possible computational load of conventional video encoders. In this study, we solve this problem by developing a flexible video encoder architecture, which reduces video encoder computational complexity by performing low-resolution coarse-step motion estimation operations in the DCT domain. When a high level of motion activity is detected, the video encoder slightly increases the computational complexity of the motion estimation operation by computing fine-search block matching for a small-size search window in a reference frame. The proposed DCT-domain video encoder architecture is based on the conventional hybrid coder and on a set of fast integer composition and decomposition DCT transforms. The set of transforms implements a technique for estimation of DCT coefficients of a block that is partitioned by the sub-blocks. Experimental results of this method were compared with the results of the conventional hybrid coder in terms of PSNR quality and computational complexity. This comparison shows that the computational complexity of the proposed encoder is lower by 26.8% with respect to the conventional hybrid video coder for the same objective PSNR quality.     

Feedback-free rate-allocation scheme for transform domain Wyner–Ziv video coding

In most existing Wyner–Ziv video coding schemes, a feedback channel (FC) is expected at the decoder in order to allocate a proper bit rate for each Wyner–Ziv frame. However, FC not only results in additional latency but also increases decoding complexity due to the several feedback-decoding iterations. Moreover, FC may be unavailable in many practical video applications. In this paper, we propose a novel feedback-free rate-allocation scheme for transform domain Wyner–Ziv video coding (TD-WZVC), which predicts the rate for each Wyner–Ziv frame at the encoder without significantly increasing the complexity of the encoder. First, a correlation estimation model is presented to characterize the relationship between the source frame and the reference frame estimated at the encoder in TD-WZVC. Then, an efficient FC-free rate-allocation algorithm is proposed and a linear model is built to avoid both overestimation and underestimation of the real rate and obtain an optimal rate-distortion performance. Experimental results show that the proposed scheme is able to achieve a good encoder rate allocation while still maintaining consistent coding efficiency.     

Development of an ultra-HD HEVC encoder using SIMD implementation and fast encoding schemes for smart surveillance system

High efficiency video coding (HEVC) is the newest video coding standard that can support powerful video compression performance with increased picture resolution for ultrahigh definition (UHD). Compared to the previous standard, HEVC achieved a coding efficiency double with a tremendous increase in encoder computational complexity, making support of commercial applications for UHD video service difficult. Especially, optimized HEVC encoder for UHD is expected to be deployed as a key technology for an emerging smart surveillance system in Internet of Things environment. Single-instruction-multiple-data implementation on an Intel x86 processor and several fast encoding schemes were investigated for the complexity reduction of the HEVC reference model (HM) encoder. Fast encoding schemes included early termination processes and data-level parallel processing. The computational complexity of the proposed HEVC encoder was decreased by approximately 192 times compared with HM encoder with an acceptable coding loss.     

Fast multiframe motion estimation algorithms by motion vector composition for the MPEG-4/AVC/H.264 standard

The MPEG-4/AVC/H.264 video coding standard adopts various coding schemes such as multiple reference frames and variable block sizes for motion estimation. Hence, MPEG-4/AVC/H.264 provides gains in compression efficiency of up to 50% over a wide range of bit rates and video resolutions compared to previous standards. However, these features result in a considerable increase in encoder complexity, mainly regarding to mode decision and motion estimation. The proposed algorithms use the stored motion vectors to compose the motion vector without performing the full search in each reference frame. Therefore, the proposed algorithms can obtain an average speed up ratio of four for encoding, thus benefiting from the prediction of the motion vector for the reference frames in advance and maintaining good performance. Any fast search algorithm can be utilized to further largely reduce the computational load.     

一种新型H.264快速模式选择算法的研究

多模式运动估计使H.264比其他视频编码算法具有更高的编码效率，但同时也使计算复杂度明显提升。该文提出了一种新的适用于H.264的快速模式选择算法。此算法提前对块的细节程度和纹理方向进行分析，根据块细节程度缩小模式选择的范围，并根据块纹理方向对模式进行进一步筛选，从而大大提高了模式搜索速度，将性能损失控制在可以忽略的范围内。仿真实验的结果证明了本算法的有效性。     

基于时空相关性的宏块编码模式预测算法

与以往的各种视频编码标准相比，H．264视频编码标准在编码效率上获得了最大50％的提高．但这是以提高编码的计算复杂度为代价的，其中运动估计和模式选择的计算复杂度最大．文中提出一种基于时空相关性的宏块编码模式预测算法，该算法首先检查根据时空相关性预测得到的模式，再根据相应的阈值判断是否结束模式选择过程。而不用对所有的模式进行全搜索．试验结果证明：在保持相近率失真性能的同时，该算法可以有效地降低编码的计算复杂度．     

Content-Aware Prediction Algorithm With Inter-View Mode Decision for Multiview Video Coding

3-D video will become one of the most significant video technologies in the next-generation television. Due to the ultra high data bandwidth requirement for 3-D video, effective compression technology becomes an essential part in the infrastructure. Thus multiview video coding (MVC) plays a critical role. However, MVC systems require much more memory bandwidth and computational complexity relative to mono-view video coding systems. Therefore, an efficient prediction scheme is necessary for encoding. In this paper, a new fast prediction algorithm, content-aware prediction algorithm (CAPA) with inter-view mode decision, is proposed. By utilizing disparity estimation (DE) to find corresponding blocks between different views, the coding information, such as rate-distortion cost, coding modes, and motion vectors, can be effectively shared and reused from the coded view channel. Therefore, the computation for motion estimation (ME) in most view channels can be greatly reduced. Experimental results show that compared with the full search block matching algorithm (FSBMA) applied to both ME and DE, the proposed algorithm saves 98.4–99.1% computational complexity of ME in most view channels with negligible quality loss of only 0.03–0.06 dB in PSNR.      

Multi-view codec with low-complexity encoding for Distributed Video Coding

The low-complexity encoding, as fundamental requirement of Distributed Video Coding, relies on performing the bulk of computation at decoder, including tasks as the generation of side information and particularly, inter-camera registration in the case of multi-view systems with complete-overlapped views and free motion of the cameras (e.g., video surveillance). In Ciobanu and Corte-Real (Multimedia Tools Appl 48(3):411–436, 2010) we introduced a codec-independent solution for such tasks at decoder. In this paper, we present a multi-view Wyner–Ziv codec (IWZ) designed for the architecture and scenarios from Ciobanu and Corte-Real (2010) (e.g., free motion of the cameras, no a priori knowledge of the instant camera positions, no feedback channel), based on transform domain (DCT), block-based coset coding. We aimed to achieve a compromise between the low encoder complexity and the rate-distortion performance. A detailed evaluation is presented for comparison with conventional coding (Intra 4×4 and Intra 16×16). Practical results show a better overall performance of the proposed codec at low bitrates.