Foveation scalable video coding with automatic fixation selection

Image and video coding is an optimization problem. A successful image and video coding algorithm delivers a good tradeoff between visual quality and other coding performance measures, such as compression, complexity, scalability, robustness, and security. In this paper, we follow two recent trends in image and video coding research. One is to incorporate human visual system (HVS) models to improve the current state-of-the-art of image and video coding algorithms by better exploiting the properties of the intended receiver. The other is to design rate scalable image and video codecs, which allow the extraction of coded visual information at continuously varying bit rates from a single compressed bitstream. Specifically, we propose a foveation scalable video coding (FSVC) algorithm which supplies good quality-compression performance as well as effective rate scalability. The key idea is to organize the encoded bitstream to provide the best decoded video at an arbitrary bit rate in terms of foveated visual quality measurement. A foveation-based HVS model plays an important role in the algorithm. The algorithm is adaptable to different applications, such as knowledge-based video coding and video communications over time-varying, multiuser and interactive networks.     

一种基于PID控制器的H.264比特率转码方案

本文基于PID控制器提出了一种高效的H.264比特率转码方案.该方案采用简化的像素域级联转码结构,在保证视频重建质量的同时降低了运算的复杂度,然后在图像层引入PID控制器保证了输出比特率与目标比特率一致.实验结果表明:本文提出的比特率转码方案具有较高的比特率转换精度,是一种高效的比特率转码方案.     

New architecture for dynamic frame-skipping transcoder

Transcoding is a key technique for reducing the bit rate of a previously compressed video signal. A high transcoding ratio may result in an unacceptable picture quality when the full frame rate of the incoming video bitstream is used. Frame skipping is often used as an efficient scheme to allocate more bits to the representative frames, so that an acceptable quality for each frame can be maintained. However, the skipped frame must be decompressed completely, which might act as a reference frame to nonskipped frames for reconstruction. The newly quantized discrete cosine transform (DCT) coefficients of the prediction errors need to be re-computed for the nonskipped frame with reference to the previous nonskipped frame; this can create undesirable complexity as well as introduce re-encoding errors. In this paper, we propose new algorithms and a novel architecture for frame-rate reduction to improve picture quality and to reduce complexity. The proposed architecture is mainly performed on the DCT domain to achieve a transcoder with low complexity. With the direct addition of DCT coefficients and an error compensation feedback loop, re-encoding errors are reduced significantly. Furthermore, we propose a frame-rate control scheme which can dynamically adjust the number of skipped frames according to the incoming motion vectors and re-encoding errors due to transcoding such that the decoded sequence can have a smooth motion as well as better transcoded pictures. Experimental results show that, as compared to the conventional transcoder, the new architecture for frame-skipping transcoder is more robust, produces fewer requantization errors, and has reduced computational complexity.     

On re-composition of motion compensated macroblocks for DCT-based video transcoding

To achieve portability between different kinds of encoding formats and network environments, heterogeneous video transcoding becomes a key technique for reducing the bitrate of a previously compressed video signal. A frame-skipping transcoder is often used to avoid an unacceptable picture quality when high transcoding ratio is required. Due to high computational complexity and quality degradation introduced by conventional frame-skipping transcoders, a DCT-based video frame-skipping transcoder has been proposed recently. However, the transcoding process of the motion compensated macroblocks in the DCT domain becomes the bottleneck since IDCT and DCT processes are required. In this paper, we propose a new architecture of the frame-skipping transcoder to reduce the computational complexity of motion compensated macroblocks in the frame-skipping process. The new architecture transcodes the dominant region of a motion compensated macroblock in the DCT domain by making use of the DCT coefficients of the incoming bistream and some pre-computed shift operators. By using a shifted version of the dominant vector, the re-encoding error introduced in the dominant region can be avoided. On the other hand, an adaptive transcoding architecture to transcode the boundary regions of MC marcoblocks and a way to perform error compensation are proposed. This architecture can further speed up the transcoding process of the motion compensated macroblocks. Half pixel accuracy related to our proposed frame skipping transcoder is also addressed. Experimental results show that, as compared to the conventional or DCT-based transocders, the new architecture is more robust to noise, gives rise to fewer requantization errors, and requires simple computational complexity.     

一种快速的DCT域MPEG-2到MPEG-1下采样转码技术

文章提出了一种快速的DCT域MPEG-2到MPEG-1准卷积下呆样算法。在转码过程中。头信忠保持不变，且要求将每四个相邻MPEG-2宏块变为一个下采样MPEG-1宏块：用最大最小距离方法确定下采样宏块的运动向量、用简单多数原则确定下采样宏块类型以及用加全平均方案确定下呆样宏块的量化参数。另外，对下采样视频转码失真来源进行了分析。实验结果表明我们提出的转码方案，在同样条件下与级联像素域转码器（TM5）相比，不仅其计算复杂性减少67．6％、PNSR提高0．1dB，而且具有很小的比特控制错误。     

Block DCT to wavelet transcoding in transform domain

The Discrete Cosine Transform (DCT) to wavelet transcoding provides input for several wavelet-based post-processing techniques of the DCT-coded image/video signals. Transcoding in domain transform avoids inverse transform and retransform operations and saves computation. In this paper, we propose a new technique for transcoding the DCT blocks to wavelet coefficients directly in the transform domain. We perform filtering, IDCT and downsampling operations in a single combined step. The proposed technique achieves the same computational result as that of a spatial domain technique. The transcoding matrices used in the proposed technique are found to satisfy certain symmetric and sparse properties, which are exploited to reduce the computational cost. As the number of zeros in the DCT coefficients is significantly higher compared to the spatial domain, computational cost reduces significantly. Also, with the proposed technique, it is possible to speedup the operation by ignoring some elements in the filtering matrices whose magnitudes are smaller than a threshold value. We demonstrate the application of the proposed transcoding for deblocking of the DCT-coded images in wavelet domain.     

DCT域视频转码技术综述

随着数字视频技术的广泛应用,实现各种视频编码格式之间转码的要求越来越迫.视频转码是数字媒体网络网关的关键技术.本文从DCT域视频转码的体系结构、关键技术及最优转码策略三个方面,介绍了现有的各种算法,分析和总结了各自的特点,提出了DCT域视频下采样帧内刷新体系结构和最优视频转码策略解的概念,并指出了下一步的研究方向.     

DCT-based video downscaling transcoder using split and merge technique.

For a conventional downscaling video transcoder, a video server has firstly to decompress the video, perform downscaling operations in the pixel domain, and then recompress it. This is computationally intensive. However, it is difficult to perform video downscaling in the discrete cosine transform (DCT)- domain since the prediction errors of each frame are computed from its immediate past higher resolution frames. Recently, a fast algorithm for DCT domain image downsampling has been proposed to obtain the downsampled version of DCT coefficients with low computational complexity. However, there is a mismatch between the downsampled version of DCT coefficients and the resampled motion vectors. In other words, significant quality degradation is introduced when the derivation of the original motion vectors and the resampled motion vector is large. In this paper, we propose a new architecture to obtain resampled DCT coefficients in the DCT domain by using the split and merge technique. Using our proposed video transcoder architecture, a macroblock is splitted into two regions: dominant region and the boundary region. The dominant region of the macroblock can be transcoded in the DCT domain with low computational complexity and re-encoding error can be avoided. By transcoding the boundary region adaptively, low computational complexity can also be achieved. More importantly, the re-encoding error introduced in the boundary region can be controlled more dynamically. Experimental results show that our proposed video downscaling transcoder can lead to significant computational savings as well as videos with high quality as compared with the conventional approach. The proposed video transcoder is useful for video servers that provide quality service in real-time for heterogeneous clients.     

Adaptive Block-size Transform based Just-Noticeable Difference model for images/videos

In this paper, we propose a novel Adaptive Block-size Transform (ABT) based Just-Noticeable Difference (JND) model for images/videos. Extension from 8×8 Discrete Cosine Transform (DCT) based JND model to 16×16 DCT based JND is firstly performed by considering both the spatial and temporal Human Visual System (HVS) properties. For still images or INTRA video frames, a new spatial selection strategy based on the Spatial Content Similarity (SCS) between a macroblock and its sub-blocks is proposed to determine the transform size to be employed to generate the JND map. For the INTER video frames, a temporal selection strategy based on the Motion Characteristic Similarity (MCS) between a macroblock and its sub-blocks is presented to decide the transform size for the JND. Compared with other JND models, our proposed scheme can tolerate more distortions while preserving better perceptual quality. In order to demonstrate the efficiency of the ABT-based JND in modeling the HVS properties, a simple visual quality metric is designed by considering the ABT-based JND masking properties. Evaluating on the image and video subjective databases, the proposed metric delivers a performance comparable to the state-of-the-art metrics. It confirms that the ABT-based JND consists well with the HVS. The proposed quality metric also is applied on ABT-based H.264/Advanced Video Coding (AVC) for the perceptual video coding. The experimental results demonstrate that the proposed method can deliver video sequences with higher visual quality at the same bit-rates.     

基于纹理分解的变换域JND模型及图像编码方法

为了提高变换域JND模型的精度,在计算对比度掩盖因子时只对纹理分量滤波并判断区域类型的方法避免了JND低估的问题。将改进的JND模型用于图像编码,考虑到辅助信息对编码效率的影响,把经过调整后的JND模型结合到量化过程中,能去除更多的视觉冗余并保持兼容性。仿真结果表明,纹理分解的方法提高了JND阈值,改进的编码方法在相似的视觉质量下能节省更多的码率并且不需要增加额外的比特开销,该编码思路也适用于视频编码。     

一种高效视频编码器的设计

提出了一个基于MPEG-4的快速高效存储的DCT视频编码器的设计原理,用它把一个H.261视频数据流转换为一个低比特率、低空间分辨率的MPEG-4流,并进行无线视频处理.和现有编码器相比,它不仅能够大大节省实际需要的存储空间,计算复杂度也降低了许多,实验结果表明,编码器得到的视频质量与像素领域方法得到的视频质量相当.     

基于视觉感知的高效视频编码标准帧内量化矩阵优化方法

该文使用一种基于离散余弦变换(DCT)的调制传递函数(MTF)改进高效视频编码标准(HEVC)帧内量化矩阵,并在计算过程中采用一种新的空间频率计算方法。HEVC整数DCT变换是在DCT变换的基础上经过伸缩扩展以及微调后得到的,与DCT变换有所区别,因此针对这种差别进一步优化校正量化矩阵。仿真过程中设计一种基于结构相似性(SSIM)的Bjontegaard比特率差值(BDBR)方法来测试量化矩阵的编码性能,结果表明在相同的视频质量下,该文设计的量化矩阵能进一步降低码率。该方法在编码过程中只需要简单地改变量化矩阵,没有影响编码模型的算法结构,也不会增加编码的复杂度。     

一种MPEG4标准的快速及高效视频编码器

提出了一个基于MPEG4的快速高效存储的DCT视频编码器，用他把一个H．261视频数据流转换为一个低比特率、低空间分辨率的MPEG4流，并进行无线视频处理。和现有编码器相比，他不仅能够大大地节省实际需要存储空间，计算复杂度也降低了许多，实验结果表明设计的编码器得到的视频质量与象素领域方法得到的视频质量相当。     

一种基于DCT系数重组的降参考3D视频质量评价方法

针对3D视频质量评价,提出了一种基于离散余弦变 换(DCT)系数重组的降参考质量评价方法。方法主要 包括4D数据集构造、DCT变换与重组和降参考(RR)质量评价3个阶段。在4D数据集构造阶段 ,通过联合视 频的空间域、视点域和时间域信息构造包含3D视频全部信息的4D数据集;在DCT变换与重 组阶段,首 先对数据集进行DCT变换,然后筛选复杂度较高的DCT系数进行重组,得到能够表征3D视 频质量的重 组系数集;在RR质量评价阶段,首先采用广义高斯分布(GGD)函数对重组后DCT系数集的每 个子带进行建模, 然后利用系数距离函数和频率比距离函数评估3D视频质量。实验结果表明,本文提出的方 法与主观评价结果有较好的相关性,符合人类视觉系统(HVS)。     

No-reference quality assessment of HEVC video streams based on visual memory modelling

Providing adequate Quality of Experience (QoE) to end-users is crucial for streaming service providers. In this paper, in order to realize automatic quality assessment, a No-Reference (NR) bitstream Human-Vision-System-(HVS)-based video quality assessment (VQA) model is proposed. Inspired by discoveries from the neuroscience community, which suggest there is a considerable overlap between active areas of the brain when engaging in video quality assessment and saliency detection tasks, saliency maps are used in the proposed method to improve the quality assessment accuracy. To this end, saliency maps are first generated from features extracted from the HEVC bitstream. Then, saliency map statistics are employed to create a model of visual memory. Finally, a support vector regression pipeline learns an estimate of the video quality from the visual memory, saliency, and frame features. Evaluations on SJTU dataset indicate that the proposed bitstream based no-reference video quality assessment algorithm achieves a competitive performance.     

Robust video watermarking based on affine invariant regions in the compressed domain

This paper proposes a novel robust video watermarking scheme based on local affine invariant features in the compressed domain. This scheme is resilient to geometric distortions and quite suitable for DCT-encoded compressed video data because it performs directly in the block DCTs domain. In order to synchronize the watermark, we use local invariant feature points obtained through the Harris-Affine detector which is invariant to affine distortions. To decode the frames from DCT domain to the spatial domain as fast as possible, a fast inter-transformation between block DCTs and sub-block DCTs is employed and down-sampling frames in the spatial domain are obtained by replacing each sub-blocks DCT of 2×2 pixels with half of the corresponding DC coefficient. The above-mentioned strategy can significantly save computational cost in comparison with the conventional method which accomplishes the same task via inverse DCT (IDCT). The watermark detection is performed in spatial domain along with the decoded video playing. So it is not sensitive to the video format conversion. Experimental results demonstrate that the proposed scheme is transparent and robust to signal-processing attacks, geometric distortions including rotation, scaling, aspect ratio changes, linear geometric transforms, cropping and combinations of several attacks, frame dropping, and frame rate conversion.     

Macroblock-Based Frequency Selective Weighting for Visual Scalable Video Coding of H.264/AVC

In this paper, a novel macroblock-based frequency selective weighting scalable video coding (MBFSW-SVC) algorithm is investigated by exploring the characteristics of the human visual system (HVS) in SVC H.264/AVC for improving the coding efficiency. Compared with previous SVC schemes, the MBFSW-SVC can extend the quality layer with different frequency weighting for each MB according to the size of region of interests. For reasonable adaptation of the frequency weighting into the quality layer extension, the spatial context characteristics and the motion activities in each picture are taken into account. The experiment results show that the proposed scheme can improve the coding performance up to a 10–20% reduction of the bit rate with almost the same subjective video quality. In particular, the subjective quality is significantly improved in lower scalable quality layers with a small number of the encoding bit budget.      

Wavelet to DCT transcoding in transform domain

Wavelet to DCT transcoding provides inter-operability between standards using the two transforms for encoding. Transcoding in transform domain avoids inverse transform and re-transform operations and saves computation. In this paper, we propose new algorithms for transcoding wavelet coefficients to block DCT coefficients. In the first step, the wavelet coefficients are transformed into upsampled DCT coefficients. Subsequently, these trans-formed coefficients are synthesized in the block DCT space for transcoding. The proposed approach restricts all operations in the DCT domain that makes filtering involved in the synthesis process computationally efficient. The proposed technique could be used by the block DCT based services when the input is available as wavelet coefficients.     

改进的降低空间分辨率视频编码转换算法

该文提出一种改进的降低空间分辨率视频编码转换算法。根据漂移误差与帧间编码块的运动活动性的关系,提出了一种自适应帧内刷新方法。该方法根据目标比特率和实际比特率的差值,动态地调整阈值,从而有效地减少了帧内编码的比特率,保持了对差错的鲁棒性,限制了差错的时间传播。同时,将率失真函数映射为线性函数,以较少计算复杂度,提出了一种线性速率控制策略。仿真结果表明:该文提出的方法有效地平滑了缓冲器的输出,同时峰值信噪比也有所提高。