Multiview coding and error correction coding for 3D video over noisy channels

We consider the joint source–channel coding problem of stereo video transmitted over AWGN and flat Rayleigh fading channels. Multiview coding (MVC) is used to encode the source, as well as a type of spatial scalable MVC. Our goal is to minimize the total number of bits, which is the sum of the number of source bits and the number of forward error correction bits, under the constraints that the quality of the left and right views must each be greater than predetermined PSNR thresholds at the receiver. We first consider symmetric coding, for which the quality thresholds are equal. Following binocular suppression theory, we also consider asymmetric coding, for which the quality thresholds are unequal. The optimization problem is solved using both equal error protection (EEP) and a proposed unequal error protection (UEP) scheme. An estimate of the expected end-to-end distortion of the two views is formulated for a packetized MVC bitstream over a noisy channel. The UEP algorithm uses these estimates for packet rate allocation. Results for various scenarios, including non-scalable/scalable MVC, symmetric/asymmetric coding, and UEP/EEP, are provided for both AWGN and flat Rayleigh fading channels. The UEP bit savings compared to EEP are given, and the performances of different scenarios are compared for a set of stereo video sequences.     

Fast macroblock encoding algorithm based on rate-distortion activity for multiview video coding

Multiview video coding (MVC) is the appendix H of H.264/AVC, and it requires a great amount of time to compress multiple viewpoints׳ video with complex prediction structures. To reduce the whole computational complexity of MVC, this paper proposes a fast macroblock (MB) encoding algorithm based on rate-distortion (RD) activity, and it includes the fast mode decision and the fast motion/disparity estimation. First, the RD activity type of the current MB is calculated by utilizing the Skip/Direct RD cost and the average RD costs of classified MB modes. Then, through utilizing the RD activity type and RD costs of the estimated modes, the selection of candidate modes, the early decision of Skip/Direct mode, and the reduction of Inter8×8 mode estimation are all presented in the fast mode decision. By using the RD activity type and the correlations of vectors, the selection of search center and the prediction of search range are introduced in the fast motion/disparity estimation. In addition, the proposed algorithm can be applied to temporal and inter-view views as well as anchor and non-anchor frames. An experiment with a wide range of video scenes, camera setups and quantization parameters was implemented, and the results confirmed that the proposed algorithm can reduce the encoding time significantly while maintaining a similar RD performance as the original MVC encoder. Compared to the state-of-the-art algorithms, the proposed algorithm also demonstrated better performances in the various test cases.     

Multiview video summarization using video partitioning and clustering

Multiview video summarization plays a crucial role in abstracting essential information form multiple videos of the same location and time. In this paper, we propose a new approach for the multiview summarization. The proposed approach uses the BIRCH clustering algorithm for the first time on the initial set of frames to get rid of the static and redundant. The work presents a new approach for shot boundary detection using frame similarity measures Jaccard and Dice. The algorithm performs effectively synchronized merging of keyframes from all camera-views to obtain the final summary. Extensive experimentation conducted on various datasets suggests that the proposed approach significantly outperforms most of the existing video summarization approaches. To state a few, a 1.5% improvement on video length reduction, 24.28% improvement in compression ratio, and 6.4% improvement in quality assessment ratio is observed on the lobby dataset.     

一种新的多视点视频编码预测结构

多视点视频编码的参考预测结构虽然具有高效的压缩效率,但是随机访问性能较差.在综合考虑了编码效率和随机访问等因素的基础上,提出了一种改进的多视点视频预测结构编码方法.首先,分析视点与视点间的相关性来调整Ⅰ视点的位置,然后在此基础上有效地增加B视点的数目,以获得较好的综合性能.实验结果表明,该方法与传统的参考预测结构编码方法相比,提高了编码效率,并且具有较好的随机访问性能.     

Overview of the MVC + D 3D video coding standard

3D video services are emerging in various application domains including cinema, TV broadcasting, Blu-ray discs, streaming and smartphones. A majority of the 3D video content in market is still based on stereo video, which is typically coded with the multiview video coding (MVC) extension of the Advanced Video Coding (H.264/AVC) standard or as frame-compatible stereoscopic video. However, the 3D video technologies face challenges as well as opportunities to support more demanding application scenarios, such as immersive 3D telepresence with numerous views and 3D perception adaptation for heterogeneous 3D devices and/or user preferences. The Multiview Video plus Depth (MVD) format enables depth-image-based rendering (DIBR) of additional viewpoints in the decoding side and hence helps in such advanced application scenarios. This paper reviews the MVC + D standard, which specifies an MVC-compatible MVD coding format.     

Multiview depth coding based on combined color/depth segmentation

In this paper, a new coding method for multiview depth video is presented. Considering the smooth structure and sharp edges of depth maps, a segmentation based approach is proposed. This allows further preserving the depth contours thus introducing fewer artifacts in the depth perception of the video. To reduce the cost associated with partition coding, an approximation of the depth partition is built using the decoded color view segmentation. This approximation is refined by sending some complementary information about the relevant differences between color and depth partitions. For coding the depth content of each region, a decomposition into orthogonal basis is used in this paper although similar decompositions may be also employed. Experimental results show that the proposed segmentation based depth coding method outperforms H.264/AVC and H.264/MVC by more than 2 dB at similar bitrates.     

Adaptive Multiview Video Coding Scheme Based on Spatiotemporal Correlation Analyses

In this paper, we propose an adaptive multiview video coding scheme based on spatiotemporal correlation analyses using hierarchical B picture (AMVC‐HBP) for the integrative encoding performances, including high compression efficiency, low complexity, fast random access, and view scalability, by integrating multiple prediction structures. We also propose an in‐coding mode‐switching algorithm that enables AMVC‐HBP to adaptively select a better prediction structure in the encoding process without any additional complexity. Experimental results show that AMVC‐HBP outperforms the previous multiview video coding scheme based on H.264/MPEG‐4 AVC using the hierarchical B picture (MVC‐HBP) on low complexity for 21.5%, on fast random access for about 20%, and on view scalability for 11% to 15% on average. In addition, distinct coding gain can be achieved by AMVC‐HBP for dense and fast‐moving sequences compared with MVC‐HBP.     

Iterative search strategy with selective bi-directional prediction for low complexity multiview video coding

The multiview video coding (MVC) extension of H.264/AVC is the emerging standard for compression of impressive 3D and free-viewpoint video. The coding structure in MVC adopts motion and disparity estimation to exploit temporal and inter-view dependencies in MVC. It results in a considerable increase in encoding complexity. Most of the computational burden comes from uni-directional and bi-directional prediction. In this paper, an iterative search strategy is designed to speed up the uni-directional prediction in MVC. It can work with an adaptive search range adjustment through a confidence measure of a loop constraint to obtain both motion and disparity vectors jointly. Furthermore, a selective bi-directional prediction algorithm is proposed to enhance the coding performance by analyzing the statistical characteristics of bi-directional prediction in MVC. Experimental results demonstrate that, by using the proposed fast search, the temporal and inter-view redundancies of multiview video can be eliminated sufficiently with low complexity.     

A VLSI architecture for the real time computation of discrete trigonometric transforms

The Discrete Trigonometric Transforms are defined as a class of transforms. An algorithm for calculating the Discrete Fourier Transform is extended to cover all members of the defined class. A VLSI architecture which provides for real time calculation of these transforms is presented. This architecture provides simple interconnections, identical processing elements and minimal control complexity.     

Low-complexity video coding via power-rate-distortion optimization

Wireless multimedia sensor networks (WMSNs) have been potentially applicable for several emerging applications. The resources, i.e., power and bandwidth available to visual sensors in a WMSN are, however, very limited. Hence, it is important but challenging to achieve efficient resource allocation and optimal video data compression while maximizing the overall network lifetime. In this paper, a power-rate-distortion (PRD) optimized resource-scalable low-complexity multiview video encoding scheme is proposed. In our video encoder, both the temporal and interview information can be exploited based on the comparisons of extracted media hashes without performing motion and disparity estimations, which are known to be time-consuming. We present a PRD model to characterize the relationship between the available resources and the RD performance of our encoder. More specifically, an RD function in terms of the percentages for different coding modes of blocks and the target bit rate under the available resource constraints is derived for optimal coding mode decision. The major goal here is to design a PRD model to optimize a “motion estimation-free” low-complexity video encoder for applications with resource-limited devices, instead of designing a general-purpose video codec to compete compression performance against current compression standards (e.g., H.264/AVC). Analytic results verify the accuracy of our PRD model, which can provide a theoretical guideline for performance optimization under limited resource constraints. Simulation results on joint RD performance and power consumption (measured in terms of encoding time) demonstrate the applicability of our video coding scheme for WMSNs.     

Object-based stereo video compression using fractals and shape-adaptive DCT

Based on the classical fractal video compression method, an improved object-based stereo video compression scheme with Shape-Adaptive DCT is proposed in this paper. Firstly, we use more effective macroblock partition scheme instead of classical quadtree partition scheme; thus reducing the block searching strategy. The stereo fractal video coding is proposed which matches the macroblock with two reference frames in left and right view results in increasing compression ratio and reducing bit rate when transmitting compressed stereo data. The stereo codec combines the Motion Compensation Prediction (MCP) and Disparity Compensation Prediction (DCP). Fractal coding is adopted and each object is encoded independently by a prior video segmentation alpha plane, which is defined exactly as in MPEG-4. The testing results with the nature monocular and stereo video sequences provide promising performances at low bit rate coding. We believe it will be a powerful and efficient technique for the object-based monocular and stereo video sequences coding.     

基于感兴趣区域的HEVC压缩性能优化

根据人类视觉系统(HVS)对纹理复杂及运动区域具有较强感知度的特点,提出了一种基于感兴趣区域的高效率视频编码(HEVC)压缩性能优化算法.首先使用Sobel梯度检测算子和运动矢量分别检测纹理复杂区域和运动区域,把检测到的纹理复杂及运动区域定义为感兴趣区域;再对感兴趣区域分级,通过调整量化参数(QP),优化比特分配.实验结果表明,与HEVC标准算法相比,所提算法码率平均减少了15.29％,时间平均节省了11.38％.     

Satellite image compression by post-transforms in the wavelet domain

This paper proposes a novel compression scheme with a tunable complexity-rate-distortion trade-off. As images increase in size and resolution, more efficient compression schemes with low complexity are required on-board Earth observation satellites. The standard of the Consultative Committee for Space Data Systems (CCSDS) defines a strip-based compression scheme with the advantages of a low complexity and an easy rate control [CCSDS, Image Data Compression Recommended Standard CCSDS 122.0-B-1 Blue Book, November 2005]. However, future mission specifications expect higher performance in terms of rate-distortion. The scheme proposed in this paper intends to perform better than the CCSDS standard while preserving low complexity and easy rate control. Moreover, to comply with existing on-board devices, the proposed core compression engine still uses the wavelet transform but in association with a linear post-processing inspired from the bandelet transform. The post-transform decomposes a small block of wavelet coefficients on a particular basis. This basis is adaptively selected within a predefined dictionary by rate-distortion optimization. The computational complexity depends upon the dictionary size and of the basis structure. An extremely simple dictionary, reduced to the Hadamard basis, is proposed. The post-transform efficiency is illustrated by experiments on various Earth observation images provided by the French Space Agency (CNES).     

一种基于频域的视频压缩算法及其实现

提出了一种基于频域的视频编码的方案并描述其如何实现，该算法利用DCT变换和运动图像前后帧在频域的三种状态，对图像快速高效编解码，运算量低，可在PC机上用软件实时实现。     

多视点与深度视频编码技术研究综述

多视点与深度视频(MVD)可以有效表示自由视点视频,减少了需传输视点的个数,该表达形式正受到越来越多的关注,如何对MVD进行高效编码尤为重要.介绍了自由视点视频的发展现状及国内外研究概况,详细讨论了深度图像的高效编码技术及多视点与深度联合编码技术,并对多视点与深度视频编码技术进行了总结及展望.     

Incentive analysis for cooperative interactive multiview video streaming

In interactive multiview video streaming (IMVS), users can periodically select one out of many captured views available for observation. In single-view video streaming, cooperative strategies where peers share received packets of the same video have proven to be effective in reducing server׳s upload burden, and incentive mechanisms are designed to stimulate user cooperation. However, exploiting user cooperation in higher dimensional IMVS is difficult, since users watching different views makes it difficult to establish partnership, and users switching views frequently and independently makes it difficult to maintain partnership over time. In this paper, we use a multiview video frame structure for IMVS to support cooperative view-switching, where peers may help each other even if they are observing different views. We then model peers׳ interaction as an indirect reciprocity game, where each user is assigned a reputation level. To gain a higher reputation level, users help others, which in turn leads to a higher likelihood to receive others׳ help later. In this work, we focus on how view switching, the key feature of IMVS, affects user cooperation. By modeling users׳ decision making as a Markov decision process, our analysis shows that users tend to cooperate at some views but not others: given peers can predict their future view navigation paths probabilistically, for a peer who is likely to enter a view-switching path not requiring others׳ help, he also has less incentive to cooperate. Furthermore, we observe that the game may have multiple Nash Equilibria corresponding to different cooperation levels, e.g., users cooperate at all views in the full cooperation equilibrium, while users only cooperate at certain views in the partial cooperation equilibrium. The particular equilibrium the game will converge to depends on the initial cooperation level of the game. To stimulate user cooperation at all views, we propose a Pay-for-Cooperation (PfC) scheme at the beginning of the game to drive the game to the full cooperation equilibrium to improve system efficiency. Our simulation results show the effectiveness of PfC.     

视频解码系统中读写带宽压缩算法设计

随着人们更多地使用携带式消费电子产品,电子产品中的电力消耗问题已经渐渐成为视频编解码器设计中关注的最主要的设计问题.特别是在最新的编码标准H.264/AVC中,由于采用了多种新的先进的压缩策略,编码器达到了更高的压缩效率的同时,由于这些新的性能,使H.264/AVC的解码器需要对外部存储进行大量的读取.所以,内存读取带宽成为对于整个系统成本的关键问题,具体如在使用电池提供高清视频播放的消费者电子产品中,需要以更低的电力提供更好更长时间的视频.在这个研究中,提出了针对于视频压缩解码系统中内存读写带宽问题所设计的可调的参考帧压缩算法设计的方案,通过降低系统读取外部内存的带宽而达到降低视频解码系统电力消耗的目的.     

Encoder adaptable difference detection for low power video compression in surveillance system

As a state-of-the-art video compression technique, H.264/AVC has been deployed in many surveillance cameras to improve the compression efficiency. However, it induces very high coding complexity, and thus high power consumption. In this paper, a difference detection algorithm is proposed to reduce the computational complexity and power consumption in surveillance video compression by automatically distributing the video data to different modules of the video encoder according to their content similarity features. Without any requirement in changing the encoder hardware, the proposed algorithm provides high adaptability to be integrated into the existing H.264 video encoders. An average of over 82% of overall encoding complexity can be reduced regardless of whether or not the H.264 encoder itself has employed fast algorithms. No loss is observed in both subjective and objective video quality.     

二维实值离散Gabor变换与DCT在图像编码中性能的比较

介绍了二维实值离散Gabor变换(RDGT)的快速算法,并着重探讨了二维实值离散Gabor变换与二维离散余弦变换在图像编码中的性能及差异.