Multiview coding and error correction coding for 3D video over noisy channels

We consider the joint source–channel coding problem of stereo video transmitted over AWGN and flat Rayleigh fading channels. Multiview coding (MVC) is used to encode the source, as well as a type of spatial scalable MVC. Our goal is to minimize the total number of bits, which is the sum of the number of source bits and the number of forward error correction bits, under the constraints that the quality of the left and right views must each be greater than predetermined PSNR thresholds at the receiver. We first consider symmetric coding, for which the quality thresholds are equal. Following binocular suppression theory, we also consider asymmetric coding, for which the quality thresholds are unequal. The optimization problem is solved using both equal error protection (EEP) and a proposed unequal error protection (UEP) scheme. An estimate of the expected end-to-end distortion of the two views is formulated for a packetized MVC bitstream over a noisy channel. The UEP algorithm uses these estimates for packet rate allocation. Results for various scenarios, including non-scalable/scalable MVC, symmetric/asymmetric coding, and UEP/EEP, are provided for both AWGN and flat Rayleigh fading channels. The UEP bit savings compared to EEP are given, and the performances of different scenarios are compared for a set of stereo video sequences.     

Adaptive packet scheduling for scalable video streaming with network coding

Over the last decade, the emergence of new multimedia devices has motivated the research on efficient media streaming mechanisms that adapt to dynamic network conditions and heterogeneous devices’ capabilities. Network coding as a rateless code has been applied to collaborative media streaming applications and brings substantial improvements regarding throughput and delay. However, little attention has been given to the recoverability of encoded data, especially for the streaming with a strict deadline. This in turn leads to severe quality of experience. In this paper, we solve the unrecoverable transmission by proposing a multi-generation packet scheduling problem, which is treated as a video quality maximization problem and solved using dynamic programming algorithm. Experimental results confirm that the proposed algorithm brings better data recoverability and better quality of service in terms of video quality, delivery ratio, lower redundancy rate under different network sizes.     

Frame distortion estimation for unequal error protection methods in scalable video coding (SVC)

In many multimedia applications, coded video is transmitted over error prone heterogeneous networks. Because of the predictive mechanism used in video coding, transmission error would propagate temporally and spatially and would result in significant quality losses. In order to address this problem, different error resilience methods have been proposed. One of the techniques, which is commonly used in video streaming, is unequal error protection (UEP) of scalable video coding (SVC). In this technique, different independent layers of an SVC stream are protected differently and based on their importance by using forward error correction (FEC) codes. Accurately analyzing the importance or utility of each video part is a critical component and would lead to a better protection and higher quality of the received video. Calculation of the utility is usually based on multiple decoding of sub-bitstreams and is highly computationally complex. In this work, we propose an accurate low complexity utility estimation technique that can be used in different applications. This technique estimates the utility of each network abstraction layer (NAL) by considering the error propagation to future frames. We utilize this method in an UEP framework with the scalable extension of H.264/AVC codec and it achieves almost the same performance as highly complex estimation techniques (an average loss of 0.05 dB). Furthermore, we propose a low delay version of this technique that can be used in delay constrained application. The estimation accuracy and performance of our proposed technique are studied extensively.     

Efficient early direct mode decision for multi-view video coding

Multi-view video coding (MVC) uses various prediction modes and exhaustive mode decision to achieve high coding efficiency. However, the introduced heavy computational complexity becomes the bottleneck of the practical application of MVC. For this, an efficient early Direct mode decision for MVC is proposed in this paper. Based on the observation that the Direct mode is highly possible to be the optimal mode, the proposed method first computes the rate distortion (RD) cost of the Direct mode and compares this RD cost value with an adaptive threshold for providing an early termination chance as follows. If this RD cost value is smaller than the adaptive threshold, the Direct mode will be selected as the optimal mode and the checking process of the remaining modes will be skipped; otherwise, all the modes will be checked to select the one with the minimum RD cost as the optimal mode. Note that the above-mentioned adaptive threshold is determined as the median prediction value of a set of thresholds, which are derived by using the spatial, temporal and inter-view correlations between the current macroblock (MB) and its neighboring MBs, respectively. Experimental results have demonstrated that the proposed method is able to significantly reduce the computational complexity of MVC with negligible loss of coding efficiency, compared with the exhaustive mode decision in MVC.     

一种基于多视点视频的低复杂度自适应环路滤波算法

自适应环路滤波(ALF)是面向高清的多视点视频编码(MVC)中的一项新的滤波技术,虽然能提高视频压缩的主观质量和客观质量,但是其复杂度过高阻碍了其实时应用。本文提出了一种低复杂度的ALF算法,充分利用了视点间相关性和层间相关性以减少亮度和色度分量的分割次数,并且能自适应跳过大部分帧的块控制(DBC)处理过程。实验结果表明,所提出的算法能在保证视频主观质量和客观质量基本不变的情况下减少61%左右的ALF处理时间。因此,所提出的方法在编码效率和复杂度上做到了很好的折中处理,并且能很好地改善ALF的性能。     

Improved video streaming using MSVC and nonoverlapping zone routing multipath propagation over MANETs

Providing real‐time video streaming in mobile ad hoc networks is difficult because of the time‐dependent channel status and stringent service requirements. The currently existing route request‐reply–based multihop overlay networks cause considerable control overheads in video transmission resulting in loss of data and communication breakdown. Such networks are more suitable to nonstreaming video applications rather than to time‐sensitive video streaming applications. Therefore, a powerful mechanism needs to be adopted to handle the channel failures amicably and reduce latency effectively in time critical video streaming applications over mobile ad hoc networks. In order to be resilient to the channel failures and reduce latency in such applications, 2 strategies, namely, multistate video coding and 2‐tier–based nonoverlapping zone routing multipath propagation through directional antennas have respectively been incorporated. The performance of the proposed nonoverlapping zone routing multipath propagation system is compared with those of the existing multicast zone routing and zone‐based hierarchical link state routing protocols with parameters average end‐to‐end delay, routing overhead and packet delivery ratio using NS 2.34. The simulation results show that latency and resilience get considerably improved. Finally, the video quality of the proposed work has been verified by subjective and objective video testing methods.     

用于多视视频加深度的错误隐藏算法

提出了一种针对多视视频加深度(MVD)的错误隐藏( EC)算法。算法充分利用MVD特有的深度 信息及当前丢失宏块周围正确解码的宏块信息,将丢失宏块分为3类不同属性的宏块。针对3类宏块的特 点,分别提出了基于候选运动矢量修正(CMVR)、基于深度的外边界匹配(DOBMA)以及自适应 权 值的EC(AWEC)等模式。实验表明,本文提出的算法在保证相同的视频主客观质量情况下, 能够快速有效地实现EC。     

基于OpenFlow的SVC流媒体时延自适应分级传输方法

针对当前互联网流媒体传输的时延敏感性问题,提出一种基于OpenFlow的SVC（scalable video coding,可分级视频编码）流媒体时延自适应分级传输方法,该方法有效结合SVC流媒体可分级和OpenFlow灵活可编程的特性,在网络带宽受限和链路拥塞的复杂网络环境下,通过构建基础层和增强层2个独立路由,实现了动态网络下SVC流媒体分级自适应高效传输。仿真结果表明,该方法在提升SVC流媒体传输效率和质量,改善用户体验方面有重要作用。     

虚拟现实视频处理与传输技术

虚拟现实(VR)是当前视频领域研究热点.介绍了VR视频内容制作以及终端显示技术现状,分析了基于用户视点的VR视频处理关键技术,最后探讨了VR视频传输模式及其对承载网络的需求.     

Comparison of video protection methods for wireless networks

An experimental comparison of video protection methods targeted for wireless networks is presented. Basic methods are the data partitioning, reversible variable length coding, and macroblock row interleaving as well as macroblock scattering for packet loss protection. An implementation is described, in which scalable video is protected unequally with forward error correcting codes and retransmissions. Comparisons are performed for simulated wideband code division multiple access channel, and measurements are carried out with wireless local area network, Bluetooth as well as with GSM high speed circuit switched data. For the measurements, point-to-point connections are used. The achieved video quality is examined in our real-time wireless video demonstrator. The performance is measured with peak-signal-to-noise-ratio of received video, data overhead, communication delay, number of lost video frames, and decoding frame rate. Results show that the quality of decoded video can be improved by 1 dB with transparent connections compared to connections designed for general packet data. As a conclusion, a video coding subsystem must have access to the error control in a wireless link for the best quality in varying conditions.     

Performance modeling of FEC‐based unequal error protection for H.264/AVC video streaming over burst‐loss channels

Unequal error protection systems are a popular technique for video streaming. Forward error correction (FEC) is one of error control techniques to improve the quality of video streaming over lossy channels. Moreover, frame‐level FEC techniques have been proposed for video streaming because of different priority video frames within the transmission rate constraint on a Bernoulli channel. However, various communication and storage systems are likely corrupted by bursts of noise in the current wireless behavior. If the burst losses go beyond the protection capacity of FEC, the efficacy of FEC can be degraded. Therefore, our proposed model allows an assessment of the perceived quality of H.264/AVC video streaming over bursty channels, and is validated by simulation experiments on the NS‐2 network simulator at a given estimate of the packet loss ratio and average burst length. The results suggest a useful reference in designing the FEC scheme for video applications, and as the video coding and channel parameters are given, the proposed model can provide a more accurate evaluation tool for video streaming over bursty channels and help to evaluate the impact of FEC performance on different burst‐loss parameters. Copyright © 2014 John Wiley & Sons, Ltd.     

Scalable 3D video streaming over P2P networks with playback length changeable chunk segmentation

3D video distribution over P2P networks has been thought as a promising way for 3D video entering home. The convergence of scalable 3D video coding and P2P streaming can provide diverse 3D experiences for heterogeneous clients with high distribution efficiencies. However, the conventional chunk segmentation and scheduling algorithms originally aiming at the non-scalable 2D video streaming are not very efficient for scalable 3D video streaming over P2P networks due to the particular data characteristics of scalable 3D video. Based on this motivation, this paper first presents a playback length changeable 3D video chunk segmentation (PLC3DCS) algorithm to provide different error resilience strengths to video and depth as well as layers with different importance levels in the 3D video transmission. Then, a hybrid-priority based chunk scheduling (HPS) algorithm is proposed to be tied in with the proposed chunk segmentation algorithm to further promote the overall 3D video P2P streaming performance. The simulation results show that the proposed PLC3DCS algorithm with the corresponding HPS can increase the success delivery rates of chunks with more important levels, and further improve the user’s quality of 3D experience.     

一种信道自适应的无线视频流差错控制机制

提出了一种自适应的无线视频流差错保护方法。该方法充分考虑MPEG-4FGS码流细粒度可扩展的特性,根据码流的重要性程度采用不等的保护措施,同时能够自适应地根据无线信道的具体状况,将信道带宽在MPEG-4FGS信源编码速率和信道编码速率之间进行最优化分配,使得接收端能获得最佳的重建视频质量。实验结果表明,在各种不同信道状况下,与均等错误保护和固定的不等错误保护方法相比,该方法均可获得更好的性能。     

3D visual experience oriented cross-layer optimized scalable texture plus depth based 3D video streaming over wireless networks

3D video streaming over the mobile Internet generally incurs the inferior 3D visual experience due to the time-varying characteristics of wireless channel. The conventional video streaming optimization methods generally neglect the harmony among different networking protocol layers. This paper proposes a cross-layer optimized texture plus depth based scalable 3D video streaming method to improve the expected 3D visual experience of the user by systematically considering the application layer texture-video/depth/FEC bit-rate allocation, MAC layer multi-channel allocation, and physical layer modulation and channel coding scheme (MCS) selection. In the cross-layer optimization, a networking-related 3D visual experience model which fuses the overlapped retinal view visual quality and depth sensation with mimicking human vision system is established to predict the 3D visual experience under the specific parameter configurations of different protocol layers. The efficiency and effectiveness of the proposed cross-layer optimized 3D video streaming method has been validated by subjective and objective experimental results.     

Models and analysis of streaming video transmission over wireless fading channels

In this paper, we first develop a simple but nevertheless accurate and fully analytical model for the sequence-level distortion–rate (D–R) performance of predictive video source encoding. This model focuses on the input–output behavior of the source encoder and requires very limited amount of empirical data to set up its working parameters, namely three pairs of rate and distortion. Then, we propose an accurate and fully analytical model for the distortion due to lost frames in wireless communication systems. The proposed model builds on long-term average properties of the video source. In our analysis, we combine the properties of the video sequence and those of the wireless transmission link. Finally, we provide an asymptotic analysis of the video transmission through a wireless channel. We compare the results to those obtained by using less sophisticated models, like a (D–R)-function that is built on a Gaussian memoryless source. It is shown that the obtained results are substantially different.