Rate-distortion optimized distributed packet scheduling of multiple video streams over shared communication resources

We consider the problem of distributed packet selection and scheduling for multiple video streams sharing a communication channel. An optimization framework is proposed, which enables the multiple senders to coordinate their packet transmission schedules, such that the average quality over all video clients is maximized. The framework relies on rate-distortion information that is used to characterize a video packet. This information consists of two quantities: the size of the packet in bits, and its importance for the reconstruction quality of the corresponding stream. A distributed streaming strategy then allows for trading off rate and distortion, not only within a single video stream, but also across different streams. Each of the senders allocates to its own video packets a share of the available bandwidth on the channel in proportion to their importance. We evaluate the performance of the distributed packet scheduling algorithm for two canonical problems in streaming media, namely adaptation to available bandwidth and adaptation to packet loss through prioritized packet retransmissions. Simulation results demonstrate that, for the difficult case of scheduling nonscalably encoded video streams, our framework is very efficient in terms of video quality, both over all streams jointly and also over the individual videos. Compared to a conventional streaming system that does not consider the relative importance of the video packets, the gains in performance range up to 6 dB for the scenario of bandwidth adaptation, and even up to 10 dB for the scenario of random packet loss adaptation.     

Failure-Aware, Open-Loop, Adaptive Video Streaming With Packet-Level Optimized Redundancy

A plethora of coding and streaming mechanisms have been proposed for real-time multimedia transmission over the Internet. However, most proposed mechanisms rely only on global (e.g. based on end-to-end measurements), delayed (at least by the round-trip-time), or statistical (often based on simplistic network models) information available about the network state. Based on recently-proposed state-of-the-art open-loop video coding schemes, we propose a new integrated streaming and routing framework for robust and efficient video transmission over networks exhibiting path failures. Our approach explicitly takes into account the network dynamics, path diversity, and the modeled video distortion at the receiver side to optimize the packet redundancy and scheduling. In the derived framework, multimedia streams can be adapted dynamically at the video server based on instantaneous routing-layer information or failure-modeling statistics. The performance of our integrated application and network-layer method is simulated against equivalent approaches that are not optimized based on routing-layer feedback and distortion modeling, and the obtained gains in video quality are quantified     

Concurrent multipath transmission combining forward error correction and path interleaving for video streaming

Video streaming is a popular application on next generation networks (NGNs). However, video streaming over NGNs has many challenges due to the high bit error rates of these networks. Forward error correction (FEC) is often applied to improve the quality of video streaming. However, continuous lost packets decrease the recovery performance of FEC protection in NGNs. To disperse continuous lost packets to different FEC blocks, we propose a concurrent multipath transmission that combines FEC with path interleaving. Our proposed control scheme adaptively adjusts the FEC block length and concurrently sends data interleaved over multiple paths. Experimental results with our approach show improved packet loss and signal to noise ratio performance.     

Video Packet Selection and Scheduling for Multipath Streaming

This paper addresses the problem of choosing the best streaming policy for distortion optimal multipath video delivery, under network bandwidth and playback delay constraints. The streaming policy consists in a joint selection of the network path and of the video packets to be transmitted, along with their sending time. A simple streaming model is introduced, which takes into account the video packet importance, and the dependencies between packets. A careful timing analysis allows to compute the quality perceived by the receiver for a constrained playback delay, as a function of the streaming policy. We derive an optimization problem based on a video abstraction model, under the assumption that the server knows, or can predict accurately the state of the network. A detailed analysis of constrained multipath streaming systems provides helpful insights to design an efficient branch and bound algorithm that finds the optimal streaming strategy. This solution allows to bound the performance of any scheduling strategy, but the complexity of the algorithm becomes rapidly intractable. We therefore propose a fast heuristic-based algorithm, built on load-balancing principles. It allows to reach close to optimal performance with a polynomial time complexity. The algorithm is then adapted to live streaming scenarios, where the server has only a partial knowledge of the packet stream, and the channel bandwidth. Extensive simulations show that the proposed algorithm only induces a negligible distortion penalty compared to the optimal strategy, even when the optimization horizon is limited, or the rate estimation is not perfect. Simulation results also demonstrate that the proposed scheduling solution performs better than common scheduling algorithms, and therefore represents a very efficient low-complexity multipath streaming algorithm, for both stored and live video services     

基于视频业务质量优化的认知无线电传输信道选择

认知无线电网络中,次级用户选择信道的传统技术基于信道特性对传输信道进行随机选择,忽略了应用层视频业务对信道质量的要求。针对该问题提出了一种基于视频业务质量优化的信道选择技术,优化视频业务端到端的传输质量。通过最小化端到端视频失真,跨层优化综合选择物理层传输信道、自适应调制与编码模式以及应用层的编码量化参数。该方法在多信道认知无线电网络下进行了大量的视频传输仿真模拟实验。实验结果表明该方法能够比不含信道选择的跨层优化方法提高认知无线电网络下次级用户的视频传输业务客观质量1.5 dB以上。     

A Queuing-Theoretic Approach to Task Scheduling and Processor Selection for Video-Decoding Applications

We propose a cross-layer design for resource-constrained systems that simultaneously decode multiple video streams on multiple parallel processors, cores, or processing elements. Our proposed design explicitly considers the coder specific application characteristics such as the decoding dependencies, decoding deadlines, and distortion impacts of different video packets (e.g., frames, slices, groups of slices etc.). The key to the cross-layer design is the resource management control plane (RMCP) that coordinates the scheduling and processor selection across the active applications. The RMCP deploys a priority-queuing model that can evaluate the system congestion and predict the total expected video quality for the set of active decoding tasks. Using this model, we develop a robust distortion-and delay-aware scheduling algorithm for video packets. This algorithm aims to maximize the sum of achieved video qualities over all of the decoded video sequences. Additionally, we propose a processor selection scheme intended to minimize the delays experienced by the queued video packets. In this way, the number of missed decoding deadlines is reduced and the overall decoded video quality is increased. We compare queuing-theoretic based scheduling strategies to media agnostic scheduling strategies (i.e., earliest-deadline-first scheduling) that do not jointly consider the decoding deadlines and distortion impacts. Our results illustrate that by directly considering the video application's properties in the design of a video decoding system, significant system performance gains on the order of 4 dB peak-signal-to-noise ratio can be achieved.     

Adaptive content-and-deadline aware chunk scheduling in mesh-based P2P video streaming

In mesh-based Peer-to-Peer (P2P) live video streaming systems packet scheduling is an important factor in overall video playback quality. In mesh based P2P video streaming systems, each video sequence is divided into chunks, which are then distributed by multiple suppliers to the receivers. The suppliers need to be coordinated by the receiver through specifying a transmission schedule for each of them. Many previous studies on scheduling of P2P streaming tend to mainly focus on networking issues which strongly depend on a particular P2P architecture such as tree or mesh. These algorithms suffer from some design issues: 1) they are too complex to deploy, 2) they do not take video characteristics into account and 3) they do not have sender-side transmission policy. To address all three of these problems, we propose a new chunk scheduling scheme which consists of two parts: i) receiver-side scheduler and ii) sender-side transmission order scheme. The proposed receiver-side scheduler considers the contribution level of each video frame as well as the frame’s urgency in order to define a priority for each video frame. It attempts to request frames with highest priority from peers which can deliver them in a shorter time. We also design a new chunk transmission order scheme that decides which requested chunk will be sent out first based on its importance to the requesting neighbor. Our simulation results show that the proposed scheduling scheme improves the overall quality of the perceived video in mesh-based P2P video streaming architectures substantially.     

Priority and delay aware packet management framework for real-time video transport over 802.11e WLANs

The rigid delay constraint is one of the most challenging issues in real-time video delivery over wireless networks. The expired video packets will become useless for the decoding and display even if they are received correctly at the receiver. Because the significance of each video packet is different, the schedulers have to take into account not only the urgency of the packet but also its importance in the real-time video applications. However, the existing QoS-based IEEE 802.11e MAC protocol leaves the urgency and the importance of video packets out of consideration. This paper proposes a Priority and Delay Aware Packet Management Framework (PDA-PMF) to improve the transmission quality of real-time video streaming over IEEE 802.11e WLANs. In the MAC layer, this framework estimates the delay of each video packet. Subsequently, video packets are sent or dropped according to both the significance of the video packets and the estimation value of the delay. Simulation results show that the proposed scheme can not only reduce the packet losses, but also protect the more important video packets, so as to improve the received video quality effectively.     

IEEE 802.11e中基于失真度度量的视频包传输方法

为改善H. 264编码的视频流在802. 11e中的传输性能, 提出了一种结合H. 264/AVC中不同类型的数据分割对视频重建质量的重要性因子和队列状态的视频包映射方法。首先定量分析H. 264/AVC中A、B、C三种分割的丢失对视频重建质量的影响, 得到其重要因子; 然后依据重要因子和队列长度将视频数据包映射到802. 11e的不同EDCA队列中。算法改进了EDCA机制中数据包的静态映射机制, 根据视频分割数据的不等重要性, 提供差异性服务。仿真结果表明, 与目前的视频包静态映射机制相比, 该算法提高了视频重建质量, 最好可提高1 dB以上。     

WiMAX网络中基于优先级的视频分组调度算法

无线视频传输是网络传输的研究热点,在WiMAX网络中,实时轮询业务(rtPS)的典型应用是实时视频传输,当前使用较多的是先进先出(FIFO)和最早到达期限数据优先(EDF)队列调度算法,其中EDF算法多用于多业务之间的资源分配,而非单个业务流队列的出对调度。在视频优先级以及WiMAX网络为单个视频流分配带宽限定的条件下进行调度算法的研究,以期待在带宽有限的情况下得到更高的视频服务质量。     

Churn-aware optimal layer scheduling scheme for scalable video distribution in super-peer overlay networks

To model a layered video streaming system in super-peer overlay networks that faces with heterogeneity and volatility of peers, we formulate a layer scheduling problem from understanding some constraints such as layer dependency, transmission rule, and bandwidth heterogeneity. To solve this problem, we propose a new layer scheduling algorithm using a real-coded messy genetic algorithm, providing a feasible solution with low complexity in decision. We also propose a peer-utility-based promotion algorithm that selects the most qualified neighbor to guarantee the sustained quality of streaming despite high intensity of churn. Simulation results show that the proposed layer scheduling scheme can achieve the most near-optimal solutions compared to the four conventional scheduling heuristics in the average streaming ratio. It also highly outperforms those with different peer selection strategies in terms of the average bandwidth (6.9 % higher at least) and the variation of utilization (11.3 % lower at least).     

A cross-layer framework for video-on-demand service in multi-hop WiMax mesh networks

In this work, we introduce a cross-layer framework to favor the video-on-demand service in multi-hop WiMax mesh networks. We first propose a joint solution of admission control and channel scheduling for video streams. The proposed approach guarantees that the required data rate is achieved for video streams, which is crucial for multimedia streaming applications. An efficient and light-weight multicast routing technique is also proposed to minimize the bandwidth cost of joining a multicast tree. Furthermore, we adopt the Patching technique in the application layer to improve the capacity of the video server. Overall, the quality of the video-on-demand service is dramatically improved with the help of the efficient cooperation between the techniques proposed in different layers of the network. Simulation study shows that with the proposed approach, true video-on-demand in WiMax mesh networks can be achieved under high video request arrival rate.     

视频流传输中的网络编码综述

网络视频应用增长迅猛,利用网络编码（NC）来提高网络吞吐量和传输可靠性进而改善视频流传输质量成为研究热点。针对如何优化网络编码进行视频流传输这一问题,必须要结合视频流自身特性作出改进并综合考虑所处的网络环境,才能充分发挥网络编码的优势。首先回顾了网络编码的基本概念和方法;然后对网络编码应用于视频流传输时需要考虑的视频业务特性,包括进行不等差错保护以优先保障重要等级视频数据包、降低数据包传输延迟以满足视频流实时性需求、增强网络差错恢复策略以提升传输可靠性等作了分析和归纳;接着阐述了基于网络编码的视频流传输策略在包括P2P、多源协作及内容中心网络等典型场景中的应用;最后对网络编码应用于视频流传输的研究趋势进行展望。     

Priority-based Media Delivery using SVC with RTP and HTTP streaming

Media delivery, especially video delivery over mobile channels may be affected by transmission bitrate variations or temporary link interruptions caused by changes in the channel conditions or the wireless interface. In this paper, we present the use of Priority-based Media Delivery (PMD) for Scalable Video Coding (SVC) to overcome link interruptions and channel bitrate reductions in mobile networks by performing a transmission scheduling algorithm that prioritizes media data according to its importance. The proposed approach comprises a priority-based media pre-buffer to overcome periods under reduced connectivity. The PMD algorithm aims to use the same transmission bitrate and overall buffer size as the traditional streaming approach, yet is more likely to overcome interruptions and reduced bitrate periods. PMD achieves longer continuous playback than the traditional approach, avoiding disruptions in the video playout and therefore improving the video playback quality. We analyze the use of SVC with PMD in the traditional RTP streaming and in the adaptive HTTP streaming context. We show benefits of using SVC in terms of received quality during interruption and re-buffering time, i.e. the time required to fill a desired pre-buffer at the receiver. We present a quality optimization approach for PMD and show results for different interruption/bitrate-reduction scenarios.     

层次编码流媒体发送调度的研究

文章针对层次编码流媒体数据在Internet上进行传输的问题,研究服务端包发送调度技术。根据发送缓冲区中层次编码流媒体数据包之间存在的多重依赖关系和传输时间限制,提出了服务端发送调度的马尔科夫模型,在该模型的基础上,用值迭代法实现一个最优的发送策略。该策略能够在带宽受限的条件下,保证重要性较高的数据包优先发送出去。通过实验证明,与顺序发送方法比较,该调度算法能够在客户端获得更好的播放质量。     

Trade-Offs in Bit-Rate Allocation for Wireless Video Streaming

One of the central problems in wireless video transmission is the choice of source and channel coding rates to allocate the available transmission rate optimally. In this paper, we present a structural distortion model for video streaming over time-varying fading channels. Based on the model, we study the end-to-end distortion for various bit-rate allocation strategies and channel conditions. We show that the robustness to channel variations is crucial for the streaming performance when frequent bit-rate adaptations are not feasible. It is achieved at the expense of higher source distortion in the encoder. Our findings are illustrated on a practical problem of distortion-optimal selection of transport formats in an adaptive modulation and coding (AMC) scheme used in HSDPA.     

Efficient Streaming Packet Video Over Differentiated Services Networks

We investigate streaming video over Differentiated Services (Diffserv) networks that can provide a number of aggregated traffic classes with increasing quality guarantee. We propose a method to measure the loss impact of a video packet on the quality of the decoded video images. We show how the optimal Quality-of-Service (QoS) mapping from the video packets into a set of traffic classes depends on the loss rates of the classes and the pricing model, and we develop an algorithm to accurately find the optimal QoS mapping. The performance of our algorithm is evaluated through computer simulations and compares favorably to an existing algorithm.     

Cross-Layer prioritized H.264 video packetization and error protection over noisy channels

Video transmission over wireless channels is affected by channel-induced packet losses. Distortion due to channel errors can be alleviated by applying forward error correction. Aggregating H.264/AVC slices to form video packets with sizes adapted to their importance can also improve transmission reliability. Larger packets are more likely to be in error but smaller packets require more overhead. We present a cross-layer dynamic programming (DP) approach to minimize the expected received video distortion by jointly addressing the priority-adaptive packet formation at the application layer and rate compatible punctured convolutional (RCPC) code rate allocation at the physical layer for prioritized slices of each group of pictures (GOP). Some low priority slices are also discarded to improve protection to more important slices and meet the channel bit-rate limitations. We propose two schemes. Our first scheme carries out joint optimization for all slices of a GOP at a time. The second scheme extends our cross-layer DP-based approach to slices of each frame by predicting the expected channel bit budget per frame for live streaming. The prediction uses a generalized linear model developed over the cumulative mean squared error per frame, channel SNR, and normalized compressed frame bit budget. The parameters are determined over a video dataset that spans high, medium and low motion complexity. The predicted frame bit budget is used to derive the packet sizes and corresponding RCPC code rates for live transmission using our DP-based approach. Simulation results show that both proposed schemes significantly improve the received video quality over contemporary error protection schemes.     

3TP: an application-Layer protocol for streaming 3-D models

This paper addresses the problem of streaming progressively compressed three-dimensional (3-D) models over lossy networks. Out of all encoded packets that can be transmitted, we intelligently choose a subset of packets to be transmitted using transport control protocol in order to meet a distortion constraint, while transmitting the remaining packets using user datagram protocol to minimize the end-to-end delay. We call this new application-layer protocol 3-D models transport protocol. We show the effectiveness of this protocol both experimentally and theoretically. We compare the performance of the proposed protocol with systems that do not optimize transmission according to the content of the encoded bitstream. When the maximum distortion is 30, measured using the Hausdorff distance, we achieve savings in delay time ranging from 39% to 68% for packet-loss rates between 1% and 19%.