Joint optimization of continuity and quality for streaming video

This paper aims to reduce the amount of prebuffering required to ensure a maximum video continuity in streaming. Current approaches do this by slowing the playout frame rate of the decoder, this is known as adaptive media playout (AMP). However, doing this introduces playout distortion to the viewers as the video is played slower than its natural playout rate. We approach this by proposing a frame rate control scheme that jointly adjusts the encoder frame generation rate of the encoder and the playout frame rate of the decoder. In addition to using AMP to improve video continuity, we also allow the encoder to increase the encoder frame generation rate. This means the encoder will be sending more frames to the decoder to quickly increase the number of frames available at the playback buffer, thus lowering the chance of buffer underflow which causes discontinuity in video playback. At the same time, the increase in the number of frames at the playback buffer may mean that the decoder does not need to use AMP to delay the playback, thus lowering the playback distortion. However, the increase in encoder frame generation rate comes at a price because frame quality will need to decrease in order to meet the constraint on available network bandwidth. This implies that the scheme needs to find the optimal trade-off between frame quality, playout distortion and video continuity. To do that, we characterize the frame rate control problem using Lyapunov optimization. We then systematically derive the optimization policies. We also show that these policies can be decoupled into separate encoder and decoder optimization policies, thus allowing for a distributed implementation. Simulation results show significant reductions in the prebuffering requirements over a scheme that perform no frame rate control and lower playout distortions compared to the AMP schemes, while exhibiting a modest drop in frame quality.     

Smooth Control of Adaptive Media Playout for Video Streaming

Client-side data buffering is a common technique to deal with media playout interruptions of streaming video caused by network jitters and packet losses of best-effort networks. However, stronger playout interruption protection inevitably amounts to larger data buffering and results in more memory requirements and longer playout delay. Adaptive media playout (AMP), also a client-side technique, can reduce the buffer requirement and avoid buffer outage but at the expense of visual quality degradation because of the fluctuation of playout speed. In this paper, we propose a novel AMP scheme to keep the video playout as smooth as possible while adapting to the channel condition. The triggering of the playout control is based on buffer variation rather than buffer fullness. Experimental results show that our AMP scheme surpasses conventional schemes in unfriendly network conditions. Unlike previous schemes that are tuned for a specific range of packet loss and network instability, the proposed AMP scheme maintains consistent performance across a wide range of network conditions.     

Adaptive VoIP playout scheduling: assessing user satisfaction

Delay and packet loss dramatically affect the quality of voice-over-IP (VoIP) calls and depend on the playout buffer scheme implemented at the receiver. The choice of playout algorithm can't be based on statistical metrics without considering the perceived end-to-end conversational speech quality. The authors present a method for evaluating various playout algorithms that extends the E-model concept by estimating user satisfaction from time-varying transmission impairments. This article evaluates several playout algorithms and shows a correspondence between the authors' results and those obtained via statistical loss and delay metrics.     

Optimization of a packet video receiver under different levels of delay jitter: an analytical approach

This paper studies the problem of analyzing and designing optimal playout adaptation policies for packet video receivers (PVRs) that operate in a delay jitter inducing best-effort network, like the current Internet. The developed system model is built around the E_k/D_i/1/N phase-type queue and allows for the effective modeling of key design and system parameters, such as: the level of delay jitter, the performance metrics and the employed playout policy. The optimal playout policy is derived under k-Erlang interarrivals by formulating and solving an optimization problem. The (theoretical) optimal solution is transformed into an approximately optimal one that utilizes observable information and it is, thus, feasible. Numerical results are derived under the optimal policy and compared against those under the optimal policy that assumes a fixed level of jitter as determined by Poisson arrivals, as well as against the deterministic service that applies no playout adaptation. Based on this work, a PVR is proposed that adapts to varying network delay jitter and tries to induce a performance that approximates the derived theoretical optimal one.     

Adaptive playout scheduling algorithm tailored for real-time packet-based voice conversations over wireless ad-hoc networks

The effective provision of real-time, packet-based voice conversations over multi-hop wireless ad-hoc networks faces several stringent constraints not found in conventional packet-based networks. Indeed, MANETs (mobile ad-hoc networks) are characterized by mobility of all nodes, bandwidth-limited channel, unreliable wireless transmission medium, etc. This environment will surely induce a high delay variation and packet loss rate impairing dramatically the user experienced quality of conversational services such as VoIP. Indeed, such services require the reception of each media unit before its deadline to guarantee a synchronous playback process. This requirement is typically achieved by artificially delaying received packets inside a de-jitter buffer. To enhance the perceptual quality the buffering delay should be adjusted dynamically throughout the vocal conversation.In this work, we describe the design of a playout algorithm tailored for real-time, packet-based voice conversations delivered over multi-hop wireless ad-hoc networks. The designed playout algorithm, which is denoted MAPA (mobility aware playout algorithm), adjusts the playout delay according to node mobility, which characterizes mobile ad-hoc networks, and talk-spurt, which is an intrinsic feature of voice signals. The detection of mobility is done in service passively at the receiver using several metrics gathered at the application layer. The perceptual quality is estimated using an augmented assessment approach relying on the ITU-T E-Model paradigm while including the time varying impairments observed by users throughout a packet-based voice conversation. Simulation results show that the tailored playout algorithm significantly outperforms conventional playout algorithms, specifically over a MANET with a high degree of mobility.     

Arrival process-controlled adaptive media playout with multiple thresholds for video streaming

To improve the playout quality of video streaming services, an arrival process-controlled adaptive media playout (AMP) mechanism is designed in this study. The proposed AMP scheme sets three threshold values, denoted by P _n , L and H, for the playout controller to start playback and dynamically adjust the playout rate based on the buffer fullness. In the preroll period, the playout can start only when the buffer fullness n is not less than the dynamic playback threshold P _n ,?which is determined by the jitters of incoming video frames. In the playback period, if the buffer fullness is below L or over H,?the playout rate will slow down or speed up in a quadratic manner. Otherwise, the playback speed depends on the instantaneous frame arrival rate, which is estimated by the proposed arrival process tracking algorithm. We employ computer simulations to demonstrate the performance of the proposed AMP scheme, and compare it with several conventional AMP mechanisms. Numerical results show that our AMP design can shorten the playout delay and reduce both buffer underflow and overflow probabilities. In addition, our proposed AMP also outperforms traditional AMP schemes in terms of the variance of distortion of playout and the playout curve. Hence, the proposed arrival process-controlled AMP is really an outstanding design.     

多媒体播放中的动态双门限自适应调整算法

针对无线网络存在的自相似特性会影响视频流的播放质量问题,提出了基于滑动窗口的接收端播放缓存调整算法,根据网络流量的变化,动态地调整双门限,并利用播放缓存的占用率来控制视频流的播放速度,平滑时延抖动.仿真实验证明,无论网络流量处于平稳状态还是处于突发状态,本文设计的算法都能够较好地保证视频流的连续播放,提高视频流的播放质量,为用户提供良好的视觉效果.     

实时语音的自适应回放算法

由于丢包和延时抖动的引入而使网络传输的实时语音质量让人难以接受，目前对丢包和延时抖动提出了很多的解决方案．但是却很少把这两者结合在一起进行研究。本文提出了一种新的自适应回放算法，通过监测接收和回放队列，结合丢包的自适应恢复技术，达到语音高质量的连续回放。实验证明，该算法能在严格的平均回放延时条件下努力减小由于超时而引起的丢包，获得较好的重建语音质量。     

Playout buffering of speech packets based on a quality maximization approach

To combat jitter problems in voice streaming over packet networks, playout buffering algorithms are used at the receiver side. Most of the proposed solutions rely on two main operations: prediction of delay statistics for future packets; setting of the end-to-end delay so as to limit or avoid packet losses. In recent years, a new approach has been presented, which is based on using a quality model to evaluate the impact of both packet loss and delay on the voice quality. Such a model is used to find the buffer setting that maximizes the expected quality. In this paper, we present a playout buffering algorithm whose main contribution is the extension of the new quality-based approach to the case of voice communications affected by bursty packet losses. This work is motivated by two main considerations: most of IP telephony applications are characterized by bursty losses instead of random ones; the human perception of the speech quality is significantly affected by the temporal correlation of losses. To this purpose, we make use of the extensions proposed in the ETSI Tiphon for the ITU-T E-Model so as to incorporate the effects of loss burstiness on the perceived quality. The resulting playout algorithm estimates the characteristics of the loss process varying the end-to-end delay, weights the loss and the delay effects on the perceived quality, and maximizes the overall quality to find the optimal setting for the playout buffer. The experimental results prove the effectiveness of the proposed technique.     

Priority-based Media Delivery using SVC with RTP and HTTP streaming

Media delivery, especially video delivery over mobile channels may be affected by transmission bitrate variations or temporary link interruptions caused by changes in the channel conditions or the wireless interface. In this paper, we present the use of Priority-based Media Delivery (PMD) for Scalable Video Coding (SVC) to overcome link interruptions and channel bitrate reductions in mobile networks by performing a transmission scheduling algorithm that prioritizes media data according to its importance. The proposed approach comprises a priority-based media pre-buffer to overcome periods under reduced connectivity. The PMD algorithm aims to use the same transmission bitrate and overall buffer size as the traditional streaming approach, yet is more likely to overcome interruptions and reduced bitrate periods. PMD achieves longer continuous playback than the traditional approach, avoiding disruptions in the video playout and therefore improving the video playback quality. We analyze the use of SVC with PMD in the traditional RTP streaming and in the adaptive HTTP streaming context. We show benefits of using SVC in terms of received quality during interruption and re-buffering time, i.e. the time required to fill a desired pre-buffer at the receiver. We present a quality optimization approach for PMD and show results for different interruption/bitrate-reduction scenarios.     

Analytical model for MPEG video frame loss rates and playback interruptions on packet networks

This paper presents and studies objective video quality evaluation techniques for a network where frame losses can be considered independent, for example a best effort not heavy loaded packet switching network. The total or partial loss of a frame’s information affects the quality of video playback, as the frame cannot be decoded and other frames that depend on it cannot be correctly decoded too. Therefore, during some time the video playback has errors in the image and the user will perceive them as interruptions. In this paper, the total number of decoded frames and the video playback interruptions duration will be considered important parameters to quantify the video quality. The analytical formulation for them will be presented and the importance of considering them together will be highlighted.     

Design and Experimental Evaluation of an Adaptive Playout Delay Control Mechanism for Packetized Audio for Use over the Internet

We describe the design and the experimental evaluation of a playout delay control mechanism we have developed in order to support unicast, voice-based audio communications over the Internet. The proposed mechanism was designed to dynamically adjust the talkspurt playout delays to the traffic conditions of the underlying network without assuming either the existence of an external mechanism for maintaining an accurate clock synchronization between the sender and the receiver during the audio communication, or a specific distribution of the audio packet transmission delays. Performance figures derived from several experiments are reported that illustrate the adequacy of the proposed mechanism in dynamically adjusting the audio packet playout delay to the network traffic conditions while maintaining a small percentage of packet loss.     

Performance evaluation for DRED discrete-time queueing network analytical model

Due to the rapid development in computer networks, congestion becomes a critical issue. Congestion usually occurs when the connection demands on network resources, i.e. buffer spaces, exceed the available ones. We propose in this paper a new discrete-time queueing network analytical model based on dynamic random early drop (DRED) algorithm to control the congestion in early stages. We apply our analytical model on two-queue nodes queueing network. Furthermore, we compare between the proposed analytical model and three known active queue management (AQM) algorithms, including DRED, random early detection (RED) and adaptive RED, in order to figure out which of them offers better quality of service (QoS). We also experimentally compare the queue nodes of the proposed analytical model and the three AQM methods in terms of different performance measures, including, average queue length, average queueing delay, throughput, packet loss probability, etc., aiming to determine the queue node that offers better performance.     

Packet audio playout delay adjustment: performance bounds and algorithms

In packet audio applications, packets are buffered at a receiving site and their playout delayed in order to compensate for variable network delays. In this paper, we consider the problem of adaptively adjusting the playout delay in order to keep this delay as small as possible, while at the same time avoiding excessive “loss” due to the arrival of packets at the receiver after their playout time has already passed. The contributions of this paper are twofold. First, given a trace of packet audio receptions at a receiver, we present efficient algorithms for computing a bound on the achievable performance of any playout delay adjustment algorithm. More precisely, we compute upper and lower bounds (which are shown to be tight for the range of loss and delay values of interest) on the optimum (minimum) average playout delay for a given number of packet losses (due to late arrivals) at the receiver for that trace. Second, we present a new adaptive delay adjustment algorithm that tracks the network delay of recently received packets and efficiently maintains delay percentile information. This information, together with a “delay spike” detection algorithm based on (but extending) our earlier work, is used to dynamically adjust talkspurt playout delay. We show that this algorithm outperforms existing delay adjustment algorithms over a number of measured audio delay traces and performs close to the theoretical optimum over a range of parameter values of interest.     

大数据量实时视频流传输拥塞控制研究

大数据量实时视频流传输系统有广泛的应用前景。大数据量、高带宽要求的视频流传输中存在许多问题。网络的负载均衡、视频的回放质量都将严重影响整个网络的安全以及传输系统的应用。通过对丢包率进行卡尔曼过滤(Kalman Filter)分析预测网络负载状况、结合接收缓冲区大小设置来平滑发送速率,在分析TCP友好拥塞控制的基础上,提出了一种基于RTCP反馈的TCP友好(TCP-Friendly)对该机制的TCP友好性、视频的回放质量进行了实验和结果分析。     

Delay and playout probability trade-off in mesh-based peer-to-peer streaming with delayed buffer map updates

In mesh-based peer-to-peer streaming systems data is distributed among the peers according to local scheduling decisions. The local decisions affect how packets get distributed in the mesh, the probability of duplicates and consequently, the probability of timely data delivery. In this paper we propose an analytic framework that allows the evaluation of scheduling algorithms. We consider four solutions in which scheduling is performed at the forwarding peer, based on the knowledge of the playout buffer content at the neighbors. We evaluate the effectiveness of the solutions in terms of the probability that a peer can play out a packet versus the playback delay, the sensitivity of the solutions to the accuracy of the knowledge of the neighbors’ playout buffer contents, and the scalability of the solutions with respect to the size of the overlay. We also show how the model can be used to evaluate the effects of node arrivals and departures on the overlay’s performance.     

Novel approaches for online playout delay prediction in VoIP applications using time series models

Voice over IP (VoIP) applications requires a buffer at the receiver to minimize the packet loss due to late arrival. Several algorithms are available in the literature to estimate the playout buffer delay. Classic estimation algorithms are non-adaptive, i.e. they differ from more recent approaches basically due to the absence of learning mechanisms. This paper introduces two new formulations of adaptive algorithms for online learning and prediction of the playout buffer delay, the first one being based on the standard Box-Jenkins autoregressive model, while the second one being based on the feedforward and recurrent neural networks. The obtained results indicate that the proposed algorithms present better overall performance than the classic ones.     

一种联合速率控制和缓冲管理的VBR视频鲁棒传输方法

In this paper we present an adaptive video transmission framework that integrates rate allocation and buffer control at the source with the playback adjustment mechanism at the receiver. A transmission rate is determined by a rate allocation algorithm which uses the program clock reference (PCR) embedded in the video streams to regulate the transmission rate in a refined way. The server side also maintains multiple buffers for packets of different importance levels to trade off random loss for controlled loss according to the source buffer size, the visual impact, and the playback deadline. An over-boundary playback adjustment mechanism based on proportional-integra (PI) controller is adopted at the receiver to maximize the visual quality of the displayed video according to the overall loss and the receiver buffer occupancy. The performance of our proposed framework is evaluated in terms of peak signal-to-noise ratio (PSNR) in the simulations, and the simulation results demonstrate the improvement of the average PSNR values as well as the better quality of the decoded frames.     

嵌入式流媒体客户端缓存技术

介绍了基于嵌入式微处理器S3C2440的嵌入式流媒体系统的硬件结构和工作流程. 服务器端通过RTP/RTCP协议将流媒体数据发送出去,客户端对收到的数据进行解压并实时播放. 将接收缓存分成接收缓冲区、播放缓冲区和DMA缓冲区,三个缓冲区的大小按1：1：2的比例设置,通过平均速率、延时抖动和解码码率等参数来约束缓冲区的容量. 在接收缓冲区设置两个临界点,通过对两个临界点的检测,来辅助调节发送端的数据发送速率. 既可以避免网络拥塞,又可以提高流媒体的传输质量.     

VOIP动态缓出时延算法研究分析

在以包为单位进行数据传输合、语音应用程序(VOIP,Voice Over Internet Protocol)中,为了补偿数据包在网络传输中不可预知的网络传输延迟,在接收端首先必须把接收到的数据包缓存起来,缓存一定的时间再播放出来,以减少通话的抖动,得到比较满意的通话质量。文章主要研究动态缓出时延算法,力求使这个缓出时延尽可能小,同时尽可能减少包的丢失率。文章提出了一个有效动态缓出时延算法,该算法主要跟踪最近到达的数据包的网络传输时延求出其近似分布函数,并利用这些信息和延迟峰的侦测算法预测下一个语音峰的缓出时延。实验结果表明利用该算法可以在缓出时延和包丢失率之间达到最佳平衡,是一种理想、有效的算法。