Unequal packet loss resilience for fine-granular-scalability video

Several embedded video coding schemes have been recently developed for multimedia streaming over IP. In particular, fine-granular-scalability (FGS) video coding has been recently adopted by the MPEG-4 standard as the core video-compression method for streaming applications. From its inception, the FGS scalability structure was designed to be packet-loss resilient especially under unequal packet-loss protection (UPP). However, since the introduction of FGS, there has not been a comprehensive study evaluating its packet-loss resilience under unrecoverable packet losses that are common in Internet streaming applications. In this paper, we evaluate two important aspects of FGS packet-loss resilience. First, we study the impact of applying UPP between the base- and enhancement-layers on FGS-based streams, and we compare equal packet-loss protection (EPP) with UPP scenarios. Second, we introduce the notion of fine-grained loss protection (FGLP), which is suitable for the FGS enhancement-layer, and we develop an analytical framework for evaluating FGLP bounds. Based on these bounds, we show the impact of applying fine-grained protection to the FGS enhancement-layer for different types of video sequences and over a wide range of bit-rates and packet-loss ratios. As illustrated by our extensive simulation results, applying 1) UPP between the base- and enhancement-layers and 2) FGLP for the FGS enhancement-layer can provide significant resilience under moderate-to-high packet-loss ratios (e.g., 5-10%). Furthermore, the merits of this new packet-loss protection technique go beyond the FGS coding scheme, because FGLP can be successfully applied to improve the resilience to packet-losses of other embedded video coding techniques     

一种压缩域中基于镜头的关键帧提取改进算法*

关键帧提取技术是基于内容视频检索的基础。针对压缩视频序列提出一种基于镜头的关键帧提取改进算法。其核心是根据压缩视频序列的编码特点,只需要对其进行部分解码,利用I帧信息的直流分量信息构造DC缩微图,并结合图像帧不同区域DC信息的重要程度差异进行相似性度量,进而实现关键帧提取。实验证明,该算法较之传统算法,在查全率和检索时间两项指标上均有显著改善,尤其适用于新闻纪录片、电影片等局部运动较为剧烈的视频序列。     

无线Ad hoc网络中的丢包分析模型研究

针对无线Ad hoc网络中的丢包现象,假设网络中节点的丢包率是与归一化吞吐量有关的,并且在此假设的基础上提出一种分析丢包率的数学模型,利用这种模型可以一定程度的解释无线Ad hoc网络中的丢包。仿真结果表明利用该模型的理论分析结果与仿真结果具有相当的相似性,表明该模型在一定程度上解释了无线Ad hoc网络中的丢包问题。     

XOR-based frame loss recovery scheme for video streaming

High coding dependencies among video frames suffer from vulnerability to packet loss, which impacts the playback quality of video streaming. In this paper, according to the characteristics of MPEG4/H.264 encoding methods, we propose a simple and low-complexity XOR-based FEC frame loss recovery scheme. Within an entire Group of Pictures (GOP), the proposed scheme shows the ability to recover simultaneously I-frame loss and one P-frame loss. The high frame loss resilience improves the playback QoS of compressed video streaming. The mathematical analysis reveals that the proposed scheme has 72.7% performance improvement than no frame loss protection in term of full GOP frames successful decoding rate.     

丢包对视频体验质量影响的分析

网络传输过程中发生丢包会降低用户对视频的体验质量。使用EvalVid仿真传输测试床来深入分析丢包率及不同位置丢包对用户体验的影响,并根据回归分析法建立了映射模型。用户使用该模型可以很方便地得出用户的体验质量。经理论分析证明该模型正确、易于操作,可实时检测丢包对视频的影响。     

An unequal packet loss resilience scheme for video over the Internet

We present an unequal packet loss resilience scheme for robust transmission of video over the Internet. By jointly exploiting the unequal importance existing in different levels of syntax hierarchy in video coding schemes, GOP-level and Resynchronization-packet-level Integrated Protection (GRIP) is designed for joint unequal loss protection (ULP) in these two levels using forward error correction (FEC) across packets. Two algorithms are developed to achieve efficient FEC assignment for the proposed GRIP framework: a model-based FEC assignment algorithm and a heuristic FEC assignment algorithm. The model-based FEC assignment algorithm is to achieve optimal allocation of FEC codes based on a simple but effective performance metric, namely distortion-weighted expected length of error propagation, which is adopted to quantify the temporal propagation effect of packet loss on video quality degradation. The heuristic FEC assignment algorithm aims at providing a much simpler yet effective FEC assignment with little computational complexity. The proposed GRIP together with any of the two developed FEC assignment algorithms demonstrates strong robustness against burst packet losses with adaptation to different channel status.     

A new approach for packet loss measurement of video streaming and its application

Packet loss is of great importance as a metric that characterizes the network’s performance, and is crucial for video applications, congestion control and routing. Most of existing measurement tools can indicate the packet loss of network links instead of the actual packet loss of individual application. On the other hand, because occurrence of packet loss behavior is relatively rare and its duration is short, active measuring methods need to inject a large number of packets and run for a long time for reporting accurate estimates, which would introduce additional intrusiveness to the network and perturb user traffic. In this paper, we present a new packet loss estimation technique by making use of user_data field of video, which is less intrusive since it does not affect video playing and does not need to inject extra probing stream. It can also provide the packet loss detailed information of I,P,B frames. The accuracy of the algorithm has been evaluated with both simulations and experiments over real-world Internet paths. In addition, we analyze the video quality distortion caused by packet loss of different frame types, and a real-time video quality monitoring system is built.     

Joint source coding and packet classification for real-time video transmission over differentiated services networks

Differentiated Services (DiffServ) is one of the leading architectures for providing quality of service in the Internet. We propose a scheme for real-time video transmission over a DiffServ network that jointly considers video source coding, packet classification, and error concealment within a framework of cost-distortion optimization. The selections of encoding parameters and packet classification are both used to manage end-to-end delay variations and packet losses within the network. We present two dual formulations of the proposed scheme: the minimum distortion problem, in which the objective is to minimize the end-to-end distortion subject to cost and delay constraints, and the minimum cost problem, which minimizes the total cost subject to end-to-end distortion and delay constraints. A solution to these problems using Lagrangian relaxation and dynamic programming is given. Simulation results demonstrate the advantage of jointly adapting the source coding and packet classification in DiffServ networks.     

一种改进的3G无线网络丢包模型及仿真方法

通信系统的可靠性仍有待提高,无线网络的丢包问题是网络应用研究中比较重要的问题。通过对通信信道建模,利用计算机进行信道仿真,可以不完全依靠实际的网络而进行相关网络应用的分析实验。基于马尔可夫Gap模型,提出了一种改进的网络丢包模型,并基于该模型进行了分析和仿真实验。该方法不仅考虑了长时间的网络丢包状况,同时考虑了相邻丢包之间的相互关系,能够比现有模型更加准确地描述3G网络的丢包特性。     

An adaptable neural-network model for recursive nonlinear traffic prediction and modeling of MPEG video sources

Multimedia services and especially digital video is expected to be the major traffic component transmitted over communication networks [such as internet protocol (IP)-based networks]. For this reason, traffic characterization and modeling of such services are required for an efficient network operation. The generated models can be used as traffic rate predictors, during the network operation phase (online traffic modeling), or as video generators for estimating the network resources, during the network design phase (offline traffic modeling). In this paper, an adaptable neural-network architecture is proposed covering both cases. The scheme is based on an efficient recursive weight estimation algorithm, which adapts the network response to current conditions. In particular, the algorithm updates the network weights so that 1) the network output, after the adaptation, is approximately equal to current bit rates (current traffic statistics) and 2) a minimal degradation over the obtained network knowledge is provided. It can be shown that the proposed adaptable neural-network architecture simulates a recursive nonlinear autoregressive model (RNAR) similar to the notation used in the linear case. The algorithm presents low computational complexity and high efficiency in tracking traffic rates in contrast to conventional retraining schemes. Furthermore, for the problem of offline traffic modeling, a novel correlation mechanism is proposed for capturing the burstness of the actual MPEG video traffic. The performance of the model is evaluated using several real-life MPEG coded video sources of long duration and compared with other linear/nonlinear techniques used for both cases. The results indicate that the proposed adaptable neural-network architecture presents better performance than other examined techniques.     

Stabilization of linear systems over networks with bounded packet loss and its use in model predictive control

This paper considers stabilization of discrete-time linear systems, where network exists for transmitting the sensor and controller information, and arbitrary bounded packet loss occurs in the sensor–controller link and the controller–actuator link. The stabilization of this system is transformed into the robust stabilization of a set of systems. The stability result for this system is specially applied on model predictive control (MPC) that explicitly considers the satisfaction of input and state constraints. Two synthesis approaches of MPC are presented, one parameterizing the infinite horizon control moves into a single state feedback law, the other into a free control move followed by the single state feedback law. Two simulation examples are given to illustrate the effectiveness of the proposed techniques.     

Joint rate control for VBR MPEG video on PVC ATM links

A joint rate control algorithm for variable bit rate (VBR) MPEG-compressed digital video on point-to-point permanent virtual circuit (PVC) ATM links is proposed. The algorithm controls the encoding mode of a number of video encoders that operate either on VBR or constant bit rate (CBR) mode. The algorithm selects the encoding mode based on the buffer occupancy of a multiplexer co-located with the encoders that interfaces them to the PVC link. VBR encoding is the predominant encoding mode used during congestion-free periods. CBR mode is used only during congested periods. When CBR is selected, new bit rates are jointly assigned to the encoders based on their relative encoding complexities. The bit rate assigned to the encoders are enforced by a CBR rate control local to each encoder. The performance of the joint rate control algorithm is evaluated through simulation of a packet multiplexer, where a number of connections are multiplexed onto a fixed-capacity channel. The performance is compared to that of multiplexing conventional CBR and open-loop VBR from the video quality and bandwidth efficiency points of view. Simulation results show that the proposed algorithm improves performance over multiplexed conventional CBR or open-loop VBR MPEG video, without significantly increasing implementation complexity. The application of the algorithm to video-on-demand over ATM is discussed.     

Texture classification and segmentation using wavelet packet frame and Gaussian mixture model

In this paper, we propose a scheme for texture classification and segmentation. The methodology involves an extraction of texture features using the wavelet packet frame decomposition. This is followed by a Gaussian-mixture-based classifier which assigns each pixel to the class. Each subnet of the classifier is modeled by a Gaussian mixture model and each texture image is assigned to the class to which pixels of the image most belong. This scheme shows high recognition accuracy in the classification of Brodatz texture images. It can also be expanded to an unsupervised texture segmentation using a Kullback-Leibler divergence between two Gaussian mixtures. The proposed method was successfully applied to Brodatz mosaic image segmentation and fabric defect detection.     

Flow-aware synthesis:A generic motion model for video frame interpolation

A popular and challenging task in video research,frame interpolation aims to increase the frame rate of video.Most existing methods employ a fixed motion model,...     

Robust speech recognition over mobile and IP networks in burst-like packet loss

This paper addresses the problem of achieving robust distributed speech recognition in the presence of burst-like packet loss. To compensate for packet loss a number of techniques are investigated to provide estimates of lost vectors. Experimental results on both a connected digits task and a large vocabulary continuous speech recognition task show that simple methods, such as repetition, are not as effective as interpolation methods which are better able to preserve the dynamics of the feature vector stream. Best performance is given by maximum a-posteriori (MAP) estimation of lost vectors which utilizes statistics of the feature vector stream. At longer burst lengths the performance of these compensation techniques deteriorates as the temporal correlation in the received feature vector stream reduces. To compensate for this interleaving is proposed which aims to disperse bursts of loss into a series of unconnected smaller bursts. Results show substantial gains in accuracy, to almost that of the no loss condition, when interleaving is combined with estimation techniques, although this is at the expense of introducing delay. This leads to the proposal that, for a distributed speech recognition application, it is more beneficial to trade delay for accuracy rather than trading bit-rate for accuracy as in forward error correction schemes.     

基于非均匀离散余弦变换的MPEG 4视频内容保护*

由于视频数据量大、处理上实时性要求高,加密方法的效率通常是视频内容安全的关键。在MPEG4框架下,提出了一种新的视频保护方法。该方法利用非均匀离散余弦变换(NDCT)取代视频编解码中的常规离散余弦变换(DCT),对MPEG4视频数据在频域上进行加扰保护和解扰,并将控制离散余弦变换非均匀性的参数作为密钥使用。由于不存在专门的密码操作模块,整个方法的时间和空间开销与正常的编解码相当,且从保护效果和安全性方面满足了大量应用的要求。     

基于DSP的MPEG-4视频解码回放

阐述以TMS320C64x为核心的DSP开发板IEKC64为开发工具,实现MPEG-4视频解码回放的方案。探讨了整个解码回放器的软件开发过程和步骤,论述了解码回放主程序的编程思路和说明了应用到的主要接口函数,以及解释解码回放程序的关键代码,对方案的进一步改进和完善提出了建议。     

基于视频传感器网络的目标跟踪模型研究

提出了一种基于视频传感器网络的模型,其核心思想是通过视频节点和非视频节点相互协作建立模型,对目标进行探测、跟踪和定位。在探测和跟踪阶段,通过非视频节点对目标进行探测。目标在移动过程中,由非视频节点实时地跟踪目标大致的移动方向和位置;在定位阶段,由视频节点完成对目标准确定位。仿真结果表明,在节点随机部署的情况下,该模型可以有效地实现对目标区域的全覆盖,通过非视频节点和视频节点的相互协作,发挥了非视频节点对目标探测和跟踪的优势,以及视频节点对目标准确定位特性。     

Encryption and watermarking for the secure distribution of copyrighted MPEG video on DVD

This paper presents a complete system for the secure distribution of a copyrighted MPEG-1/2 video stored on a DVD-ROM disc. A combined selective watermarking and encryption method that operates in the compressed MPEG domain is introduced. Watermarking resistant to a number of attacks is used for copyright protection. The video quality deteriorates significantly due to encryption, thus restraining unauthorized viewers from viewing it. The video can only be viewed using the developed Secure MPEG Player, which performs real-time decryption of the encrypted video. The decryption requires a secret key that is extracted from the DVD-ROM disc in a cryptographically secure manner.This work was supported by the EU IST Project "ASPIS"