基于压缩域的关键帧快速提取方法

关键帧提取技术是基于内容检索和视频分析的基础。关键帧的使用减少了视频索引的数据量,同时也为视频摘要和检索提供了一个组织框架。首先介绍了目前的关键帧提取技术,然后提出了一种基于运动特征利用模糊推理算法从MPEG视频流中提取关键帧的方法。由于处理过程是直接从MPEG的压缩视频提取,不需对其解压,所以计算复杂度低,提高了提取速度。实验证明该方法效率高,可以比较好地代表视频内容。     

Unequal packet loss resilience for fine-granular-scalability video

Several embedded video coding schemes have been recently developed for multimedia streaming over IP. In particular, fine-granular-scalability (FGS) video coding has been recently adopted by the MPEG-4 standard as the core video-compression method for streaming applications. From its inception, the FGS scalability structure was designed to be packet-loss resilient especially under unequal packet-loss protection (UPP). However, since the introduction of FGS, there has not been a comprehensive study evaluating its packet-loss resilience under unrecoverable packet losses that are common in Internet streaming applications. In this paper, we evaluate two important aspects of FGS packet-loss resilience. First, we study the impact of applying UPP between the base- and enhancement-layers on FGS-based streams, and we compare equal packet-loss protection (EPP) with UPP scenarios. Second, we introduce the notion of fine-grained loss protection (FGLP), which is suitable for the FGS enhancement-layer, and we develop an analytical framework for evaluating FGLP bounds. Based on these bounds, we show the impact of applying fine-grained protection to the FGS enhancement-layer for different types of video sequences and over a wide range of bit-rates and packet-loss ratios. As illustrated by our extensive simulation results, applying 1) UPP between the base- and enhancement-layers and 2) FGLP for the FGS enhancement-layer can provide significant resilience under moderate-to-high packet-loss ratios (e.g., 5-10%). Furthermore, the merits of this new packet-loss protection technique go beyond the FGS coding scheme, because FGLP can be successfully applied to improve the resilience to packet-losses of other embedded video coding techniques     

Feedback priority control for MPEG video transmitted through ATM networks

Although ATM networks are suitable for transporting variable bit rate (VBR) traffic, bursty moving picture exports group (MPEG) traffic is tending to make the networks congested and results in cell loss. It is suggested that cells should be assigned different priorities according to the significance of the contained information such that the networks can try to protect the high-priority cells and obtain high video quality. Considering the payload behind the slice header, losing any cell would disable the decoding for the following part of the same slice. As a result, if any cell has to be discarded, the farther the position is, the lesser the following data are affected. In this article, a simple mechanism called ‘feedback control priority partition with RC-shaper (FBCPP&RC)’ is proposed to prevent losing cells at the former position by adaptively assigning high priority to these cells. The decision policy is simple and quick. It also employs an RC-shaper to smooth the burstiness of the cells so as to reduce cell-loss probability. The mechanism is expected to facilitate transmitting MPEG video as a cheaper available bit rate (ABR) traffic rather than VBR traffic. According to the simulation results, it performs well in improving the quality of single-frame as well as continuous pictures. It can also be applied to the new standard MPEG-II.     

基于小波变换的MPEG视频流预测模型

提出了一种基于小波变换的MPEG视频流的预测模型,该模型对原始数据做Haar小波变换以后,充分考虑尺度系数和乘子的不同统计特性,对尺度系数和乘子采用不同的预测模型分别预测,最终通过多重分形小波模型建模过程合成MPEG视频流序列。利用仿真和数学分析的方法对预测模型在刻画MPEG视频流的主要统计特性方面进行性能分析,从误差、数据的概率分布特性以及自相似特性等方面,证实了该预测模型的优越性能。     

Collusion,MPEG4 compression and frame dropping resistant video watermarking

We present in this article a new video watermarking which resists collusion, MPEG4 compression and frame dropping attacks. This scheme is based on video mosaicing. For that, we are going to start by describing the mosaicing technique in order to illustrate the contribution of this technique in video watermarking. In fact, mosaicing allows to select an interesting area where the mark should be embedded. The idea is to insert the same mark into the same pixels which represent the same physical point. This is exactly the information which can be provided by a mosaic image at least for the located points in the scene background. Next, we present extensive experimental simulations which prove the watermark imperceptibility and robustness against several video attacks.     

Impact of packet loss on 4K UHD video for portable devices

Multimedia Tools and Applications - Ultra High Definition (UHD) video streaming to portable devices has become topical. Two standardized codecs are current, H.264/Advanced Video Coding (AVC) and...     

An unequal packet loss resilience scheme for video over the Internet

We present an unequal packet loss resilience scheme for robust transmission of video over the Internet. By jointly exploiting the unequal importance existing in different levels of syntax hierarchy in video coding schemes, GOP-level and Resynchronization-packet-level Integrated Protection (GRIP) is designed for joint unequal loss protection (ULP) in these two levels using forward error correction (FEC) across packets. Two algorithms are developed to achieve efficient FEC assignment for the proposed GRIP framework: a model-based FEC assignment algorithm and a heuristic FEC assignment algorithm. The model-based FEC assignment algorithm is to achieve optimal allocation of FEC codes based on a simple but effective performance metric, namely distortion-weighted expected length of error propagation, which is adopted to quantify the temporal propagation effect of packet loss on video quality degradation. The heuristic FEC assignment algorithm aims at providing a much simpler yet effective FEC assignment with little computational complexity. The proposed GRIP together with any of the two developed FEC assignment algorithms demonstrates strong robustness against burst packet losses with adaptation to different channel status.     

Vector-quantization-based video codec for software-only playback on personal computers

This paper discusses a video compression and decompression method based on vector quantization (VQ) for use on general purpose computer systems without specialized hardware. After describing basic VQ coding, we survey common VQ variations and discuss their impediments in light of the target application. We discuss how the proposed video codec was designed to reduce computational complexity in every principal task of the video codec process. We propose a classified VQ scheme that satisfies the data rate, image quality, decoding speed, and encoding speed objectives for software-only video playback. The functional components of the proposed VQ method are covered in detail. The method employs a pseudo-YUV color space and criteria to detect temporal redundancy and low spatial frequency regions. A treestructured-codebook generation algorithm is proposed to reduce encoding execution time while preserving image quality. Two separate vector codebooks, each generated with the treestructured search, are employed for detail and low spatial frequency blocks. Codebook updating and sharing are proposed to further improve encoder speed and compression.     

A new approach for packet loss measurement of video streaming and its application

Packet loss is of great importance as a metric that characterizes the network’s performance, and is crucial for video applications, congestion control and routing. Most of existing measurement tools can indicate the packet loss of network links instead of the actual packet loss of individual application. On the other hand, because occurrence of packet loss behavior is relatively rare and its duration is short, active measuring methods need to inject a large number of packets and run for a long time for reporting accurate estimates, which would introduce additional intrusiveness to the network and perturb user traffic. In this paper, we present a new packet loss estimation technique by making use of user_data field of video, which is less intrusive since it does not affect video playing and does not need to inject extra probing stream. It can also provide the packet loss detailed information of I,P,B frames. The accuracy of the algorithm has been evaluated with both simulations and experiments over real-world Internet paths. In addition, we analyze the video quality distortion caused by packet loss of different frame types, and a real-time video quality monitoring system is built.     

Adaptive bandwidth allocation based on playback tunnel to support video on demand service over ATM networks

This paper focuses on adaptive bandwidth allocation (ABA) for transporting pre-stored VBR compressed video data across networks in support of video-on-demand (VoD) service. Two key issues are addressed. Firstly, a simplified dynamic programming method based on playback tunnel is proposed to predict the bandwidth which can be dynamically allocated for transporting video segment to guarantee that the playback buffer neither underflows nor overflows. Secondly, ABA schemes with different criteria including middle-tunnel ABA (MT-ABA) and reduced-bandwidth-variability ABA (RBV-ABA) are investigated. The proposed ABA schemes are evaluated by a set of real-life MPEG video traces. The obtained results show that the RBV-ABA scheme outperforms MT-ABA scheme in terms of peak rate, traffic burstiness and bandwidth variability. On the other hand, the MT-ABA scheme is more significant for avoiding the playback buffer overflow or underflow than that of RBV-ABA.     

Joint source coding and packet classification for real-time video transmission over differentiated services networks

Differentiated Services (DiffServ) is one of the leading architectures for providing quality of service in the Internet. We propose a scheme for real-time video transmission over a DiffServ network that jointly considers video source coding, packet classification, and error concealment within a framework of cost-distortion optimization. The selections of encoding parameters and packet classification are both used to manage end-to-end delay variations and packet losses within the network. We present two dual formulations of the proposed scheme: the minimum distortion problem, in which the objective is to minimize the end-to-end distortion subject to cost and delay constraints, and the minimum cost problem, which minimizes the total cost subject to end-to-end distortion and delay constraints. A solution to these problems using Lagrangian relaxation and dynamic programming is given. Simulation results demonstrate the advantage of jointly adapting the source coding and packet classification in DiffServ networks.     

An adaptable neural-network model for recursive nonlinear traffic prediction and modeling of MPEG video sources

Multimedia services and especially digital video is expected to be the major traffic component transmitted over communication networks [such as internet protocol (IP)-based networks]. For this reason, traffic characterization and modeling of such services are required for an efficient network operation. The generated models can be used as traffic rate predictors, during the network operation phase (online traffic modeling), or as video generators for estimating the network resources, during the network design phase (offline traffic modeling). In this paper, an adaptable neural-network architecture is proposed covering both cases. The scheme is based on an efficient recursive weight estimation algorithm, which adapts the network response to current conditions. In particular, the algorithm updates the network weights so that 1) the network output, after the adaptation, is approximately equal to current bit rates (current traffic statistics) and 2) a minimal degradation over the obtained network knowledge is provided. It can be shown that the proposed adaptable neural-network architecture simulates a recursive nonlinear autoregressive model (RNAR) similar to the notation used in the linear case. The algorithm presents low computational complexity and high efficiency in tracking traffic rates in contrast to conventional retraining schemes. Furthermore, for the problem of offline traffic modeling, a novel correlation mechanism is proposed for capturing the burstness of the actual MPEG video traffic. The performance of the model is evaluated using several real-life MPEG coded video sources of long duration and compared with other linear/nonlinear techniques used for both cases. The results indicate that the proposed adaptable neural-network architecture presents better performance than other examined techniques.     

Stabilization of linear systems over networks with bounded packet loss and its use in model predictive control

This paper considers stabilization of discrete-time linear systems, where network exists for transmitting the sensor and controller information, and arbitrary bounded packet loss occurs in the sensor–controller link and the controller–actuator link. The stabilization of this system is transformed into the robust stabilization of a set of systems. The stability result for this system is specially applied on model predictive control (MPC) that explicitly considers the satisfaction of input and state constraints. Two synthesis approaches of MPC are presented, one parameterizing the infinite horizon control moves into a single state feedback law, the other into a free control move followed by the single state feedback law. Two simulation examples are given to illustrate the effectiveness of the proposed techniques.     

Texture classification and segmentation using wavelet packet frame and Gaussian mixture model

In this paper, we propose a scheme for texture classification and segmentation. The methodology involves an extraction of texture features using the wavelet packet frame decomposition. This is followed by a Gaussian-mixture-based classifier which assigns each pixel to the class. Each subnet of the classifier is modeled by a Gaussian mixture model and each texture image is assigned to the class to which pixels of the image most belong. This scheme shows high recognition accuracy in the classification of Brodatz texture images. It can also be expanded to an unsupervised texture segmentation using a Kullback-Leibler divergence between two Gaussian mixtures. The proposed method was successfully applied to Brodatz mosaic image segmentation and fabric defect detection.     

Flow-aware synthesis:A generic motion model for video frame interpolation

A popular and challenging task in video research,frame interpolation aims to increase the frame rate of video.Most existing methods employ a fixed motion model,...     

Impact of video scheduling on bandwidth allocation for multiplexed MPEG streams

We present efficient schemes for scheduling the delivery of variable-bit-rate MPEG-compressed video with stringent quality-of-service (QoS) requirements. Video scheduling is being used to improve bandwidth allocation at a video server that uses statistical multiplexing to aggregate video streams prior to transporting them over a network. A video stream is modeled using a traffic envelope that provides a deterministic time-varying bound on the bit rate. Because of the periodicity in which frame types in an MPEG stream are typically generated, a simple traffic envelope can be constructed using only five parameters. Using the traffic-envelope model, we show that video sources can be statistically multiplexed with an effective bandwidth that is often less than the source peak rate. Bandwidth gain is achieved without sacrificing the stringency of the requested QoS. The effective bandwidth depends on the arrangement of the multiplexed streams, which is a measure of the lag between the GOP periods of various streams. For homogeneous streams, we give an optimal scheduling scheme for video sources at a video-on-demand server that results in the minimum effective bandwidth. For heterogeneous sources, a sub-optimal scheduling scheme is given, which achieves acceptable bandwidth gain. Numerical examples based on traces of MPEG-coded movies are used to demonstrate the effectiveness of our schemes.     

Joint rate control for VBR MPEG video on PVC ATM links

A joint rate control algorithm for variable bit rate (VBR) MPEG-compressed digital video on point-to-point permanent virtual circuit (PVC) ATM links is proposed. The algorithm controls the encoding mode of a number of video encoders that operate either on VBR or constant bit rate (CBR) mode. The algorithm selects the encoding mode based on the buffer occupancy of a multiplexer co-located with the encoders that interfaces them to the PVC link. VBR encoding is the predominant encoding mode used during congestion-free periods. CBR mode is used only during congested periods. When CBR is selected, new bit rates are jointly assigned to the encoders based on their relative encoding complexities. The bit rate assigned to the encoders are enforced by a CBR rate control local to each encoder. The performance of the joint rate control algorithm is evaluated through simulation of a packet multiplexer, where a number of connections are multiplexed onto a fixed-capacity channel. The performance is compared to that of multiplexing conventional CBR and open-loop VBR from the video quality and bandwidth efficiency points of view. Simulation results show that the proposed algorithm improves performance over multiplexed conventional CBR or open-loop VBR MPEG video, without significantly increasing implementation complexity. The application of the algorithm to video-on-demand over ATM is discussed.     

Robust speech recognition over mobile and IP networks in burst-like packet loss

This paper addresses the problem of achieving robust distributed speech recognition in the presence of burst-like packet loss. To compensate for packet loss a number of techniques are investigated to provide estimates of lost vectors. Experimental results on both a connected digits task and a large vocabulary continuous speech recognition task show that simple methods, such as repetition, are not as effective as interpolation methods which are better able to preserve the dynamics of the feature vector stream. Best performance is given by maximum a-posteriori (MAP) estimation of lost vectors which utilizes statistics of the feature vector stream. At longer burst lengths the performance of these compensation techniques deteriorates as the temporal correlation in the received feature vector stream reduces. To compensate for this interleaving is proposed which aims to disperse bursts of loss into a series of unconnected smaller bursts. Results show substantial gains in accuracy, to almost that of the no loss condition, when interleaving is combined with estimation techniques, although this is at the expense of introducing delay. This leads to the proposal that, for a distributed speech recognition application, it is more beneficial to trade delay for accuracy rather than trading bit-rate for accuracy as in forward error correction schemes.