整帧丢失时基于多帧参考的差错掩盖算法

压缩视频对于信道误码十分敏感,可导致重组视频的质量严重下降。在低码率的情况下,当网络传输中出现数据包丢失的情况时,通常对应于一整帧图像内容的丢失。为此本文提出了两种针对整帧图像丢失的差错掩盖算法一基于多帧参考的差错掩盖算法。研究结果表明,该算法不仅能够恢复整个丢失帧,而且其掩盖效果比传统的掩盖算法更好。     

基于分层多播的视频传输拥塞控制算法研究

分层多播以多速率方式解决了多播接收者异构性问题,对于提高网络服务质量具有重要意义。本文分析了分层多播传输特性,通过提取MPEG视频流中的I帧、P帧、B帧组成3个帧流,分别放到分层多播的基层、增强层1和增强层2上传输,并在中间节点采用优先级队列机制,提出了一种面向视频流传输的分层多播拥塞控制(VLMCC)算法。仿真实验表明,本文提出的VLMCC算法能够适应视频多播接收者的异构性,大大提高了视频多播传输质量。     

一种整帧丢失时的视频差错掩盖算法

压缩视频对于信道误码十分敏感,可导致重组视频的质量严重下降。在低码率的情况下,当网络传输中数据包丢失时,通常对应于一整帧图像内容的丢失。为了恢复丢失帧,提出了一种针对整帧图像丢失时的基于运动向量外推法的空时域相结合的差错掩盖算法。试验结果表明,该算法不仅能够恢复整个丢失帧,而且在主观上与客观上其掩盖效果均优于传统的掩盖算法。     

结合显著性的MVD视频整帧丢失错误隐藏

基于多视点视频加深度（Multiview Video plus Depth,MVD）格式的三维视频能给用户提供多视角的沉浸式视觉体验.针对MVD视频流传输中彩色帧发生整帧丢失的情况,提出了一种结合视觉显著性的三维视频整帧丢失错误隐藏算法.先将丢失帧各区域划分为三种不同的显著等级,然后对低、中、高显著区域,分别采用时域直接拷贝、时域和视点间像素自适应搜索法和新型的运动矢量补偿法进行恢复.相较于目前的对比算法,经提出算法重建后的丢失帧的峰值信噪比值（Peak Signal to Noise Ratio,PSNR）和结构相似度值（Structural Similarity Index Measure,SSIM）能分别提高0.99～2.61 dB和0.005～0.012,并且重建图像主观视觉质量更佳.     

基于区域划分的立体视频整帧丢失错误隐藏新算法

针对立体视频传输中右视点整帧丢失,提出了一种基于区域划分的立体视频整帧丢失错误隐藏新算法。首先通过丢失帧视点内和视点间相邻帧的运动信息,估计丢失帧的运动区域与静止区域;然后利用视点间的全局视差,将丢失帧运动区域进一步划分为遮挡边界区域和非遮挡边界区域,并对遮挡边界区域和非遮挡边界区域分别采用运动补偿预测和视差补偿预测进行恢复,而对静止区域则直接采用帧拷贝的方法恢复丢失信息。实验结果表明,本文算法有效提高了所恢复的立体视频丢失帧图像的主客观质量。     

一种有效降低误差扩散的数据分割方法

提出一种有效降低误差扩散的数据分割方法.该方法改进了H.264划分子流的做法,采用当前编码宏块被错误解码时对下一编码帧产生的误差扩散程度来判别该宏块的优先级,以此划分不同的传输子流.在给定网络带宽和丢包率的情况下,采用率失真优化的码率分配算法对不同传输子流分别施以不同的信道编码保护力度,以保证在相同传输条件下,优先级高的宏块被正确接收的概率最大化,使重建视频流的质量达到最优.该方法实现简单,对宏块的分类更加合理,有效降低了由于包丢失引起的误差扩散效应,改善了重建视频的质量.实验结果表明在相同的带宽和丢包率条件下,本文方法比H.264方法提高视频接收质量大约0.3~0.6dB.     

OFDM系统中基于内容感知的SVC视频流调度算法

针对多用户视频流在下行正交频分多址(Orthogonal Frequency Division Multiplexing,OFDM)系统中的资源调度问题,重点研究基于可伸缩视频编码(Scalable Video Coding,SvC)的视频流调度机制.SVC作为一种具有空间、时间和质量3个维度的可伸缩性视频编码方案,能够为视频流在无线环境中提供灵活的传输方式,因此,研究提出一种OFDM系统中基于内容感知的SVC视频流调度算法,充分考虑不同SVC视频流分组的优先级与延迟约束,并利用OFDM技术在无线资源(时域、频域及功率)分配上的灵活性对分组进行调度.最后,研究通过仿真实验,对比了基于内容感知的SVC视频流调度算法与传统算法的性能特性,仿真结果表明所提出的调度机制相对于传统算法最高有3.2dB的平均峰值信噪比(PSNR,Peak Signal to Noise Ratio)增益.     

基于3D ResNet-LSTM的多视角人体动作识别方法

在基于视频图像的动作识别中,由于固定视角相机所获取的不同动作视频存在视角差异,会造成识别准确率降低等问题。使用多视角视频图像是提高识别准确率的方法之一,提出基于三维残差网络（3D Residual Network,3D ResNet）和长短时记忆（Long Short-term Memory,LSTM）网络的多视角人体动作识别算法,通过3D ResNet学习各视角动作序列的融合时空特征,利用多层LSTM网络继续学习视频流中的长期活动序列表示并深度挖掘视频帧序列之间的时序信息。在NTU RGB+D 120数据集上的实验结果表明,该模型对多视角视频序列动作识别的准确率可达83.2%。     

一种新的用于无线Ad hoc网络中视频传输的多路径包调度算法

该文提出了一种新的在无线Ad hoc网络中进行视频传输的多路径包调度算法(MPPA)。该算法充分考虑到视频流传输的特殊QoS要求,针对视频流中的每个数据包进行操作,通过计算将其指定在某条路径上传输。避免了传统多路径传输中的接收端数据包乱序和重新排序的问题,有效地减小了接收端的启动延时和缓冲区需求。同时仿真结果验证了上述理论的正确性。     

时空域自适应错误隐藏算法

由于解码器后处理错误隐藏技术主要在解码器端完成,不会在码流信息中加入冗余信息,所以解码器后处理错误隐藏技术在实践中得到广泛应用.目前解码器后处理错误隐藏方面已经提出了多种有效的方法,在对比现有的几种时空域错误隐藏的算法的基础上提出了针对视频流中I帧,P帧图像中不同的出错区域采用自适应的时空域错误隐藏算法来恢复原图像,模拟测试表明,该算法能够有效提高视频质量.     

Performance modeling of FEC‐based unequal error protection for H.264/AVC video streaming over burst‐loss channels

Unequal error protection systems are a popular technique for video streaming. Forward error correction (FEC) is one of error control techniques to improve the quality of video streaming over lossy channels. Moreover, frame‐level FEC techniques have been proposed for video streaming because of different priority video frames within the transmission rate constraint on a Bernoulli channel. However, various communication and storage systems are likely corrupted by bursts of noise in the current wireless behavior. If the burst losses go beyond the protection capacity of FEC, the efficacy of FEC can be degraded. Therefore, our proposed model allows an assessment of the perceived quality of H.264/AVC video streaming over bursty channels, and is validated by simulation experiments on the NS‐2 network simulator at a given estimate of the packet loss ratio and average burst length. The results suggest a useful reference in designing the FEC scheme for video applications, and as the video coding and channel parameters are given, the proposed model can provide a more accurate evaluation tool for video streaming over bursty channels and help to evaluate the impact of FEC performance on different burst‐loss parameters. Copyright © 2014 John Wiley & Sons, Ltd.     

Efficient PFD-Based Networking and Buffering Models for Improving Video Quality over Congested Links

Video traffic over the Internet becomes increasingly popular and is expected to comprise the largest proportion of the traffic carried by wired and wireless networks. On the other hand, videos are usually compressed by exploiting spatial and temporal redundancy for the reason of increasing the number of video streams that can be simultaneously carried over links. Unfortunately, receiving high-quality video streaming over the Internet remains a challenge due to the packet loss encountered in the congested wired and wireless links. In addition, the problem is more apparent in wireless links due to not only employing limited system capacity, but also some of the major drawbacks of wireless networks, out of which the bandwidth limitations and link asymmetry which refers to the situation where the forward and reverse paths of a transmission have different channel capacities. Therefore, the wireless hops may be congested which result in dropping many video frames. Additionally, as a result of compressing videos, dependencies among frames and within a frame arise. Consequently, the overall video quality tends to be degraded dramatically. The main challenge is to support the growth of video traffic while keeping the perceived quality of the delivered videos high. In this paper, we extend our previous work concerning improving video traffic over wireless networks through professionally studying the dependencies between video frames and their implications on the overall network performance. In other words, we propose very efficient network and buffer models proportionately to novel algorithms that aim to minimize the cost of aforementioned possible losses by selectively discarding frames based on their contribution to picture quality, namely, partial and selective partial frame discarding policies considering the dependencies between video frames. The performance metrics that are employed to evaluate the performance of the proposed algorithms include the rate of non-decodable frames, peak signal-to-noise ratio, frameput, average buffer occupancy, average packet delay, as well as jitter. Our results are so promising and show significant improvements in the perceived video quality over what is relevant in the current literature. We do not end up to this extent, but rather the effect of producing different bit-stream rates by the FFMPEG codecs on aforementioned performance metrics has been extensively studied.     

End-to-end TCP-friendly streaming protocol and bit allocation for scalable video over wireless Internet

With the convergence of wired-line Internet and mobile wireless networks, as well as the tremendous demand on video applications in mobile wireless Internet, it is essential to an design effective video streaming protocol and resource allocation scheme for video delivery over wireless Internet. Taking both network conditions in the Internet and wireless networks into account, in this paper, we first propose an end-to-end transmission control protocol (TCP)-friendly multimedia streaming protocol for wireless Internet, namely WMSTFP, where only the last hop is wireless. WMSTFP can effectively differentiate erroneous packet losses from congestive losses and filter out the abnormal round-trip time values caused by the highly varying wireless environment. As a result, WMSTFP can achieve higher throughput in wireless Internet and can perform rate adjustment in a smooth and TCP-friendly manner. Based upon WMSTFP, we then propose a novel loss pattern differentiated bit allocation scheme, while applying unequal loss protection for scalable video streaming over wireless Internet. Specifically, a rate-distortion-based bit allocation scheme which considers both the wired and the wireless network status is proposed to minimize the expected end-to-end distortion. The global optimal solution for the bit allocation scheme is obtained by a local search algorithm taking the characteristics of the progressive fine granularity scalable video into account. Analytical and simulation results demonstrate the effectiveness of our proposed schemes.     

A handover-aware seamless video streaming scheme in heterogeneous wireless networks

With the development of wireless technologies, video streaming services over heterogeneous wireless networks have become more popular in recent years. Video streaming schemes for heterogeneous networks should consider vertical handover in which the link capacity is varied significantly, because the quality experienced for a video streaming service is affected by the network status. When a vertical handover occurs, an abrupt bandwidth change and substantial handover latency lead to bursty packet loss and discontinuity of the video playback. In this paper, we propose a handover-aware video streaming scheme to provide seamless video streaming services over heterogeneous wireless networks. The proposed scheme adjusts its sending rate and the quality level of the transmitted video streams according to the significant bandwidth variation that occurs in a vertical handover. To expedite the response to the bandwidth variation due to a handover, our scheme uses an explicit notification message that informs the streaming server of a client's handover occurrence. In order to evaluate the performance, we use a simulation environment for a vertical handover between wireless local area networks and cellular networks. Through the simulation results, we prove that our scheme improves the experienced quality of video streaming in vertical handovers.     

Adaptive video protection in large scale peer‐to‐peer video streaming over mobile wireless mesh networks

Because video streaming over mobile handheld devices has been of great interest, the necessity of introducing new methods with low implementation cost and scalable infrastructures is a strong demand of the service. In particular, these requirements are present in popular wireless networks such as wireless mesh networks (WMN). Peer‐to‐peer (P2P) networks promise an efficient scalable network infrastructure for video streaming over wired and wireless networks. Limited resources of the peers in P2P networks and high error rate in wireless channels make it more challenging to run P2P streaming applications over WMNs. Therefore, it is necessary to design efficient and improved error protection methods in P2P video streaming applications over WMNs. In this paper, we propose a new adaptive unequal video protection method specially intended for large scale P2P video streaming over mobile WMNs. Using this method, different frames have different priorities in receivers along the recovery process. Moreover, we precisely and completely evaluate different aspects related to frame protection in these networks using five important performance metrics including video distortion, late arrival distortion, end‐to‐end delay, overhead and initial start‐up delay. The results obtained from a precise simulation in OMNeT++ show that the proposed adaptive method significantly outperforms other solutions by providing better video quality on mobile wireless nodes. Copyright © 2015 John Wiley & Sons, Ltd.     

Downlink media streaming with wireless fountain coding in wireline‐cum‐WiFi networks

This paper addresses the problem of streaming packetized media data in a combined wireline/802.11 network. Since the wireless channel is normally the bottleneck for media streaming in such a network, we propose that wireless fountain coding (WFC) be used over the wireless downlink in order to efficiently utilize the wireless bandwidth and exploit the broadcast nature of the channel. Forward error correction (FEC) is also used to combat errors at the application‐layer. We analytically obtain the moment generating function (MGF) for the wireless link‐layer delay incurred by WFC. With the MGF, the expected value of this wireless link‐layer delay is found and used by the access point (AP), who has no knowledge of the buffer contents of wireless receivers, to make a coding‐based decision. We then derive the end‐to‐end packet loss/late probability based on the MGF. We develop an integrated ns‐3/EvalVid simulator to evaluate our proposed system and compare it with the traditional 802.11e scheme which is without WFC capability but equipped with application‐ and link‐layer retransmission mechanisms. Through extensive simulations of video streaming, we show that streaming with WFC is able to support more concurrent video flows compared to the traditional scheme. When the deadlines imposed on video packets are relatively stringent, streaming with WFC also shows superior performance in terms of packet loss/late probability, video distortion, and video frame delay, over the traditional scheme. Copyright © 2011 John Wiley & Sons, Ltd.     

CASHeW: Cluster-based Adaptive Scheme for Multimedia Delivery in Heterogeneous Wireless Networks

Multimedia streaming over heterogeneous wireless networks has attracted significant interest in recent years from both telecom network operators and end users. However, the heterogeneity of the wireless network makes it very difficult to synchronize real-time multimedia streaming to different types of end-user devices across different wireless networks. In addition, with different delay and packet loss across different networks, multimedia delivery over the heterogeneous wireless networks cannot provide good quality streaming video. This paper proposes CASHeW—a novel cluster-based design with an in-built feedback-based adaptive mechanism that results in a higher video perceived quality in two-hop heterogeneous wireless network environments. CASHeW employs a proxy-client-server mechanism between the base station (BS) and the end-user; and importantly uses a quality-oriented adaptive scheme for efficient multimedia delivery. Simulation-based tests indicate that the performance of CASHeW not only outperforms transport layer adaptive delivery protocols like the TCP-Friendly Rate Control Protocol (TFRCP) and Loss Delay Adaptation (LDA+), but also is better than that of medium access control (MAC) layer protocols such as the Receiver Based Auto Rate (RBAR) and Enhanced Distributed Channel Access (EDCA) in terms of average perceived quality, average bit rate and loss rate.     

Performance modeling of MPEG‐4 video streaming over IEEE 802.11 using distribution coordination function

Delivering video streaming over wireless Internet is becoming increasingly popular. However, most of the research studies focused on the modeling analysis of system performance such as saturation throughput and channel utilization. Perceived quality of video streaming cannot be assessed solely based on the results of analytical models. In this paper, we propose a model to assess the perceived quality of MPEG‐4 video streaming over IEEE 802.11 distribution coordination function (DCF)‐based wireless local area networks. The analysis of our proposed model considers not only effects of losses such as collision loss from channel access competition but also wireless loss caused by wireless interferences. Moreover, the impact of the loss of specific MPEG‐4 video frames is also taken into account in the performance analysis. The model was validated by comparing our performance results with results obtained from simulation and analytical models. The results show that our proposed model is able to predict the perceived quality of MPEG‐4 video streaming over DCF‐based WLAN more accurately than other models. Copyright © 2010 John Wiley & Sons, Ltd.     

Design and Implementation of a Network‐Adaptive Mechanism for HTTP Video Streaming

This paper proposes a network‐adaptive mechanism for HTTP‐based video streaming over wireless/mobile networks. To provide adaptive video streaming over wireless/mobile networks, the proposed mechanism consists of a throughput estimation scheme in the time‐variant wireless network environment and a video rate selection algorithm used to increase the streaming quality. The adaptive video streaming system with proposed modules is implemented using an open source multimedia framework and is validated over emulated wireless/mobile networks. The emulator helps to model and emulate network conditions based on data collected from actual experiments. The experiment results show that the proposed mechanism provides higher video quality than the existing system provides and a rate of video streaming almost void of freezing.     

基于视频自然统计特性的无参考移动终端视频质量评价

针对无线网络中压缩编码及无线丢包等因素对移动终端视频的降质影响,在分析视频相邻帧差信号空-时感知统计特性的基础上,该文提出一种基于视频自然统计特性的无参考移动终端视频质量评价(NMVQA)算法。进行视频帧差空-时自然统计规律分析,确定移动终端视频失真类型对视频相邻帧差系数统计特性的影响;计算水平、垂直、主对角线和副对角线方向的帧差相邻系数乘积分布参数的时域统计特性;以多尺度帧差相邻系数的时域统计特性相关程度来衡量移动终端视频失真程度。在LIVE移动视频数据库上的实验结果表明,该文所提算法的结果与主观评价具有较好的一致性,能够准确反映人类对视频失真程度的视觉感知效果,可为实时在线调节信源码率和无线信道参数提供参考依据。