Joint In-network Video Rate Adaptation and Measurement-Based Admission Control: Algorithm Design and Evaluation

The important new revenue opportunities that multimedia services offer to network and service providers come with important management challenges. For providers, it is important to control the video quality that is offered and perceived by the user, typically known as the quality of experience (QoE). Both admission control and scalable video coding techniques can control the QoE by blocking connections or adapting the video rate but influence each other’s performance. In this article, we propose an in-network video rate adaptation mechanism that enables a provider to define a policy on how the video rate adaptation should be performed to maximize the provider’s objective (e.g., a maximization of revenue or QoE). We discuss the need for a close interaction of the video rate adaptation algorithm with a measurement based admission control system, allowing to effectively orchestrate both algorithms and timely switch from video rate adaptation to the blocking of connections. We propose two different rate adaptation decision algorithms that calculate which videos need to be adapted: an optimal one in terms of the provider’s policy and a heuristic based on the utility of each connection. Through an extensive performance evaluation, we show the impact of both algorithms on the rate adaptation, network utilisation and the stability of the video rate adaptation. We show that both algorithms outperform other configurations with at least 10 %. Moreover, we show that the proposed heuristic is about 500 times faster than the optimal algorithm and experiences only a performance drop of approximately 2 %, given the investigated video delivery scenario.     

Subjective and objective quality assessment of videos in error-prone network environments

Compression and transmission are two fundamental stages involved in wireless video communications, each of which may cause degradation of the quality of experience (QoE) of end users by producing compression artifacts and packet loss artifacts, respectively. They have their own unique perceptual influences. To provide insight for designing QoE-aware content delivery applications, this paper studies subjective and objective quality of videos containing both types of artifacts. First, subjective quality assessment is conducted, from which interaction between the two types of artifacts during quality perception is investigated. Second, using the subjective data, the performance of the state-of-the-art objective quality metrics is evaluated, with the aim of examining suitability of the existing metrics for their use in error-prone video communication applications. Finally, the developed data set is made publicly available for the community.     

QoE-Aware wireless video communications for emotion-aware intelligent systems: A multi-layered collaboration approach

With the ever increasing demand on high-quality visual information for emotion-aware intelligent systems, wireless video traffic explosively grows and causes great energy consumption. Therefore, providing high quality of experience (QoE) for connected users becomes increasingly important. Aiming to establish a new paradigm to solve this challenging problem, in this article we propose a multi-layered collaboration approach to provide energy-efficient QoE-aware wireless video communications by efficiently utilizing the limited transmission resources of wireless networks for 5G. We first investigate the emotion-aware intelligent system QoE measurement based on objective metrics of quality of service (QoS). Then, we utilize the multi-layered collaborations of physical, network and application layers among the connected users to achieve energy-efficient QoE-aware video communications. By developing a profound understanding of the interplay between the video applications and wireless networks, we qualitatively analyze how QoE can benefit from the multi-layered collaborations, and quantitatively assess the achievable gains in a typical wireless-connected emotion-aware application scenario.     

Quality assessment for a visual and automatic license plate recognition

Video transmission and analysis is often utilized in applications outside of the entertainment sector, and generally speaking this class of video is used to perform specific tasks. Examples of these applications include security and public safety. The Quality of Experience (QoE) concept for video content used for entertainment differs significantly from the QoE of surveillance video used for recognition tasks. This is because, in the latter case, the subjective satisfaction of the user depends on achieving a given functionality. Recognizing the growing importance of video in delivering a range of public safety services, we focused on developing critical quality thresholds in license plate recognition tasks based on videos streamed in constrained networking conditions. Since the number of surveillance cameras is still growing it is obvious that automatic systems will be used to do the tasks. Therefore, the presented research includes also analysis of automatic recognition algorithms.     

QoE prediction model for mobile video telephony

Interactive online video applications, such as video telephony, are known for their vulnerability to network condition. With the increasing usage of hand-held wireless mobile devices, which are capable of capturing and processing good quality videos, combined with the flexibility in an end-user movements have added new challenging factors for application providers and network operators. These factors affect the perceived video quality of mobile video telephony applications, unlike conventional video telephony over desktop computers. We investigate this impact on video quality of mobile video telephony in varying network conditions and end-users movement scenarios. Based on 312 live traces, we quantitatively derive the correlation between the perceived video quality and the network Quality of Service (QoS) and user mobility. With the results, we develop a Quality of Experience (QoE) prediction model for mobile video telephony using Support Vector Regression techniques. The prediction models display ≈ 0.8 pearson correlation with experimental data. Our methodology and findings can be used to guide the video telephony application providers and network operators to work towards satisfying end-user experience.     

A Multicast-Enabled Delivery Framework for QoE Assurance of Over-The-Top Services in Multimedia Access Networks

Over-The-Top (OTT) video services are becoming more and more important in today’s broadband access networks. While original OTT services only offered short duration medium quality videos, more recently, premium content such as high definition full feature movies and live video are offered as well. For operators, who see the potential in providing Quality of Experience (QoE) assurance for an increased revenue, this introduces important new network management challenges. Traditional network management paradigms are often not suited for ensuring QoE guarantees as the provider does not have any control on the content’s origin. In this article, we focus on the management of an OTT-based video service. We present a loosely coupled architecture that can be seamlessly integrated into an existing OTT-based video delivery architecture. The framework has the goal of resolving the network bottleneck that might occur from high peaks in the requests for OTT video services. The proposed approach groups the existing Hypertext Transfer Protocol (HTTP) based video connections to be multicasted over an access network’s bottleneck and then splits them again to reconstruct the original HTTP connections. A prototype of this architecture is presented, which includes the caching of videos and incorporates retransmission schemes to ensure robust transmission. Furthermore, an autonomic algorithm is presented that allows to intelligently select which OTT videos need to be multicasted by making a remote assessment of the cache state to predict the future availability of content. The approach was evaluated through both simulation and large scale emulation and shows a significant gain in scalability of the prototype compared to a traditional video delivery architecture.     

一种基于视频分类的码率自适应算法

流媒体的码率自适应算法依据网络状态动态调节视频块的码率,提升用户体验质量,但忽略了视频类型的差异对用户体验质量的影响,导致算法性能下降。提出区分视频类型特征的码率选择算法C-ABR。设计相应的用户体验质量效用函数,使用强化学习算法训练模型A3C,提升用户体验质量。实验结果说明,相对于典型的码率自适应算法Pensieve和MPC,C-ABR算法用户体验质量分别提升22.7%和50.4%。     

Effect of kinesthetic and tactile haptic feedback on the quality of experience of edutainment applications

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user’s sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application’s Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.     

异构环境下层次编码多视频源多共享信道分层组播

视频组播是许多当前和将来网络服务的重要组成部分,如视频会议,远程学习、远地展示及视频点播,随着网络传送基础设施的改善和端系统处理能力的增强,组播视频应用日益变得可行,组播视频传输中存在的主要问题是网络送资源的异构性和动态性,其使得视频流的多个接收方都达到可接受的流量特性变得异常困难,目前该问题的一个有效解决方式就是利用自适应的分层视频传输机制,在该机制中,各源产生层次媒体流,并在多个网络信道中传输。对视频会议类的多点到多点视频组播应用,信道往往被所有潜在的发送方共享,任何发送方都可在任何一个共享信道中发送其视频层次。在该多点到多点、共享信道、分层视频组播模型下,一个关键问题就是如何动态确定各视频源层次到各共享组播信道的映射,映射策略直接影响到会话整体视频接收质量和网络带宽利用率。典型的方式是顺序映射,该映射方式同等对待各发送方,但利用该方式,随源数目的增加,在各共享网络信度上会出现带宽可伸缩性问题,而且顺序映射方式无法适应网络传送资源和会话状态的动态变化。为此,该文设计了一种基于接收方反馈信息的自适应的层次映射算法,接收方周期性地将其当前感兴趣的发送方及接收速率的信息反馈给某控制节点,而控制节点就利用当前反馈信息动态地调整映射策略。经证实,该算法始终能比顺序层次映射算法获得更高的整体视频接收质量,并具有高的带宽利用率和很小的复杂度。     

大规模客户端视频流带宽调度

视频流服务的迅猛发展, 大规模用户共享带宽链路的场景不断增多. 现存的DASH视频流采用的ABR算法多用于提高单客户端用户的体验质量(quality of experience, QoE), 还有一些算法仅针对数个客户端的情况. 本文提出一种应用于大规模客户端场景的带宽调度算法, 通过聚类算法减小调度规模, 再将带宽分...     

Dynamic Resource Allocation for Robust Distributed Multi-Point Video Conferencing

This paper proposes a distributed multi-point video conferencing system over packet erasure channels, where the aggregation of multiple video streams and resource allocation are performed in a distributed manner. Video stream combiners, which are located in different geographical areas and serve as portals for conferees, aggregate incoming streams supplied by local users with other streams aggregated from nearby video stream combiners. A packet-division multiple-access (PDMA)-based error protection scheme is proposed to be performed at each video stream combiner to minimize the maximal expected video distortion among aggregated streams. The proposed error protection scheme for multi-stream aggregation also supports user preference. In order to deliver video streams to end users with different preferred quality, a consensus algorithm is proposed to adaptively perform resource allocation based on user preference. Simulation results show that the proposed multi-stream aggregation and error protection scheme has significant gains over traditional multi-stream error protection schemes for a multi-point video conferencing system.      

移动视频业务QoE自适应测量工具

针对不同无线环境（3G、WiFi）下获取用户体验质量（Quality of Experience,QoE）数据困难和不精确的问题,提出一种基于安卓（Android）移动终端视频业务QoE的自适应测量方法.通过实时测量并评估用户在线视频业务体验质量,提高用户体验质量评价的准确性和实用性.为此开发了能自动测量视频QoE的工具,测量服务质量（Quality of Service,QoS）客观参数,通过效用函数映射到主观QoE（MOS值）.通过对理论QoE评价模型（取自文献）与用户实际反馈相关性研究改进理论模型.结果表明,无线环境下改进的模型测量结果更接近用户实际反馈,可以更好地评价QoE.     

Enabling broadcast of user-generated live video without servers

We are witnessing the unprecedented popularity of User-Generated-Content (UGC) on the Internet. While YouTube hosts pre-recorded video clips, in near future, we expect to see the emergence of User-Generated Live Video, for which any user can create its own temporary live video channel from a webcam or a hand-held wireless device. Hosting a large number of UG live channels on commercial servers can be very expensive. Server-based solutions also involve various economic, copyright and content control issues between users and the companies hosting their content. In this paper, leveraging on the recent success of P2P video streaming, we study the strategies for end users to directly broadcast their own live channels to a large number of audiences without resorting to any server support. The key challenge is that end users are normally bandwidth constrained and can barely send out one complete video stream to the rest of the world. Existing P2P streaming solutions cannot maintain a high level of user Quality-of-Experience (QoE) with such a highly constrained video source. We propose a novel Layered P2P Streaming (LPS) architecture, to address this challenge. LPS introduces playback delay differentiation and constructs virtual servers out of peers to boost end users’ capability of driving large-scale video streaming. Through detailed packet-level simulations and PlanetLab experiments, we show that LPS enables a source with upload bandwidth slightly higher than the video streaming rate to stream video to tens of thousands of peers with premium quality of experience.     

一种视频质量监测系统的设计与实现

网络服务提供商希望能从用户的角度了解目前网络所提供的服务质量,而用户也希望获得定量的指标来评价当前网络服务质量。为此,以视频质量监测为研究对象,提出一种面向用户体验质量的网络监测系统。通过实验分析了网络传输过程中QoS参数对视频QoE的影响;提出一种将视频流转化为测试序列的视频丢包测量方法,该方法能低入侵、准确测量视频传输过程中的丢包情况;基于以上的研究成果,通过对MIB库的扩展和对MIB库轮询机制的研究,构建了面向QoE的视频服务监测系统,该监测系统结构简单、可行性强,实验表明可实时对网络中的视频服务质量进行监测。     

Compressed-domain-based no-reference video quality assessment model considering fast motion and scene change

Due to the variability of wireless channel state, video quality monitoring became very important for guaranteeing users’ Quality of Experience (QoE). QoE presents the overall perceptual quality of service from the subjective users’ perspective. However, because of diverse characteristics of video content, Human Visual System (HVS) cannot give the same attention to whole scene simultaneously when facing video sequence. In this paper, we proposed a video quality assessment model by considering the influence of fast motion and scene change. The motion change contribution factor and scene change contribution factor are defined to quantify the characteristics of video content, which is closely related to the users’ QoE. Based on G.1070, our proposed model considers the influential factors of loss nature of video coding, variability of practical network and video features. Also, the proposed model owns low computational complexity due to the compressed domain approach for the estimation of the model parameters. Therefore, the video quality is assessed without fully decoding the video stream. The performance of our proposed model has been compared with five existing models and the results also shown that our model has high prediction accuracy closing to human perception.     

面向视频会议的运动估计快速算法

针对H.264/AVC视频压缩算法中运动估计模块计算复杂度高的问题,提出了一种基于积分图像的运动估计快速算法.算法通过对单元块进行预计算求和,在块匹配运算时使用单元块级绝对差值.该算法能够在保证视频编码质量的前提下,大幅度提高运动估计模块的计算效率.算法适用于视频会议等要求视频实时压缩的应用.     

Perceptual VVC quantization refinement with ensemble learning

Compressing videos while maintaining an acceptable level of Quality of Experience (QoE) is indispensable. To this aim, a feasible method is to further increase the Quantization Parameter (QP) of video stream to eliminate visual redundancy, simultaneously utilizing perceptual characteristics of Human Visual System (HVS) to impose a threshold constraint on the maximum QP. In this paper, we employ Just Noticeable Distortion (JND) to characterize the aforementioned threshold constraint, thereby avoiding perceptual loss during QP refinement process. We propose an effective JND-based algorithm for QP optimization, in which a video saliency detection is introduced to extract regions of interest, a refinement model based on a lightweight network is designed to predict QP value and an ensemble learning method to improve generalization performance. Theoretical analysis and experimental results demonstrate that the proposed algorithm has been successfully applied to Versatile Video Coding (VVC) to achieve significant bitrate reduction without sacrificing perceived quality.     

Adaptation logic for HTTP dynamic adaptive streaming using geo-predictive crowdsourcing for mobile users

The increasing demand for video streaming services with a high Quality of Experience (QoE) has prompted considerable research on client-side adaptation logic approaches. However, most algorithms use the client’s previous download experience and do not use a crowd knowledge database generated by users of a professional service. We propose a new crowd algorithm that maximizes the QoE. We evaluate our algorithm against state-of-the-art algorithms on large, real-life, crowdsourcing datasets. There are six datasets, each of which contains samples of a single operator (T-Mobile, AT&T or Verizon) from a single road (I100 or I405). All measurements were from Android cellphones. The datasets were provided by WeFi LTD and are public for academic users. Our new algorithm outperforms all other methods in terms of QoE (eMOS).     

Efficient content-based resynchronization approach for wireless video

Recent advances in technology have caused a significant growth in wireless communications, which have resulted in a strong demand for reliable transmission of video data. The challenge of robust video transmission is to protect the compressed data against hostile channel conditions while bringing little impact on bandwidth efficiency. In this paper, using results from a simplified macroblock-based segmentation algorithm, we propose a framework called content-based resynchronization for the effective positioning of resynchronization markers such that the image quality of foreground can be improved at the expense of sacrificing unimportant background. We do this because, in applications such as video telephony and video conferencing, foreground is typically the most important image region for viewers. Experimental results demonstrate that this scheme significantly improve the perceptual quality of video sequences for robust video transmission.     

Content-adaptive wireless streaming of instructional videos

Video streaming over wireless networks is becoming increasingly important for a variety of applications. To accommodate the dynamic change of wireless network bandwidths, Quality of Service (QoS) scalable video streams need to be provided. This paper presents a system of content-adaptive streaming of instructional (lecture) videos over wireless networks for E-learning applications. We first provide a real-time content analysis method to detect and extract content regions from instructional videos, then apply a “leaking-video-buffer” model to adjust QoS of video streams dynamically based on video content. In content-adaptive video streaming, an adaptive feedback control scheme is also developed to transmit properly compressed video streams to video clients not only based on network bandwidth, but also based on video content and the preferences of users. Finally, we demonstrate the scalability and content adaptiveness of the proposed video streaming system with experimental results on several instructional videos.