A novel adaptive logic for dynamic adaptive streaming over HTTP

In this paper, we propose an estimation method that estimates the throughput of upcoming video segments based on variations in the network throughput observed during the download of previous video segments. Then, we propose a rate-adaptive algorithm for Hypertext Transfer Protocol (HTTP) streaming. The proposed algorithm selects the quality of the video based on the estimated throughput and playback buffer occupancy. The proposed method selects high-quality video segments, while minimizing video quality changes and the risk of playback interruption, improving user’s experience. We evaluate the algorithm for single- and multi-user environments and demonstrate that it performs remarkably well under varying network conditions. Furthermore, we determine that it efficiently utilizes network resources to achieve a high video rate; competing HTTP clients achieve equitable video rates. We also confirm that variations in the playback buffer size and segment duration do not affect the performance of the proposed algorithm.     

Quality perception when streaming video on tablet devices

The proposed work aims at analyzing the quality perceived by the user when streaming video on tablet devices. The contributions of this paper are: (i) to analyze the results of subjective quality assessments to determine which Quality of Service (QoS) parameters mainly affect the users’ Quality of Experience (QoE) in video streaming over tablet devices; (ii) to define a parametric quality model useful in system control and optimization for the considered scenarios; (iii) to compare the performance of the proposed model with subjective quality results obtained in alternative state-of-the-art studies and investigate whether other models could be applied to our case and vice versa.     

Video transcoding for adaptive bitrate streaming over edge-cloud continuum

Video transcoding is to create multiple representations of a video for content adaptation. It is deemed as a core technique in Adaptive BitRate (ABR) streaming. How to manage video transcoding affects the performance of ABR streaming in various aspects, including operational cost, streaming delays, Quality of Experience (QoE), etc. Therefore, the problems of implementing video transcoding in ABR streaming must be systematically studied to improve the overall performance of the streaming services. These problems become more worthy of investigation with the emergence of the edge-cloud continuum, which makes the resource allocation for video transcoding more complicated. To this end, this paper provides an investigation of the main technical problems related to video transcoding in ABR streaming, including designing a rate profile for video transcoding, providing resources for video transcoding in clouds, and caching multi-bitrate video contents in networks, etc. We analyze these problems from the perspective of resource allocation in the edge-cloud continuum and cast them into resource and Quality of Service (QoS) optimization problems. The goal is to minimize resource consumption while guaranteeing the QoS for ABR streaming. We also discuss some promising research directions for the ABR streaming services.     

QoE optimization for HTTP adaptive streaming: Performance evaluation of MEC-assisted and client-based methods

Seamless streaming of high quality video under unstable network condition is a big challenge. HTTP adaptive streaming (HAS) provides a solution that adapts the video quality according to the network conditions. Traditionally, HAS algorithm runs at the client side while the clients are unaware of bottlenecks in the radio channel and competing clients. The traditional adaptation strategies do not explicitly coordinate between the clients, servers, and cellular networks. The lack of coordination has been shown to lead to suboptimal user experience. As a response, multi-access edge computing (MEC)-assisted adaptation techniques emerged to take advantage of computing and content storage capabilities in mobile networks. In this study, we investigate the performance of both MEC-assisted and client-side adaptation methods in a multi-client cellular environment. Evaluation and comparison are performed in terms of not only the video rate and dynamics of the playback buffer but also the fairness and bandwidth utilization. We conduct extensive experiments to evaluate the algorithms under varying client, server, dataset, and network settings. Results demonstrate that the MEC-assisted algorithms improve fairness and bandwidth utilization compared to the client-based algorithms for most settings. They also reveal that the buffer-based algorithms achieve significant quality of experience; however, these algorithms perform poorly compared with throughput-based algorithms in protecting the playback buffer under rapidly varying bandwidth fluctuations. In addition, we observe that the preparation of the representation sets affects the performance of the algorithms, as does the playback buffer size and segment duration. Finally, we provide suggestions based on the behaviors of the algorithms in a multi-client environment.     

MDVQM: A novel multidimensional no-reference video quality metric for video transcoding

In this paper, we study the impact of quantization, frame dropping and spatial down-sampling on the perceived quality of compressed video streams. Based on the analysis of quality ratings obtained from extensive subjective tests, we propose a no-reference metric (named MDVQM) for video quality estimation in the presence of both spatial and temporal quality impairments. The proposed metric is based on the per-pixel bitrate of the encoded stream and selected spatial and temporal activity measures extracted from the video content. All the values required to compute the proposed video quality metric can be obtained without using the original reference video which makes the metric for instance useful for making transcoding decisions in a wireless video transmission scenario. Different from comparable metrics in the literature, we have also considered the case when both frame rate and frame size are changed simultaneously. The validation results show that the proposed metric provides more accurate estimation of the video quality than the state of the art metrics.     

Eliminating bandwidth estimation from adaptive video streaming in wireless networks

Dynamic Adaptive Streaming over HTTP (DASH) is the state-of-the-art technology for video streaming and has been widely deployed in both wired and wireless environments. However, mobile DASH users often suffer from video quality oscillation and even video freeze in wireless environments, which results in poor user experience. This is mainly because most quality adaptation algorithms in DASH rely highly on bandwidth estimation to adjust the video quality while wireless network bandwidth is unstable in nature and changes frequently according to wireless channel contention and condition. To provide stable performance, even during severe bandwidth fluctuation, this paper proposes the Wireless Quality Adaptation (WQUAD) algorithm, which eliminates bandwidth estimation from quality adaptation. Thanks to the Scalable Video Codec (SVC), the proposed scheme always prioritizes to lower layers over higher ones as long as the play-out buffer is not completely filled by the lower layers. As a result, the client always fills the buffer with the base layers first and then the upper enhancement layers sequentially. This horizontal adaptation is straightforward and does not require any bandwidth estimation. Through NS-2 simulations, we show that WQUAD achieves (i) stable performance, keeping the video quality level with respect to the long-term network bandwidth, (ii) effective video freeze prevention, and (iii) high video quality on average.     

No-reference pixel based video quality assessment for HEVC decoded video

This paper proposes a No-Reference (NR) Video Quality Assessment (VQA) method for videos subject to the distortion given by the High Efficiency Video Coding (HEVC) scheme. The assessment is performed without access to the bitstream. The proposed analysis is based on the transform coefficients estimated from the decoded video pixels, which is used to estimate the level of quantization. The information from this analysis is exploited to assess the video quality. HEVC transform coefficients are modeled with a joint-Cauchy probability density function in the proposed method. To generate VQA features the quantization step used in the Intra coding is estimated. We map the obtained HEVC features using an Elastic Net to predict subjective video quality scores, Mean Opinion Scores (MOS). The performance is verified on a dataset consisting of HEVC coded 4 K UHD (resolution equal to 3840 × 2160) video sequences at different bitrates and spanning a wide range of content. The results show that the quality scores computed by the proposed method are highly correlated with the mean subjective assessments.     

一个基于速率控制的Internet视频流服务方案

由于视频流服务对于网络服务质量有着较高的要求,而现有的Internet所提供的是尽力而为的服务,无法保证数据的实时传输。该文设计了一个用于Internet上视频流的端到端传输方案.整个方案设计的目的是在网络本身缺乏服务质量保证的条件下尽可能达到最好的视频传输质量。根据可用带宽估计和网络信息反馈,系统对发送速率进行调整,并提供两种视频流服务:存储视频和实时视频。仿真结果表明方案的性能良好,能满足Internet视频流的需求。     

Discrete quality assessment in IPTV content distribution networks

Maintaining the quality of videos in resource-intensive IPTV services is challenging due to the nature of packet-based content distribution networks (CDN). Network impairments are unpredictable and highly detrimental to the quality of video content. Quality of the end user experience (QoE) has become a critical service differentiator. An efficient real-time quality assessment service in distribution networks is the foundation of service quality monitoring and management. The perceptual impact of individual impairments varies significantly and is influenced by complex impact factors. Without differentiating the impact of quality violation events to the user experience, existing assessment methodologies based on network QoS such as packet loss rate cannot provide adequate supports for the IPTV service assessment. A discrete perceptual impact evaluation quality assessment (DEQA) framework is introduced in this paper. The proposed framework enables a real-time, non-intrusive assessment service by efficiently recognising and assessing individual quality violation events in the IPTV distribution network. The discrete perceptual impacts to a media session are aggregated for the overall user level quality evaluation. With its deployment scheme the DEQA framework also facilitates efficient network diagnosis and QoE management. To realise the key assessment function of the framework and investigate the proposed advanced packet inspection mechanism, we also introduce the dedicated evaluation testbed—the LA2 system. A subjective experiment with data analysis is also presented to demonstrate the development of perceptual impact assessment functions using analytical inference, the tools of the LA2 system, subjective user tests and statistical modelling.     

Adaptive video streaming testbed design for performance study and assessment of QoE

Hypertext Transfer Protocol adaptive streaming switches between different video qualities, adapting to the network conditions, and avoids stalling streamed frames over high‐oscillation client's throughput improving the users' quality of experience (QoE). Quality of experience has become the most important parameter to lead the service providers to know about the end‐user feedback. Implementing Hypertext Transfer Protocol adaptive streaming applications to find out QoE in real‐life scenarios of vast networks becomes more challenging and complex task regarding to cost, agile, time, and decisions. In this paper, a virtualized network testbed to virtualize various machines to support implementing experiments of adaptive video streaming has been developed. Within the test study, the metrics which demonstrate performance of QoE are investigated, respectively, including initial delay (ie, startup delay at the beginning of playback a video), frequency switches (ie, number of times the quality is changed), accumulative video time (ie, number and length of stalls), CPU usage, and battery energy consumption. Furthermore, the relation between effective parameters of QoS on the aforementioned metrics for different segment length is investigated. Experimental results show that the proposed virtualized system is agile, easy to install and use, and costs less than real testbeds. Moreover, the subjective and objective performance studies of QoE evaluation in the system have proven that the segment lengths of 6 to 8 seconds were faired and more efficient than others according to the investigated parameters.     

Blind video quality assessment based on multilevel video perception

Blind video quality assessment (VQA) metrics predict the quality of videos without the presence of reference videos. This paper proposes a new blind VQA model based on multilevel video perception, abbreviated as MVP. The model fuses three levels of video features occurring in natural video scenes to predict video quality: natural video statistics (NVS) features, global motion features and motion temporal correlation features. They represent video scene characteristics, video motion types, and video temporal correlation variations. In the process of motion feature extraction, motion compensation filtering video enhancement is adopted to highlight the motion characteristics of videos so as to improve the perceptual correlations of the video features. The experimental results on the LIVE and CSIQ video databases show that the predicted video scores of the new model are highly correlated with human perception and have low root mean square errors. MVP obviously outperforms state-of-art blind VQA metrics, and particularly demonstrates competitive performance even compared against top-performing full reference VQA metrics.     

基于深层特征学习的无参考立体图像质量评价

提出了一种基于深层特征学习的无参考(NR)立体图 像质量评价方 法。与传统人工提取图像特征不同,采用卷积神经网络(CNN)自动提取图像特征,评价过程 分为训练和 测试两阶段。在训练阶段,将图像分块训练CNN网络,利用CNN提取图像块特征,并结合不同 的整合方式 得到图像的全局特征,通过支持向量回归(SVR)建立主观质量与全局特征的回归模型;在测 试阶段,由已训练的CNN网 络和回归模型,得到左右图像和独眼图的质量。最后,根据人眼双目视觉特性融合左图像、 右图像和独眼 图的质量,得到立体图像质量。本文方法在LIVE-I和LIVE-II数据库上的Spearman等级系 数(SROCC)分别达 到了0.94,评价结果准确,与人眼的主 观感受一致。     

Robust image retrieval by cascading a deep quality assessment network

The performance of computer vision algorithms can severely degrade in the presence of a variety of distortions. While image enhancement algorithms have evolved to optimize image quality as measured according to human visual perception, their relevance in maximizing the success of computer vision algorithms operating on the enhanced image has been much less investigated. We consider the problem of image enhancement to combat Gaussian noise and low resolution with respect to the specific application of image retrieval from a dataset. We define the notion of image quality as determined by the success of image retrieval and design a deep convolutional neural network (CNN) to predict this quality. This network is then cascaded with a deep CNN designed for image denoising or super resolution, allowing for optimization of the enhancement CNN to maximize retrieval performance. This framework allows us to couple enhancement to the retrieval problem. We also consider the problem of adapting image features for robust retrieval performance in the presence of distortions. We show through experiments on distorted images of the Oxford and Paris buildings datasets that our algorithms yield improved mean average precision when compared to using enhancement methods that are oblivious to the task of image retrieval. ¹     

On quality of experience of scalable video adaptation

In this paper, we study the quality of experience (QoE) issues in scalable video coding (SVC) for its adaptation in video communications. A QoE assessment database is developed according to SVC scalabilities. Based on the subjective evaluation results, we derive the optimal scalability adaptation track for the individual video and further summarize common scalability adaptation tracks for videos according to their spatial information (SI) and temporal information (TI). Based on the summarized adaptation tracks, we conclude some general guidelines for the effective SVC video adaptation. A rate-QoE model for SVC adaptation is derived accordingly. Experimental results show that the proposed QoE-aware scalability adaptation scheme significantly outperforms the conventional adaptation schemes in terms of QoE. Moreover, the proposed QoE model reflects the rate and QoE relationship in SVC adaptation and thus, provides a useful methodology to estimate video QoE which is important for QoE-aware scalable video streaming.     

Hierarchical content importance-based video quality assessment for HEVC encoded videos transmitted over LTE networks

To improve the accuracy of assessment, many previous works take into account the video content. However, these previous works just only consider the video content, but do not consider the location and importance of the degraded content. Thus, this paper takes into account not only the video content, but also the location and importance of the degraded content, and proposes a hierarchical content importance-based video quality assessment. Firstly, we propose to use the hierarchical content importance-based frame degradation rate (HFDR) metric to quantify the importance of degraded content hierarchically. Secondly, we propose to use the intra random access point (IRAP) loss rate (ILR) metric to quantify the impact of IRAP. Finally, the proposed HFDR metric and ILR metric are subsequently used to develop an objective video quality assessment model. The experimental results show that the predicted mean opinion score (MOS) of the proposed method highly correlates with the actual MOS.     

Link efficiency and quality of experience aware routing protocol to improve video streaming in urban VANETs

In a vehicular ad‐hoc network (VANET), vehicles can play an essential role in monitoring areas of a smart city by transmitting data or multimedia content of environmental circumstances like disasters or road conditions. Multimedia content communication with quality of experience (QoE) guarantees is a challenging undertaking in an environment such as that of a VANET. Indeed, a VANET is characterized by numerous varying conditions, significantly impacting its topology, quality of communication channels, and paths with respect to bandwidth, loss, and delay. This paper introduces a link efficiency and quality of experience aware routing protocol (LEQRV) to improve video streaming provisioning in urban vehicular ad‐hoc networks. LEQRV uses an enhanced greedy forwarding‐based approach to create and maintain stable high quality routes for video streaming delivery. It improves the performance of the quality of experience by increasing the achieved QoE scores and reducing the forwarding end‐to‐end delay and frame loss.     

Evaluation of temporal variation of video quality in packet loss networks

We examine the effect that variations in the temporal quality of videos have on global video quality. We also propose a general framework for constructing temporal video quality assessment (QA) algorithms that seek to assess transient temporal errors, such as packet losses. The proposed framework modifies simple frame-based quality assessment algorithms by incorporating a temporal quality variance factor. We use packet loss from channel errors as a specific study of practical significance. Using the PSNR and the SSIM index as exemplars, we are able to show that the new video QA algorithms are highly responsive to packet loss errors.     

A low-complexity psychometric curve-fitting approach for the objective quality assessment of streamed game videos

The increasing popularity of video gaming competitions, the so called eSports, has contributed to the rise of a new type of end-user: the passive game video streaming (GVS) user. This user acts as a passive spectator of the gameplay rather than actively interacting with the content. This content, which is streamed over the Internet, can suffer from disturbing network and encoding impairments. Therefore, assessing the user’s perceived quality, i.e the Quality of Experience (QoE), in real-time becomes fundamental. For the case of natural video content, several approaches already exist that tackle the client-side real-time QoE evaluation. The intrinsically different expectations of the passive GVS user, however, call for new real-time quality models for these streaming services. Therefore, this paper presents a real-time Reduced-Reference (RR) quality assessment framework based on a low-complexity psychometric curve-fitting approach. The proposed solution selects the most relevant, low-complexity objective feature. Afterwards, the relationship between this feature and the ground-truth quality is modelled based on the psychometric perception of the human visual system (HVS). This approach is validated on a publicly available dataset of streamed game videos and is benchmarked against both subjective scores and objective models. As a side contribution, a thorough accuracy analysis of existing Objective Video Quality Metrics (OVQMs) applied to passive GVS is provided. Furthermore, this analysis has led to interesting insights on the accuracy of low-complexity client-based metrics as well as to the creation of a new Full-Reference (FR) objective metric for GVS, i.e. the Game Video Streaming Quality Metric (GVSQM).     

Evaluation of objective quality metrics for multidimensional video scalability

Multidimensional video scalability refers to the possibility that a video sequence can be adapted according to given conditions of video consumption by adjusting one or more of its features such as frame size, frame rate, and spatial quality. An important issue in implementing an adaptive video distribution scheme using scalability is how to maximize the quality of experience for the delivered contents, which raises a more fundamental issue, that is, how to estimate perceived quality of scalable video contents. This paper evaluates existing state-of-the-art objective quality metrics, including both generic image/video metrics and ones particularly developed for scalable videos, on the problem of quality assessment of multidimensional video scalability. It is shown that, on the whole, some recently developed metrics targeting scalability perform best. The results are thoroughly discussed in relation to the nature of the problem in comparison to what has been reported in existing studies for other problems.     

Evaluating feedback devices for time-continuous mobile multimedia quality assessment

In January 2014, the new ITU-T P.913 recommendation for measuring subjective video, audio and multimedia quality in any environment has been published. This document does not contain any time-continuous subjective method. However, environmental parameter values are changing continuously in a majority of outdoor and also most indoor environments. To be aware of their impact on the perceived quality, a time-continuous quality assessment methodology is necessary. In previous standards, targeting laboratory-based test settings, a desk-mounted slider of substantial size is recommended. Unfortunately, there are many environments where such a device cannot be used.In this paper, new feedback tools for mobile time-continuous rating are presented and analysed. We developed several alternatives to the generally adopted desk-mounted slider as a rating device. In order to compare the tools, we defined a number of performance measures that can be used in further studies. The suitability and efficacy of the rating scheme based on measurable parameters as well as user opinions is compared. One method, the finger count, seems to outperform the others from all points of view. It was been judged to be easy to use with low potential for distractions. Furthermore, it reaches a similar precision level as the slider, while requiring lower user reaction and scoring times. Low reaction times are particularly important for time-continuous quality assessment, where the reliability of a mapping between impairments and user ratings plays an essential role.