Evaluation of objective quality metrics for multidimensional video scalability

Multidimensional video scalability refers to the possibility that a video sequence can be adapted according to given conditions of video consumption by adjusting one or more of its features such as frame size, frame rate, and spatial quality. An important issue in implementing an adaptive video distribution scheme using scalability is how to maximize the quality of experience for the delivered contents, which raises a more fundamental issue, that is, how to estimate perceived quality of scalable video contents. This paper evaluates existing state-of-the-art objective quality metrics, including both generic image/video metrics and ones particularly developed for scalable videos, on the problem of quality assessment of multidimensional video scalability. It is shown that, on the whole, some recently developed metrics targeting scalability perform best. The results are thoroughly discussed in relation to the nature of the problem in comparison to what has been reported in existing studies for other problems.     

A brief survey on adaptive video streaming quality assessment

Quality of experience (QoE) assessment for adaptive video streaming plays a significant role in advanced network management systems. It is especially challenging in case of dynamic adaptive streaming schemes over HTTP (DASH) which has increasingly complex characteristics including additional playback issues. In this paper, we provide a brief overview of adaptive video streaming quality assessment. Upon our review of related works, we analyze and compare different variations of objective QoE assessment models with or without using machine learning techniques for adaptive video streaming. Through the performance analysis, we observe that hybrid models perform better than both quality-of-service (QoS) driven QoE approaches and signal fidelity measurement. Moreover, the machine learning-based model slightly outperforms the model without using machine learning for the same setting. In addition, we find that existing video streaming QoE assessment models still have limited performance, which makes it difficult to be applied in practical communication systems. Therefore, based on the success of deep learned feature representations for traditional video quality prediction, we also apply the off-the-shelf deep convolutional neural network (DCNN) to evaluate the perceptual quality of streaming videos, where the spatio-temporal properties of streaming videos are taken into consideration. Experiments demonstrate its superiority, which sheds light on the future development of specifically designed deep learning frameworks for adaptive video streaming quality assessment. We believe this survey can serve as a guideline for QoE assessment of adaptive video streaming.     

On the performance of objective quality metrics for lightfields

Lightfield (LF) technology has attained significant attention in recent years due to its capability to capture much richer textural and geometric information in the scene compared to the classical 2D representation. The resampling and compression operations on LFs often lead to visual quality degradation, thus, sophisticated visual quality assessment methods play a crucial role to ensure a pleasant viewing experience. To this end, it is necessary to examine the performance of quality assessment methods for LF contents. The paper provides a comprehensive study on the reliability of various objective algorithms for LF quality prediction. Three subjectively-annotated LF data sets were selected and an extensive quality estimation analysis has been conducted using several objective quality assessment methods. In total, 250 LFs (more than 48000 perspective images) were evaluated. The results were compared against human opinion scores using various correlation indices and their statistical significance. Next, a decision-making strategy was adopted to choose the most reliable quality metrics for evaluation of LFs and finally, a metric fusion framework was proposed to further improve the quality prediction accuracy. To best of our knowledge, the benchmark and the analytical methodologies used in this paper are the most comprehensive study on the objective quality assessment methods for LF application.     

MDVQM: A novel multidimensional no-reference video quality metric for video transcoding

In this paper, we study the impact of quantization, frame dropping and spatial down-sampling on the perceived quality of compressed video streams. Based on the analysis of quality ratings obtained from extensive subjective tests, we propose a no-reference metric (named MDVQM) for video quality estimation in the presence of both spatial and temporal quality impairments. The proposed metric is based on the per-pixel bitrate of the encoded stream and selected spatial and temporal activity measures extracted from the video content. All the values required to compute the proposed video quality metric can be obtained without using the original reference video which makes the metric for instance useful for making transcoding decisions in a wireless video transmission scenario. Different from comparable metrics in the literature, we have also considered the case when both frame rate and frame size are changed simultaneously. The validation results show that the proposed metric provides more accurate estimation of the video quality than the state of the art metrics.     

A low-complexity psychometric curve-fitting approach for the objective quality assessment of streamed game videos

The increasing popularity of video gaming competitions, the so called eSports, has contributed to the rise of a new type of end-user: the passive game video streaming (GVS) user. This user acts as a passive spectator of the gameplay rather than actively interacting with the content. This content, which is streamed over the Internet, can suffer from disturbing network and encoding impairments. Therefore, assessing the user’s perceived quality, i.e the Quality of Experience (QoE), in real-time becomes fundamental. For the case of natural video content, several approaches already exist that tackle the client-side real-time QoE evaluation. The intrinsically different expectations of the passive GVS user, however, call for new real-time quality models for these streaming services. Therefore, this paper presents a real-time Reduced-Reference (RR) quality assessment framework based on a low-complexity psychometric curve-fitting approach. The proposed solution selects the most relevant, low-complexity objective feature. Afterwards, the relationship between this feature and the ground-truth quality is modelled based on the psychometric perception of the human visual system (HVS). This approach is validated on a publicly available dataset of streamed game videos and is benchmarked against both subjective scores and objective models. As a side contribution, a thorough accuracy analysis of existing Objective Video Quality Metrics (OVQMs) applied to passive GVS is provided. Furthermore, this analysis has led to interesting insights on the accuracy of low-complexity client-based metrics as well as to the creation of a new Full-Reference (FR) objective metric for GVS, i.e. the Game Video Streaming Quality Metric (GVSQM).     

Hierarchical content importance-based video quality assessment for HEVC encoded videos transmitted over LTE networks

To improve the accuracy of assessment, many previous works take into account the video content. However, these previous works just only consider the video content, but do not consider the location and importance of the degraded content. Thus, this paper takes into account not only the video content, but also the location and importance of the degraded content, and proposes a hierarchical content importance-based video quality assessment. Firstly, we propose to use the hierarchical content importance-based frame degradation rate (HFDR) metric to quantify the importance of degraded content hierarchically. Secondly, we propose to use the intra random access point (IRAP) loss rate (ILR) metric to quantify the impact of IRAP. Finally, the proposed HFDR metric and ILR metric are subsequently used to develop an objective video quality assessment model. The experimental results show that the predicted mean opinion score (MOS) of the proposed method highly correlates with the actual MOS.     

Visual-PSNR measure of image quality

Objective assessment of image quality is important in numerous image and video processing applications. Many objective measures of image quality have been developed for this purpose, of which peak signal-to-noise ratio PSNR is one of the simplest and commonly used. However, it sometimes does not match well with objective mean opinion scores (MOS). This paper presents a novel objective full-reference measure of image quality (VPSNR), which is a modified PSNR measure. It will be shown that VPSNR takes into account some features of the human visual system (HVS). The performance of VPSNR is validated using a data set of four image databases, and in this article it is shown that for images compressed by block-based compression algorithms (like JPEG) the proposed measure in the pixel domain matches well with MOS.     

Calculation of average coding efficiency based on subjective quality scores

The Bjøntegaard model is widely used to calculate the coding efficiency between different codecs. However, this model might not be an accurate predictor of the true coding efficiency as it relies on PSNR measurements. Therefore, in this paper, we propose a model to calculate the average coding efficiency based on subjective quality scores, i.e., mean opinion scores (MOS). We call this approach Subjective Comparison of ENcoders based on fItted Curves (SCENIC). To consider the intrinsic nature of bounded rating scales, a logistic function is used to fit the rate–distortion (R–D) values. The average MOS and bit rate differences are computed between the fitted R–D curves. The statistical property of subjective scores is considered to estimate corresponding confidence intervals on the calculated average MOS and bit rate differences. The proposed model is expected to report more realistic coding efficiency as PSNR is not always correlated with perceived visual quality.     

Q2ABR: QoE‐aware adaptive video bit rate solution

In this paper, we propose a new adaptive bit rate (ABR) streaming method. This method is based on estimating and monitoring users' video streaming experience, their quality of experience (QoE). This ensures a good user QoE and optimises bandwidth utilisation by monitoring video buffer fill rate to ensure minimal data traffic. First, we achieve a QoE evaluation model based on network bandwidth, video segment representation, and dropped video frame rate parameters. Second, following our QoE evaluation model, we formulate an ABR method using the reinforcement learning (RL) paradigm to select video representations and using a breakpoint detection mechanism to monitor end‐user QoE variation. The proposed ABR method is called “QoE‐aware adaptive bit rate (Q2ABR)” and is composed of three individual modules, one for QoE estimation using machine learning methods, one for QoE variation monitoring using the breakpoint detection mechanism, and one for video representation selection using reinforcement learning. The design objective of Q2ABR is to ensure the overall QoE of these users while maintaining a minimum variation in the standard deviation of the users' QoE values. Third, the performance of the Q2ABR method is evaluated and compared with several existing ABR approaches in the literature using real traces that we collect on different transport scenarios (such as bus and train, among others). Since this method considers the user's perception of video quality as a regulator for optimising the overall video distribution network, good results are ensured in terms of the user's experience and buffer fill rate.