Blind video quality assessment based on multilevel video perception

Blind video quality assessment (VQA) metrics predict the quality of videos without the presence of reference videos. This paper proposes a new blind VQA model based on multilevel video perception, abbreviated as MVP. The model fuses three levels of video features occurring in natural video scenes to predict video quality: natural video statistics (NVS) features, global motion features and motion temporal correlation features. They represent video scene characteristics, video motion types, and video temporal correlation variations. In the process of motion feature extraction, motion compensation filtering video enhancement is adopted to highlight the motion characteristics of videos so as to improve the perceptual correlations of the video features. The experimental results on the LIVE and CSIQ video databases show that the predicted video scores of the new model are highly correlated with human perception and have low root mean square errors. MVP obviously outperforms state-of-art blind VQA metrics, and particularly demonstrates competitive performance even compared against top-performing full reference VQA metrics.     

Evaluation of objective quality metrics for multidimensional video scalability

Multidimensional video scalability refers to the possibility that a video sequence can be adapted according to given conditions of video consumption by adjusting one or more of its features such as frame size, frame rate, and spatial quality. An important issue in implementing an adaptive video distribution scheme using scalability is how to maximize the quality of experience for the delivered contents, which raises a more fundamental issue, that is, how to estimate perceived quality of scalable video contents. This paper evaluates existing state-of-the-art objective quality metrics, including both generic image/video metrics and ones particularly developed for scalable videos, on the problem of quality assessment of multidimensional video scalability. It is shown that, on the whole, some recently developed metrics targeting scalability perform best. The results are thoroughly discussed in relation to the nature of the problem in comparison to what has been reported in existing studies for other problems.     

基于四元数奇异值分解的视频质量评价方法

准确的客观视频质量评价方法对于视频应用发展是至关重要的.近年来,图像质量评价方法已经比较成熟,而视频质量评价方法与图像质量评价方法在性能上的差距仍然较大.本文提出一种基于四元数奇异值分解的客观视频质量评价方法,该方法将像素的亮度、色度、边缘能量和残差能量作为四元数的四个部分,并用熵作为视觉感兴趣系数对块加权.在视频质量...     

A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms

Measurement of visual quality is of fundamental importance for numerous image and video processing applications, where the goal of quality assessment (QA) algorithms is to automatically assess the quality of images or videos in agreement with human quality judgments. Over the years, many researchers have taken different approaches to the problem and have contributed significant research in this area and claim to have made progress in their respective domains. It is important to evaluate the performance of these algorithms in a comparative setting and analyze the strengths and weaknesses of these methods. In this paper, we present results of an extensive subjective quality assessment study in which a total of 779 distorted images were evaluated by about two dozen human subjects. The “ground truth” image quality data obtained from about 25 000 individual human quality judgments is used to evaluate the performance of several prominent full-reference image quality assessment algorithms. To the best of our knowledge, apart from video quality studies conducted by the Video Quality Experts Group, the study presented in this paper is the largest subjective image quality study in the literature in terms of number of images, distortion types, and number of human judgments per image. Moreover, we have made the data from the study freely available to the research community . This would allow other researchers to easily report comparative results in the future.     

Blind omnidirectional image quality assessment with representative features and viewport oriented statistical features

With the development of information technologies, various types of streaming images are generated, such as videos, graphics, Virtual Reality (VR)/omnidirectional images (OIs), etc. Among them, the OIs usually have a broader view and a higher resolution, which provides human an immersive visual experience in a head-mounted display. However, the current image quality assessment works cannot achieve good performance without considering representative human visual features and visual viewing characteristics of OIs, which limited OIs’ further development. Motivated by the above problem, this work proposes a blind omnidirectional image quality assessment (BOIQA) model based on representative features and viewport oriented statistical features. Specifically, we apply the local binary pattern operator to encoder the cross-channel color information, and apply the weighted LBP to extract the structural features. Then the local natural scene statistics (NSS) features are extracted by using the viewport sampling to boost the performance. Finally, we apply support vector regression to predict the OIs’ quality score, and experimental results on CVIQD2018 and OIQA2018 Databases prove that the proposed model achieves better performance than state-of-the-art OIQA models.     

注意力模型指导下的视频质量评价方法

在视频质量评价方法中,常常需要对人眼视觉系统做出合理的假设,其中注意力模型就是一个很重要的因素。提出了一种在注意力模型指导下的视频质量评价方法,在图像帧的质量评价中加入了显著性区域信息,使之更能符合人眼视觉特性,并兼顾了视频中的运动信息,在一定程度上提高了客观质量评价方法的性能。     

No-reference quality assessment of dynamic sports videos based on a spatiotemporal motion model

This paper proposes an approach to improve the performance of no-reference video quality assessment for sports videos with dynamic motion scenes using an efficient spatiotemporal model. In the proposed method, we divide the video sequences into video blocks and apply a 3D shearlet transform that can efficiently extract primary spatiotemporal features to capture dynamic natural motion scene statistics from the incoming video blocks. The concatenation of a deep residual bidirectional gated recurrent neural network and logistic regression is used to learn the spatiotemporal correlation more robustly and predict the perceptual quality score. In addition, conditional video block-wise constraints are incorporated into the objective function to improve quality estimation performance for the entire video. The experimental results show that the proposed method extracts spatiotemporal motion information more effectively and predicts the video quality with higher accuracy than the conventional no-reference video quality assessment methods.     

No-reference pixel based video quality assessment for HEVC decoded video

This paper proposes a No-Reference (NR) Video Quality Assessment (VQA) method for videos subject to the distortion given by the High Efficiency Video Coding (HEVC) scheme. The assessment is performed without access to the bitstream. The proposed analysis is based on the transform coefficients estimated from the decoded video pixels, which is used to estimate the level of quantization. The information from this analysis is exploited to assess the video quality. HEVC transform coefficients are modeled with a joint-Cauchy probability density function in the proposed method. To generate VQA features the quantization step used in the Intra coding is estimated. We map the obtained HEVC features using an Elastic Net to predict subjective video quality scores, Mean Opinion Scores (MOS). The performance is verified on a dataset consisting of HEVC coded 4 K UHD (resolution equal to 3840 × 2160) video sequences at different bitrates and spanning a wide range of content. The results show that the quality scores computed by the proposed method are highly correlated with the mean subjective assessments.     

Ambiguity of objective image quality metrics: A new methodology for performance evaluation

Objective image quality metrics try to estimate the perceptual quality of the given image by considering the characteristics of the human visual system. However, it is possible that the metrics produce different quality scores even for two images that are perceptually indistinguishable by human viewers, which have not been considered in the existing studies related to objective quality assessment. In this paper, we address the issue of ambiguity of objective image quality assessment. We propose an approach to obtain an ambiguity interval of an objective metric, within which the quality score difference is not perceptually significant. In particular, we use the visual difference predictor, which can consider viewing conditions that are important for visual quality perception. In order to demonstrate the usefulness of the proposed approach, we conduct experiments with 33 state-of-the-art image quality metrics in the viewpoint of their accuracy and ambiguity for three image quality databases. The results show that the ambiguity intervals can be applied as an additional figure of merit when conventional performance measurement does not determine superiority between the metrics. The effect of the viewing distance on the ambiguity interval is also shown.     

基于运动和视差信息的立体视频质量客观评价

在研究人类 立体视觉特性及现有立体图像/视频质量评价算法的基础上,提出了一种基于运动信息和视 差信息的立 体视频质量的客观评价方法。方法包括视频质量评价(VQA)和视频立体感评价(VSSA)两个指 标,其中VQA的估计基于梯度的结构相似度(GSSIM) 算法,并充分考虑了帧内的亮度信息和结构信息、帧间运动信息以及人眼的感知特性对视频 质量的影响, 特别是根据人类的视觉特性,对左右视点的质量赋予了不同的权重;VSSA的估计 是通过计算参考 视频的绝对差值图和降质视频的绝对差值图之间的峰值信噪比(PSNR)而得到。实验结果表明,本文方法对基于H.264 编码的失真视频的评价结果与主观测试有较高的一致性,很好地体现人眼的视觉特性。     

A brief survey on adaptive video streaming quality assessment

Quality of experience (QoE) assessment for adaptive video streaming plays a significant role in advanced network management systems. It is especially challenging in case of dynamic adaptive streaming schemes over HTTP (DASH) which has increasingly complex characteristics including additional playback issues. In this paper, we provide a brief overview of adaptive video streaming quality assessment. Upon our review of related works, we analyze and compare different variations of objective QoE assessment models with or without using machine learning techniques for adaptive video streaming. Through the performance analysis, we observe that hybrid models perform better than both quality-of-service (QoS) driven QoE approaches and signal fidelity measurement. Moreover, the machine learning-based model slightly outperforms the model without using machine learning for the same setting. In addition, we find that existing video streaming QoE assessment models still have limited performance, which makes it difficult to be applied in practical communication systems. Therefore, based on the success of deep learned feature representations for traditional video quality prediction, we also apply the off-the-shelf deep convolutional neural network (DCNN) to evaluate the perceptual quality of streaming videos, where the spatio-temporal properties of streaming videos are taken into consideration. Experiments demonstrate its superiority, which sheds light on the future development of specifically designed deep learning frameworks for adaptive video streaming quality assessment. We believe this survey can serve as a guideline for QoE assessment of adaptive video streaming.     

视频用户体验理论与实践

随着视频服务逐渐成为人们获取信息的主要途径之一,消费者对观看体验的要求不断提高,视频用户体验质量已经成为视频服务的主要竞争因素.首先对用户体验质量理论进行了系统的阐述,指出了用户体验质量与服务质量之间的差别和联系,同时给出了用户体验评价方法的主要步骤和视频质量评价的具体方法.进一步地,对我国现阶段视频用户体验评价研究及标准化的进展进行详细介绍,概括了目前我国视频服务的用户体验现状并给出了相关的改进方向意见.     

Image information and visual quality.

Measurement of visual quality is of fundamental importance to numerous image and video processing applications. The goal of quality assessment (QA) research is to design algorithms that can automatically assess the quality of images or videos in a perceptually consistent manner. Image QA algorithms generally interpret image quality as fidelity or similarity with a "reference" or "perfect" image in some perceptual space. Such "full-reference" QA methods attempt to achieve consistency in quality prediction by modeling salient physiological and psychovisual features of the human visual system (HVS), or by signal fidelity measures. In this paper, we approach the image QA problem as an information fidelity problem. Specifically, we propose to quantify the loss of image information to the distortion process and explore the relationship between image information and visual quality. QA systems are invariably involved with judging the visual quality of "natural" images and videos that are meant for "human consumption." Researchers have developed sophisticated models to capture the statistics of such natural signals. Using these models, we previously presented an information fidelity criterion for image QA that related image quality with the amount of information shared between a reference and a distorted image. In this paper, we propose an image information measure that quantifies the information that is present in the reference image and how much of this reference information can be extracted from the distorted image. Combining these two quantities, we propose a visual information fidelity measure for image QA. We validate the performance of our algorithm with an extensive subjective study involving 779 images and show that our method outperforms recent state-of-the-art image QA algorithms by a sizeable margin in our simulations. The code and the data from the subjective study are available at the LIVE website.     

Stereoscopic video quality assessment based on visual attention and just-noticeable difference models

With the consideration that incorporating visual saliency information appropriately can benefit image quality assessment metrics, this paper proposes an objective stereoscopic video quality assessment (SVQA) metric by incorporating stereoscopic visual attention (SVA) to SVQA metric. Specifically, based upon the multiple visual masking characteristics of HVS, a stereoscopic just-noticeable difference model is proposed to compute the perceptual visibility for stereoscopic video. Next, a novel SVA model is proposed to extract stereoscopic visual saliency information. Then, the quality maps are calculated by the similarity of the original and distorted stereoscopic videos’ perceptual visibility. Finally, the quality score is obtained by incorporating visual saliency information to the pooling of quality maps. To evaluate the proposed SVQA metric, a subjective experiment is conducted. The experimental result shows that the proposed SVQA metric achieves better performance in comparison with the existing SVQA metrics.     

视觉信号质量评价的挑战与新进展

While quality assessment is essential for testing, optimizing, benchmarking, monitoring, and inspecting related systems and services, it also plays an essential role in the design of virtually all visual signal processing and communication algorithms, as well as various related decision-making processes. In this pa-per, we first provide an overview of recently derived quality assessment approaches for traditional visual signals (i.e., 2D im-ages/videos), with highlights for new trends (such as machine learning approaches). On the other hand, with the ongoing development of devices and multimedia services, newly emerged visual signals (e.g., mobile/3D videos) are becoming more and more popular. This work focuses on recent progresses of quality metrics, which have been reviewed for the newly emerged forms of visual signals, which include scalable and mobile videos, High Dynamic Range (HDR) images, image segmentation results, 3D images/videos, and retargeted images.     

Objective quality assessment of synthesized images by local variation measurement

Due to the rapid development of free-viewpoint television (FVT), Depth-Image-Based Rendering (DIBR) technology has been widely used to synthesize images of virtual view-points. However, the types of distortions in the synthesized images are different from those of natural images, such as discontinuity, flickering, stretching, etc. To measure the distortion occurred in the synthesized images, we propose a full-reference (FR) quality assessment method by local variation measurement consisting of three-modules. Firstly, since the distortion in the synthesized image mainly occurs in the region with high-frequency structure information, the Neutrosophic domain is employed to evaluate the degradation of local image structure. Secondly, by considering that the texture of the synthesized image might be damaged due to the warping of 2D image or the loss of information in the occlusion region, we evaluate the visual quality of local texture by using the features obtained from frequency domain. Thirdly, to measure the stretching distortion which is unique in the synthesized image, the visual quality of extracted stretching area is measured by entropy. Finally, a pooling operation is used to combine the quality scores of the three modules to obtain the final predicted quality score. Experimental results show that the performance of the proposed algorithm is competitive with state-of-the-art FR and no-reference image quality assessment metrics.     

An objective and subjective quality assessment study of passive gaming video streaming

Passive gaming video‐streaming applications have recently gained much attention as evident with the rising popularity of many Over The Top (OTT) providers such as Twitch.tv and YouTube Gaming. For the continued success of such services, it is imperative that the user Quality of Experience (QoE) remains high, which is usually assessed using subjective and objective video quality assessment methods. Recent years have seen tremendous advancement in the field of objective video quality assessment (VQA) metrics, with the development of models that can predict the quality of the videos streamed over the Internet. A study on the performance of objective VQA on gaming videos, which are artificial and synthetic and have different streaming requirements than traditionally streamed videos, is still missing. Towards this end, we present in this paper an objective and subjective quality assessment study on gaming videos considering passive streaming applications. Subjective ratings are obtained for 90 stimuli generated by encoding six different video games in multiple resolution‐bitrate pairs. Objective quality performance evaluation considering eight widely used VQA metrics is performed using the subjective test results and on a data set of 24 reference videos and 576 compressed sequences obtained by encoding them in 24 resolution‐bitrate pairs. Our results indicate that Video Multimethod Assessment Fusion (VMAF) predicts subjective video quality ratings the best, while Naturalness Image Quality Evaluator (NIQE) turns out to be a promising alternative as a no‐reference metric in some scenarios.     

融合空时感知特性的无参考视频质量评估算法

本文通过简化视频质量评估中人眼感知模型的复杂性,提出了一种新的无参考视频质量评估模型.首先通过分别抽取视频的空间域和时间域特征,然后按照视频局部块、视频帧、视频段等从细到粗的不同粒度,模拟人眼感知特性进行多重加权汇聚,最终得到整段视频的特征向量描述.本方法以支持向量回归器为评估模型训练工具,通过有监督的视频样本库训练,以无参考方式完成未知视频的质量评估.实验结果表明,该评估算法的性能不但要优于当前已知最经典的无参考评估算法Video BLLINDS,而且与部分参考评估算法相当.     

基于块编码视频的无参考质量评估

该文提出了一种适用于基于块编码视频的无参考质量评估方法。首先结合人类视觉的亮度掩盖和对比度掩盖特性提出了一个符合主观视觉感知的方块效应测度,然后根据滤波对方块效应的影响,给出了一种适合于使用不同压缩和处理算法的基于块编码重构视频的质量评估方法。实验表明该质量评估测度与主观质量评估有较好的一致性。