变权无参考视频质量评价模型研究

传输信道失真是导致视频图像质量损失的重要原因,而不需要任何附加传输信息的无参考客观评价是监测视频传输质量受损的主要方法.为了提高无参考客观评价模型的准确性和效率,提出模拟人类视觉特性的变权评价模型.变权评价模型综合考虑视频的空域和时域两类质量指标,引入运动强度来量化视频内容中的运动变化程度,根据统计学习的非线性回归法建立变权控制函数;通过变权控制函数动态调整清晰度和平滑度权值来模仿人类视觉特性.实验结果表明该变权评价模型与主观评价符合度高,优于现有评价方案.     

Quality-distinguishing and patch-comparing no-reference image quality assessment

No-reference image quality assessment (NR-IQA) based on deep learning attracts a great research attention recently. However, its performance in terms of accuracy and efficiency is still under exploring. To address these issues, in this paper, we propose a quality-distinguishing and patch-comparing NR-IQA approach based on convolutional neural network (QDPC-CNN). We improve the prediction accuracy by two proposed mechanisms: quality-distinguishing adaption and patch-comparing regression. The former trains multiple models from different subsets of a dataset and adaptively selects one for predicting quality score of a test image according to its quality level, and the latter generates patch pairs for regression under different combination strategies to make better use of reference images in network training and enlarge training data at the same time. We further improve the efficiency of network training by a new patch sampling way based on the visual importance of each patch. We conduct extensive experiments on several public databases and compare our proposed QDPC-CNN with existing state-of-the-art methods. The experimental results demonstrate that our proposed method outperforms the others both in terms of accuracy and efficiency.

     

无参考图像质量评价研究进展

图像质量评价一直是图像处理和计算机视觉领域的一个基础问题,图像质量评价模型也广泛应用于图像/视频编码、超分辨率重建和图像/视频视觉质量增强等相关领域。图像质量评价主要包括全参考图像质量评价、半参考图像质量评价和无参考图像质量评价。全参考图像质量评价和半参考图像质量评价分别指预测图像质量时参考信息完全可用和部分可用,而无参考图像质量评价是指预测图像质量时参考信息不可用。虽然全参考和半参考图像质量评价模型较为可靠,但在计算过程中必须依赖参考信息,使得应用场景极为受限。无参考图像质量评价模型因不需要依赖参考信息而有较强的适用性,一直都是图像质量评价领域研究的热点。本文主要概述2012—2020年国内外公开发表的无参考图像质量评价模型,根据模型训练过程中是否需要用到主观分数,将无参考图像质量评价模型分为有监督学习和无监督学习的无参考图像质量评价模型。同时,每类模型分成基于传统机器学习算法的模型和基于深度学习算法的模型。对基于传统机器学习算法的模型,重点介绍相应的特征提取策略及思想;对基于深度学习算法的模型,重点介绍设计思路。此外,本文介绍了图像质量评价在新媒体数据中的研究工作及图像质量评价的应用。最后对介绍的无参考图像质量评价模型进行总结,并指出未来可能的发展方向。     

Neural network solution for a real-time no-reference video quality assessment of H.264/AVC video bitstreams

The ever-growing video streaming services require accurate quality assessment with often no reference to the original media. One primary challenge in developing no-reference (NR) video quality metrics is achieving real-timeliness while retaining the accuracy. A real-time no-reference video quality assessment (VQA) method is proposed for videos encoded by H.264/AVC codec. Temporal and spatial features are extracted from the encoded bit-stream and pixel values to train and validate a fully connected neural network. The hand-crafted features and network dynamics are designed in a manner to ensure a high correlation with human judgment of quality as well as minimizing the computational complexities. Proof-of-concept experiments are conducted via comparison with: 1) video sequences rated by a full-reference quality metric, and 2) H.264-encoded sequences from the LIVE video dataset which are subjectively evaluated through differential mean opinion scores (DMOS). The performance of the proposed method is verified by correlation measurements with the aforementioned objective and subjective scores. The framework achieves real-time execution while outperforming state-of-art full-reference and no-reference video quality assessment methods.

     

DFT-based no-reference quality assessment of blurred images

Multimedia Tools and Applications - In this paper, blind image quality assessment (IQA) of Gaussian blurred images based on Discrete Fourier Transform (DFT) is proposed. The proposed work is based...     

基于机器学习的无参考图像质量评价综述

无参考图像质量评价（NRIQA）因其广泛的应用需求一直以来都是计算机视觉及其交叉领域的研究热点。回顾近十几年来基于机器学习的典型NRIQA模型,介绍图像质量评价的常用数据库、算法性能指标、NRIQA主要难点和现有的解决方法;分析了不同模型的思想、实现、特点;最后统计对比多个数据库上的测试结果。总结研究现状、分析发展趋势,为这一领域的研究者提供文献参考。     

变权无参考视频质量评价模型

由于视频图像在传输过程中信道噪声将导致质量下降,在无需增加传输信息的前提下客观无参评价方法可实现视频质量的自动评估,因此成为一个重要研究课题。为了提高无参考评价方法准确性,提出了符合人类视觉特性的变权评价模型,综合考虑空域中的清晰度和时域中的平滑度两类指标,利用视频内容的运动信息控制权重变化,模型评价结果与主观评价符合度高,简单相关系数为0.85。实验结果表明,符合视觉特性的连续变权方法比固定权值模型准确,计算复杂度比同类研究方案小,具有更大的应用价值。     

红外模糊图像的无参考质量评价方法

图像质量评价是对图像处理算法的优劣给出合理的评估,在很多无法获取原始参考图像的应用场合中使用无参考质量评价方法。通过对红外图像结构分析得知图像所具有的不确定性往往是模糊性,而不是随机性,因此将模糊集理论中模糊熵的概念引入到红外图像质量评价中,提出一种针对红外模糊图像的无参考质量评价方法,并从算法的有效性、一致性和准确性三个方面进行比较分析。仿真实验结果表明,该方法具有计算复杂度低、运算速度快和主客观评价一致等特点,且在总体性能上优于均方误差(MSE)和峰值信噪比(PSNR)全参考图像质量评价方法。     

无参考模糊图像质量评价改进算法

针对传统无参考模糊图像质量评价算法存在高计算复杂度的问题,通过改进经典的二次模糊处理算法,提出一种快速有效的无参考模糊图像质量评价方法。该算法基于人眼视觉系统(HVS)特性,利用局部方差选取人眼感兴趣图像块代替整体图像,并将感兴趣图像块通过低通滤波处理,构造模糊图像块,通过计算滤波前后图像块相邻像素差值变化大小获取原始整体图像的客观质量评价参数。仿真测试结果表明,该算法与传统整体图像二次模糊算法相比,皮尔逊相关系数提高0.01,与主观评价结果更为一致;运算速度提高一倍,降低了运算复杂度。     

Retina inspired no-reference image quality assessment for blur and noise

Multimedia Tools and Applications - A blind image quality assessment technique with no-training is proposed in this paper. The proposed technique considers two important types of distortions viz....     

New feature selection algorithms for no-reference image quality assessment

No reference image quality assessment (NR-IQA) is a challenging task since reference images are usually unavailable in real world scenarios. The performance of NR-IQA techniques is vastly dependent on the features utilized to predict the image quality. Many NR-IQA techniques have been proposed that extract features in different domains like spatial, discrete cosine transform and wavelet transform. These NR-IQA techniques have the possibility to contain redundant features, which result in degradation of quality score prediction. Recently impact of general purpose feature selection algorithms on NR-IQA techniques has shown promising results. But these feature selection algorithms have the tendency to select irrelevant features and discard relevant features. This paper presents fifteen new feature selection algorithms specifically designed for NR-IQA, which are based on Spearman rank ordered correlation constant (SROCC), linear correlation constant (LCC), Kendall correlation constant (KCC) and root mean squared error (RMSE). The proposed feature selection algorithms are applied on the extracted features of existing NR-IQA techniques. Support vector regression (SVR) is then applied to selected features to predict the image quality score. The fifteen newly proposed feature selection algorithms are evaluated using eight different NR-IQA techniques over three commonly used image quality assessment databases. Experimental results show that the proposed feature selection algorithms not only reduce the number of features but also improve the performance of NR-IQA techniques. Moreover, features selection algorithms based on SROCC and its combination with LCC, KCC and RMSE perform better in comparison to other proposed algorithms.     

Multi-view aggregation transformer for no-reference point cloud quality assessment

With the increasing maturity of 3D point cloud acquisition, storage, and transmission technologies, a large number of distorted point clouds without original reference exist in practical applications. Hence, it is necessary to design a no-reference point cloud quality assessment (PCQA) for point cloud systems. However, the existing no-reference PCQA metrics ignore the content differences and positional context among the projected images. For this, we propose a Multi-View Aggregation Transformer (MVAT) with two different fusion modules to extract the comprehensive feature representation of PCQA. Specifically, considering the content differences of different projected images, we first design a Content Fusion Module (CFM) to fuse multiple projected image features by adaptive weighting. Then, we design a Bidirectional Context Fusion Module (BCFM) to extract context features for reflecting the contextual relationship among projected images. Finally, we joint the above two fusion modules via Content-Position Fusion Module (CPFM) to fully mine the feature representation of point clouds. Experimental results show that our MVAT can achieve comparable or better performance than state-of-the-art metrics on three open point cloud datasets.     

通用型无参考图像质量评价算法综述

图像质量评价可有效评估图像采集和传输过程引起的失真或退化,在数字多媒体领域具有广阔的应用前景,无参考图像质量评价算法由于不需要参考图像先验知识,近年来成为图像质量评价领域研究的热点。在对国内外文献进行广泛调研的基础上,从评价算法原理和性能比较两个方面,系统综述了BIQI、DIIVINE、BLIINDS、BLIINDS-II、BRISQUE、NIQE和GRNN等当前性能较优的几种无参考图像质量评价算法。介绍了各种算法的特征提取和质量评价原理,在LIVE数据库上对上述评价方法进行仿真评估,并分析和比较了各种算法的评价性能和执行速度,提出了无参考评价方法的进一步研究方向。综述的几种无参考图像质量评价算法虽然已具有很好的效果,但在评价时严重依赖数据库中的主观评价数据,并且在评价精度和算法复杂度方面还存在一些不足,需要进行深入研究。     

场景切换视频自适应帧间码率控制

针对视频帧中可能出现的大量场景切换,提出一种基于非连接点的场景切换检测算法,提高编码性能,该场景检测算法复杂度低,在运动估计的同时,完成视频场景切换检测。场景切换将导致GOP(group of pictures)长度的变化,并可能出现GOP长度太短的情况。提出改进的自适应GOP时域滤波技术,避免由于GOP太短引起的编码性能下降。针对视频场景切换检测分割出的不同长度的GOP,提出一种基于率失真模型的帧间码率控制算法,利用视频的失真与码率及视频帧复杂度的关系,对帧间码率分配进行优化,提高重构视频帧的总质量。实验结果表明,基于场景检测的自适应帧间码率控制算法能够获得较好的编码性能。     

Blind image quality assessment with improved natural scene statistics model

No-reference (NR)/blind image quality assessment (IQA) metrics play an important role in the area of image processing. Natural scene statistics (NSS) model assumes that natural images possess certain regular statistical properties and is widely used in NR IQA metrics. Most existing NSS-based NR algorithms are achieved by measuring the variation of image statistics, which are characterized by the fitting parameters of NSS model, across different distortions. However, distortions not only change the image statistics, but also disturb the statistical regularity held by natural images. As a result, the distribution of distorted images can not well follow the NSS model. There exists fitting error between the real distribution of the distorted image and the fitted one under certain NSS model. In this paper, the statistical distributions of the distorted images are discussed in detail. We suggest to take the fitting errors into account as well as the fitting parameters for feature extraction, and propose a novel NR IQA algorithm. Experimental results on several image databases demonstrate that the proposed metric performs highly consistent with human visual perception.     

Saliency-based deep convolutional neural network for no-reference image quality assessment

In this paper, we proposed a novel method for No-Reference Image Quality Assessment (NR-IQA) by combining deep Convolutional Neural Network (CNN) with saliency map. We first investigate the effect of depth of CNNs for NR-IQA by comparing our proposed ten-layer Deep CNN (DCNN) for NR-IQA with the state-of-the-art CNN architecture proposed by Kang et al. (2014). Our results show that the DCNN architecture can deliver a higher accuracy on the LIVE dataset. To mimic human vision, we introduce saliency maps combining with CNN to propose a Saliency-based DCNN (SDCNN) framework for NR-IQA. We compute a saliency map for each image and both the map and the image are split into small patches. Each image patch is assigned with a patch importance value based on its saliency patch. A set of Salient Image Patches (SIPs) are selected according to their saliency and we only apply the model on those SIPs to predict the quality score for the whole image. Our experimental results show that the SDCNN framework is superior to other state-of-the-art approaches on the widely used LIVE dataset. The TID2008 and the CISQ image quality datasets are utilised to report cross-dataset results. The results indicate that our proposed SDCNN can generalise well on other datasets.     

NDNetGaming - development of a no-reference deep CNN for gaming video quality prediction

Multimedia Tools and Applications - Gaming video streaming services are growing rapidly due to new services such as passive video streaming of gaming content, e.g. Twitch.tv, as well as cloud...     

基于空域自然场景统计的无参考立体图像质量评价模型

针对现有的评价方法大都将图像变换到不同的坐标域问题,提出一种基于空域自然场景统计(NSS)的通用型无参考立体图像质量评价模型。在评价中为了更好地结合人类双目视觉特性, 将左右图像融合成一幅独眼图;评价模型首先统计独眼图归一化亮度(CMSCN)系数分布规律,进而对独眼图提取空域自然场景统计特征;其次,统计视差图归一化亮度(DMSCN)系数的分布规律,并对用光流法得到的视差图提取同样的特征;最后,通过支持向量回归(SVR)建立立体图像特征信息与主观评价值(DMOS)之间的关系,从而预测得到图像质量的客观评价值。实验结果表明,该评价模型对立体数据测试库进行评价,其Pearson线性相关系数(PLCC)和Spearman等级相关系数(SROCC)值均在0.94以上;对于非对称立体图像库,PLCC和SROCC值分别接近0.91和0.93。该模型能够很好地预测人眼对立体图像的主观感知。     

Foveated video quality assessment

Most image and video compression algorithms that have been proposed to improve picture quality relative to compression efficiency have either been designed based on objective criteria such as signal-to-noise-ratio (SNR) or have been evaluated, post-design, against competing methods using an objective sample measure. However, existing quantitative design criteria and numerical measurements of image and video quality both fail to adequately capture those attributes deemed important by the human visual system, except, perhaps, at very low error rates. We present a framework for assessing the quality of and determining the efficiency of foveated and compressed images and video streams. Image foveation is a process of nonuniform sampling that accords with the acquisition of visual information at the human retina. Foveated image/video compression algorithms seek to exploit this reduction of sensed information by nonuniformly reducing the resolution of the visual data. We develop unique algorithms for assessing the quality of foveated image/video data using a model of human visual response. We demonstrate these concepts on foveated, compressed video streams using modified (foveated) versions of H.263 that are standard-compliant. We rind that quality vs. compression is enhanced considerably by the foveation approach