No reference image blurriness assessment with local binary patterns

In this paper, we put forward an effective and efficient no reference image blurriness assessment metric on the basis of local binary pattern (LBP) features. In this proposal, we reveal that part of the LBP histogram bins present monotonously with the degree of blurriness. The proposed method contains the following steps. Firstly, the LBP maps of an input image are extracted with multiple radiuses. And then, the frequency of pattern histogram is analyzed before part of bins are chosen as the features. In addition, we also take the entropy of these bins as another feature. Finally, we learn the extracted features to predict the image blurriness score. Validation of the proposed method is conducted on the blurred images of LIVE-II, CSIQ, TID2008, TID2013, LIVE3D IQA Phase I and LIVE3D IQA Phase II. Experimental results demonstrate that compared with the state-of-the-art image quality assessment (IQA) methods, the proposed algorithm has notable advantage in correlation with subjective perception and computational complexity.     

基于亮度统计的无参考图像质量评价

基于亮度均值减损对比归一化(MSCN) 系数统计特性及其8方 向邻域系数间的相关性,提出了一种通用无参考图像质量评价方法.首先,分别利用非 对称广义高斯分布(AGGD)模型拟合MSCN系数及其8方 向邻域系数,并估计 相应AGGD 模型参数作为亮度统计特征;其次,计算8方向邻域MSCN系数间的互信息(MI),作为描述方向相 关性的统计特征;进而,分别利用支持向量回归机(SVR)和支 持向量分类机(SVC)构建无参考图像质量评价模型和图像失真类型识别模型; 最后, 在LIVE 等图像质量 评价数据库上进行了算法与DMOS的相关性、失真类型识别、模型 鲁棒性及计算复杂性等方面的实验。实 验结果表明,本文方法的评价结果与人类主观评价具有高度的一致性,在LIVE图像质量评 价数据库上的斯 皮尔曼等级相关系数(SROCC)和皮尔逊线性相关系数 (PLCC)均在0.945以上;而且,图像失真 类型识别模型的识别准确率也高达到92.95%,明显高于 当今主流无参考图像质量评价方法。     

No-reference image quality assessment based on neighborhood co-occurrence matrix

No-reference image quality assessment (NR-IQA) aims to develop models that can predict the quality of distorted image automatically and accurately in the absent of reference image. Previous NR-IQA methods based on natural scene statistics (NSS) always focus on the luminance contrast of image but attach limited attention to pixel-wise relationship. However, human visual system (HVS) is highly adaptive to extract spatial correlation according to relative position within visual field. In this paper, a new approach is proposed for NR-IQA, in which the neighborhood co-occurrence matrix (NCM) is introduced to describe spatial correlation of pixels for quality assessment. The NCM is constructed based on spatial correlation of every pixel and its neighborhood through a mapping to highlight the one-to-many pixel-wise relationship. Moreover, a series of tailored statistical metrics are designed to quantify the unnaturalness extent of NCM effectively, which is combined with others natural scene statistics to predict image quality. Extensive experiments demonstrate the proposed method has superior performance against compared methods, and achieves significant improvements on distortions associated with color or locality.     

No-reference image quality assessment using structural activity

Presuming that human visual perception is highly sensitive to the structural information in a scene, we propose the concept of structural activity (SA) together with a model of SA indicator in a new framework for no-reference (NR) image quality assessment (QA) in this study. The proposed framework estimates image quality based on the quantification of the SA information of different visual significance. We propose some alternative implementations of SA indicator in this paper as examples to demonstrate the effectiveness of the SA-motivated framework. Comprehensive testing demonstrates that the model of SA indicator exhibits satisfactory performance in comparison with subjective quality scores as well as representative full-reference (FR) image quality measures.     

Gaussian Process-based Feature-Enriched Blind Image Quality Assessment

The objective of blind-image quality assessment (BIQA) research is the prediction of perceptual quality of images, without reference information. The human’s perceptual assessment of quality of an image is the backbone of BIQA research. Therefore, human-provided, mean opinion score (perceptual quality) has been analyzed in detail, and it has been observed to follow the Gaussian distribution and thus can be ideally modeled by the same. In this paper, we have proposed an integrated two-stage Gaussian process-based hybrid-feature selection algorithm for the BIQA problem. Moreover, a new consolidated feature set (obtained from the proposed algorithm), consisting of momentous Natural Scene Statistics (NSS)-based features is used in combination with the Gaussian process regression algorithm for the design of a new blind-image quality evaluator, referred to as GPR-BIQA. The proposed evaluator is tested on eight IQA legacy databases, and it is found that the proposed evaluator proficiently correlate with the human opinion, and outperformed a substantial number of existing approaches.     

BGT: A blind image quality evaluator via gradient and texture statistical features

Blind image quality assessment (BIQA) aims to design a model that can accurately evaluate the quality of the distorted image without any information about its reference image. Previous studies have shown that gradients and textures of image is widely used in image quality evaluation tasks. However, few studies used the joint statistics of gradient and texture information to evaluate image quality. Considering the visual perception characteristics of the human visual system, we develop a novel general-purpose BIQA model via two sets of complementary perception features. Specifically, the joint statistical histograms of gradient and texture are extracted as the first set of features, and the second set of features is extracted using the local binary pattern (LBP) operator. After extracting two groups of complementary quality-aware features, the feature vectors are sent to the support vector regression machine to establish the nonlinear relationship between quality-aware features and quality scores. A large number of experiments on seven large benchmark databases show that the proposed BIQA model has higher accuracy, better generalization properties and lower computational complexity than the relevant state-of-the-art BIQA metrics.     

No-reference image quality assessment based on spatial and spectral entropies

We develop an efficient general-purpose no-reference (NR) image quality assessment (IQA) model that utilizes local spatial and spectral entropy features on distorted images. Using a 2-stage framework of distortion classification followed by quality assessment, we utilize a support vector machine (SVM) to train an image distortion and quality prediction engine. The resulting algorithm, dubbed Spatial–Spectral Entropy-based Quality (SSEQ) index, is capable of assessing the quality of a distorted image across multiple distortion categories. We explain the entropy features used and their relevance to perception and thoroughly evaluate the algorithm on the LIVE IQA database. We find that SSEQ matches well with human subjective opinions of image quality, and is statistically superior to the full-reference (FR) IQA algorithm SSIM and several top-performing NR IQA methods: BIQI, DIIVINE, and BLIINDS-II. SSEQ has a considerably low complexity. We also tested SSEQ on the TID2008 database to ascertain whether it has performance that is database independent.     

No-reference blur image quality measure based on multiplicative multiresolution decomposition

A new approach for analyzing the blur effect on real images is proposed. This approach is based on the Multiplicative Multi-resolution Decomposition MMD. From MMD image-content analysis, a blind image quality measure dedicated to blur is then derived. The proposed measure has been applied on Gaussian-blurred and JPEG2000-compressed images from the LIVE, TID and IVC databases. The performance of the proposed measure is evaluated and compared with some referenced image quality metrics. The experimental results measured in terms of correlation with the subjective assessment of the images, demonstrate the efficiency of the proposed image quality measure in predicting the amount of blur.     

MLSIM: A Multi-Level Similarity index for image quality assessment

Image quality assessment (IQA) is of great importance to numerous image processing applications, and various methods have been proposed for it. In this paper, a Multi-Level Similarity (MLSIM) index for full reference IQA is proposed. The proposed metric is based on the fact that human visual system (HVS) distinguishes the quality of an image mainly according to the details given by low-level gradient information. In the proposed metric, the Prewitt operator is first utilized to get gradient information of both reference and distorted images, then the gradient information of reference image is segmented into three levels (3LSIM) or two levels (2LSIM), and the gradient information of distorted image is segmented by the corresponding regions of reference image, therefore we get multi-level information of these two images. Riesz transform is utilized to get corresponding features of different levels and the corresponding 1st-order and 2nd-order coefficients are combined together by regional mutual information (RMI) and weighted to obtain a single quality score. Experimental results demonstrate that the proposed metric is highly consistent with human subjective evaluations and achieves good performance.     

基于图像相关性和结构信息的无参考图像质量评价

分析了非下采样Contourlet变换(NSCT)方向子带亲 戚系数 间和父子系数间的强相关性及其包含的结构信息,并基于图像发生失真会 影响这些系数间的强相关性和结构信息的假设,提出了一种新的无参考图像质量评价(NR-I QA)方法。首先,分别计算 NSCT方向子带亲戚系数间和父子系数间的互信息(MI),以此作为描述这些系数 间相关性的统计特征;其次, 分别计算NSCT方向子带亲戚系数间和父子系数间的结构相似度(SSIM),以此作为描述 图像结构信息的统计特征; 进而,结合这些系数间的MI和SSIM等统计特征,构造了相应的NR-I QA模型和图像 失真类型识别模型;最后,在LIVE及LIVE Multiply Distorted图像质量评价数据库上进行 了大量的实验仿 真。结果表明,本文评价模型的评价结果与人类主观评价具有非常高的相关性,LIVE图 像质量评价数据 库上的斯皮尔曼等级相关系数和皮尔逊线性相关系数均在0.931以上 。无论总体评价效果还是各失真类型 评价效果,与当今主流评价算法相比非常具有竞争性;而且,失真类型识别模型的识别精度 可以达到92.31%,明显高于这些主流算法。     

A new SVM based emotional classification of image

How high-level emotional representation of art paintings can be inferred from perceptual level features suited for the particular classes (dynamic vs. static classification) is presented. The key points are feature selection and classification. According to the strong relationship between notable lines of image and human sensations, a novel feature vector WLDLV (Weighted Line Direction-Length Vector) is proposed, which includes both orientation and length information of lines in an image. Classification is performed by SVM (Support Vector Machine) and images can be classified into dynamic and static. Experimental results demonstrate the effectiveness and superiority of the algorithm.     

Attention integrated hierarchical networks for no-reference image quality assessment

Quality assessment of natural images is influenced by perceptual mechanisms, e.g., attention and contrast sensitivity, and quality perception can be generated in a hierarchical process. This paper proposes an architecture of Attention Integrated Hierarchical Image Quality networks (AIHIQnet) for no-reference quality assessment. AIHIQnet consists of three components: general backbone network, perceptually guided neck network, and head network. Multi-scale features extracted from the backbone network are fused to simulate image quality perception in a hierarchical manner. The attention and contrast sensitivity mechanisms modelled by an attention module capture essential information for quality perception. Considering that image rescaling potentially affects perceived quality, appropriate pooling methods in the non-convolution layers in AIHIQnet are employed to accept images with arbitrary resolutions. Comprehensive experiments on publicly available databases demonstrate outstanding performance of AIHIQnet compared to state-of-the-art models. Ablation experiments were performed to investigate the variants of the proposed architecture and reveal importance of individual components.     

No-reference quality assessment for DCT-based compressed image

A blind/no-reference (NR) method is proposed in this paper for image quality assessment (IQA) of the images compressed in discrete cosine transform (DCT) domain. When an image is measured by structural similarity (SSIM), two variances, i.e. mean intensity and variance of the image, are used as features. However, the parameters of original copies are actually unavailable in NR applications; hence SSIM is not widely applicable. To extend SSIM in general cases, we apply Gaussian model to fit quantization noise in spatial domain, and directly estimate noise distribution from the compressed version. Benefit from this rearrangement, the revised SSIM does not require original image as the reference. Heavy compression always results in some zero-value DCT coefficients, which need to be compensated for more accurate parameter estimate. By studying the quantization process, a machine-learning based algorithm is proposed to estimate quantization noise taking image content into consideration. Compared with state-of-the-art algorithms, the proposed IQA is more heuristic and efficient. With some experimental results, we verify that the proposed algorithm (provided no reference image) achieves comparable efficacy to some full reference (FR) methods (provided the reference image), such as SSIM.     

Image quality assessment for contrast enhancement evaluation

Although the contrast enhancement (CE) is a great challenge, few efforts have been conducted on evaluation of the contrast changes. In this paper, we propose a contrast-changed image quality (CCIQ) metric including a local index, named edge-based contrast criterion (ECC), and three global measures. In the global measures, entropy, correlation coefficient and mean intensity are exploited. Particle swarm optimization (PSO) algorithm is utilized for obtaining an optimal combination of these quantities. Although the presented method utilizes the original image, it cannot be considered as a full-reference metric, since the original image is not regarded to have the ideal quality. Hence, it can be concluded that it follows a new paradigm in image quality assessment. Experimental results on the three benchmark databases, CID2013, TID2013 and TID2008 demonstrate that the proposed metric outperforms the-state-of-the-art methods.     

基于双目能量响应的无参考立体图像质量评价

为了实现对不同失真类型立体图像的质量评价,提出了一种基于双目能量响应的无参考立体图像质量评价(NR-IAQ)方法。首先,通过对各失真图像进行Gabor滤波,提取出不同频率、不同方向、不同视差响应下的局部特征矢量,作为立体图像特征信息;然后,利用支持向量回归(SVR)建立立体图像特征与主观评价值的关系,从而预测得到立体图像质量的客观评价值。实验结果表明,对于NBU-3D测试库,Pearson线性相关系数值在0.92以上,Spearman等级相关系数值在0.93以上;对于LIVE-3D测试库,Pearson线性相关系数值在0.96以上,Spearman等级相关系数值在0.96以上;与现有的全参考(FR)和(NR)质量评价方法相比,本方法得到的客观评价值与主观评价结果有较好的相关性,更加符合人眼视觉系统。     

No-reference image quality assessment using statistical characterization in the shearlet domain

Image and video quality measurements are crucial for many applications, such as acquisition, compression, transmission, enhancement, and reproduction. Nowadays, no-reference (NR) image quality assessment (IQA) methods have drawn extensive attention because it does not rely on any information of original images. However, most of the conventional NR-IQA methods are designed only for one or a set of predefined specific image distortion types, which are unlikely to generalize for evaluating image/video distorted with other types of distortions. In order to estimate a wide range of image distortions, in this paper, we present an efficient general-purpose NR-IQA algorithm which is based on a new multiscale directional transform (shearlet transform) with a strong ability to localize distributed discontinuities. This is mainly based on distorted natural image that leads to significant variation in the spread discontinuities in all directions. Thus, the statistical property of the distorted image is significantly different from that of natural images in fine scale shearlet coefficients, which are referred to as ‘distorted parts’. However, some ‘natural parts’ are reserved in coarse scale shearlet coefficients. The algorithm relies on utilizing the natural parts to predict the natural behavior of distorted parts. The predicted parts act as ‘reference’ and the difference between the reference and distorted parts is used as an indicator to predict the image quality. In order to achieve this goal, we modify the general sparse autoencoder to serve as a predictor to get the predicted parts from natural parts. By translating the NR-IQA problem into classification problem, the predicted parts and distorted parts are utilized to form features and the differences between them are identified by softmax classifier. The resulting algorithm, which we name SHeArlet based No-reference Image quality Assessment (SHANIA), is tested on several database (LIVE, Multiply Distorted LIVE and TID2008) and shown to be suitable for many common distortions, consistent with subjective assessment and comparable to full-reference IQA methods and state-of-the-art general purpose NR-IQA algorithms.     

基于支持向量回归的无参考模糊和噪声图像质量评价方法

基于支持向量回归(SVR)和图像奇异值分解,提出了一种新的无参考(NR,no-reference)模糊和噪声图像质量评价(IQA)方法。首先通过对待评价图像进行高斯低通滤波生成再模糊图像,然后分别对它们进行奇异值分解并计算奇异值的改变量,最后使用奇异值的改变量作为SVR的输入,训练预并测得到图像的质量评分。在3个公开的模糊和噪声数据库上的实验结果表明,新方法预测得分与主观得分有较好的一致性,获得了较好的评价指标;对于模糊失真类型和噪声失真类型,在LIVE2数据库上的性能评价指标斯皮尔曼等级相关系数(SROCC)分别达到0.961 3和0.965 9。     

基于结构相似度的自适应图像质量评价

考虑到在结构相似度(SSIM,structural similarity)模型中,亮度、对比度和结构度3个评价因子对不同失真类型图像质量评价(QA)的贡献程度不同,本文提出了根据图像失真类型分析的自适应SSIM(ASSIM)的IQA方法。首先,分析失真图像和参考图像的小波子带能量、傅里叶功率谱和幅度谱的数据特点,据此判定图像失真类型,包括高斯白噪声(WN)、JPEG压缩(JPEM)、高斯模糊(Gblur)及类JP2K4类失真;接着,通过优化算法确定SSIM在评价不同失真类型图像时最佳的评价因子权重;最后,将图像的失真类型判别和评价因子的调整相结合,实现对图像的自适应评价。实验结果表明,由于失真类型的判断和评价因子权值的优化,ASSIM对各类失真图像的评价效果都要优于SSIM,特别是对Gblur失真的图像进行评价时,Pearson系数(CC)值提高了0.05,Spearman等级相关系数(SROCC)值的提高超过0.039。     

Blind quality assessment of tone-mapped images using multi-exposure sequences

The tone mapping operator (TMO) enables high dynamic range (HDR) images to be presented on low dynamic range (LDR) consumer electronic devices. However, the results obtained by this method are not always ideal due to the reduced number of bits. In comparison, the multi-exposure image fusion (MEF) bypasses the intermediate HDR image composition and directly produces an image presented on standard devices. Inspired by this, this paper proposes a quality assessment method for tone-mapped image (TMI) based on generating multi-exposure sequences. Specifically, the method uses a generative adversarial network (GAN) to generate a set of sequences with different exposure levels based on the TMIs. Then a two-branch convolutional neural network (CNN) is used to extract features from the tone-mapped images and the multi-exposure reference sequences, respectively. Finally, the transformer is used to mine the intrinsic connections between TMIs and multi-exposure sequences and learn the mapping relationships from feature space to quality space. We conducted extensive experiments on the ESPL-LIVE HDR database. The applicability and effectiveness of the proposed method are verified by comparing and analyzing relevant features and model configurations with existing mainstream evaluation algorithms.