Sparse representation-based image quality assessment

A highly promising approach to assess the quality of an image involves comparing the perceptually important structural information in this image with that in its reference image. The extraction of the perceptually important structural information is however a challenging task. This paper employs a sparse representation-based approach to extract such structural information. It proposes a new metric called the sparse representation-based quality (SPARQ) index that measures the visual quality of an image. The proposed approach learns the inherent structures of the reference image as a set of basis vectors. These vectors are obtained such that any structure in the image can be efficiently represented by a linear combination of only a few of these basis vectors. Such a sparse strategy is known to generate basis vectors that are qualitatively similar to the receptive field of the simple cells present in the mammalian primary visual cortex. To estimate the visual quality of the distorted image, structures in the visually important areas in this image are compared with those in the reference image, in terms of the learnt basis vectors. Our approach is evaluated on six publicly available subject-rated image quality assessment datasets. The proposed SPARQ index consistently exhibits high correlation with the subjective ratings of all datasets and overall, performs better than a number of popular image quality metrics.     

Kurtosis-based no-reference quality assessment of JPEG2000 images

No-reference (NR) image quality assessment (QA) presumes no prior knowledge of reference (distortion-free) images and seeks to quantitatively predict visual quality solely from the distorted images. We develop kurtosis-based NR quality measures for JPEG2000 compressed images in this paper. The proposed measures are based on either 1-D or 2-D kurtosis in the discrete cosine transform (DCT) domain of general image blocks. Comprehensive testing demonstrates their good consistency with subjective quality scores as well as satisfactory performance in comparison with both the representative full-reference (FR) and state-of-the-art NR image quality measures.     

基于多尺度边缘结构相似性的图像质量评价

基于结构相似度(SSIM)的图像质量评价方法是从视觉区域提取图像的结构性信息,但在评价模糊较严重的图像时存在其局限性,因此本文将图像边缘和SSIM相结合,提出了基于多尺度边缘结构相似度(MESS)的图像质量评价方法。实验结果表明,由于MESS考虑了边缘信息对于人眼感知结构信息的重要性,评价结果比SSIM更加符合人眼视觉感知特性。     

Image quality/distortion metric based on α-stable model similarity in wavelet domain

Quality assessment is of central importance in numerous image processing tasks. State-of-the-art objective image quality assessment (IQA) algorithms are generally devised for specific distortion types or based on training procedure of large databases. In this work, we propose a general-purpose full-reference/no-reference (FR/NR) IQA framework for image distortions, nominated by Image Quality/Distortion Metric (IQDM). The leptokurtic and heavy-tailed behaviors of image wavelet coefficients are characterized by symmetric α-stable (SαS) density, and the statistical studies indicate that the model parameters may be altered because of the presence of distortion. This important priori knowledge of original image’s distribution is then used to gauge the distortion between degraded and reference SαS models in multi-scale wavelet sub-bands. We investigate the relationship between original and degraded parameters over scales, accordingly infer the original parameters from the degraded ones. A characteristic probability density function for SαS and its closed-form Kullback–Leibler distance are derived for FR/NR-IQDM using the model parameters. Extensive experiments and comparisons demonstrate that the proposed FR/NR-IQDM scheme is efficacious to most common types of distortion, and leads to a highly comparable performance to the benchmarks and prevalent competitors in consistency with subjective judgements.     

C-DIIVINE: No-reference image quality assessment based on local magnitude and phase statistics of natural scenes

It is widely known that the wavelet coefficients of natural scenes possess certain statistical regularities which can be affected by the presence of distortions. The DIIVINE (Distortion Identification-based Image Verity and Integrity Evaluation) algorithm is a successful no-reference image quality assessment (NR IQA) algorithm, which estimates quality based on changes in these regularities. However, DIIVINE operates based on real-valued wavelet coefficients, whereas the visual appearance of an image can be strongly determined by both the magnitude and phase information.In this paper, we present a complex extension of the DIIVINE algorithm (called C-DIIVINE), which blindly assesses image quality based on the complex Gaussian scale mixture model corresponding to the complex version of the steerable pyramid wavelet transform. Specifically, we applied three commonly used distribution models to fit the statistics of the wavelet coefficients: (1) the complex generalized Gaussian distribution is used to model the wavelet coefficient magnitudes, (2) the generalized Gaussian distribution is used to model the coefficients׳ relative magnitudes, and (3) the wrapped Cauchy distribution is used to model the coefficients׳ relative phases. All these distributions have characteristic shapes that are consistent across different natural images but change significantly in the presence of distortions. We also employ the complex wavelet structural similarity index to measure degradation of the correlations across image scales, which serves as an important indicator of the subbands׳ energy distribution and the loss of alignment of local spectral components contributing to image structure. Experimental results show that these complex extensions allow C-DIIVINE to yield a substantial improvement in predictive performance as compared to its predecessor, and highly competitive performance relative to other recent no-reference algorithms.     

Video quality assessment using space–time slice mappings

We develop a full-reference (FR) video quality assessment framework that integrates analysis of space–time slices (STSs) with frame-based image quality measurement (IQA) to form a high-performance video quality predictor. The approach first arranges the reference and test video sequences into a space–time slice representation. To more comprehensively characterize space–time distortions, a collection of distortion-aware maps are computed on each reference–test video pair. These reference-distorted maps are then processed using a standard image quality model, such as peak signal-to-noise ratio (PSNR) or Structural Similarity (SSIM). A simple learned pooling strategy is used to combine the multiple IQA outputs to generate a final video quality score. This leads to an algorithm called Space–TimeSlice PSNR (STS-PSNR), which we thoroughly tested on three publicly available video quality assessment databases and found it to deliver significantly elevated performance relative to state-of-the-art video quality models. Source code for STS-PSNR is freely available at: http://live.ece.utexas.edu/research/Quality/STS-PSNR_release.zip.     

An optimized CNN-based quality assessment model for screen content image

Most existing convolutional neural network (CNN) based models designed for natural image quality assessment (IQA) employ image patches as training samples for data augmentation, and obtain final quality score by averaging all predicted scores of image patches. This brings two problems when applying these methods for screen content image (SCI) quality assessment. Firstly, SCI contains more complex content compared to natural image. As a result, qualities of SCI patches are different, and the subjective differential mean opinion score (DMOS) is not appropriate as qualities of all image patches. Secondly, the average score of image patches does not represent the quality of entire SCI since the human visual system (HVS) is sensitive to image patches containing texture and edge information. In this paper, we propose a novel quadratic optimized model based on the deep convolutional neural network (QODCNN) for full-reference (FR) and no-reference (NR) SCI quality assessment to overcome these two problems. The contribution of our algorithm can be concluded as follows: 1) Considering the characteristics of SCIs, a valid network architecture is designed for both NR and FR visual quality evaluation of SCIs, which makes the networks learn the feature differences for FR-IQA; 2) with the consideration of correlation between local quality and DMOS, a training data selection method is proposed to fine-tune the pre-trained model with valid SCI patches; 3) an adaptive pooling approach is employed to fuse patch quality to obtain image quality, owns strong noise robust and effects on both FR and NR IQA. Experimental results verify that our model outperforms both current no-reference and full-reference image quality assessment methods on the benchmark screen content image quality assessment database (SIQAD). Cross-database evaluation shows high generalization ability and high effectiveness of our model.     

Web image concept annotation with better understanding of tags and visual features

This paper focuses on improving the semi-manual method for web image concept annotation. By sufficiently studying the characteristics of tag and visual feature, we propose the Grouping-Based-Precision & Recall-Aided (GBPRA) feature selection strategy for concept annotation. Specifically, for visual features, we construct a more robust middle level feature by concatenating the k-NN results for each type of visual feature. For tag, we construct a concept-tag co-occurrence matrix, based on which the probability of an image belonging to certain concept can be calculated. By understanding the tags’ quality and groupings’ semantic depth, we propose a grouping based feature selection method; by studying the tags’ distribution, we adopt Precision and Recall as a complementary indicator for feature selection. In this way, the advantages of both tags and visual features are boosted. Experimental results show our method can achieve very high Average Precision, which greatly facilitates the annotation of large-scale web image dataset.     

M2OVQA: Multi-space signal characterization and multi-channel information aggregation for quality assessment of compressed omnidirectional videos

Considering the high requirements for omnidirectional video compression, we propose an objective quality evaluation method to assess quality loss in encoding omnidirectional videos. According to characteristics of 360° videos, we consider multi-space signal characterization (MSSC) to fully characterize the distortions of video signals from spatial/image domains to frequency domains and from image content to motion information, and further consider multi-channel information aggregation (MCIA) to fuse scores from multiple projection planes and temporal divided groups. The main innovation of our method is to establish a universal framework in bridging the connection between typical quality assessment and 360° quality assessment to measure 360° video quality effectively and efficiently. Experimental results show that our method outperforms state-of-the-art 2D quality metrics and quality metrics for omnidirectional images.     

基于结构信息和像素失真的客观图像质量评价

为了度量多种失真类型的图像质量,根据人类视觉系统(HVS)对图像空域结构信息高度敏感和任一类型的失真都会产生像素失真理论,提出一种基于结构信息和像素失真的无参考的质量评价方法.该方法利用色彩信息提取能够表征图像结构信息的视觉内容结构图,并加权像素失真来度量图像质量,同时对部分失真类型进行修正.该方法不涉及任何参数设置也无需训练过程.实验结果表明,该方法能够较好地评价白噪声、JPEG压缩、高斯模糊、JPEG2000压缩和FastFading等失真图像的质量,并与主观评价方法有较好的一致性.     

注意力模型指导下的视频质量评价方法

在视频质量评价方法中,常常需要对人眼视觉系统做出合理的假设,其中注意力模型就是一个很重要的因素。提出了一种在注意力模型指导下的视频质量评价方法,在图像帧的质量评价中加入了显著性区域信息,使之更能符合人眼视觉特性,并兼顾了视频中的运动信息,在一定程度上提高了客观质量评价方法的性能。     

Objective quality assessment of synthesized images by local variation measurement

Due to the rapid development of free-viewpoint television (FVT), Depth-Image-Based Rendering (DIBR) technology has been widely used to synthesize images of virtual view-points. However, the types of distortions in the synthesized images are different from those of natural images, such as discontinuity, flickering, stretching, etc. To measure the distortion occurred in the synthesized images, we propose a full-reference (FR) quality assessment method by local variation measurement consisting of three-modules. Firstly, since the distortion in the synthesized image mainly occurs in the region with high-frequency structure information, the Neutrosophic domain is employed to evaluate the degradation of local image structure. Secondly, by considering that the texture of the synthesized image might be damaged due to the warping of 2D image or the loss of information in the occlusion region, we evaluate the visual quality of local texture by using the features obtained from frequency domain. Thirdly, to measure the stretching distortion which is unique in the synthesized image, the visual quality of extracted stretching area is measured by entropy. Finally, a pooling operation is used to combine the quality scores of the three modules to obtain the final predicted quality score. Experimental results show that the performance of the proposed algorithm is competitive with state-of-the-art FR and no-reference image quality assessment metrics.     

基于人眼视觉特性的彩色图像质量评价

图像处理系统的性能优劣的评判往往需要一个合理迅速的图像质量评价算法作为支撑.传统的图像质量评价算法由于没有充分考虑人眼的视觉特性,使得质量评价结果与实际图像的人眼感知质量不符.根据人眼对图像边缘信息非常敏感这一人眼视觉特性,提出一种综合图像边缘和背录相似度的算法(EBS)来评价彩色图像质量,即通过比较失真彩色图像与原始参考图像的边缘以及除边缘之外的背景相似程度最终确定失真图像的质量.应用于由779幅包含五种类型失真的图像质量评价库的实验结果表明,该算法的评价结果相比PSNR,MSSIM,IFC以及基于像素域的VIF等算法与图像的主观评价结果(由DMOS值表示--将背景不同的一组观察者对失真图像的评分进行统计平均后所得到的评价结果)更一致,也即该算法的评价结果更接近图像的实际视觉感知质量.     

基于视觉感知和零水印的部分参考立体图像质量评价模型

通过分析人类视觉系统的纹理方向特性和立体感知特性,并结合数字水印的半脆弱性和支持向量回归(Support Vector Regression, SVR)的泛化学习能力,该文提出一种基于视觉感知和零水印的部分参考立体图像质量客观评价模型。该模型利用立体图像左右视点经小波分解后在同一空间频率的水平和垂直方向子带系数关系构造反映图像纹理方向特征的视点零水印,同时,利用左右视点视差值与自适应阈值的大小关系构造反映立体感质量的视差零水印,然后利用SVR来学习两类零水印恢复率(视觉加权视点零水印恢复率和视差零水印恢复率)与主观评价值的关系,最后用训练好的SVR完成立体图像质量预测。实验结果表明该模型符合人眼视觉特性,所得到的客观评价值与主观评价值具有较好的一致性。     

An enhanced image quality assessment by synergizing superpixels and visual saliency

Superpixel and saliency-based evaluation methods play important roles in full reference image quality assessment (FR IQA). However, we find that these methods have one complementary principle and three limitations: (1) the weighted maps of superpixel-based methods conflict with the perception of the human visual system; (2) saliency-based methods are inefficient in terms of the block distortion; (3) the general two-direction gradient extraction factor must be extended to be multidirectional. To address these limitations, we propose an enhanced image quality assessment by synergizing superpixels and visual saliency. Specifically, the calculation of a newly proposed framework involves three similarities and two strategies: the saliency, superpixel and multidirectional gradient similarities of the neighborhoods, and the saliency pooling strategy, the fusion strategy of these similarities. Theoretical analysis and experimental results show that the proposed method can effectively address the limitations noted above and outperform the existing methods.     

基于双目特征联合的无参考立体图像质量评价

通过模拟人类视觉系统(HVS)的双目视觉行为,提 出一种基于双目特征联合的无参考立 体图像质量评价(NR-SIQA)方法。首先分析立体视觉感知中的双目联合行为,提出 可应用于立体图像质量预 测的双目联合模型;然后采用学习和统计分析的方法,分别提取局部和全局特征并联合作 为感知特征; 最后采用机器学习算法,建立特征和质量的关系模型,并结合基于特征的双目联合模型预测 立体图像质量。实验结果表明,本文方法在对称立体图像库上的Pearson线性相关系数(PLCC)和Spearman等级系数(SRCC)高于0.93,在非对称库上高于0.87,优 于现有评价方法。     

基于结构相似度的图像去噪新方法

将结构相似度作为一种刻画忠诚项的度量用于图像去噪模型中。针对经典ROF模型忠诚项的约束项L2度量未考虑图像空间结构性而导致恢复图像视觉效果差的缺陷,引入结构相似度来改进模型的忠诚项,提出了一种新的去噪模型。为在去噪过程中,更好地保护图像的边缘,在此模型的基础上,文中还做了进一步改进,用非凸正则项代替TV正则项,得到推广模型。实验结果表明,相对于ROF模型,两个模型在有效去除噪声的同时,能更好地保持图像的结构信息,提高图像的视觉效果,且推广模型在图像边缘保护方面的性能更好。     

Image quality assessment: from error visibility to structural similarity

Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a Structural Similarity Index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000.