No-reference image quality assessment based on non-subsample shearlet transform and natural scene statistics

A novel no-reference (NR) image quality assessment (IQA) method is proposed for assessing image quality across multifarious distortion categories. The new method transforms distorted images into the shearlet domain using a non-subsample shearlet transform (NSST), and designs the image quality feature vector to describe images utilizing natural scenes statistical features:coefficient distribution, energy distribution and structural correlation (SC) across orientations and scales. The final image quality is achieved from distortion classification and regression models trained by a support vector machine (SVM). The experimental results on the LIVE2 IQA database indicate that the method can assess image quality effectively, and the extracted features are susceptive to the category and severity of distortion. Furthermore, our proposed method is database independent and has a higher correlation rate and lower root mean squared error (RMSE) with human perception than other high performance NR IQA methods.     

A no-reference perceptual image quality assessment database for learned image codecs

The drastic growth of research in image compression, especially deep learning-based image compression techniques, poses new challenges to objective image quality assessment (IQA). Typical artifacts encountered in the emerging image codecs are significantly different from that produced by traditional block-based codecs, leading to inapplicability of the existing objective IQA algorithms. Towards advancing the development of objective IQA algorithms for recent compression artifacts, we built a learning-based compressed image quality assessment (LCIQA) database involving traditional block-based image codecs, hybrid neural network based image codecs, convolutional neural network based and generative adversarial network (GAN) based end-to-end optimized image coding approaches. Our study confirms the statistical difference and human perception difference between reconstructions of learned compression and traditional block-based compression. We propose a two-step deep learning model for learning-based compressed image quality assessment. Extensive experiments on LCIQA database demonstrate that our proposed model performs better than other counterparts on learning-based compressed images, especially on GAN compressed images, and achieves competitive performance to the state-of-the-art IQA metrics on traditional compressed images.     

一个图像质量盲评估的统计测度

本文提出了一个图像质量盲评估的统计测度.该测度首先根据自然图像的统计性质与失真图像的模型,实现对图像小波系数分布参数的盲估计;再利用估计的分布参数来计算失真图像与参考图像之间的互信息,以量化失真图像对参考图像的保真度,进而实现对图像质量的评估.本文提出的测度避免了对参考图像的依赖,且克服了现有图像质量盲评估对特征选择与提取、机器学习等过程的依赖.LIVE图像质量评估数据库的总体评估结果表明：本文提出的盲评估统计测度对图像质量评估结果与数据库的主观评估结果高度一致,且优于文献中报道的盲评估测度.     

Learning to predict the quality of distorted-then-compressed images via a deep neural network

Being captured by amateur photographers, reciprocally propagated through multimedia pipelines, and compressed with different levels, real-world images usually suffer from a wide variety of hybrid distortions. Faced with this scenario, full-reference (FR) image quality assessment (IQA) algorithms can not deliver promising predictions due to the inferior references. Meanwhile, existing no-reference (NR) IQA algorithms remain limited in their efficacy to deal with different distortion types. To address this obstacle, we explore a NR-IQA metric by predicting the perceptual quality of distorted-then-compressed images using a deep neural network (DNN). First, we propose a novel two-stream DNN to handle both authentic distortions and synthetic compressions and adopt effective strategies to pre-train the two branches of the network. Specifically, we transfer the knowledge learned from in-the-wild images to account for authentic distortions by utilizing a pre-trained deep convolutional neural network (CNN) to provide meaningful initializations. Meanwhile, we build a CNN for synthetic compressions and pre-train it on a dataset including synthetic compressed images. Subsequently, we bilinearly pool these two sets of features as the image representation. The overall network is fine-tuned on an elaborately-designed auxiliary dataset, which is annotated by a reliable objective quality metric. Furthermore, we integrate the output of the authentic-distortion-aware branch with that of the overall network following a two-step prediction manner to boost the prediction performance, which can be applied in the distorted-then-compressed scenario when the reference image is available. Extensive experimental results on several databases especially on the LIVE Wild Compressed Picture Quality Database show that the proposed method achieves state-of-the-art performance with good generalizability and moderate computational complexity.     

Screen content image quality assessment based on convolutional neural networks

Screen content image (SCI) is a composite image including textual and pictorial regions resulting in many difficulties in image quality assessment (IQA). Large SCIs are divided into image patches to increase training samples for CNN training of IQA model, and this brings two problems: (1) local quality of each image patch is not equal to subjective differential mean opinion score (DMOS) of an entire image; (2) importance of different image patches is not same for quality assessment. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN) for assessing the perceptual quality of SCIs. Our model conducts two designs solving problems which benefits from two strategies. For the first strategy, to imitate full-reference (FR) CNN-based model behavior, a CNN-based model is designed for both FR and NR IQA, and performance of NR-IQA part improves when the image patch scores predicted by FR-IQA part are adopted as the ground-truth to train NR-IQA part. For the second strategy, image patch qualities of one entire SCI are fused to obtain the SCI quality with an adaptive weighting method taking account the effect of the different image patch contents. Experimental results verify that our model outperforms all test NR IQA methods and most FR IQA methods on the screen content image quality assessment database (SIQAD). On the cross-database evaluation, the proposed method outperforms the existing NR IQA method in terms of at least 2.4 percent in PLCC and 2.8 percent in SRCC, which shows high generalization ability and high effectiveness of our model.     

A weighted full-reference image quality assessment based on visual saliency

In full reference image quality assessment (IQA), the images without distortion are usually employed as reference, while the structures in both reference images and distorted images are ignored and all pixels are equally treated. In addition, the role of human visual system (HVS) is not taken account into subjective IQA metric. In this paper, a weighted full-reference image quality metric is proposed, where a weight imposed on each pixel indicates its importance in IQA. Furthermore, the weights can be estimated via visual saliency computation, which can approximate the subjective IQA via exploiting the HVS. In the experiments, the proposed metric is compared with several objective IQA metrics on LIVE release 2 and TID 2008 database. The results demonstrate that SROCC and PLCC of the proposed metric are 0.9647 and 0.9721, respectively,which are higher than other methods and it only takes 427.5 s, which is lower than that of most other methods.     

Reduced-reference image quality assessment using moment method

Reduced-reference image quality assessment (RR IQA) aims to evaluate the perceptual quality of a distorted image through partial information of the corresponding reference image. In this paper, a novel RR IQA metric is proposed by using the moment method. We claim that the first and second moments of wavelet coefficients of natural images can have approximate and regular change that are disturbed by different types of distortions, and that this disturbance can be relevant to human perceptions of quality. We measure the difference of these statistical parameters between reference and distorted image to predict the visual quality degradation. The introduced IQA metric is suitable for implementation and has relatively low computational complexity. The experimental results on Laboratory for Image and Video Engineering (LIVE) and Tampere Image Database (TID) image databases indicate that the proposed metric has a good predictive performance.     

No reference image blurriness assessment with local binary patterns

In this paper, we put forward an effective and efficient no reference image blurriness assessment metric on the basis of local binary pattern (LBP) features. In this proposal, we reveal that part of the LBP histogram bins present monotonously with the degree of blurriness. The proposed method contains the following steps. Firstly, the LBP maps of an input image are extracted with multiple radiuses. And then, the frequency of pattern histogram is analyzed before part of bins are chosen as the features. In addition, we also take the entropy of these bins as another feature. Finally, we learn the extracted features to predict the image blurriness score. Validation of the proposed method is conducted on the blurred images of LIVE-II, CSIQ, TID2008, TID2013, LIVE3D IQA Phase I and LIVE3D IQA Phase II. Experimental results demonstrate that compared with the state-of-the-art image quality assessment (IQA) methods, the proposed algorithm has notable advantage in correlation with subjective perception and computational complexity.     

No-reference image quality assessment based on spatial and spectral entropies

We develop an efficient general-purpose no-reference (NR) image quality assessment (IQA) model that utilizes local spatial and spectral entropy features on distorted images. Using a 2-stage framework of distortion classification followed by quality assessment, we utilize a support vector machine (SVM) to train an image distortion and quality prediction engine. The resulting algorithm, dubbed Spatial–Spectral Entropy-based Quality (SSEQ) index, is capable of assessing the quality of a distorted image across multiple distortion categories. We explain the entropy features used and their relevance to perception and thoroughly evaluate the algorithm on the LIVE IQA database. We find that SSEQ matches well with human subjective opinions of image quality, and is statistically superior to the full-reference (FR) IQA algorithm SSIM and several top-performing NR IQA methods: BIQI, DIIVINE, and BLIINDS-II. SSEQ has a considerably low complexity. We also tested SSEQ on the TID2008 database to ascertain whether it has performance that is database independent.     

Unified quality assessment of natural and screen content images via adaptive weighting on double scales

In multimedia communication systems, digital images may contain various visual contents, among which the natural scene images (NSIs) and screen content images (SCIs) are two important and common types. The existing full-reference image quality assessment (IQA) metrics are designed for only one type of images, but cannot precisely perceive the visual quality of another type. It is still unclear what the different characteristics are between NSIs and SCIs resulting in this failure. Inspired by some psychological studies, we figure out that it is due to the different structural scale levels between NSIs and SCIs. Given this observation, this paper introduces the gradient degradation of Gaussians (GDoG) to analyze the images’ structural scale level, proposes a fast unified IQA index for both NSIs and SCIs by incorporating an adaptive weighting strategy on double scales. Experimental results conducted on several databases verify the effectiveness and efficiency of the proposed unified IQA index for both types of images, also demonstrate that the adaptive weighting strategy based on GDoG can improve the existing models for cross-content-type images.     

Image quality assessment based on regions of interest

Most methods in the literature of image quality assessment (IQA) use whole image information for measuring image quality. However, human perception does not always use this criterion to assess the quality of images. Individuals usually provide their opinions by considering only some parts of an image, called regions of interest. Based on this hypothesis, in this research work, a segmentation technique is initially employed to obtain a bi-level image map composed of the foreground and background information. A patch selection strategy is then proposed to choose some particular patches based on the foreground information as the regions of interest for IQA. Three recent IQA methods in the literature are considered to demonstrate the improvement in IQA when using only the extracted regions of interest. To evaluate the impact of the proposed patch selection strategy in various IQA metrics, three publicly available datasets were used for experiments. Experimental results have revealed that our proposal, based on the regions of interest, can improve quality measures of three IQA methods.     

图像质量评价方法研究进展

图像质量评价是图像处理领域的研究热点。该文综合论述了图像质量的主观和客观评价方法,重点阐述了单视点图像质量的客观评价方法。对目前比较常用的峰值信噪比和均方误差全参考评价算法进行了分析并指出其存在的问题。然后,对基于误差敏感度和基于结构相似度的评价算法进行了论述和分析,并对质降和无参考评价方法进行了综述。根据视点的个数,图像质量评价可分为对传统单视点图像和立体图像的评价。该文还对立体图像质量评价算法进行了分析讨论。最后,就图像质量评价算法的进一步发展提出了若干技术与研究方向的展望。     

互补色小波域图像质量盲评价方法

图像色彩空间的RGB通道具有密切的关系,图像质量的改变会改变这样的关系.然而传统图像质量评价方法大多基于灰度图像统计特性,忽略了颜色通道间关系信息.为充分利用颜色信息,本文基于新近提出的互补色小波变换提出一种图像质量盲评价方法.文章建立了图像互补色域自然场景统计、多尺度和方向性能量分布等模型.分析表明：这些模型不仅涵盖了传统灰度方法所能描述的信息,而且还能借助于互补色来有效表示彩色图像各通道之间的信息联系,提供表征图像质量的一组高效特征.基于这些特征,我们提出的图像质量盲评价的方法能有效提取图像的失真统计特征,能给出与人眼主观评价图像质量结果保持高度一致、优于现有文献报道盲方法、且可与非盲（全参考）方法相比拟的评价结果.     

Image quality assessment based on Prewitt magnitude

The goal of image quality assessment (IQA) research is to use computational models to calculate the quality of images consistently with subjective evaluations. In this paper, we propose a new image quality assessment (IQA) algorithm by combining Prewitt magnitude and regional mutual information (RMI) in HSV color space. The Prewitt operator is usually used for edge detection and can extract vertical edge more accurately than other operators. The HSV color space encapsulates information about a color in terms that are more natural and intuitive to humans. The proposed method PMRMI first transforms reference and distorted images from RGB color space into HSV color space and Prewitt magnitude is introduced to extract key edge features of each channel. Then the regional mutual information is calculated to measure the similarity of the two images. After that, a weighting method is utilized for better consistency with subjective evaluations. Therefore we get a single quality score. Experiments on various image distortion types demonstrate that the proposed algorithm can achieve better consistency with the subjective evaluations than PSNR and SSIM.     

基于空谱联合的多假设预测高光谱图像压缩感知重构算法

为充分利用高光谱图像的空间相关性和谱间相关性,该文提出一种基于空谱联合的多假设预测压缩感知重构算法。将高光谱图像分组为参考波段图像和非参考波段图像,参考波段图像利用光滑Landweber投影算法重构,对于非参考波段图像,引入空谱联合的多假设预测模型,提高重构精度。非参考波段图像中每个图像块的预测值不仅来自非参考波段图像未经预测的初始重构值的相邻图像块,而且来自参考波段重构图像相应位置及其邻近的图像块,利用预测值得到测量域中的残差,然后对残差进行重构并对预测值进行修正,此残差比原图像更稀疏,且算法采用迭代方式提高重构图像的精度。借助Tikhonov正则化方法求解多假设预测的权重系数,并基于结构相似性判断是否改变多假设预测搜索窗口大小,最后利用交叉验证计算重构算法终止迭代的判据参数。实验结果表明,所提算法优于仅利用空间相关性或谱间相关性进行预测和不预测的重构算法,其重构图像的峰值信噪比提高2 dB以上。     

Color image quality assessment based on sparse representation and reconstruction residual

Image quality assessment (IQA) is a fundamental problem in image processing. While in practice almost all images are represented in the color format, most of the current IQA metrics are designed in gray-scale domain. Color influences the perception of image quality, especially in the case where images are subject to color distortions. With this consideration, this paper presents a novel color image quality index based on Sparse Representation and Reconstruction Residual (SRRR). An overcomplete color dictionary is first trained using natural color images. Then both reference and distorted images are represented using the color dictionary, based on which two feature maps are constructed to measure structure and color distortions in a holistic manner. With the consideration that the feature maps are insensitive to image contrast change, the reconstruction residuals are computed and used as a complementary feature. Additionally, luminance similarity is also incorporated to produce the overall quality score for color images. Experiments on public databases demonstrate that the proposed method achieves promising performance in evaluating traditional distortions, and it outperforms the existing metrics when used for quality evaluation of color-distorted images.     

An ensemble image quality assessment algorithm based on deep feature clustering

To effective handle image quality assessment (IQA) where the images might be with sophisticated characteristics, we proposed a deep clustering-based ensemble approach for image quality assessment toward diverse images. Our approach is based on a convolutional DAE-aware deep architecture. By leveraging a layer-by-layer pre-training, our proposed deep feature clustering architecture extracted a fixed number of high-level features at first. Then, it optimally splits image samples into different clusters by using the fuzzy C-means algorithm based on the engineered deep features. For each cluster, we simulated a particular fitting function of differential mean opinion scores with each assessed image’s PSNR, SIMM, and VIF scores. Comprehensive experimental results on TID2008, TID2013 and LIVE databases have demonstrated that compared to the state-of-the-art counterparts, our proposed IQA method can reflect the subjective quality of images more accurately by seamlessly integrating the advantages of three existed IQA methods.