MLSIM: A Multi-Level Similarity index for image quality assessment

Image quality assessment (IQA) is of great importance to numerous image processing applications, and various methods have been proposed for it. In this paper, a Multi-Level Similarity (MLSIM) index for full reference IQA is proposed. The proposed metric is based on the fact that human visual system (HVS) distinguishes the quality of an image mainly according to the details given by low-level gradient information. In the proposed metric, the Prewitt operator is first utilized to get gradient information of both reference and distorted images, then the gradient information of reference image is segmented into three levels (3LSIM) or two levels (2LSIM), and the gradient information of distorted image is segmented by the corresponding regions of reference image, therefore we get multi-level information of these two images. Riesz transform is utilized to get corresponding features of different levels and the corresponding 1st-order and 2nd-order coefficients are combined together by regional mutual information (RMI) and weighted to obtain a single quality score. Experimental results demonstrate that the proposed metric is highly consistent with human subjective evaluations and achieves good performance.     

A weighted full-reference image quality assessment based on visual saliency

In full reference image quality assessment (IQA), the images without distortion are usually employed as reference, while the structures in both reference images and distorted images are ignored and all pixels are equally treated. In addition, the role of human visual system (HVS) is not taken account into subjective IQA metric. In this paper, a weighted full-reference image quality metric is proposed, where a weight imposed on each pixel indicates its importance in IQA. Furthermore, the weights can be estimated via visual saliency computation, which can approximate the subjective IQA via exploiting the HVS. In the experiments, the proposed metric is compared with several objective IQA metrics on LIVE release 2 and TID 2008 database. The results demonstrate that SROCC and PLCC of the proposed metric are 0.9647 and 0.9721, respectively,which are higher than other methods and it only takes 427.5 s, which is lower than that of most other methods.     

Full-reference IPTV image quality assessment by deeply learning structural cues

Image quality assessment (IQA) attempts to quantify the quality-aware visual attributes perceived by humans. They can be divided into subjective and objective image quality assessment. Subjective IQA algorithms rely on human judgment of image quality, where the human visual perception functions as the dominant factor However, they cannot be widely applied in practice due to the heavy reliance on different individuals. Motivated by the fact that objective IQA largely depends on image structural information, we propose a structural cues-based full-reference IPTV IQA algorithm. More specifically, we first design a grid-based object detection module to extract multiple structural information from both the reference IPTV image (i.e., video frame) and the test one. Afterwards, we propose a structure-preserved deep neural networks to generate the deep representation for each IPTV image. Subsequently, a new distance metric is proposed to measure the similarity between the reference image and the evaluated image. A test IPV image with a small calculated distance is considered as a high quality one. Comprehensive comparative study with the state-of-the-art IQA algorithms have shown that our method is accurate and robust.     

Reduced reference image quality assessment based on statistics in empirical mode decomposition domain

This paper deals with the image quality assessment (IQA) task using a natural image statistics approach. A reduced reference (RRIQA) measure based on the bidimensional empirical mode decomposition is introduced. First, we decompose both, reference and distorted images, into intrinsic mode functions (IMF) and then we use the generalized Gaussian density (GGD) to model IMF coefficients of the reference image. Finally, we measure the impairment of a distorted image by fitting error between the IMF coefficients histogram of the distorted image and the estimated IMF coefficients distribution of the reference image, using the Kullback–Leibler divergence (KLD). Furthermore, to predict the quality, we propose a new support vector machine-based (SVM) classification approach as an alternative to logistic function-based regression. In order to validate the proposed measure, three benchmark datasets are involved in our experiments. Results demonstrate that the proposed metric compare favorably with alternative solutions for a wide range of degradation encountered in practical situations.     

一个图像质量盲评估的统计测度

本文提出了一个图像质量盲评估的统计测度.该测度首先根据自然图像的统计性质与失真图像的模型,实现对图像小波系数分布参数的盲估计;再利用估计的分布参数来计算失真图像与参考图像之间的互信息,以量化失真图像对参考图像的保真度,进而实现对图像质量的评估.本文提出的测度避免了对参考图像的依赖,且克服了现有图像质量盲评估对特征选择与提取、机器学习等过程的依赖.LIVE图像质量评估数据库的总体评估结果表明：本文提出的盲评估统计测度对图像质量评估结果与数据库的主观评估结果高度一致,且优于文献中报道的盲评估测度.     

AdaBoost neural network and cyclopean view for no-reference stereoscopic image quality assessment

Stereoscopic imaging is widely used in many fields. In many scenarios, stereo images quality could be affected by various degradations, such as asymmetric distortion. Accordingly, to guarantee the best quality of experience, robust and accurate reference-less metrics are required for quality assessment of stereoscopic content. Most existing stereo no-reference Image Quality Assessment (IQA) models are not consistent with asymmetrical distortions. This paper presents a new no-reference stereoscopic image quality assessment metric using a human visual system (HVS) modeling and an advanced machine-learning algorithm. The proposed approach consists of two stages. In the first stage, cyclopean image is constructed considering the presence of binocular rivalry in order to cover the asymmetrically distorted part. In the second stage, gradient magnitude, relative gradient magnitude, and gradient orientation are extracted. These are used as a predictive source of information for the quality. In order to obtain the best overall performance against different databases, Adaptive Boosting (AdaBoost) idea of machine learning combined with artificial neural network model has been adopted. The benchmark LIVE 3D phase-I, phase-II, and IRCCyN/IVC 3D databases have been used to evaluate the performance of the proposed approach. Experimental results have demonstrated that the proposed metric performance achieves high consistency with subjective assessment and outperforms the blind stereo IQA over various types of distortion.     

Perceptual visual quality assessment using deeply-learned gaze shifting kernel

Image quality assessment (IQA) is a useful technique in computer vision and machine intelligence. It is widely applied in image retrieval, image clustering and image recognition. IQA algorithms generally rely on human visual system (HVS), which can reflect how human perceive salient regions in the image. In this paper, we leverage both low-level features and high-level semantic features to select salient regions, which will be concatenated to form GSPs by the designed saliency-constraint algorithm to mimic human visual system. We design an enhanced IQA index based on the GSPs to calculate the simialrity between reference image and test image to achieve image quality assessment. Experiments demonstrate that our IQA method can achieve satisfactory performance.     

Toward an unsupervised blind stereoscopic 3D image quality assessment using joint spatial and frequency representations

Existing blind stereoscopic 3D (S3D) image quality assessment (IQA) metrics usually require supervised learning methods to predict S3D image quality, which limits their applicability in practice. In this paper, we propose an unsupervised blind S3D IQA metric that utilizes the joint spatial and frequency representations of visual perception. The metric proposed in this work was inspired by the binocular visual mechanism; furthermore, it is unsupervised and does not require subject-rated samples for training. To be more specific, first, the various binocular quality-aware features in spatial and frequency domains are extracted from the monocular and cyclopean views of natural S3D image patches. Subsequently, these features are utilized to establish a pristine multivariate Gaussian (MVG) model to characterize natural S3D image regularities. Finally, with the learned MVG model, the final quality score for a distorted S3D image can be yielded using a Bhattacharyya-like distance. Our experimental results illustrate that, compared to related existing metrics, the devised metric achieves competitive prediction performance.     

No-reference image quality assessment using statistical characterization in the shearlet domain

Image and video quality measurements are crucial for many applications, such as acquisition, compression, transmission, enhancement, and reproduction. Nowadays, no-reference (NR) image quality assessment (IQA) methods have drawn extensive attention because it does not rely on any information of original images. However, most of the conventional NR-IQA methods are designed only for one or a set of predefined specific image distortion types, which are unlikely to generalize for evaluating image/video distorted with other types of distortions. In order to estimate a wide range of image distortions, in this paper, we present an efficient general-purpose NR-IQA algorithm which is based on a new multiscale directional transform (shearlet transform) with a strong ability to localize distributed discontinuities. This is mainly based on distorted natural image that leads to significant variation in the spread discontinuities in all directions. Thus, the statistical property of the distorted image is significantly different from that of natural images in fine scale shearlet coefficients, which are referred to as ‘distorted parts’. However, some ‘natural parts’ are reserved in coarse scale shearlet coefficients. The algorithm relies on utilizing the natural parts to predict the natural behavior of distorted parts. The predicted parts act as ‘reference’ and the difference between the reference and distorted parts is used as an indicator to predict the image quality. In order to achieve this goal, we modify the general sparse autoencoder to serve as a predictor to get the predicted parts from natural parts. By translating the NR-IQA problem into classification problem, the predicted parts and distorted parts are utilized to form features and the differences between them are identified by softmax classifier. The resulting algorithm, which we name SHeArlet based No-reference Image quality Assessment (SHANIA), is tested on several database (LIVE, Multiply Distorted LIVE and TID2008) and shown to be suitable for many common distortions, consistent with subjective assessment and comparable to full-reference IQA methods and state-of-the-art general purpose NR-IQA algorithms.     

Color image quality assessment based on sparse representation and reconstruction residual

Image quality assessment (IQA) is a fundamental problem in image processing. While in practice almost all images are represented in the color format, most of the current IQA metrics are designed in gray-scale domain. Color influences the perception of image quality, especially in the case where images are subject to color distortions. With this consideration, this paper presents a novel color image quality index based on Sparse Representation and Reconstruction Residual (SRRR). An overcomplete color dictionary is first trained using natural color images. Then both reference and distorted images are represented using the color dictionary, based on which two feature maps are constructed to measure structure and color distortions in a holistic manner. With the consideration that the feature maps are insensitive to image contrast change, the reconstruction residuals are computed and used as a complementary feature. Additionally, luminance similarity is also incorporated to produce the overall quality score for color images. Experiments on public databases demonstrate that the proposed method achieves promising performance in evaluating traditional distortions, and it outperforms the existing metrics when used for quality evaluation of color-distorted images.     

基于非局部梯度的图像质量评价算法

图像质量评价研究的目标在于模拟人类视觉系统对图像质量的感知过程,构建与主观评价结果尽可能一致的客观评价算法。现有的很多算法都是基于局部结构相似设计的,但人对图像的主观感知是高级的、语义的过程,而语义信息本质上是非局部的,因此图像质量评价应该考虑图像的非局部信息。该文突破了经典的基于局部信息的算法框架,提出一种基于非局部信息的框架,并在此框架内构建了一种基于非局部梯度的图像质量评价算法,该算法通过度量参考图像与失真图像的非局部梯度之间的相似性来预测图像质量。在公开测试数据库TID2008, LIVE, CSIQ上的数值实验结果表明,该算法能获得较好的评价效果。

     

适应于图像复原的无参考图像质量评价方法

由于图像降质过程的复杂性、成像获取条件限制,以及图像本身的复杂性和图像复原过程的病态性,图像复原解大多都是近似的或畸变的,一种适应于图像复原质量评价的计算方法将大大提升图像复原的应用范围。针对图像复原过程的病态性,提出了一种针对图像复原图像质量评价的计算方法,该算法通过在图像质量算子中引入图像相似矩阵和图像复原趋势矩阵,使其能适应复原对于图像结构或噪声结构的变化。该图像质量评价算子计算无需参考图像,可以很好地反映图像的模糊程度和噪声程度,并且计算简单。实验证明了该图像质量评价算子的有效性。     

No-reference image quality assessment based on spatial and spectral entropies

We develop an efficient general-purpose no-reference (NR) image quality assessment (IQA) model that utilizes local spatial and spectral entropy features on distorted images. Using a 2-stage framework of distortion classification followed by quality assessment, we utilize a support vector machine (SVM) to train an image distortion and quality prediction engine. The resulting algorithm, dubbed Spatial–Spectral Entropy-based Quality (SSEQ) index, is capable of assessing the quality of a distorted image across multiple distortion categories. We explain the entropy features used and their relevance to perception and thoroughly evaluate the algorithm on the LIVE IQA database. We find that SSEQ matches well with human subjective opinions of image quality, and is statistically superior to the full-reference (FR) IQA algorithm SSIM and several top-performing NR IQA methods: BIQI, DIIVINE, and BLIINDS-II. SSEQ has a considerably low complexity. We also tested SSEQ on the TID2008 database to ascertain whether it has performance that is database independent.     

CNN Based No-Reference HDR Image Quality Assessment

Motivated by the problems of non-universality and over-reliance on the original reference image in High dynamic range (HDR) Image quality assessment (IQA), a convolutional neural network-based algorithm for no-reference HDR image quality assessment is proposed. The Salience detection by self-resemblance (SDSR) algorithm which extracts the salient regions of the HDR image, is used to simulate the human visual attention mechanism. Then a visual quality perception network for training quality prediction models is designed according to the visual characteristics of luminance and contrast sensitivity. And this network consists of an Error estimation network (Error-net), a Perceptual resistance network (PR-net) and a mixing function. The experimental results indicate that the method proposed has high consistency with subjective perception, and the value of assessment metrics Spearman rank-order correlation coefficient (SROCC), Pearson product-moment correlation coefficient (PLCC) and Root mean square error (RMSE) correspondingly reaches 0.941, 0.910 and 8.176 as well. It is comparable with classic full-reference HDR IQA methods.     

基于局部最大梯度的无参考图像质量评价

图像质量评价在数字图像处理中应用广泛,无参考图像质量评价更是近些年来的研究热点。该文提出一种基于局部结构的无参考图像质量评价方法,该方法首先利用局部梯度选择强边缘区域,然后通过强边缘的信息来评价图像的质量。该方法的创新之处在于：基于局部最大梯度的像素点质量评价;利用强边缘点的局部质量来估计全局图像质量。该方法可以同时评价噪声图像和模糊图像,图像失真越严重,该方法的评价分数就越低。与图像质量评价数据库的主观评价结果比较表明,该文方法与主观评价结果相关性很强,能很好地反映图像质量的视觉感知效果。     

Image quality assessment based on Prewitt magnitude

The goal of image quality assessment (IQA) research is to use computational models to calculate the quality of images consistently with subjective evaluations. In this paper, we propose a new image quality assessment (IQA) algorithm by combining Prewitt magnitude and regional mutual information (RMI) in HSV color space. The Prewitt operator is usually used for edge detection and can extract vertical edge more accurately than other operators. The HSV color space encapsulates information about a color in terms that are more natural and intuitive to humans. The proposed method PMRMI first transforms reference and distorted images from RGB color space into HSV color space and Prewitt magnitude is introduced to extract key edge features of each channel. Then the regional mutual information is calculated to measure the similarity of the two images. After that, a weighting method is utilized for better consistency with subjective evaluations. Therefore we get a single quality score. Experiments on various image distortion types demonstrate that the proposed algorithm can achieve better consistency with the subjective evaluations than PSNR and SSIM.     

No-reference image quality assessment based on non-subsample shearlet transform and natural scene statistics

A novel no-reference (NR) image quality assessment (IQA) method is proposed for assessing image quality across multifarious distortion categories. The new method transforms distorted images into the shearlet domain using a non-subsample shearlet transform (NSST), and designs the image quality feature vector to describe images utilizing natural scenes statistical features:coefficient distribution, energy distribution and structural correlation (SC) across orientations and scales. The final image quality is achieved from distortion classification and regression models trained by a support vector machine (SVM). The experimental results on the LIVE2 IQA database indicate that the method can assess image quality effectively, and the extracted features are susceptive to the category and severity of distortion. Furthermore, our proposed method is database independent and has a higher correlation rate and lower root mean squared error (RMSE) with human perception than other high performance NR IQA methods.