MLSIM: A Multi-Level Similarity index for image quality assessment

Image quality assessment (IQA) is of great importance to numerous image processing applications, and various methods have been proposed for it. In this paper, a Multi-Level Similarity (MLSIM) index for full reference IQA is proposed. The proposed metric is based on the fact that human visual system (HVS) distinguishes the quality of an image mainly according to the details given by low-level gradient information. In the proposed metric, the Prewitt operator is first utilized to get gradient information of both reference and distorted images, then the gradient information of reference image is segmented into three levels (3LSIM) or two levels (2LSIM), and the gradient information of distorted image is segmented by the corresponding regions of reference image, therefore we get multi-level information of these two images. Riesz transform is utilized to get corresponding features of different levels and the corresponding 1st-order and 2nd-order coefficients are combined together by regional mutual information (RMI) and weighted to obtain a single quality score. Experimental results demonstrate that the proposed metric is highly consistent with human subjective evaluations and achieves good performance.     

Image quality assessment based on deep learning with FPGA implementation

To improve image quality assessment (IQA) methods, it is believable that we have to extract image features that are highly representative to human visual perception. In this paper, we propose a novel IQA algorithm by leveraging an optimized convolutional neural network architecture that is designed to automatically extract discriminative image quality features. And the IQA algorithm uses local luminance coefficient normalization, dropout and the other advanced techniques to further improve the network learning ability. At the same time the proposed IQA algorithm is implemented based on Field Programmable Gate Array (FPGA) and further evaluated on two public databases. Extensive experimental results have shown that our method outperforms many existing IQA algorithms in terms of accuracy and speed.     

No-reference image quality assessment using Prewitt magnitude based on convolutional neural networks

No-reference image quality assessment is of great importance to numerous image processing applications, and various methods have been widely studied with promising results. These methods exploit handcrafted features in the transformation or space domain that are discriminated for image degradations. However, abundant a priori knowledge is required to extract these handcrafted features. The convolutional neural network (CNN) is recently introduced into the no-reference image quality assessment, which integrates feature learning and regression into one optimization process. Therefore, the network structure generates an effective model for estimating image quality. However, the image quality score obtained by the CNN is based on the mean of all of the image patch scores without considering the human visual system, such as edges and contour of images. In this paper, we combine the CNN and the Prewitt magnitude of segmented images and obtain the image quality score using the mean of all the products of the image patch scores and weights based on the result of segmented images. Experimental results on various image distortion types demonstrate that the proposed algorithm achieves good performance.     

基于亮度统计的无参考图像质量评价

基于亮度均值减损对比归一化(MSCN) 系数统计特性及其8方 向邻域系数间的相关性,提出了一种通用无参考图像质量评价方法.首先,分别利用非 对称广义高斯分布(AGGD)模型拟合MSCN系数及其8方 向邻域系数,并估计 相应AGGD 模型参数作为亮度统计特征;其次,计算8方向邻域MSCN系数间的互信息(MI),作为描述方向相 关性的统计特征;进而,分别利用支持向量回归机(SVR)和支 持向量分类机(SVC)构建无参考图像质量评价模型和图像失真类型识别模型; 最后, 在LIVE 等图像质量 评价数据库上进行了算法与DMOS的相关性、失真类型识别、模型 鲁棒性及计算复杂性等方面的实验。实 验结果表明,本文方法的评价结果与人类主观评价具有高度的一致性,在LIVE图像质量评 价数据库上的斯 皮尔曼等级相关系数(SROCC)和皮尔逊线性相关系数 (PLCC)均在0.945以上;而且,图像失真 类型识别模型的识别准确率也高达到92.95%,明显高于 当今主流无参考图像质量评价方法。     

Blind image quality assessment in the contourlet domain

No-reference/blind image quality assessment (NR-IQA/BIQA) algorithms play an important role in image evaluation, as they can assess the quality of an image automatically, only using the distorted image whose quality is being assessed. Among the existing NR-IQA/BIQA methods, natural scene statistic (NSS) models which can be expressed in different bandpass domains show good consistency with human subjective judgments of quality.In this paper, we create new ‘quality-aware’ features: the energy differences of the sub-band coefficients across scales via contourlet transform, and propose a new NR-IQA/BIQA model that operates on natural scene statistics in the contourlet domain. Prior to applying the contourlet transform, we apply two preprocessing steps that help to create more information-dense, low-entropy representations. Specifically, we transform the picture into the CIELAB color space and gradient magnitude map. Then, a number of ‘quality-aware’ features are discovered in the contourlet transform domain: the energy of the sub-band coefficients within scales, and the energy differences between scales, as well as measurements of the statistical relationships of pixels across scales. A detailed analysis is conducted to show how different distortions affect the statistical characteristics of these features, and then features are fed to a support vector regression (SVR) model which learns to predict image quality. Experimental results show that the proposed method has high linearity against human subjective perception, and outperforms the state-of-the-art NR-IQA models.     

C-DIIVINE: No-reference image quality assessment based on local magnitude and phase statistics of natural scenes

It is widely known that the wavelet coefficients of natural scenes possess certain statistical regularities which can be affected by the presence of distortions. The DIIVINE (Distortion Identification-based Image Verity and Integrity Evaluation) algorithm is a successful no-reference image quality assessment (NR IQA) algorithm, which estimates quality based on changes in these regularities. However, DIIVINE operates based on real-valued wavelet coefficients, whereas the visual appearance of an image can be strongly determined by both the magnitude and phase information.In this paper, we present a complex extension of the DIIVINE algorithm (called C-DIIVINE), which blindly assesses image quality based on the complex Gaussian scale mixture model corresponding to the complex version of the steerable pyramid wavelet transform. Specifically, we applied three commonly used distribution models to fit the statistics of the wavelet coefficients: (1) the complex generalized Gaussian distribution is used to model the wavelet coefficient magnitudes, (2) the generalized Gaussian distribution is used to model the coefficients׳ relative magnitudes, and (3) the wrapped Cauchy distribution is used to model the coefficients׳ relative phases. All these distributions have characteristic shapes that are consistent across different natural images but change significantly in the presence of distortions. We also employ the complex wavelet structural similarity index to measure degradation of the correlations across image scales, which serves as an important indicator of the subbands׳ energy distribution and the loss of alignment of local spectral components contributing to image structure. Experimental results show that these complex extensions allow C-DIIVINE to yield a substantial improvement in predictive performance as compared to its predecessor, and highly competitive performance relative to other recent no-reference algorithms.     

Image quality assessment based on adaptive multiple Skyline query

Non-reference image quality assessment has attracted great emphasis in recent years. Traditional image quality assessment algorithms based on structural similarity cannot make full use of the image gradient features, and the contrast similarity features often ignore the consistency of continuous color blocks within the image, which leads to large discrepancy between the evaluation result and the subjective judgment of human vision system. In this paper, we propose a deep model for image quality assessment where the spatial and visual features of image are both considered. For better feature fusion, we design an adaptive multiple Skyline query algorithm named MSFF, which takes as input multiple features of images, and learns the feature weights through end-to-end training. Extensive experiments on image quality assessment tasks prove that the proposed model exhibits superior performance compared with existing solutions.     

Image quality assessment based on self-supervised learning and knowledge distillation

Deep neural networks have achieved great success in a wide range of machine learning tasks due to their excellent ability to learn rich semantic features from high-dimensional data. Deeper networks have been successful in the field of image quality assessment to improve the performance of image quality assessment models. The success of deep neural networks majorly comes along with both big models with hundreds of millions of parameters and the availability of numerous annotated datasets. However, the lack of large-scale labeled data leads to the problems of over-fitting and poor generalization of deep learning models. Besides, these models are huge in size, demanding heavy computation power and failing to be deployed on edge devices. To deal with the challenge, we propose an image quality assessment based on self-supervised learning and knowledge distillation. First, the self-supervised learning of soft target prediction given by the teacher network is carried out, and then the student network is jointly trained to use soft target and label on knowledge distillation. Experiments on five benchmark databases show that the proposed method is superior to the teacher network and even outperform the state-of-the-art strategies. Furthermore, the scale of our model is much smaller than the teacher model and can be deployed in edge devices for smooth inference.     

基于改进Prewitt算子的最佳阈值选择

边缘作为图像的一种基本特征,在图像识别、图像分割等领域中有着广泛的应用。文中在传统的Prewitt算子改进的基础上,提出利用边缘的最大后验概率估计,找出边缘检测的最佳阈值。并将实验结果与Sobel,Canny等算子进行了对比。结果显示,文中算法能检测出最佳阈值。     

基于多尺度边缘结构相似性的图像质量评价

基于结构相似度(SSIM)的图像质量评价方法是从视觉区域提取图像的结构性信息,但在评价模糊较严重的图像时存在其局限性,因此本文将图像边缘和SSIM相结合,提出了基于多尺度边缘结构相似度(MESS)的图像质量评价方法。实验结果表明,由于MESS考虑了边缘信息对于人眼感知结构信息的重要性,评价结果比SSIM更加符合人眼视觉感知特性。     

基于波前像差的图像质量评价方法研究

图像质量评价的研究已成为图像信息工程的基础技术之一。由于图像的最终接受者是人,所以评价图像质量应反映出人类的主观视觉感知。为构造一种符合人眼视觉特性的图像质量评价方法,利用点扩散函数针对人眼建立了含有波前像差信息的图像视觉评价模型,并用此模型分别对添加不同噪声的图像进行图像质量评价。实验结果表明,该方法是可行的、有效的,不同的人眼对同一幅图像的评价存在有差异,人眼波前像差越小观察到的图像越清晰。该方法不仅能够在评价图像质量时准确反映人眼的主观感知,而且能够直观地呈现不同人眼实际看到的图像。     

Image quality assessment for contrast enhancement evaluation

Although the contrast enhancement (CE) is a great challenge, few efforts have been conducted on evaluation of the contrast changes. In this paper, we propose a contrast-changed image quality (CCIQ) metric including a local index, named edge-based contrast criterion (ECC), and three global measures. In the global measures, entropy, correlation coefficient and mean intensity are exploited. Particle swarm optimization (PSO) algorithm is utilized for obtaining an optimal combination of these quantities. Although the presented method utilizes the original image, it cannot be considered as a full-reference metric, since the original image is not regarded to have the ideal quality. Hence, it can be concluded that it follows a new paradigm in image quality assessment. Experimental results on the three benchmark databases, CID2013, TID2013 and TID2008 demonstrate that the proposed metric outperforms the-state-of-the-art methods.     

基于Harris角点和质量评价的图像篡改检测

复制粘贴是图像篡改的常用手段,经典伪造检测方法将所有重叠图像块作为检测区域,算法时间复杂度高,邻近区域误检率大.为解决以上问题,提出将扩展Harris角点作为检测区域以降低算法复杂度.由于图像经过复制粘贴检测后往往会进行模糊、添加杂色、色彩调整等后处理,使得图像质量下降,本文结合NR图像质量评价给出更为准确的检测结果.实验证明本文算法对经过模糊、添加杂色、JPEG压缩等后处理的复制粘贴图像检测效率高,检测效果好.     

基于模糊度和噪声水平的图像质量评价方法

针对图像质量评价的重要性,提出了一种新的无参考图像质量客观评价方式。算法考虑了模糊度与噪声水平两方面,用平均边缘宽度衡量图像的模糊度,通过比较去噪前后的图像预测图像受噪声污染的程度,最后通过两者的综合作为无参考图像质量评价指标。实验结果表明:将模糊度和噪声评价相结合,具有很强的抗噪性和广泛的适用范围;与峰值信噪比(PSNR)和结构类似性(SSIM)等算法比较,本文算法可以很好地区分各种失真类型图像的质量好坏,其结果接近人眼的主观感受。     

基于OpenCL的Prewitt算法的并行实现

Prewitt算法是数字图像分割中最常用的边缘检测算法。采用传统CPU上的串行方法实现该算法需要较大的计算量、耗时较长,因此,通过GPU对其进行性能加速有着重要的意义。然而由于GPU硬件体系结构的差异性,跨平台移植是一件非常困难的工作。针对上述问题,提出了一种基于OpenCL异构框架的Prewitt图像边缘检测并行算法。实验结果表明,该并行算法比CPU上的串行算法运行速度快,加速比可达30倍,有效地提高了大规模数据处理的效率,可移植性好,具有较高的应用价值。     

Screen content image quality assessment based on convolutional neural networks

Screen content image (SCI) is a composite image including textual and pictorial regions resulting in many difficulties in image quality assessment (IQA). Large SCIs are divided into image patches to increase training samples for CNN training of IQA model, and this brings two problems: (1) local quality of each image patch is not equal to subjective differential mean opinion score (DMOS) of an entire image; (2) importance of different image patches is not same for quality assessment. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN) for assessing the perceptual quality of SCIs. Our model conducts two designs solving problems which benefits from two strategies. For the first strategy, to imitate full-reference (FR) CNN-based model behavior, a CNN-based model is designed for both FR and NR IQA, and performance of NR-IQA part improves when the image patch scores predicted by FR-IQA part are adopted as the ground-truth to train NR-IQA part. For the second strategy, image patch qualities of one entire SCI are fused to obtain the SCI quality with an adaptive weighting method taking account the effect of the different image patch contents. Experimental results verify that our model outperforms all test NR IQA methods and most FR IQA methods on the screen content image quality assessment database (SIQAD). On the cross-database evaluation, the proposed method outperforms the existing NR IQA method in terms of at least 2.4 percent in PLCC and 2.8 percent in SRCC, which shows high generalization ability and high effectiveness of our model.     

Color image quality assessment based on sparse representation and reconstruction residual

Image quality assessment (IQA) is a fundamental problem in image processing. While in practice almost all images are represented in the color format, most of the current IQA metrics are designed in gray-scale domain. Color influences the perception of image quality, especially in the case where images are subject to color distortions. With this consideration, this paper presents a novel color image quality index based on Sparse Representation and Reconstruction Residual (SRRR). An overcomplete color dictionary is first trained using natural color images. Then both reference and distorted images are represented using the color dictionary, based on which two feature maps are constructed to measure structure and color distortions in a holistic manner. With the consideration that the feature maps are insensitive to image contrast change, the reconstruction residuals are computed and used as a complementary feature. Additionally, luminance similarity is also incorporated to produce the overall quality score for color images. Experiments on public databases demonstrate that the proposed method achieves promising performance in evaluating traditional distortions, and it outperforms the existing metrics when used for quality evaluation of color-distorted images.     

Stereoscopic image quality assessment considering visual mechanism and multi-loss constraints

In this paper, a convolutional neural network (CNN) with multi-loss constraints is designed for stereoscopic image quality assessment (SIQA). A stereoscopic image not only contains monocular information, but also provides binocular information which is as identically crucial as the former. So we take the image patches of left-view images, right-view images and the difference images as the inputs of the network to utilize monocular information and binocular information. Moreover, we propose a method to obtain proxy label of each image patch. It preserves the quality difference between different regions and views. In addition, the multiple loss functions with adaptive loss weights are introduced in the network, which consider both local features and global features and constrain the feature learning from multiple perspectives. And the adaptive loss weights also make the multi-loss CNN more flexible. The experimental results on four public SIQA databases show that the proposed method is superior to other existing SIQA methods with state-of-the-art performance.     

No-reference image quality assessment using structural activity

Presuming that human visual perception is highly sensitive to the structural information in a scene, we propose the concept of structural activity (SA) together with a model of SA indicator in a new framework for no-reference (NR) image quality assessment (QA) in this study. The proposed framework estimates image quality based on the quantification of the SA information of different visual significance. We propose some alternative implementations of SA indicator in this paper as examples to demonstrate the effectiveness of the SA-motivated framework. Comprehensive testing demonstrates that the model of SA indicator exhibits satisfactory performance in comparison with subjective quality scores as well as representative full-reference (FR) image quality measures.