No-reference image quality assessment using statistical characterization in the shearlet domain

Image and video quality measurements are crucial for many applications, such as acquisition, compression, transmission, enhancement, and reproduction. Nowadays, no-reference (NR) image quality assessment (IQA) methods have drawn extensive attention because it does not rely on any information of original images. However, most of the conventional NR-IQA methods are designed only for one or a set of predefined specific image distortion types, which are unlikely to generalize for evaluating image/video distorted with other types of distortions. In order to estimate a wide range of image distortions, in this paper, we present an efficient general-purpose NR-IQA algorithm which is based on a new multiscale directional transform (shearlet transform) with a strong ability to localize distributed discontinuities. This is mainly based on distorted natural image that leads to significant variation in the spread discontinuities in all directions. Thus, the statistical property of the distorted image is significantly different from that of natural images in fine scale shearlet coefficients, which are referred to as ‘distorted parts’. However, some ‘natural parts’ are reserved in coarse scale shearlet coefficients. The algorithm relies on utilizing the natural parts to predict the natural behavior of distorted parts. The predicted parts act as ‘reference’ and the difference between the reference and distorted parts is used as an indicator to predict the image quality. In order to achieve this goal, we modify the general sparse autoencoder to serve as a predictor to get the predicted parts from natural parts. By translating the NR-IQA problem into classification problem, the predicted parts and distorted parts are utilized to form features and the differences between them are identified by softmax classifier. The resulting algorithm, which we name SHeArlet based No-reference Image quality Assessment (SHANIA), is tested on several database (LIVE, Multiply Distorted LIVE and TID2008) and shown to be suitable for many common distortions, consistent with subjective assessment and comparable to full-reference IQA methods and state-of-the-art general purpose NR-IQA algorithms.     

No-reference image quality assessment based on neighborhood co-occurrence matrix

No-reference image quality assessment (NR-IQA) aims to develop models that can predict the quality of distorted image automatically and accurately in the absent of reference image. Previous NR-IQA methods based on natural scene statistics (NSS) always focus on the luminance contrast of image but attach limited attention to pixel-wise relationship. However, human visual system (HVS) is highly adaptive to extract spatial correlation according to relative position within visual field. In this paper, a new approach is proposed for NR-IQA, in which the neighborhood co-occurrence matrix (NCM) is introduced to describe spatial correlation of pixels for quality assessment. The NCM is constructed based on spatial correlation of every pixel and its neighborhood through a mapping to highlight the one-to-many pixel-wise relationship. Moreover, a series of tailored statistical metrics are designed to quantify the unnaturalness extent of NCM effectively, which is combined with others natural scene statistics to predict image quality. Extensive experiments demonstrate the proposed method has superior performance against compared methods, and achieves significant improvements on distortions associated with color or locality.     

基于自然统计特征分布的无参考图像质量评价

针对目前的无参考评价方法无法准确反映人类对图像质量的视觉感知效果,该文提出一种基于自然统计特征分布(DIstribution Characteristics of Natural, DICN)的无参考图像质量评价方法。其原理是用小波变换将图像分解为低频子带和高频子带部分,再将高频子带部分分成 的小块,提取每一子块的幅值和信息熵,并分别计算其分布直方图均值和斜度作为特征,利用支持向量回归思想对特征进行训练,建立5种不同失真类型的质量预测模型。在此基础上,采用支持向量机针对图像特征构造分类器并进行失真判断以确定不同失真的权重,结合5种失真评价模型可得到自然统计特征分布的无参考评价模型。实验结果分析表明,该算法的评价效果优于现有的经典算法,与主观评价具有较好一致性,能够准确反映人类对图像质量的视觉感知效果。     

基于局部最大梯度的无参考图像质量评价

图像质量评价在数字图像处理中应用广泛,无参考图像质量评价更是近些年来的研究热点。该文提出一种基于局部结构的无参考图像质量评价方法,该方法首先利用局部梯度选择强边缘区域,然后通过强边缘的信息来评价图像的质量。该方法的创新之处在于:基于局部最大梯度的像素点质量评价;利用强边缘点的局部质量来估计全局图像质量。该方法可以同时评价噪声图像和模糊图像,图像失真越严重,该方法的评价分数就越低。与图像质量评价数据库的主观评价结果比较表明,该文方法与主观评价结果相关性很强,能很好地反映图像质量的视觉感知效果。     

Multi-layer and Multi-scale feature aggregation for DIBR-Synthesized image quality assessment

Depth-Image-Based Rendering (DIBR) is one of the main fundamental techniques for generating new viewpoints in 3D video applications such as multi-viewpoint video (MVV), free viewpoint video (FVV) and virtual reality (VR). Due to the imperfections of color images, depth maps or texture restoration techniques, several types of distortions occur in synthesized views. However, most of related works evaluated the quality of DIBR-synthesized views by only detecting a specific type of distortion, such as stretching, black holes, blurring, etc., which were unable to accurately evaluate the quality of DIBR-synthesized views. In this paper, a new no-reference image quality assessment method is proposed to evaluate the quality of DIBR-synthesized images by combining multi-layer and multi-scale features of images. To be specific, the distortions introduced by different stages of virtual viewpoint synthesis are first analyzed, and then multi-layer and multi-scale features are extracted to estimate the degree of texture and structure distortions. As a result, individual quality scores associated with two types of distortions (e.g., structural distortion and texture distortion) are aggregated to an overall image quality. Experimental results on two publicly available DIBR datasets show that the method has better performance than the state-of-the-art models.Index Terms: image quality assessment, DIBR-synthesized image, distortion correction, BIQA.     

Multiplication free vector quantization usingL1 distortion measure and its variants

The author considers vector quantization that uses the L (1) distortion measure for its implementation. A gradient-based approach for codebook design that does not require any multiplications or median computation is proposed. Convergence of this method is proved rigorously under very mild conditions. Simulation examples comparing the performance of this technique with the LBG algorithm show that the gradient-based method, in spite of its simplicity, produces codebooks with average distortions that are comparable to the LBG algorithm. The codebook design algorithm is then extended to a distortion measure that has piecewise-linear characteristics. Once again, by appropriate selection of the parameters of the distortion measure, the encoding as well as the codebook design can be implemented with zero multiplications. The author applies the techniques in predictive vector quantization of images and demonstrates the viability of multiplication-free predictive vector quantization of image data.     

Image database TID2013: Peculiarities,results and perspectives

This paper describes a recently created image database, TID2013, intended for evaluation of full-reference visual quality assessment metrics. With respect to TID2008, the new database contains a larger number (3000) of test images obtained from 25 reference images, 24 types of distortions for each reference image, and 5 levels for each type of distortion. Motivations for introducing 7 new types of distortions and one additional level of distortions are given; examples of distorted images are presented. Mean opinion scores (MOS) for the new database have been collected by performing 985 subjective experiments with volunteers (observers) from five countries (Finland, France, Italy, Ukraine, and USA). The availability of MOS allows the use of the designed database as a fundamental tool for assessing the effectiveness of visual quality. Furthermore, existing visual quality metrics have been tested with the proposed database and the collected results have been analyzed using rank order correlation coefficients between MOS and considered metrics. These correlation indices have been obtained both considering the full set of distorted images and specific image subsets, for highlighting advantages and drawbacks of existing, state of the art, quality metrics. Approaches to thorough performance analysis for a given metric are presented to detect practical situations or distortion types for which this metric is not adequate enough to human perception. The created image database and the collected MOS values are freely available for downloading and utilization for scientific purposes.     

基于ViT和多任务自监督学习的图像质量评价

针对现有的基于深度学习的图像质量评价方法,因为标注数据不足而存在的过拟合与泛化性能不足的问题,提出了一种基于多任务自监督学习的图像质量评价方法。首先,通过算法合成17种失真类型图像,并以全参考MDSI(mean deviation similarity index)得分和失真类型作为合成失真图像的2个标签;随后,在ViT(vision transformer)上进行预测MDSI得分和失真类型的多任务自监督学习;最后,将训练得到的模型在下游任务上进行微调,将上游任务学习到的语义特征迁移到下游任务。将本文方法与主流无参考图像质量评价(no reference image quality assessment,NR-IQA) 方法在多个公开的图像质量评价数据集上进行了充分比较,在LIVE、CSIQ、TID2013以及CID2013等数据集上的测试 结果相比于表现最好的算法均提升了1—2个百分点,这表明提出的算法优于大多数主流的NR-IQA算法。     

基于结构信息和像素失真的客观图像质量评价

为了度量多种失真类型的图像质量,根据人类视觉系统(HVS)对图像空域结构信息高度敏感和任一类型的失真都会产生像素失真理论,提出一种基于结构信息和像素失真的无参考的质量评价方法.该方法利用色彩信息提取能够表征图像结构信息的视觉内容结构图,并加权像素失真来度量图像质量,同时对部分失真类型进行修正.该方法不涉及任何参数设置也无需训练过程.实验结果表明,该方法能够较好地评价白噪声、JPEG压缩、高斯模糊、JPEG2000压缩和FastFading等失真图像的质量,并与主观评价方法有较好的一致性.     

Objective quality assessment of synthesized images by local variation measurement

Due to the rapid development of free-viewpoint television (FVT), Depth-Image-Based Rendering (DIBR) technology has been widely used to synthesize images of virtual view-points. However, the types of distortions in the synthesized images are different from those of natural images, such as discontinuity, flickering, stretching, etc. To measure the distortion occurred in the synthesized images, we propose a full-reference (FR) quality assessment method by local variation measurement consisting of three-modules. Firstly, since the distortion in the synthesized image mainly occurs in the region with high-frequency structure information, the Neutrosophic domain is employed to evaluate the degradation of local image structure. Secondly, by considering that the texture of the synthesized image might be damaged due to the warping of 2D image or the loss of information in the occlusion region, we evaluate the visual quality of local texture by using the features obtained from frequency domain. Thirdly, to measure the stretching distortion which is unique in the synthesized image, the visual quality of extracted stretching area is measured by entropy. Finally, a pooling operation is used to combine the quality scores of the three modules to obtain the final predicted quality score. Experimental results show that the performance of the proposed algorithm is competitive with state-of-the-art FR and no-reference image quality assessment metrics.     

结合结构信息和亮度统计的无参考图像质量评价

基于非下采样Contourlet变换(Nonsubsampled Contourlet Transform,NSCT)子带系数间的结构相关性,本文提出了一种通用的无参考图像质量评价方法.首先,利用互信息分析NSCT子带系数间的相关性,确定出相关性比较强的子带系数;其次,分别计算这些子带系数间的结构信息比较算子,以此作为描述图像结构相关性的统计特征;进而,结合空间域亮度均值减损对比归一化(Mean Subtracted Contrast Normalized,MSCN)系数及其邻域系数的统计特征,分别构造相应的无参考图像质量评价模型和图像失真类型识别模型;最后,在LIVE等图像质量评价数据库上进行了大量的实验仿真.仿真结果表明,评价模型的评价结果与人类主观评价具有非常高的相关性,与当今主流评价算法相比非常具有竞争性.     

Perceptually optimized blind repair of natural images

We define the new idea of blind image repair as a process of correcting one or more different and unknown types of distortions afflicting an image. These distortions could introduce linear or non-linear degradations, compression artifacts, noise, etc., or combinations of these. Thus the concept encompasses denoising, deblurring, deblocking, deringing, and other post-acquisition image improvement processes that address distortions. The problem is distortion-blind when the natures of the distortion processes are unknown prior to analyzing the image. Towards solving this problem, we describe a new framework for repairing an image that has undergone an unknown set of distortions, based on identifying the distortion(s) present in the image (if any) and applying possibly multiple distortion-specific image repair algorithms. Our philosophy is based on the principle that the task of general purpose image repair is one of agglomeration, i.e., the algorithm should embody multiple high-performing distortion-specific repair modules such that seamless general purpose image repair is achieved. Our proposed framework – the GEneral-purpose No-reference Image Improver (GENII) – enables the design of algorithms that are blind to distortion type as well as to distortion parameters, and only requires as input the distorted image to be repaired. The GENII framework is modular and easily extensible to image repair problems beyond those considered here. GENII operates by using natural scene statistic models to identify distortion, to perceptually optimize the distortion parameter(s), to assess the quality of the intermediate repaired images, and to perceptually optimize the repair processes. We explain the general purpose image repair framework and one specific realization, dubbed GENII-1, which assumes that the image has been affected by one or more of four possible distortion types.The performance of GENII-1 is evaluated on 4000 distorted images, and shown to deliver substantial improvements in both quantitative and qualitative visual quality.     

Learning to predict the quality of distorted-then-compressed images via a deep neural network

Being captured by amateur photographers, reciprocally propagated through multimedia pipelines, and compressed with different levels, real-world images usually suffer from a wide variety of hybrid distortions. Faced with this scenario, full-reference (FR) image quality assessment (IQA) algorithms can not deliver promising predictions due to the inferior references. Meanwhile, existing no-reference (NR) IQA algorithms remain limited in their efficacy to deal with different distortion types. To address this obstacle, we explore a NR-IQA metric by predicting the perceptual quality of distorted-then-compressed images using a deep neural network (DNN). First, we propose a novel two-stream DNN to handle both authentic distortions and synthetic compressions and adopt effective strategies to pre-train the two branches of the network. Specifically, we transfer the knowledge learned from in-the-wild images to account for authentic distortions by utilizing a pre-trained deep convolutional neural network (CNN) to provide meaningful initializations. Meanwhile, we build a CNN for synthetic compressions and pre-train it on a dataset including synthetic compressed images. Subsequently, we bilinearly pool these two sets of features as the image representation. The overall network is fine-tuned on an elaborately-designed auxiliary dataset, which is annotated by a reliable objective quality metric. Furthermore, we integrate the output of the authentic-distortion-aware branch with that of the overall network following a two-step prediction manner to boost the prediction performance, which can be applied in the distorted-then-compressed scenario when the reference image is available. Extensive experimental results on several databases especially on the LIVE Wild Compressed Picture Quality Database show that the proposed method achieves state-of-the-art performance with good generalizability and moderate computational complexity.     

利用梯度奇异值分解的图像结构相似度评价

针对传统的图像质量评价方法中对图像结构信息的表征能力不足的问题,在研究了基于结构相似度和奇异值分解的两种图像评价方法的基础上,结合其不同特点提出了基于奇异值分解的结构相似度质量评价方法.该算法分别将参考图像和失真图像的梯度图像分成8×8大小的图像块,并对每一个图像块进行奇异值分解后计算对应图像块的奇异值相似性和各图像块局部方差分布的相似性,最后结合各图像块的奇异值相似性和图像的局部方差分布的相似来表征图像的畸变程度.对LIVE库中包括5种失真类型的982幅图像进行验证,其结果表明该评价方法能很好地对各种失真类型的图像进行评价,比峰值信噪比(PSNR)、结构相似度(SSIM)等算法的主客观一致性更好,更加符合人眼的视觉特性.     

Dynamic contrast-based quantization for lossy wavelet image compression.

This paper presents a contrast-based quantization strategy for use in lossy wavelet image compression that attempts to preserve visual quality at any bit rate. Based on the results of recent psychophysical experiments using near-threshold and suprathreshold wavelet subband quantization distortions presented against natural-image backgrounds, subbands are quantized such that the distortions in the reconstructed image exhibit root-mean-squared contrasts selected based on image, subband, and display characteristics and on a measure of total visual distortion so as to preserve the visual system's ability to integrate edge structure across scale space. Within a single, unified framework, the proposed contrast-based strategy yields images which are competitive in visual quality with results from current visually lossless approaches at high bit rates and which demonstrate improved visual quality over current visually lossy approaches at low bit rates. This strategy operates in the context of both nonembedded and embedded quantization, the latter of which yields a highly scalable codestream which attempts to maintain visual quality at all bit rates; a specific application of the proposed algorithm to JPEG-2000 is presented.     

No-reference assessment of blur and noise impacts on image quality

The quality of images may be severely degraded in various situations such as imaging during motion, sensing through a diffusive medium, and low signal to noise. Often in such cases, the ideal un-degraded image is not available (no reference exists). This paper overviews past methods that dealt with no-reference (NR) image quality assessment, and then proposes a new NR method for the identification of image distortions and quantification of their impacts on image quality. The proposed method considers both noise and blur distortion types that may exist in the image. The same methodology employed in the spatial frequency domain is used to evaluate both distortion impacts on image quality, while noise power is further independently estimated in the spatial domain. Specific distortions addressed here include additive white noise, Gaussian blur and de-focus blur. Estimation results are compared to the true distortion quantities, over a set of 75 different images.     

Pair comparison based progressive subjective quality ranking for underwater images

Underwater images contain an interacting mixture of distortions due to the physicochemical properties of the water, suspended organic matter and floating particles in water. Unlike images in traditional natural image quality databases, underwater images are often difficult to acquire with reference images and sets of images with gradient distortion. Therefore, it is even more difficult for the viewers to assign an absolute psychophysical scale to the quality of underwater images. In this paper, we propose a pairwise subjective comparison procedure for underwater images quality ranking inspired by the intuitive suppression and competence mechanisms in visual perception. In the proposed method, we construct a preselection based initial image quality dataset by full pairwise comparison, which also enables online adaptive new image updating. The proposed method is not constrained by the lack of reference images, and is reliable and sensitive to images with discriminable distortion level and various image contents. The proposed pairwise comparison further allows an uncertain choice, which does not require a reinforce human opinion. To the best of our knowledge, this is the first implementation for underwater image subjective quality ranking, and a new approach to the image quality ranking for different image contents with unknown distortion level. We demonstrate that the obtained subjective image ranking correlates well with the human perception of quality difference among the underwater images than that of the single stimuli image quality assessment with finite labor burden. Moreover, our proposed method accurately characterize the gradual degradation in the underwater image sequence taken in controlled conditions. The proposed progressive learning ranking is also an alternative way to realize adaptive extension of the existing image quality databases.     

Blind image quality assessment in the contourlet domain

No-reference/blind image quality assessment (NR-IQA/BIQA) algorithms play an important role in image evaluation, as they can assess the quality of an image automatically, only using the distorted image whose quality is being assessed. Among the existing NR-IQA/BIQA methods, natural scene statistic (NSS) models which can be expressed in different bandpass domains show good consistency with human subjective judgments of quality.In this paper, we create new ‘quality-aware’ features: the energy differences of the sub-band coefficients across scales via contourlet transform, and propose a new NR-IQA/BIQA model that operates on natural scene statistics in the contourlet domain. Prior to applying the contourlet transform, we apply two preprocessing steps that help to create more information-dense, low-entropy representations. Specifically, we transform the picture into the CIELAB color space and gradient magnitude map. Then, a number of ‘quality-aware’ features are discovered in the contourlet transform domain: the energy of the sub-band coefficients within scales, and the energy differences between scales, as well as measurements of the statistical relationships of pixels across scales. A detailed analysis is conducted to show how different distortions affect the statistical characteristics of these features, and then features are fed to a support vector regression (SVR) model which learns to predict image quality. Experimental results show that the proposed method has high linearity against human subjective perception, and outperforms the state-of-the-art NR-IQA models.     

基于视觉显著失真度的图像质量自适应评价方法

针对结构相似(SSIM)图像质量评价算法没有考虑人眼视觉多通道性和对图像高失真评价的不稳定性,提出一种基于视觉显著失真度的图像质量自适应融合(VSAP)评价方法。该方法首先采用log-Gabor滤波提取图像的高频、中频及低频3层视觉特征,基于log-Gabor变换尺度和方向权重系数计算特征值的相似度;然后基于视觉阈值多分辨性迭加计算出特征值的失真度;最后,根据视觉失真度自适应融合相似度评价与失真度评价获得图像质量的最终客观评价。实验结果表明,VSAP方法不但对图像不同类型失真的客观评价与主观感知具有更高的相关性,而且3个主要指标斯皮尔曼等级相关系数(SROCC)、曲线拟合相关系数(CC)和均方根误差(RMSE)对图像不同水平失真的整体评价性能更稳定,明显优于其它评价方法。