Blind image quality assessment in the contourlet domain

No-reference/blind image quality assessment (NR-IQA/BIQA) algorithms play an important role in image evaluation, as they can assess the quality of an image automatically, only using the distorted image whose quality is being assessed. Among the existing NR-IQA/BIQA methods, natural scene statistic (NSS) models which can be expressed in different bandpass domains show good consistency with human subjective judgments of quality.In this paper, we create new ‘quality-aware’ features: the energy differences of the sub-band coefficients across scales via contourlet transform, and propose a new NR-IQA/BIQA model that operates on natural scene statistics in the contourlet domain. Prior to applying the contourlet transform, we apply two preprocessing steps that help to create more information-dense, low-entropy representations. Specifically, we transform the picture into the CIELAB color space and gradient magnitude map. Then, a number of ‘quality-aware’ features are discovered in the contourlet transform domain: the energy of the sub-band coefficients within scales, and the energy differences between scales, as well as measurements of the statistical relationships of pixels across scales. A detailed analysis is conducted to show how different distortions affect the statistical characteristics of these features, and then features are fed to a support vector regression (SVR) model which learns to predict image quality. Experimental results show that the proposed method has high linearity against human subjective perception, and outperforms the state-of-the-art NR-IQA models.     

BGT: A blind image quality evaluator via gradient and texture statistical features

Blind image quality assessment (BIQA) aims to design a model that can accurately evaluate the quality of the distorted image without any information about its reference image. Previous studies have shown that gradients and textures of image is widely used in image quality evaluation tasks. However, few studies used the joint statistics of gradient and texture information to evaluate image quality. Considering the visual perception characteristics of the human visual system, we develop a novel general-purpose BIQA model via two sets of complementary perception features. Specifically, the joint statistical histograms of gradient and texture are extracted as the first set of features, and the second set of features is extracted using the local binary pattern (LBP) operator. After extracting two groups of complementary quality-aware features, the feature vectors are sent to the support vector regression machine to establish the nonlinear relationship between quality-aware features and quality scores. A large number of experiments on seven large benchmark databases show that the proposed BIQA model has higher accuracy, better generalization properties and lower computational complexity than the relevant state-of-the-art BIQA metrics.     

Gaussian Process-based Feature-Enriched Blind Image Quality Assessment

The objective of blind-image quality assessment (BIQA) research is the prediction of perceptual quality of images, without reference information. The human’s perceptual assessment of quality of an image is the backbone of BIQA research. Therefore, human-provided, mean opinion score (perceptual quality) has been analyzed in detail, and it has been observed to follow the Gaussian distribution and thus can be ideally modeled by the same. In this paper, we have proposed an integrated two-stage Gaussian process-based hybrid-feature selection algorithm for the BIQA problem. Moreover, a new consolidated feature set (obtained from the proposed algorithm), consisting of momentous Natural Scene Statistics (NSS)-based features is used in combination with the Gaussian process regression algorithm for the design of a new blind-image quality evaluator, referred to as GPR-BIQA. The proposed evaluator is tested on eight IQA legacy databases, and it is found that the proposed evaluator proficiently correlate with the human opinion, and outperformed a substantial number of existing approaches.     

A new objective stereoscopic image assessment model based on stereoscopic perception

In order to establish a stereoscopic image quality assessment method which is consistent with human visual perception, we propose an objective stereoscopic image quality assessment method. It takes into account the strong correlation and high degree of structural between pixels of image. This method contains two models. One is the quality synthetic assessment of left-right view images, which is based on human visual characteristics, we use the Singular Value Decomposition （SVD） that can represent the degree of the distortion, and combine the qualities of left and right images by the characteristics of binocular superposition. The other model is stereoscopic perception quality as- sessment, due to strong stability of image＇s singular value characteristics, we calculate the distance of the singular values and structural characteristic similarity of the absolute difference maps, and utilize the statistical value of the global error to evaluate stereoscopic perception. Finally, we combine two models to describe the stereoscopic image quality. Experimental results show that the correlation coefficients of the proposed assessment method and the human subjective perception are above 0.93, and the mean square errors are all less than 6.2, under JPEG, JP2K compression, Gaussian blurring, Gaussian white noise, H.264 coding distortion, and hybrid cross distortion. It indicates that the proposed stereoscopic objective method is consistent with human visual properties and also of availability.     

An efficient and effective blind camera image quality metric via modeling quaternion wavelet coefficients

As an extension of Discrete and Complex Wavelet Transform, Quaternion Wavelet Transform (QWT) has attracted extensive attention in the past few years, because it can provide better analytic representation for 2D images. The QWT of an image consists of four parts, i.e., one magnitude part and three phase parts. The magnitude is nearly shift-invariant, which characterizes features at any spatial location, and the three phases represent the structure of these features. This indicates that QWT is more powerful in representing image structures, and thus is suitable for image quality evaluation. In this paper, an efficient and effective Camera Image Quality Metric (CIQM) is proposed based on QWT, which is utilized to describe the intrinsic structures of an image. For an image, it is first decomposed by QWT with three scales. Then, for each scale, the magnitude and entropy of the subband coefficients, and natural scene statistics of the third phase are calculated. The magnitude is utilized to describe the generalized spectral behavior, and the entropy is used to encode the generalized information of distortions. Since the third phase of QWT is considered to be texture feature, the natural scene statistics of the third phase of QWT is used to measure structure degradations in the proposed method. All these features reflect the self-similarity and independency of image content, which can effectively reflect image distortions. Finally, random forest is utilized to build the quality model. Experiments conducted on three camera image databases and two multiply distorted image databases have proved that CIQM outperforms the relevant state-of-the-art models for both authentically distorted images and multiply distorted images.     

Multi-layer and Multi-scale feature aggregation for DIBR-Synthesized image quality assessment

Depth-Image-Based Rendering (DIBR) is one of the main fundamental techniques for generating new viewpoints in 3D video applications such as multi-viewpoint video (MVV), free viewpoint video (FVV) and virtual reality (VR). Due to the imperfections of color images, depth maps or texture restoration techniques, several types of distortions occur in synthesized views. However, most of related works evaluated the quality of DIBR-synthesized views by only detecting a specific type of distortion, such as stretching, black holes, blurring, etc., which were unable to accurately evaluate the quality of DIBR-synthesized views. In this paper, a new no-reference image quality assessment method is proposed to evaluate the quality of DIBR-synthesized images by combining multi-layer and multi-scale features of images. To be specific, the distortions introduced by different stages of virtual viewpoint synthesis are first analyzed, and then multi-layer and multi-scale features are extracted to estimate the degree of texture and structure distortions. As a result, individual quality scores associated with two types of distortions (e.g., structural distortion and texture distortion) are aggregated to an overall image quality. Experimental results on two publicly available DIBR datasets show that the method has better performance than the state-of-the-art models.Index Terms: image quality assessment, DIBR-synthesized image, distortion correction, BIQA.     

Dual-branch vision transformer for blind image quality assessment

Blind image quality assessment (BIQA) has always been a challenging problem due to the absence of reference images. In this paper, we propose a novel dual-branch vision transformer for BIQA, which simultaneously considers both local distortions and global semantic information. It first extracts dual-scale features from the backbone network, and then each scale feature is fed into one of the transformer encoder branches as a local feature embedding to consider the scale-variant local distortions. Each transformer branch obtains the context of global image distortion as well as the local distortion by adopting content-aware embedding. Finally, the outputs of the dual-branch vision transformer are combined by using multiple feed-forward blocks to predict the image quality scores effectively. Experimental results demonstrate that the proposed BIQA method outperforms the conventional methods on the six public BIQA datasets.     

Toward an unsupervised blind stereoscopic 3D image quality assessment using joint spatial and frequency representations

Existing blind stereoscopic 3D (S3D) image quality assessment (IQA) metrics usually require supervised learning methods to predict S3D image quality, which limits their applicability in practice. In this paper, we propose an unsupervised blind S3D IQA metric that utilizes the joint spatial and frequency representations of visual perception. The metric proposed in this work was inspired by the binocular visual mechanism; furthermore, it is unsupervised and does not require subject-rated samples for training. To be more specific, first, the various binocular quality-aware features in spatial and frequency domains are extracted from the monocular and cyclopean views of natural S3D image patches. Subsequently, these features are utilized to establish a pristine multivariate Gaussian (MVG) model to characterize natural S3D image regularities. Finally, with the learned MVG model, the final quality score for a distorted S3D image can be yielded using a Bhattacharyya-like distance. Our experimental results illustrate that, compared to related existing metrics, the devised metric achieves competitive prediction performance.     

Unsupervised image classification, segmentation, and enhancementusing ICA mixture models

An unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent, non-Gaussian densities. The algorithm estimates the data density in each class by using parametric nonlinear functions that fit to the non-Gaussian structure of the data. This improves classification accuracy compared with standard Gaussian mixture models. When applied to images, the algorithm can learn efficient codes (basis functions) for images that capture the statistically significant structure intrinsic in the images. We apply this technique to the problem of unsupervised classification, segmentation, and denoising of images. We demonstrate that this method was effective in classifying complex image textures such as natural scenes and text. It was also useful for denoising and filling in missing pixels in images with complex structures. The advantage of this model is that image codes can be learned with increasing numbers of classes thus providing greater flexibility in modeling structure and in finding more image features than in either Gaussian mixture models or standard independent component analysis (ICA) algorithms.     

Reduced reference image quality assessment based on statistics in empirical mode decomposition domain

This paper deals with the image quality assessment (IQA) task using a natural image statistics approach. A reduced reference (RRIQA) measure based on the bidimensional empirical mode decomposition is introduced. First, we decompose both, reference and distorted images, into intrinsic mode functions (IMF) and then we use the generalized Gaussian density (GGD) to model IMF coefficients of the reference image. Finally, we measure the impairment of a distorted image by fitting error between the IMF coefficients histogram of the distorted image and the estimated IMF coefficients distribution of the reference image, using the Kullback–Leibler divergence (KLD). Furthermore, to predict the quality, we propose a new support vector machine-based (SVM) classification approach as an alternative to logistic function-based regression. In order to validate the proposed measure, three benchmark datasets are involved in our experiments. Results demonstrate that the proposed metric compare favorably with alternative solutions for a wide range of degradation encountered in practical situations.     

C-DIIVINE: No-reference image quality assessment based on local magnitude and phase statistics of natural scenes

It is widely known that the wavelet coefficients of natural scenes possess certain statistical regularities which can be affected by the presence of distortions. The DIIVINE (Distortion Identification-based Image Verity and Integrity Evaluation) algorithm is a successful no-reference image quality assessment (NR IQA) algorithm, which estimates quality based on changes in these regularities. However, DIIVINE operates based on real-valued wavelet coefficients, whereas the visual appearance of an image can be strongly determined by both the magnitude and phase information.In this paper, we present a complex extension of the DIIVINE algorithm (called C-DIIVINE), which blindly assesses image quality based on the complex Gaussian scale mixture model corresponding to the complex version of the steerable pyramid wavelet transform. Specifically, we applied three commonly used distribution models to fit the statistics of the wavelet coefficients: (1) the complex generalized Gaussian distribution is used to model the wavelet coefficient magnitudes, (2) the generalized Gaussian distribution is used to model the coefficients׳ relative magnitudes, and (3) the wrapped Cauchy distribution is used to model the coefficients׳ relative phases. All these distributions have characteristic shapes that are consistent across different natural images but change significantly in the presence of distortions. We also employ the complex wavelet structural similarity index to measure degradation of the correlations across image scales, which serves as an important indicator of the subbands׳ energy distribution and the loss of alignment of local spectral components contributing to image structure. Experimental results show that these complex extensions allow C-DIIVINE to yield a substantial improvement in predictive performance as compared to its predecessor, and highly competitive performance relative to other recent no-reference algorithms.     

Image quality assessment based on Prewitt magnitude

The goal of image quality assessment (IQA) research is to use computational models to calculate the quality of images consistently with subjective evaluations. In this paper, we propose a new image quality assessment (IQA) algorithm by combining Prewitt magnitude and regional mutual information (RMI) in HSV color space. The Prewitt operator is usually used for edge detection and can extract vertical edge more accurately than other operators. The HSV color space encapsulates information about a color in terms that are more natural and intuitive to humans. The proposed method PMRMI first transforms reference and distorted images from RGB color space into HSV color space and Prewitt magnitude is introduced to extract key edge features of each channel. Then the regional mutual information is calculated to measure the similarity of the two images. After that, a weighting method is utilized for better consistency with subjective evaluations. Therefore we get a single quality score. Experiments on various image distortion types demonstrate that the proposed algorithm can achieve better consistency with the subjective evaluations than PSNR and SSIM.     

Entropy-based image registration method using the curvelet transform

Registration is a prerequisite for fusion of geometrically distorted images. Traditionally, intensity-based image registration methods are preferred to feature-based ones due to higher accuracy of the former than that of the latter. To reduce computational load, image registration is often carried out using the approximate-level coefficients of a wavelet-like transform. Directional selectivity of the transform and the objective function used for the coefficients play vital roles in the alignment process of images. This paper introduces an image registration algorithm that uses the approximate-level coefficients of the curvelet transform, directional selectivity of which is better than many wavelet-like transforms. A conditional entropy-based objective function is developed for registration using a suitable probabilistic model of the curvelet coefficients of images. Suitability of the probability distribution of the coefficients is validated using a standard method to assess goodness of fit. To align the distorted images, the affine transformation that possesses parameters related to the translation, rotation, scaling, and shearing is used. Extensive experimentations are carried out to test the performance of the proposed registration method considering that the images are synthetically or naturally distorted. Experimental results show that performance of the proposed registration method is superior to existing methods in terms of commonly used performance metrics.     

Learning-based image restoration for compressed images

In this paper, we propose a novel learning-based image restoration scheme for compressed images by suppressing compression artifacts and recovering high frequency (HF) components based upon the priors learnt from a training set of natural images. The JPEG compression process is simulated by a degradation model, represented by the signal attenuation and the Gaussian noise addition process. Based on the degradation model, the input image is locally filtered to remove Gaussian noise. Subsequently, the learning-based restoration algorithm reproduces the HF component to handle the attenuation process. Specifically, a Markov-chain based mapping strategy is employed to generate the HF primitives based on the learnt codebook. Finally, a quantization constraint algorithm regularizes the reconstructed image coefficients within a reasonable range, to prevent possible over-smoothing and thus ameliorate the image quality. Experimental results have demonstrated that the proposed scheme can reproduce higher quality images in terms of both objective and subjective quality.     

基于多特征扩展pLSA模型的场景图像分类

场景图像分类近年来受到人们的广泛关注,而基于统计模型的方法更是场景分类中的研究热点。我们提出了一种新的基于多特征融合和扩展pLSA模型的场景图像分类框架。对每幅图像首先用多尺度规则分割确定局部基元,然后提取每个局部基元的多分辨率直方图矩特征和SIFT特征,最后用扩展的概率生成模型对图像集进行建模,测试。我们的方法不仅能够很好的表示图像的语义特性而且在模型的训练阶段是无监督的。我们针对目前常用的3个数据库,做了三组对比实验,均取得了比以前的方法更好的识别结果。      

基于稀疏表示的立体图像客观质量评价方法

提出了一种基于稀疏表示的立体图像质量评价方法 ,分为训练和测试两个部分。在训练部 分,通过训练不同频带的立体图像获得立体图像的稀疏字典;在测试部分,根据稀疏字典计 算得到立体图 像的稀疏特征,定义了稀疏特征相似度衡量原始和失真图像信息的差异,并根据稀疏字典计 算了频带增益和左右视点的融合权值,最后融合稀疏特征相似度作为立体图像质量的 客观评价值。在立体图像测试库上的实验结果表明,本文方法的评价结果与主观评价结果有 较好的相关性,符合人类视觉系统的感知。     

基于三维结构张量的立体图像质量客观评价方法

根据梯度结构张量能够表示图像结构信息的特点, 提出了一种基于三维结构张 量的立体图像客观质量评价方法。首先分别求取原始和失真的立体图像水平、垂直和 视点方向的梯度信息,以及敏感区域,并构造出立体图像中每个像素的三维结构张量矩 阵;然后,提 取三维结构张量矩阵的特征值和特征向量信息;最后,根据特征值和特征向量预测得到立体 图像质量的客 观评价值。实验结果表明,采用本文提出的客观评价方法对立体图像测试库进行评价,总体 评价的Pearson 线性相关系数(PLCC)和Spearman等级相关系数(SROCC)值均在0.92左右,Kendall相关系数 (KROCC)值 接近0.80,均方根误差(RMSE)值均在6.00左右;与其他方法相 比,本方法具有较高的预测精确性。     

Mining discriminative spatial cues for aerial image quality assessment towards big data

Evaluating massive-scale aerial/satellite images quality is useful in computer vision and intelligent applications. Traditional local features-based algorithms have achieved impressive performance. However, spatial cues, i.e., geometric property and topological structure, have not been exploited effectively and explicitly. Thus, in this paper, we propose a novel method for image quality assessment towards aerial/satellite images, where discriminative spatial cues are well encoded. More specifically, in order to mine inherent spatial structure of aerial images, each image is segmented into several basic components such as buildings, airport and playground. Afterwards, a weighted region adjacency graph (RAG) is built based on the basic components to represent the spatial feature of each aerial image. We integrate the spatial feature with other transform domain features, and train a support vector regression model to achieve image quality assessment. Experiments demonstrate that our method shows competitive or even better performance compared with several state-of-the-art algorithms.     

MLSIM: A Multi-Level Similarity index for image quality assessment

Image quality assessment (IQA) is of great importance to numerous image processing applications, and various methods have been proposed for it. In this paper, a Multi-Level Similarity (MLSIM) index for full reference IQA is proposed. The proposed metric is based on the fact that human visual system (HVS) distinguishes the quality of an image mainly according to the details given by low-level gradient information. In the proposed metric, the Prewitt operator is first utilized to get gradient information of both reference and distorted images, then the gradient information of reference image is segmented into three levels (3LSIM) or two levels (2LSIM), and the gradient information of distorted image is segmented by the corresponding regions of reference image, therefore we get multi-level information of these two images. Riesz transform is utilized to get corresponding features of different levels and the corresponding 1st-order and 2nd-order coefficients are combined together by regional mutual information (RMI) and weighted to obtain a single quality score. Experimental results demonstrate that the proposed metric is highly consistent with human subjective evaluations and achieves good performance.     

Quality-aware images.

We propose the concept of quality-aware image, in which certain extracted features of the original (high-quality) image are embedded into the image data as invisible hidden messages. When a distorted version of such an image is received, users can decode the hidden messages and use them to provide an objective measure of the quality of the distorted image. To demonstrate the idea, we build a practical quality-aware image encoding, decoding and quality analysis system, which employs: 1) a novel reduced-reference image quality assessment algorithm based on a statistical model of natural images and 2) a previously developed quantization watermarking-based data hiding technique in the wavelet transform domain.