Quality-distinguishing and patch-comparing no-reference image quality assessment

No-reference image quality assessment (NR-IQA) based on deep learning attracts a great research attention recently. However, its performance in terms of accuracy and efficiency is still under exploring. To address these issues, in this paper, we propose a quality-distinguishing and patch-comparing NR-IQA approach based on convolutional neural network (QDPC-CNN). We improve the prediction accuracy by two proposed mechanisms: quality-distinguishing adaption and patch-comparing regression. The former trains multiple models from different subsets of a dataset and adaptively selects one for predicting quality score of a test image according to its quality level, and the latter generates patch pairs for regression under different combination strategies to make better use of reference images in network training and enlarge training data at the same time. We further improve the efficiency of network training by a new patch sampling way based on the visual importance of each patch. We conduct extensive experiments on several public databases and compare our proposed QDPC-CNN with existing state-of-the-art methods. The experimental results demonstrate that our proposed method outperforms the others both in terms of accuracy and efficiency.

     

New feature selection algorithms for no-reference image quality assessment

No reference image quality assessment (NR-IQA) is a challenging task since reference images are usually unavailable in real world scenarios. The performance of NR-IQA techniques is vastly dependent on the features utilized to predict the image quality. Many NR-IQA techniques have been proposed that extract features in different domains like spatial, discrete cosine transform and wavelet transform. These NR-IQA techniques have the possibility to contain redundant features, which result in degradation of quality score prediction. Recently impact of general purpose feature selection algorithms on NR-IQA techniques has shown promising results. But these feature selection algorithms have the tendency to select irrelevant features and discard relevant features. This paper presents fifteen new feature selection algorithms specifically designed for NR-IQA, which are based on Spearman rank ordered correlation constant (SROCC), linear correlation constant (LCC), Kendall correlation constant (KCC) and root mean squared error (RMSE). The proposed feature selection algorithms are applied on the extracted features of existing NR-IQA techniques. Support vector regression (SVR) is then applied to selected features to predict the image quality score. The fifteen newly proposed feature selection algorithms are evaluated using eight different NR-IQA techniques over three commonly used image quality assessment databases. Experimental results show that the proposed feature selection algorithms not only reduce the number of features but also improve the performance of NR-IQA techniques. Moreover, features selection algorithms based on SROCC and its combination with LCC, KCC and RMSE perform better in comparison to other proposed algorithms.     

Retina inspired no-reference image quality assessment for blur and noise

Multimedia Tools and Applications - A blind image quality assessment technique with no-training is proposed in this paper. The proposed technique considers two important types of distortions viz....     

Saliency-based deep convolutional neural network for no-reference image quality assessment

In this paper, we proposed a novel method for No-Reference Image Quality Assessment (NR-IQA) by combining deep Convolutional Neural Network (CNN) with saliency map. We first investigate the effect of depth of CNNs for NR-IQA by comparing our proposed ten-layer Deep CNN (DCNN) for NR-IQA with the state-of-the-art CNN architecture proposed by Kang et al. (2014). Our results show that the DCNN architecture can deliver a higher accuracy on the LIVE dataset. To mimic human vision, we introduce saliency maps combining with CNN to propose a Saliency-based DCNN (SDCNN) framework for NR-IQA. We compute a saliency map for each image and both the map and the image are split into small patches. Each image patch is assigned with a patch importance value based on its saliency patch. A set of Salient Image Patches (SIPs) are selected according to their saliency and we only apply the model on those SIPs to predict the quality score for the whole image. Our experimental results show that the SDCNN framework is superior to other state-of-the-art approaches on the widely used LIVE dataset. The TID2008 and the CISQ image quality datasets are utilised to report cross-dataset results. The results indicate that our proposed SDCNN can generalise well on other datasets.     

Compressed-domain-based no-reference video quality assessment model considering fast motion and scene change

Due to the variability of wireless channel state, video quality monitoring became very important for guaranteeing users’ Quality of Experience (QoE). QoE presents the overall perceptual quality of service from the subjective users’ perspective. However, because of diverse characteristics of video content, Human Visual System (HVS) cannot give the same attention to whole scene simultaneously when facing video sequence. In this paper, we proposed a video quality assessment model by considering the influence of fast motion and scene change. The motion change contribution factor and scene change contribution factor are defined to quantify the characteristics of video content, which is closely related to the users’ QoE. Based on G.1070, our proposed model considers the influential factors of loss nature of video coding, variability of practical network and video features. Also, the proposed model owns low computational complexity due to the compressed domain approach for the estimation of the model parameters. Therefore, the video quality is assessed without fully decoding the video stream. The performance of our proposed model has been compared with five existing models and the results also shown that our model has high prediction accuracy closing to human perception.     

变权无参考视频质量评价模型研究

传输信道失真是导致视频图像质量损失的重要原因,而不需要任何附加传输信息的无参考客观评价是监测视频传输质量受损的主要方法.为了提高无参考客观评价模型的准确性和效率,提出模拟人类视觉特性的变权评价模型.变权评价模型综合考虑视频的空域和时域两类质量指标,引入运动强度来量化视频内容中的运动变化程度,根据统计学习的非线性回归法建立变权控制函数;通过变权控制函数动态调整清晰度和平滑度权值来模仿人类视觉特性.实验结果表明该变权评价模型与主观评价符合度高,优于现有评价方案.     

Paired comparison-based subjective quality assessment of stereoscopic images

As 3D image and video content has gained significant popularity, subjective 3D quality assessment has become an important issue for the creation, processing, and distribution of high quality 3D content. Reliable subjective quality assessment of 3D content is often difficult due to the subjects’ limited 3D experience, the interaction of multiple quality factors, minor quality differences between stimuli, etc. Among subjective evaluation methodologies, paired comparison has the advantage of improved simplicity and reliability, which can be useful to tackle the aforementioned difficulties. In this paper, we propose a new method to analyze the results of paired comparison-based subjective tests. We assume that ties convey information about the significance of quality score differences between two stimuli. Then, a maximum likelihood estimation is performed to obtain confidence intervals providing intuitive measures of significance of the quality differences. We describe the complete test procedure using the proposed method, from subjective experiment design to outlier detection and score analysis for 3D image quality assessment. Especially, we design the test procedure in a way that quality comparison across different contents is enabled while the number of pair-wise comparisons is minimized. Experimental results on a stereoscopic image database with varying camera distances demonstrate the usefulness of the proposed method and enhanced quality discriminability of paired comparison in comparison to the conventional single stimulus methodology.     

Restoration of blurred images with conditional total variation method

Image deblurring based on a conditional total variation approach is presented. The restored image minimizes the total variation in the class of images which satisfy the constraints given by the Kronrod’s variations. Computer simulation results using a real image are provided and discussed.     

Spoofing detection on hand images using quality assessment

Multimedia Tools and Applications - Recent research on biometrics focuses on achieving a high success rate of authentication and addressing the concern of various spoofing attacks. Although hand...     

Neural network solution for a real-time no-reference video quality assessment of H.264/AVC video bitstreams

The ever-growing video streaming services require accurate quality assessment with often no reference to the original media. One primary challenge in developing no-reference (NR) video quality metrics is achieving real-timeliness while retaining the accuracy. A real-time no-reference video quality assessment (VQA) method is proposed for videos encoded by H.264/AVC codec. Temporal and spatial features are extracted from the encoded bit-stream and pixel values to train and validate a fully connected neural network. The hand-crafted features and network dynamics are designed in a manner to ensure a high correlation with human judgment of quality as well as minimizing the computational complexities. Proof-of-concept experiments are conducted via comparison with: 1) video sequences rated by a full-reference quality metric, and 2) H.264-encoded sequences from the LIVE video dataset which are subjectively evaluated through differential mean opinion scores (DMOS). The performance of the proposed method is verified by correlation measurements with the aforementioned objective and subjective scores. The framework achieves real-time execution while outperforming state-of-art full-reference and no-reference video quality assessment methods.

     

Curved text detection in blurred/non-blurred video/scene images

Multimedia Tools and Applications - Text detection in video/images is challenging due to the presence of multiple blur caused by defocus and motion. In this paper, we present a new method for...     

Application of blurred circular 3D images on the human vision model

Microsystem Technologies - This study was developed based on the lateral inhibition effect and the Mach band effect. In the study, various human vision experiments involving blurred circular 3D...     

Parametric characterization of the form of the human pupil from blurred noisy images

The fluctuation of the human pupil is an important parameter in order to make non-invasive diagnosis of many different diseases and in several clinical applications. The relevant measurement device, the pupillometer, consists in a CCD camera, which shoots the pupil. We suppose that the measured image is blurred by a Gaussian kernel and corrupted by an additive white noise; moreover an elliptic shape for the pupil is assumed. We here present the extension of a multiscale approach for edge detection to identify some parameters of the pupil: the location of its centre, the length of the semi-axes and the orientation of the corresponding ellipse. The chosen method requires knowledge about the degradation parameters of the assumed model; so we first present a simple but efficient method to determine such quantities for the measured image. Then we apply the edge detection procedure to identify points close to the pupil edge, within a chosen probability. Finally we find the optimal ellipse fitting a suitable subset of the previously detected edge points. Results are presented, with comparisons to other approaches for edge finding.     

Reduced-reference quality assessment of DIBR-synthesized images based on multi-scale edge intensity similarity

Multimedia Tools and Applications - Depth-image-based-rendering (DIBR) plays an important role in view synthesis for free-viewpoint videos. The warping process in DIBR causes geometric...     

NDNetGaming - development of a no-reference deep CNN for gaming video quality prediction

Multimedia Tools and Applications - Gaming video streaming services are growing rapidly due to new services such as passive video streaming of gaming content, e.g. Twitch.tv, as well as cloud...     

Compression assessment based on medical image quality concepts using computer-generated test images

Compression algorithms are widely used in medical imaging systems for efficient image storage, transmission, and display. In the acceptance of lossy compression algorithms in the clinical environment, important factors are the assessment of 'visually lossless' compression thresholds, as well as the development of assessment methods requiring fewer data and time than observer performance based studies. In this study a set of quantitative measurements related to medical image quality parameters is proposed for compression assessment. Measurements were carried out using region of interest (ROI) operations on computer-generated test images, with characteristics similar to radiographic images. As a paradigm, the assessment of the lossy Joint Photographic Expert Group (JPEG) algorithm, available in a telematics application for healthcare, is presented. A compression ratio of 15 was found as the visually lossless threshold for the JPEG lossy algorithm, in agreement with previous observer performance studies. Up to this ratio low contrast discrimination is not affected, image noise level is decreased, high contrast line-pair amplitude is decreased by less than 3%, and input/output gray level differences are minor (less than 1%). This type of assessment provides information regarding the type of loss, offering cost and time benefits, in parallel with the advantages of test image adaptation to the requirements of a certain imaging modality and clinical study.     

Perceptual quality assessment of stereoscopic images based on local and global visual characteristics

The quality assessment of stereoscopic images has attracted considerable attention and become an important issue in 3D multimedia applications. The 3D image quality assessment (IQA) encounters many challenges and simple extension of the 2D quality metrics to the 3D case is not satisfying. In this paper, we propose a new perceptual quality assessment scheme for stereoscopic 3D images by considering the local and global visual characteristics. The design of this scheme is motivated by studies on the perception of distorted stereoscopic images. To be more specific, after the log-Gabor filter processing, the local amplitude and phase from the left and right views of the reference and distorted 3D images are utilized as features in local quality evaluation. Meanwhile, the global structure changes of the left and right views are also incorporated into the final quality pooling. The overall 3D quality score is obtained by combining the local and global quality indexes together. The effectiveness of the designed metric is verified on publicly available 3D image quality assessment databases. Experimental results show that the proposed scheme exhibits better performance than other related algorithms in terms of consistency with subjective assessment of stereoscopic 3D images.

     

基于偏度和结构特征的无参考虚拟视点图像质量评价

针对虚拟视点图像存在与普通图像不同失真的问题,提出了一种基于偏度和结构特征的无参考虚拟视点图像质量评价方法.首先,将输入图像分为H、S、V三个通道,每个通道都均分为九个小块,对每个小块提取偏度特征;然后,利用局部二值模式算子对结构特征映射进行编码,计算质量感知分数;最后,将偏度特征和结构特征输入支持向量机进行训练,得到无参考的虚拟视点图像回归与视觉质量预测模型来预测图像质量.在IRCCyN/IVC和MCL-3D两个公共虚拟视点图像数据库上对提出方法进行了实验.实验结果表明,提出方法的皮尔逊线性相关系数(PLCC)分别为0.8538和0.9534,斯皮尔曼秩相关系数(SROCC)分别为0.7966和0.9159.提出方法的PLCC和SROCC均高于自回归加阈值(APT)等10个虚拟视点图像质量评价方法和BRISQUE等6个通用的无参考质量评价方法.该方法采用的偏度和结构特征能很好地评价虚拟视点图像的视觉质量,评价结果与主观感知有较好的一致性.