一种基于最小可察觉失真的立体图像质量客观评价方法

立体图像质量是评价立体视频系统性能的有效途径,而如何利用人类视觉特性对立体图像质量进行有效的评价是目前的研究难点。本文通过分析最小可察觉失真(JND,just noticeable distortion)视觉感知模型,并结合反映图像结构信息的奇异值矢量,提出了一种基于JND的立体图像质量客观评价方法。评价方法由图像质量评价和深度感知评价两部分组成,首先提取反映图像质量和深度感知的特征信息作为立体图像特征信息,然后根据立体图像的不同失真类型情况对其特征进行融合,通过支持向量回归(SVR,support vector Regression)预测得出立体图像质量的客观评价值。实验结果表明,采用本文提出的客观评价方法对立体数据测试库进行评价,在不同失真类型或混合失真评价结果中,Pearson线性相关系数(CC)值均在0.94以上,Spearman等级相关系数(SROCC)值均在0.92以上,符合人眼视觉特性,能够很好地预测人眼对立体图像的主观感知。     

基于支持向量回归的立体图像客观质量评价模型

立体图像质量评价是评价立体视频系统性能的有效途径,而如何利用人类视觉特性对立体图像质量进行有效评价是目前的研究难点。该文根据图像奇异值有较强稳定性的特点,结合立体图像的主观视觉特性,提出了一种基于支持向量回归(Support Vector Regression, SVR)的立体图像客观质量评价模型。该模型通过分析立体图像的视觉特性,提取左右图像的奇异值作为立体图像的特征信息,然后根据立体图像的不同失真类型情况对其特征进行融合,通过SVR预测得到立体图像质量的客观评价值。实验结果表明,采用该文提出的客观评价模型对立体数据测试库进行评价,Pearson线性相关系数值在0.93以上,Spearman等级相关系数值在0.94以上,均方根误差值接近6,异常值比率值为0.00%,符合人眼视觉特性,能够很好地预测人眼对立体图像的主观感知。     

基于稀疏表示的立体图像客观质量评价方法

提出了一种基于稀疏表示的立体图像质量评价方法 ,分为训练和测试两个部分。在训练部 分,通过训练不同频带的立体图像获得立体图像的稀疏字典;在测试部分,根据稀疏字典计 算得到立体图 像的稀疏特征,定义了稀疏特征相似度衡量原始和失真图像信息的差异,并根据稀疏字典计 算了频带增益和左右视点的融合权值,最后融合稀疏特征相似度作为立体图像质量的 客观评价值。在立体图像测试库上的实验结果表明,本文方法的评价结果与主观评价结果有 较好的相关性,符合人类视觉系统的感知。     

基于视差空间图的立体图像质量客观评价方法

立体图像质量评价是评价立体视频系统性能的有 效途径,而如何利用人类视觉特性对立体图像质量 进行有效评价是目前的研究难点。本文提出了一种基于视差空间图(DSI) 的立体图像质量客观评价方法。首先, 分别构造原始立体图像和失真立体图像的DSI图;然后,通过三维离散余弦变换(3D-DCT)提取出反映图像质量 和深度感知的特征信息,并采用主成分分析(PCA)进行特征降维,形成立体图像特征信息; 最后,通过支持向量 回归(SVR)建立立体图像特征与主观评价值的关系,从而预测得到立体图像质量的客观评价 值。实验表明, 对于对称立体图像库,Pearson线性相关系数(PLCC)和Spe arman等级相关系数(SROCC)值均达到0.94以上;对于非 对称立体图像库,PLCC和SROCC值分别达到0.94以上。结果表明,本文方法能够很好地预测人眼对立体图像的主观感 知。     

Reduced-reference stereoscopic image quality assessment based on view and disparity zero-watermarks

As a practical and novel application of watermarking, this paper presents a zero-watermarking based objective reduced-reference stereoscopic image quality assessment (RR-SIQA) method. In the proposed method, two kinds of zero-watermarks are constructed according to the characteristics of image structure and stereoscopic perception. Concretely, two view zero-watermarks, which are constructed by judging the relation of the horizontal and vertical components of gradient vectors with respect to the two views, are used to reflect the image structure variation of the stereoscopic image. Meanwhile, a disparity zero-watermark, which is constructed with disparity map of the stereoscopic image, is used to reflect the stereoscopic perception quality variation. Then, the quality of stereoscopic image is objectively assessed by pooling the recovering rates of the detected zero-watermarks. The experimental results show that the stereoscopic image quality evaluation results assessed with the proposed RR-SIQA method are well consistent with subjective assessment, and the proposed method achieves better performance than the widely used full-reference stereoscopic image quality assessment method PSNR in assessing quality of stereoscopic images compressed with JPEG and JPEG2000.     

基于三维结构张量的立体图像质量客观评价方法

根据梯度结构张量能够表示图像结构信息的特点, 提出了一种基于三维结构张 量的立体图像客观质量评价方法。首先分别求取原始和失真的立体图像水平、垂直和 视点方向的梯度信息,以及敏感区域,并构造出立体图像中每个像素的三维结构张量矩 阵;然后,提 取三维结构张量矩阵的特征值和特征向量信息;最后,根据特征值和特征向量预测得到立体 图像质量的客 观评价值。实验结果表明,采用本文提出的客观评价方法对立体图像测试库进行评价,总体 评价的Pearson 线性相关系数(PLCC)和Spearman等级相关系数(SROCC)值均在0.92左右,Kendall相关系数 (KROCC)值 接近0.80,均方根误差(RMSE)值均在6.00左右;与其他方法相 比,本方法具有较高的预测精确性。     

基于多特征融合的三维网格质量评价方法

三维网格客观质量方法研究对三维网格处理方法设计有重要指导意义。现有的三维网格质量评价方法大多采用单一特征来表征三维网格的失真程度,难以全面描述人眼对三维网格失真的视觉感知。在提出基于分区的曲率奇异值分解的结构相似性(CSSIM)特征的基础上,进一步提出多特征融合的三维网格质量评价方法。提取的特征通过随机森林的聚合,训练得到三维网格的客观质量评价模型,预测评价分数。经过在两个公开的三维网格数据库(LIRIS/EPFL general-purpose和LIRIS Masking)上的验证,其结果表明相比现有代表性方法,所提出方法的评价结果与人眼主观感知评价结果的一致性更好,表面其符合人眼的视觉感知。     

基于视觉感知特性的红外图像质量评价方法

根据红外成像特点,设计了一种基于视觉感知特性的红外图像质量评价算法。该算法结合人眼视觉和红外图像的结构信息对图像的失真程度进行描述,通过提取图像的边缘特征、对比度特征,然后利用视觉显著模型对特征进行差异融合,从而实现对红外失真图像的质量评测。实现结果表明,本文方法可对失真红外图像进行有效评价,与传统方法相较,此评价指标与人眼主观感知更一致。     

基于双目能量响应的无参考立体图像质量评价

为了实现对不同失真类型立体图像的质量评价,提出了一种基于双目能量响应的无参考立体图像质量评价(NR-IAQ)方法。首先,通过对各失真图像进行Gabor滤波,提取出不同频率、不同方向、不同视差响应下的局部特征矢量,作为立体图像特征信息;然后,利用支持向量回归(SVR)建立立体图像特征与主观评价值的关系,从而预测得到立体图像质量的客观评价值。实验结果表明,对于NBU-3D测试库,Pearson线性相关系数值在0.92以上,Spearman等级相关系数值在0.93以上;对于LIVE-3D测试库,Pearson线性相关系数值在0.96以上,Spearman等级相关系数值在0.96以上;与现有的全参考(FR)和(NR)质量评价方法相比,本方法得到的客观评价值与主观评价结果有较好的相关性,更加符合人眼视觉系统。     

Quality assessment of multiply and singly distorted stereoscopic images via adaptive construction of cyclopean views

A challenging problem confronted when designing a blind/no-reference (NR) stereoscopic image quality assessment (SIQA) algorithm is to simulate the quality assessment (QA) behavior of the human visual system (HVS) during binocular vision. An effective way to solve this problem is to estimate the quality of the merged single view created in the human brain which is also referred to as the cyclopean image. However, due to the difficulty in modeling the binocular fusion and rivalry properties of the HVS, obtaining effective cyclopean images for QA is non-trivial, and consequently previous NR SIQA algorithms either require the MOS/DMOS values of the distorted 3D images for training or ignore the quality analysis of the merged cyclopean view. In this paper, we focus on (1) constructing accurate and appropriate cyclopean views for QA of stereoscopic images by adaptively analyzing the distortion information of two monocular views, and (2) training NR SIQA models without requiring the assistance of the MOS/DMOS values in existing databases. Accordingly, we present an effective opinion-unaware SIQA algorithm called MUSIQUE-3D, which blindly assesses the quality of multiply and singly distorted stereoscopic images by analyzing quality degradations of both monocular and cyclopean views. The monocular view quality is estimated by an extended version of the MUSIQUE algorithm, and the cyclopean view quality is computed from the distortion parameter values predicted by a two-layer classification-regression model trained on a large 3D image dataset. Tests on various 3D image databases demonstrate the superiority of our method as compared with other state-of-the-art SIQA algorithms.     

利用梯度奇异值分解的图像结构相似度评价

针对传统的图像质量评价方法中对图像结构信息的表征能力不足的问题,在研究了基于结构相似度和奇异值分解的两种图像评价方法的基础上,结合其不同特点提出了基于奇异值分解的结构相似度质量评价方法.该算法分别将参考图像和失真图像的梯度图像分成8×8大小的图像块,并对每一个图像块进行奇异值分解后计算对应图像块的奇异值相似性和各图像块局部方差分布的相似性,最后结合各图像块的奇异值相似性和图像的局部方差分布的相似来表征图像的畸变程度.对LIVE库中包括5种失真类型的982幅图像进行验证,其结果表明该评价方法能很好地对各种失真类型的图像进行评价,比峰值信噪比(PSNR)、结构相似度(SSIM)等算法的主客观一致性更好,更加符合人眼的视觉特性.     

基于三维感知的立体虚拟视点图像质量评价方法

为了评价立体虚拟视点图像的质量,提出了一种基 于三维感知的客观评价方法。综合考虑了立体虚拟视点图像两大最主要失真类型:单视点绘 制失真和立体视点不匹配失真。针对单视点绘制失真,先提取 当前视点失真图与无失真图的差异性区域,再针对该差异性区域计算平均结构相似度(MSSIM ),最后将左 右视点平均池化作为单目纹理特征值;针对立体视点不匹配失真,先对左右视点失真图分别 进行视差映射, 再提取映射图与该视点失真图的差异区域作为双目不匹配区域,然后针对不匹配区域计算MS SIM 值,最 后将左右视点平均池化作为双目竞争特征值;最终将两个特征值幂次融合,作为最终的立体 虚拟视点图像 质量评价客观指标。实验结果表明本方法有效匹配主观打分的DMOS值,皮尔森线性相关系数 和斯皮尔曼 秩相关系数分别为0.911和0.900,正确反映了 立体虚拟视点图像质量。     

基于双目特征联合的无参考立体图像质量评价

通过模拟人类视觉系统(HVS)的双目视觉行为,提 出一种基于双目特征联合的无参考立 体图像质量评价(NR-SIQA)方法。首先分析立体视觉感知中的双目联合行为,提出 可应用于立体图像质量预 测的双目联合模型;然后采用学习和统计分析的方法,分别提取局部和全局特征并联合作 为感知特征; 最后采用机器学习算法,建立特征和质量的关系模型,并结合基于特征的双目联合模型预测 立体图像质量。实验结果表明,本文方法在对称立体图像库上的Pearson线性相关系数(PLCC)和Spearman等级系数(SRCC)高于0.93,在非对称库上高于0.87,优 于现有评价方法。     

A ParaBoost stereoscopic image quality assessment (PBSIQA) system

The problem of stereoscopic image quality assessment, which finds applications in 3D visual content delivery such as 3DTV, is investigated in this work. Specifically, we propose a new ParaBoost (parallel-boosting) stereoscopic image quality assessment (PBSIQA) system. The system consists of two stages. In the first stage, various distortions are classified into a few types, and individual quality scorers targeting at a specific distortion type are developed. These scorers offer complementary performance in face of a database consisting of heterogeneous distortion types. In the second stage, scores from multiple quality scorers are fused to achieve the best overall performance, where the fuser is designed based on the parallel boosting idea borrowed from machine learning. Extensive experimental results are conducted to compare the performance of the proposed PBSIQA system with those of existing stereo image quality assessment (SIQA) metrics. The developed quality metric can serve as an objective function to optimize the performance of a 3D content delivery system.     

No-reference Stereoscopic Image Quality Assessment Using Binocular Self-similarity and Deep Neural Network

Quality assessment of three-dimensional (3D) images is more challenging than that of 2D images. The quality of 3D visual experience is one of the most challenging areas of human binocular perception and is affected by multiple factors such as asymmetric stereo image/video compression, depth perception, visual discomfort, and single view quality. In this paper, we propose a new no-reference quality assessment method for stereoscopic images based on Binocular Self-similarity (BS) and Deep Neural Networks (DNN). To be more specific, a BS index is defined and computed according to binocular rivalry and suppression based on the depth image-based rendering technique. Then, a DNN is trained in an opinion unaware way to predict local quality. Binocular integration (BI) index is calculated by using the trained DNN, accounting for binocular integration behaviors. Finally, the final quality score of stereoscopic image is obtained by combining the BS and BI indexes together. Experimental results on four public 3D image quality assessment databases demonstrate that compared with existing methods, the proposed method can achieve high consistency with subjective perception on stereoscopic images with both symmetric and asymmetric distortions.     

No-reference stereoscopic image quality evaluator based on human visual characteristics and relative gradient orientation

Stereoscopic image quality assessment (SIQA) is of great significance to the development of modern three-dimensional (3D) display technology. In this work, by further mining the relationship between visual features and stereoscopic image quality perception, we build a new no-reference SIQA model, which combines the monocular and binocular features. Statistical quality-aware structural features from relative gradient orientation (RGO) map and texture features from the histogram of the weighted local binary pattern (LBP) in the texture image (TLBP) are not only extracted from both monocular view, but also extracted from binocular views to predict binocular quality perception. Meanwhile, the color statistical features ignored by most models and the binocularity feature is extracted to complement the monocular features and the above binocular features, respectively. Finally, all the extracted features and subjective scores are used to predict the objective quality score through the support vector regression (SVR) model. Experiments on four popular stereoscopic image databases show that the proposed model achieves high consistency with subjective assessment, and the performance of the model is very competitive with the latest models.     

基于双眼视觉感知特征一致性的立体图像体验质量评价

3D图像被认为是多媒体技术的重要标志,其中,立体图像质量对3D图像发展起到至关重要的作用。不同于传统的2D图像质量评价,在3D图像质量评价中引入关于体验质量（ QoE）问题的新挑战,因此,本文提出一个基于双眼视觉感知特征一致性的立体图像体验质量评价算法。具体地,先对2个视点图像提取像素梯度作为视觉感知的低层次特征,再用梯度方向直方图特征（ HOG）建立立体图像的视觉感知特征向量,然后,由支持向量回归（ SVR）方法来学习视觉感知特征与立体图像体验质量得分的关系,最后,通过训练好的SVR模型来预测立体图像体验质量。实验结果表明所提算法能够有效地预测立体图像体验质量。     

AdaBoost neural network and cyclopean view for no-reference stereoscopic image quality assessment

Stereoscopic imaging is widely used in many fields. In many scenarios, stereo images quality could be affected by various degradations, such as asymmetric distortion. Accordingly, to guarantee the best quality of experience, robust and accurate reference-less metrics are required for quality assessment of stereoscopic content. Most existing stereo no-reference Image Quality Assessment (IQA) models are not consistent with asymmetrical distortions. This paper presents a new no-reference stereoscopic image quality assessment metric using a human visual system (HVS) modeling and an advanced machine-learning algorithm. The proposed approach consists of two stages. In the first stage, cyclopean image is constructed considering the presence of binocular rivalry in order to cover the asymmetrically distorted part. In the second stage, gradient magnitude, relative gradient magnitude, and gradient orientation are extracted. These are used as a predictive source of information for the quality. In order to obtain the best overall performance against different databases, Adaptive Boosting (AdaBoost) idea of machine learning combined with artificial neural network model has been adopted. The benchmark LIVE 3D phase-I, phase-II, and IRCCyN/IVC 3D databases have been used to evaluate the performance of the proposed approach. Experimental results have demonstrated that the proposed metric performance achieves high consistency with subjective assessment and outperforms the blind stereo IQA over various types of distortion.     

基于深层特征学习的无参考立体图像质量评价

提出了一种基于深层特征学习的无参考(NR)立体图 像质量评价方 法。与传统人工提取图像特征不同,采用卷积神经网络(CNN)自动提取图像特征,评价过程 分为训练和 测试两阶段。在训练阶段,将图像分块训练CNN网络,利用CNN提取图像块特征,并结合不同 的整合方式 得到图像的全局特征,通过支持向量回归(SVR)建立主观质量与全局特征的回归模型;在测 试阶段,由已训练的CNN网 络和回归模型,得到左右图像和独眼图的质量。最后,根据人眼双目视觉特性融合左图像、 右图像和独眼 图的质量,得到立体图像质量。本文方法在LIVE-I和LIVE-II数据库上的Spearman等级系 数(SROCC)分别达 到了0.94,评价结果准确,与人眼的主 观感受一致。     

Image quality assessment using a SVD-based structural projection

The development of objective image quality assessment (IQA) metrics aligned with human perception is of fundamental importance to numerous image-processing applications. Recently, human visual system (HVS)-based engineering algorithms have received widespread attention for their low computational complexity and good performance. In this paper, we propose a new IQA model by incorporating these available engineering principles. A local singular value decomposition (SVD) is first utilised as a structural projection tool to select local image distortion features, and then, both perceptual spatial pooling and neural networks (NN) are employed to combine feature vectors to predict a single perceptual quality score. Extensive experiments and cross-validations conducted with three publicly available IQA databases demonstrate the accuracy, consistency, robustness, and stability of the proposed approach compared to state-of-the-art IQA methods, such as Visual Information Fidelity (VIF), Visual Signal to Noise Ratio (VSNR), and Structural Similarity Index (SSIM).