Image super-resolution based on deep neural network of multiple attention mechanism

At present, the main super-resolution (SR) method based on convolutional neural network (CNN) is to increase the layer number of the network by skip connection so as to improve the nonlinear expression ability of the model. However, the network also becomes difficult to be trained and converge. In order to train a smaller but better performance SR model, this paper constructs a novel image SR network of multiple attention mechanism(MAMSR), which includes channel attention mechanism and spatial attention mechanism. By learning the relationship between the channels of the feature map and the relationship between the pixels in each position of the feature map, the network can enhance the ability of feature expression and make the reconstructed image more close to the real image. Experiments on public datasets show that our network surpasses some current state-of-the-art algorithms in PSNR, SSIM, and visual effects.     

Dual-branch vision transformer for blind image quality assessment

Blind image quality assessment (BIQA) has always been a challenging problem due to the absence of reference images. In this paper, we propose a novel dual-branch vision transformer for BIQA, which simultaneously considers both local distortions and global semantic information. It first extracts dual-scale features from the backbone network, and then each scale feature is fed into one of the transformer encoder branches as a local feature embedding to consider the scale-variant local distortions. Each transformer branch obtains the context of global image distortion as well as the local distortion by adopting content-aware embedding. Finally, the outputs of the dual-branch vision transformer are combined by using multiple feed-forward blocks to predict the image quality scores effectively. Experimental results demonstrate that the proposed BIQA method outperforms the conventional methods on the six public BIQA datasets.     

基于QDCT和贝叶斯谱熵的盲彩色图像质量评价

图像质量评价算法在评价彩色图像质量时,往往会因损失色彩信息或者破坏彩色图像结构的整体性,而使得评价结果与人眼观测结果不一致.由于图像越模糊其频谱的高频分量分布越不均匀,基于四元数离散余弦变换(QDCT)和贝叶斯谱熵,提出了一种无参考模糊彩色图像质量评价算法.首先,利用四元数矩阵对彩色图像进行表示并分解成不重叠的8×8 ...     

Blind tone mapped image quality assessment with image segmentation and visual perception

With tone mapping, high dynamic range (HDR) image contents can be displayed on low dynamic range (LDR) display devices, in which some important visual information may be distorted. Thus, the tone mapped image (TMI) quality assessment is one of important issues in HDR image/video processing fields. Considering the difference of visual distortion degrees between the flat and complex regions in TMI, and considering that high-quality TMI should preserve as much information as possible of its original HDR image especially in the high/low luminance regions, this paper proposes a new blind TMI quality assessment method with image segmentation and visual perception. First, we design different features to describe the distortion of TMI’s different regions with two kinds of TMI segmentation. Then, considering that there lacks an efficient algorithm to quantify the importance of features, a feature clustering scheme is designed to eliminate the poor effect feature components in the extracted features to improve the effectiveness of the selected features. Finally, considering the diversity of tone mapping operator (TMO), which may cause global and local distortion of TMI, some other global features are also combined. At last, a final feature vector is formed to synthetically describe the distortion in TMI and used to blindly predict the TMI’s quality. Experimental results in the public ESPL-LIVE HDR database show that the Pearson linear correlation coefficient and Spearman rank order correlation coefficient of the proposed method reach 0.8302 and 0.7887, respectively, which is superior to the state-of-the-art blind TMI quality assessment methods, and it means that the proposed method is highly consistent with human visual perception.     

基于深层特征学习的无参考立体图像质量评价

提出了一种基于深层特征学习的无参考(NR)立体图 像质量评价方 法。与传统人工提取图像特征不同,采用卷积神经网络(CNN)自动提取图像特征,评价过程 分为训练和 测试两阶段。在训练阶段,将图像分块训练CNN网络,利用CNN提取图像块特征,并结合不同 的整合方式 得到图像的全局特征,通过支持向量回归(SVR)建立主观质量与全局特征的回归模型;在测 试阶段,由已训练的CNN网 络和回归模型,得到左右图像和独眼图的质量。最后,根据人眼双目视觉特性融合左图像、 右图像和独眼 图的质量,得到立体图像质量。本文方法在LIVE-I和LIVE-II数据库上的Spearman等级系 数(SROCC)分别达 到了0.94,评价结果准确,与人眼的主 观感受一致。     

融合半监督降维与稀疏表示的人脸识别方法

由于人脸图像数据的维数都较高,将稀疏表示分类用于人脸识别时计算量很大,为了提高人脸识别系统的效率,提出了一种融合半监督降维和稀疏表示的人脸识别方法。首先利用半监督降维算法对图像进行降维处理,在较低的维数空间快速取得较高的识别率,然后利用稀疏表示分类进行人脸识别,取得比传统的最近邻分类器更高的识别率,最后在ORL人脸库上进行实验验证。结果表明,利用该融合算法可快速有效地提高人脸图像的识别效果。     

Blind image quality assessment in the contourlet domain

No-reference/blind image quality assessment (NR-IQA/BIQA) algorithms play an important role in image evaluation, as they can assess the quality of an image automatically, only using the distorted image whose quality is being assessed. Among the existing NR-IQA/BIQA methods, natural scene statistic (NSS) models which can be expressed in different bandpass domains show good consistency with human subjective judgments of quality.In this paper, we create new ‘quality-aware’ features: the energy differences of the sub-band coefficients across scales via contourlet transform, and propose a new NR-IQA/BIQA model that operates on natural scene statistics in the contourlet domain. Prior to applying the contourlet transform, we apply two preprocessing steps that help to create more information-dense, low-entropy representations. Specifically, we transform the picture into the CIELAB color space and gradient magnitude map. Then, a number of ‘quality-aware’ features are discovered in the contourlet transform domain: the energy of the sub-band coefficients within scales, and the energy differences between scales, as well as measurements of the statistical relationships of pixels across scales. A detailed analysis is conducted to show how different distortions affect the statistical characteristics of these features, and then features are fed to a support vector regression (SVR) model which learns to predict image quality. Experimental results show that the proposed method has high linearity against human subjective perception, and outperforms the state-of-the-art NR-IQA models.     

基于注意力机制的多分支特征级联图像去雨网络

针对已有去雨网络在不同环境中去雨不彻底和图像细节信息损失严重的问题,本文提出一种基于注意力机制的多分支特征级联图像去雨网络。该模型结合多种注意力机制,形成不同类型的多分支网络,将图像空间细节和上下文特征信息在整体网络中自下而上地进行传递并级联融合,同时在网络分支间构建的阶段注意融合机制,可以减少特征提取过程中图像信息的损失,更大限度地保留特征信息,使图像去雨任务更加高效。实验结果表明,本文算法的客观评价指标优于其他对比算法,主观视觉效果得以有效提升,去雨能力更强,准确性更加突出,能够去除不同密度的雨纹,并且能够更好地保留图像背景中的细节信息。     

基于推理注意力机制的二阶段网络图像修复

现有的图像修复方法在处理大面积缺失或高度纹理化的图像时,通常会产生扭曲的结构或与周围区域不一致的模糊纹理,无法重建合理的图像结构。为此,提出了一种基于推理注意力机制的二阶段网络图像修复方法。首先通过边缘生成网络生成合理的幻觉边缘信息,然后在图像补全网络完成图像的重建工作。为了进一步生成视觉效果更逼真的图像,提高图像修复的精确度,在图像补全网络采用推理注意力机制,有效控制了生成特征的不一致性,从而生成更有效的信息。所提方法在多个数据集上进行了实验验证,结果表明该图像修复方法的结构相似性指数达到了88.9%,峰值信噪比达到了25.56 dB,与现有的图像修复方法相比,该方法具有更高的图像修复精确度,生成的图像更逼真。     

Blind video quality assessment based on multilevel video perception

Blind video quality assessment (VQA) metrics predict the quality of videos without the presence of reference videos. This paper proposes a new blind VQA model based on multilevel video perception, abbreviated as MVP. The model fuses three levels of video features occurring in natural video scenes to predict video quality: natural video statistics (NVS) features, global motion features and motion temporal correlation features. They represent video scene characteristics, video motion types, and video temporal correlation variations. In the process of motion feature extraction, motion compensation filtering video enhancement is adopted to highlight the motion characteristics of videos so as to improve the perceptual correlations of the video features. The experimental results on the LIVE and CSIQ video databases show that the predicted video scores of the new model are highly correlated with human perception and have low root mean square errors. MVP obviously outperforms state-of-art blind VQA metrics, and particularly demonstrates competitive performance even compared against top-performing full reference VQA metrics.     

基于多尺度特征融合和注意力机制的医学图像分割网络

针对传统编解码结构的医学图像分割网络存在特征信息利用率低、泛化能力不足等问题,该文提出了一种结合编解码模式的多尺度语义感知注意力网络(multi-scale semantic perceptual attention network,MSPA-Net) 。首先,该网络在解码路径加入双路径多信息域注意力模块(dual-channel multi-information domain attention module,DMDA) ,提高特征信息的提取能力;其次,网络在级联处加入空洞卷积模块(dense atrous convolution module,DAC) ,扩大卷积感受野;最后,借鉴特征融合思想,设计了可调节多尺度特征融合模块 (adjustable multi-scale feature fusion,AMFF) 和双路自学习循环连接模块(dual self-learning recycle connection module,DCM) ,提升网络的泛化性和鲁棒性。为验证网络的有效性,在CVC-ClinicDB、ETIS-LaribPolypDB、COVID-19 CHEST X-RAY、Kaggle_3m、ISIC2017和Fluorescent Neuronal Cells等数据 集上进行验证,实验结果表明,相似系数分别达到了94.96%、92.40%、99.02%、90.55%、92.32%和75.32%。因此,新的分割网络展现了良好的泛化能力,总体性能优于现有网络,能够较好实现通用医学图像的有效分割。     

Deep network based stereoscopic image quality assessment via binocular summing and differencing

With the development of deep networks in dealing with various visual tasks, the deep network based on binocular vision is expected to tackle the issue of stereoscopic image quality assessment. Here, we present a stereoscopic image quality assessment method using the deep network with four channels together, which takes the left view, right view, binocular summing view, and binocular differencing view as the inputs of the network. The visual features are enhanced through the concatenation in a weighted way, so that the binocular vision can be adequately included in the binocular addition and subtraction information. Compared with the state-of-the-art metrics, the proposed method exhibits relatively high performances on four benchmark databases.     

基于边缘保持和注意力生成对抗网络的红外与可见光图像融合

由于红外与可见光图像特征差异大,并且不存在理想的融合图像监督网络学习源图像与融合图像之间的映射关系,深度学习在图像融合领域的应用受到了限制。针对此问题,提出了一个基于注意力机制和边缘损失函数的生成对抗网络框架,应用于红外与可见光图像融合。通过引入对抗训练和注意力机制的思想,将融合问题视为源图像和融合图像对抗的关系,并结合了通道注意力和空间注意力机制学习特征通道域和空间域的非线性关系,增强了显著性目标特征表达。同时提出了一种边缘损失函数,将源图像与融合图像像素之间的映射关系转化为边缘之间的映射关系。多个数据集的测试结果表明,该方法能有效融合红外目标和可见光纹理信息,锐化图像边缘,显著提高图像清晰度和对比度。     

Unsupervised blind image quality assessment based on joint structure and natural scene statistics features

Compared with the widely used supervised blind image quality assessment (BIQA) models, unsupervised BIQA models require little prior knowledge for calculating the objective quality scores of distorted images. In this paper, we propose an unsupervised BIQA method that aims to achieve both good performance and generalization capability with low computational complexity. Carefully selected and extensive structure and natural scene statistics (NSS) features can better represent image quality. First, we employ phase congruency (PC) and finely selected gradient magnitude map and Laplacian of Gaussian response (GM-LOG) features to represent image structure information. Second, we calculate the local mean-subtracted and contrast-normalized (MSCN) coefficients and the Karhunen–Loéve transform (KLT) coefficients to represent the naturalness of the distorted images. Last, multivariate Gaussian (MVG) model with joint features extracted from both the pristine images and the distorted images is adopted to calculate the objective image quality. Extensive experiments conducted on nine IQA databases demonstrate that the proposed method achieves better performance than the state-of-the-art BIQA methods.     

Color image quality assessment based on sparse representation and reconstruction residual

Image quality assessment (IQA) is a fundamental problem in image processing. While in practice almost all images are represented in the color format, most of the current IQA metrics are designed in gray-scale domain. Color influences the perception of image quality, especially in the case where images are subject to color distortions. With this consideration, this paper presents a novel color image quality index based on Sparse Representation and Reconstruction Residual (SRRR). An overcomplete color dictionary is first trained using natural color images. Then both reference and distorted images are represented using the color dictionary, based on which two feature maps are constructed to measure structure and color distortions in a holistic manner. With the consideration that the feature maps are insensitive to image contrast change, the reconstruction residuals are computed and used as a complementary feature. Additionally, luminance similarity is also incorporated to produce the overall quality score for color images. Experiments on public databases demonstrate that the proposed method achieves promising performance in evaluating traditional distortions, and it outperforms the existing metrics when used for quality evaluation of color-distorted images.     

结合高效通道注意模块的残差网络骨龄评估

本文针对现有骨龄评估数据集数据规模小,样本分布不均匀以及现有方法评估准确度较低的问题,提出了一种新的结合高效通道注意模块的残差网络骨龄评估模型.通过结合深度残差网络和高效通道注意模块来提高卷积效率,并改进损失函数,缓解样本分布不均匀问题的影响;然后运用迁移学习的方法微调训练骨龄评估模型,提高模型训练效率;最后引入随机深...     

基于深度学习网络的可见光图像重构质量评价研究

可见光图像重构质量评价一直是一个难点,因此,设计了基于深度学习网络的可见光图像重构质量评价方法.通过卷积神经网络(CNN)与图像质量评价方法(IQA)相结合,构成IQA-CNN模型,引入信息熵构建改进IQA-CNN模型,向该模型内输入重构可见光图像,归一化预处理后划分成数个分块,经有监督学习法训练该模型后,获取到该模型...     

Robust image retrieval by cascading a deep quality assessment network

The performance of computer vision algorithms can severely degrade in the presence of a variety of distortions. While image enhancement algorithms have evolved to optimize image quality as measured according to human visual perception, their relevance in maximizing the success of computer vision algorithms operating on the enhanced image has been much less investigated. We consider the problem of image enhancement to combat Gaussian noise and low resolution with respect to the specific application of image retrieval from a dataset. We define the notion of image quality as determined by the success of image retrieval and design a deep convolutional neural network (CNN) to predict this quality. This network is then cascaded with a deep CNN designed for image denoising or super resolution, allowing for optimization of the enhancement CNN to maximize retrieval performance. This framework allows us to couple enhancement to the retrieval problem. We also consider the problem of adapting image features for robust retrieval performance in the presence of distortions. We show through experiments on distorted images of the Oxford and Paris buildings datasets that our algorithms yield improved mean average precision when compared to using enhancement methods that are oblivious to the task of image retrieval. ¹     

基于视觉感知特性的红外图像质量评价方法

根据红外成像特点,设计了一种基于视觉感知特性的红外图像质量评价算法。该算法结合人眼视觉和红外图像的结构信息对图像的失真程度进行描述,通过提取图像的边缘特征、对比度特征,然后利用视觉显著模型对特征进行差异融合,从而实现对红外失真图像的质量评测。实现结果表明,本文方法可对失真红外图像进行有效评价,与传统方法相较,此评价指标与人眼主观感知更一致。     

融合显著图和保真图的全参考图像质量评价

提出一种融合显著图(SM)和保真图(FM)的全参考图 像质量 评价算法,用于评价质降图像的失真度。利用亮度和色度的相似度提取质降图像相对于 参考图像的FM;对参考图像进行区域划分、全局显著性提取和纹理边缘补充得到SM,将SM与 质降图像的FM融合得到基 于感知的显著保真图(PSM),计算质降图像的客观评价得分。在标准数据库上的实验结果表 明,本文方法与主观评价能够很好保持一致,并对LIVE图像库中的5种失真图像均有很好的 表现。