基于深度学习的低照度图像增强方法

在低照度环境下采集的图像往往亮度不足,导致在后续视觉任务中难以有效利用.针对这一问题,过去的低照度图像增强方法大多在极度低光场景中表现失败,甚至放大了图像中的底层噪声.为了解决这一难题,本文提出了 一种新的基于深度学习的端到端神经网络,该网络主要通过空间和通道双重注意力机制来抑制色差和噪声,其中空间注意力模块利用图像的...     

基于双重迭代的零样本低照度图像增强

针对低光照条件下拍摄图像质量低下的问题,该文提出一种基于双重迭代的零样本低照度图像增强方法。其外层迭代通过卷积神经网络估计增强参数,再由内层迭代进行图像增强,增强结果进一步用于计算损失函数并反馈更新外层的参数估计网络,最终通过多轮迭代生成高质量的图像。在该框架下,还设计了多尺度增强系数估计模块、基于注意力的像素级大气光估计模块,并提出了基于亮度对比度、大气光、颜色均衡以及图像平滑性先验的无监督损失函数。大量实验结果表明,该方法可有效将低光照图像增强为高质量的清晰图像,其性能优于现有的同类方法。同时该方法基于零样本学习,不需任何训练数据集,具有良好的普适性。     

Event-guided low light image enhancement via a dual branch GAN

In the low light conditions, images are corrupted by low contrast and severe noise, but event cameras capture event streams with clear edge structures. Therefore, we propose an Event-Guided Low Light Image Enhancement method using a dual branch generative adversarial networks and recover clear structure with the guide of events. To overcome the lack of paired training datasets, we first synthesize three datasets containing low-light event streams, low-light images, and the ground truth normal-light images. Then, in the generator network, we develop an end-to-end dual branch network consisting of a image enhancement branch and a gradient reconstruction branch. The image enhancement branch is employed to enhance the low light images, and the gradient reconstruction branch is utilized to learn the gradient from events. Moreover, we develops the attention based event-image feature fusion module which selectively fuses the event and low-light image features, and the fused features are concatenated into the image enhancement branch and gradient reconstruction branch, which respectively generate the enhanced images with clear structure and more accurate gradient images. Extensive experiments on synthetic and real datasets demonstrate that the proposed event guided low light image enhancement method produces visually more appealing enhancement images, and achieves a good performance in structure preservation and denoising over state-of-the-arts.     

基于双注意力机制的雾天图像清晰化算法研究

针对传统去雾算法容易依赖先验知识以及恢复出来的清晰图像会产生颜色失真等问题,本文提出一种基于双注意力机制的雾天图像清晰化算法。首先将雾图输入编码器,经过下采样后得到特征图像;特征提取模块将多个特征提取基本块联结在一起,每个基本块由局部残差学习和特征注意模块组成,提高图像质量以及图像特征信息的利用率,增加网络训练的稳定性;然后通过通道注意力与多尺度空间注意力并行的结构处理特征图像,使得网络更加关注细节特征,提取更多关键信息,同时提高网络效率;最后将融合后的特征图像输入解码器中,经过多级映射,得到与输入大小匹配的雾密度图。实验结果表明,不论是对合成雾天图像或者真实雾天图像,本文算法能够高效地进行去雾处理,得到更自然的清晰图像。     

Low-light image enhancement via deep Retinex decomposition and bilateral learning

Low-light images enhancement is a challenging task because enhancing image brightness and reducing image degradation should be considered simultaneously. Although existing deep learning-based methods improve the visibility of low-light images, many of them tend to lose details or sacrifice naturalness. To address these issues, we present a multi-stage network for low-light image enhancement, which consists of three sub-networks. More specifically, inspired by the Retinex theory and the bilateral grid technique, we first design a reflectance and illumination decomposition network to decompose an image into reflectance and illumination maps efficiently. To increase the brightness while preserving edge information, we then devise an attention-guided illumination adjustment network. The reflectance and the adjusted illumination maps are fused and refined by adversarial learning to reduce image degradation and improve image naturalness. Experiments are conducted on our rebuilt SICE low-light image dataset, which consists of 1380 real paired images and a public dataset LOL, which has 500 real paired images and 1000 synthetic paired images. Experimental results show that the proposed method outperforms state-of-the-art methods quantitatively and qualitatively.     

基于移位窗口多头自注意力U型网络的低照度图像增强方法

针对低照度图像增强模型中的亮度提升、噪声抑制以及保持纹理颜色一致性等难点问题,该文提出一种基于移位窗口自注意力机制的低照度图像增强方法。该文以U型结构为基本框架,以移位窗口多头自注意力模型为基础,构建了由编码器、解码器以及跳跃连接组成的图像增强网络。该网络将自注意力机制的特征提取优势应用到低照度图像增强领域,建立图像特征信息之间的长期依赖关系,能够有效获取全局特征。将所提方法与当前流行的算法进行定量和定性对比试验,主观感受上,该文方法显著提升了图像亮度,抑制图像噪声效果明显并较好地保持了纹理细节和颜色信息;在峰值信噪比(PSNR)、结构相似性(SSIM)和图像感知相似度(LPIPS)等客观指标方面,该方法较其他方法的最优值分别提高了0.35 dB, 0.041和0.031。实验结果表明,该文所提方法能够有效提升低照度图像的主观感受质量和客观评价指标,具有一定的应用价值。     

融合注意力机制的高分辨人脸识别图像重建

针对由于人脸姿势、光照不均、拍摄环境、拍摄设备等内外部因素造成图像分辨率低的问题,提出融合注意力机制的高分辨人脸识别图像重建模型.首先以低分辨率人脸图像对作为两个生成器输入,通过残差块和注意力模块堆叠网络提取人脸特征信息,进而生成高分辨率人脸图像.训练中使用一个鉴别器来监督两个生成器的训练过程.利用Adam算法对鉴别器...     

DSE-Net: Deep simultaneous estimation network for low-light image enhancement

This paper presents a novel approach for low-light image enhancement. We propose a deep simultaneous estimation network (DSE-Net), which simultaneously estimates the reflectance and illumination for low-light image enhancement. The proposed network contains three modules: image decomposition, illumination adjustment, and image refinement module. The DSE-Net uses a novel branched encoder–decoder based image decomposition module for simultaneous estimation. The proposed decomposition module uses a separate decoder to estimate illumination and reflectance. DSE-Net improves the estimated illumination using the illumination adjustment module and feeds it to the proposed refinement module. The image refinement module aims to produce sharp and natural-looking output. Extensive experiments conducted on a range of low-light images demonstrate the efficacy of the proposed model and show its supremacy over various state-of-the-art alternatives.     

Deep image compression based on multi-scale deformable convolution

Deep image compression efficiency has been improved in the past years. However, to fully exploit context information for compressing image objects of different scales and shapes, more adaptive geometric structure of inputs should be considered. In this paper, we novelly introduce deformable convolution and its spatial attention extension into deep image compression task to fully exploit the context information. Specifically, a novel deep image compression network with Multi-Scale Deformable Convolution and Spatial Attention, named MS-DCSA, is proposed to better extract compact and efficient latent representation as well as reconstruct higher-quality images. First, multi-scale deformable convolution is presented to provide multi-scale receptive fields for learning spatial sampling offsets in deformable operations. Subsequently, multi-scale deformable spatial attention module is developed to generate attention masks to re-weight extracted features according to their importance. In addition, the multi-scale deformable convolution is applied to design delicate up/down sampling modules. Extensive experiments demonstrate that the proposed MS-DCSA network achieves improved performance on both PSNR and MS-SSIM quality metrics, compared to conventional as well as competing deep image compression methods.     

Multi-scale fusion residual encoder-decoder approach for low illumination image enhancement

The sensing light source of the line scan camera cannot be fully exposed in a low light environment due to the extremely small number of photons and high noise, which leads to a reduction in image quality. A multi-scale fusion residual encoder-decoder (FRED) was proposed to solve the problem. By directly learning the end-to-end mapping between light and dark images, FRED can enhance the image's brightness with the details and colors of the original image fully restored. A residual block (RB) was added to the network structure to increase feature diversity and speed up network training. Moreover, the addition of a dense context feature aggregation module (DCFAM) made up for the deficiency of spatial information in the deep network by aggregating the context's global multi-scale features. The experimental results show that the FRED is superior to most other algorithms in visual effect and quantitative evaluation of peak signa-to-noise ratio (PSNR) and structural similarity index measure (SSIM). For the factor that FRED can restore the brightness of images while representing the edge and color of the image effectively, a satisfactory visual quality is obtained under the enhancement of low-light.     

Global guidance-based integration network for salient object detection in low-light images

Most of current salient object detection (SOD) methods focus on well-lit scenes, and their performance drops when generalized into low-light scenes due to limitations such as blurred boundaries and low contrast. To solve this problem, we propose a global guidance-based integration network (G2INet) customized for low-light SOD. First, we propose a Global Information Flow (GIF) to extract comprehensive global information, for guiding the fusion of multi-level features. To facilitate information integration, we design a Multi-level features Cross Integration (MCI) module, which progressively fuses low-level details, high-level semantics, and global information by interweaving. Furthermore, a U-shaped Attention Refinement (UAR) module is proposed to further refine edges and details for accurate saliency predictions. In terms of five metrics, extensive experimental results demonstrate that our method outperforms the existing twelve state-of-the-art models.     

基于注意力的多尺度水下图像增强网络

水下图像往往会因为光的吸收和散射而出现颜色退化与细节模糊的现象,进而影响水下视觉任务。该文通过水下成像模型合成更接近水下图像的数据集,以端到端的方式设计了一个基于注意力的多尺度水下图像增强网络。在该网络中引入像素和通道注意力机制,并设计了一个多尺度特征提取模块,在网络开始阶段提取不同层次的特征,通过带跳跃连接的卷积层和注意力模块后得到输出结果。多个数据集上的实验结果表明,该方法在处理合成水下图像和真实水下图像时都能有很好的效果,与现有方法相比能更好地恢复图像颜色和纹理细节。     

一种有效融合多尺度特征的图像语义分割方法

卷积神经网络在高级计算机视觉任务中展现出强 大的特征学习能力,已经在图像语义 分割任务 中取得了显著的效果。然而,如何有效地利用多尺度的特征信息一直是个难点。本文提出一 种有效 融合多尺度特征的图像语义分割方法。该方法包含4个基础模块,分别为特征融合模块(feature fusion module,FFM)、空 间信息 模块(spatial information module,SIM)、全局池化模块(global pooling module,GPM)和边界细化模块(boundary refinement module,BRM)。FFM采用了注意力机制和残差结构,以提高 融合多 尺度特征的效率,SIM由卷积和平均池化组成,为模型提供额外的空间细节信息以 辅助定 位对象的边缘信息,GPM提取图像的全局信息,能够显著提高模型的性能,BRM以残差结构为核心,对特征图进行边界细化。本文在全卷积神经网络中添加4个基础模块, 从而有 效地利用多尺度的特征信息。在PASCAL VOC 2012数据集上的实验结 果表明该方法相比全卷积神 经网络的平均交并比提高了8.7%,在同一框架下与其他方法的对比结 果也验证了其性能的有效性。     

并行多特征提取网络的红外图像增强方法

为解决低质量红外图像细节模糊、对比度低等问题,提出了并行多特征提取网络的红外图像增强方法,设计了结构特征映射网络和双尺度特征提取网络。结构特征映射网络用于建立全局结构特征权重,以保持原始图像的空间结构信息。双尺度特征提取网络采用多尺度卷积层和融合多空洞卷积的注意力,增强网络对上下文信息的关注力,提升网络对感兴趣区域的特征提取能力,同时学习不同尺度的特征信息,完成双尺度间信息的交换,生成目标增强映射,实现目标区域细节纹理自适应增强。实验证明,所提方法能有效提高对比度,避免过增强,丰富图像细节纹理,减少伪影和光晕现象,在BSD200数据集上的PSNR与SSIM较典型的传统方法和深度学习方法分别提升了约37.35%、2.1%与25.94%、3.15%,在真实红外数据集上分别提升了约30.62%、1.04%与24.83%、2.08%,且对不同对比度因子的低质量图像,文中方法也具有良好的增强效果。     

Single image deblurring with cross-layer feature fusion and consecutive attention

Single image deblurring aims to restore the single blurry image to its sharp counterpart and remains an active topic of enduring interest. Recently, deep Convolutional Neural Network (CNN) based methods have achieved promising performance. However, two primary limitations mainly exist on those CNNs-based image deblurring methods: most of them simply focus on increasing the complexity of the network, and rarely make full use of features extracted by encoder. Meanwhile, most of the methods perform the deblurred image reconstruction immediately after the decoder, and the roles of the decoded features are always underestimated. To address these issues, we propose a single image deblurring method, in which two modules to fuse multiple features learned in encoder (the Cross-layer Feature Fusion (CFF) module) and manipulate the features after decoder (the Consecutive Attention Module (CAM)) are specially designed, respectively. The CFF module is to concatenate different layers of features from encoder to enhance rich structural information to decoder, and the CAM module is able to generate more important and correlated textures to the reconstructed sharp image. Besides, the ranking content loss is employed to further restore more realistic details in the deblurred images. Comprehensive experiments demonstrate that our proposed method can generate less blur and more textures in deblurred image on both synthetic datasets and real-world image examples.     

结合多尺度深度学习网络和Retinex理论的低光照图像增强算法

针对低光照增强任务缺乏参考图像及现有算法存在的色彩失真、纹理丢失、细节模糊、真值图像获取难等问题,本文提出了一种基于Retinex理论与注意力机制的多尺度加权特征低光照图像增强算法。该算法通过基于Unet架构的特征提取模块对低光照图像进行多尺度的特征提取,生成高维度的多尺度特征图;建立注意力机制模块凸显对增强图像有利的不同尺度的特征信息,得到加权的高维特征图;最后反射估计模块中利用Retinex理论建立网络模型,通过高维特征图生成最终的增强图像。设计了一个端到端的网络架构并利用一组自正则损失函数对网络模型进行约束,摆脱了参考图像的约束,实现了无监督学习。最终实验结果表明本文算法在增强图像的对比度与清晰度的同时维持了较高的图像细节与纹理,具有良好的视觉效果,能够有效增强低光照图像,视觉质量得到较大改善;并与其他多种增强算法相比,客观指标PSNR和SSIM得到了提高。     

Stereoscopic image quality assessment considering visual mechanism and multi-loss constraints

In this paper, a convolutional neural network (CNN) with multi-loss constraints is designed for stereoscopic image quality assessment (SIQA). A stereoscopic image not only contains monocular information, but also provides binocular information which is as identically crucial as the former. So we take the image patches of left-view images, right-view images and the difference images as the inputs of the network to utilize monocular information and binocular information. Moreover, we propose a method to obtain proxy label of each image patch. It preserves the quality difference between different regions and views. In addition, the multiple loss functions with adaptive loss weights are introduced in the network, which consider both local features and global features and constrain the feature learning from multiple perspectives. And the adaptive loss weights also make the multi-loss CNN more flexible. The experimental results on four public SIQA databases show that the proposed method is superior to other existing SIQA methods with state-of-the-art performance.     

基于渐进式特征增强网络的超分辨率重建算法

为了在图像重建质量和网络参数之间取得较好的平衡,本文提出一种基于渐进式特征增强网络的超分辨率（Super-Resolution,SR）重建算法。该方法主要包含两个模块:浅层信息增强模块和深层信息增强模块。在浅层信息增强模块中,首先利用单层卷积层提取低分辨率（Low-Resolution,LR）图像的浅层信息,再通过我们设计的多尺度注意力块来实现特征的提取和增强。深层信息增强模块先利用残差学习块学习图像的深度信息,然后将得到的深层信息通过设计的多尺度注意力块来获得增强后的深层多尺度信息。最后我们利用跳转连接的方式将首层得到的浅层信息和深层多尺度信息进行像素级相加得到融合特征图,再对其进行上采样操作,得到最终的高分辨率（High-Resolution, HR）图像。实验结果表明,相比于一些主流的深度学习超分辨率方法,本文方法重建得到的图像无论是主观效果还是客观指标,都取得了更好的效果。      

红外与可见光图像多尺度Transformer融合方法

目前主流的深度融合方法仅利用卷积运算来提取图像局部特征,但图像与卷积核之间的交互过程与内容无关,且不能有效建立特征长距离依赖关系,不可避免地造成图像上下文内容信息的丢失,限制了红外与可见光图像的融合性能。为此,本文提出了一种红外与可见光图像多尺度Transformer融合方法。以Swin Transformer为组件,架构了Conv Swin Transformer Block模块,利用卷积层增强图像全局特征的表征能力。构建了多尺度自注意力编码-解码网络,实现了图像全局特征提取与全局特征重构;设计了特征序列融合层,利用SoftMax操作计算特征序列的注意力权重系数,突出了源图像各自的显著特征,实现了端到端的红外与可见光图像融合。在TNO、Roadscene数据集上的实验结果表明,该方法在主观视觉描述和客观指标评价都优于其他典型的传统与深度学习融合方法。本方法结合自注意力机制,利用Transformer建立图像的长距离依赖关系,构建了图像全局特征融合模型,比其他深度学习融合方法具有更优的融合性能和更强的泛化能力。     

CCFNet: Cross-Complementary fusion network for RGB-D scene parsing of clothing images

Schemes to complement context relationships by cross-scale feature fusion have appeared in many RGB-D scene parsing algorithms; however, most of these works conduct multi-scale information interaction after multi-modal feature fusion, which ignores the information loss of the two modes in the original coding. Therefore, a cross-complementary fusion network (CCFNet) is designed in this paper to calibrate the multi-modal information before feature fusion, so as to improve the feature quality of each mode and the information complementarity ability of RGB and the depth map. First, we divided the features into low, middle, and high levels, among which the low-level features contain the global details of the image and the main learning features include texture, edge, and other features. The middle layer features contain not only some global detail features but also some local semantic features. Additionally, the high-level features contain rich local semantic features. Then, the feature information lost in the coding process of low and middle level features is supplemented and extracted through the designed cross feature enhancement module, and the high-level features are extracted through the feature enhancement module. In addition, the cross-modal fusion module is designed to integrate multi-modal features of different levels. The experimental results verify that the proposed CCFNet achieves excellent performance on the RGB-D scene parsing dataset containing clothing images, and the generalization ability of the model is verified by the dataset NYU Depth V2.