Coupled adversarial variational autoencoder

Generating image is a hot research topic in the field of deep learning, and it is challenging for generating high quality image pairs. The image pair refers to the corresponding image tuples with the same high-level features and different low-level features, generating high-quality image pairs has important applications in some specific fields. Currently, there are many methods to generate high quality images, but these methods cannot produce higher resolution image pairs. To address this problem, we proposed a novel model which consists of two adversarial variational autoencoders, each one aim at generating an image of pairs more accurately. We called this model CoAdVAE (coupled adversarial variational autoencoders), it can generate high quality image pairs due to introducing adversarial learning to the model. In the experiments, we applied the proposed model to three learning tasks, i.e., generating image pairs with different attributes, converting image attributes, and image dehazing. We show by experiments compared with related approaches on four datasets, Mnist, Celeba, AFHQ, and Fog_data that the proposed model can achieve the-state-of-the-art results.     

Blind compression artifact reduction using dense parallel convolutional neural network

Different artifacts will manifest, whenever an image is compressed by a lossy compression algorithm. Higher frequency details present in the image may tend to be eliminated by compression. In certain cases, compression may introduce small image structures and noise. This phenomenon will limit the image quality thereby making images to appear much less pleasant to the human eye. Furthermore, other machine learning tasks like object detectors performance will be reduced due to compression. In this paper, we introduce a novel deep neural network with densely connected parallel convolutions to remove such artifacts and to recover the original image from its perturbed version. The proposed algorithm is named as densely connected parallel convolutional neural network in short DPCNN. Parallel convolution provides model parallelism and reduce the training burden. Furthermore, the dense skip connections provide short paths for gradient back-propagation and alleviates the gradient vanishing problem. Moreover, skip connections reduce the feature redundancy by combining features from different levels and increases the learning efficiency. However, these skip connections increase the model complexity. A bottleneck layer is used to keep the model compactness and to reduce the model complexity. The proposed approach can be used as a preprocessing step in different computer vision tasks where images are degraded by compression. Different from other methods, the proposed method is able to remove compression artifacts generated at any quality factor (QF). The experiments on benchmark datasets show the superiority of the proposed method over other methods quantitatively and qualitatively.     

基于非局部先验的深度压缩感知图像重构网络

传统的基于迭代的压缩感知(CS)图像重构算法易于集成图像先验信息,但存在性能不足、计算复杂度高等缺点。基于深度学习的图像重构算法重构性能通常优于传统的重构算法,并且具有更低的重构计算成本。因此,为了设计出一种更有效利用先验信息的深度学习图像重构算法,该文提出基于非局部先验的深度压缩感知图像重构网络。首先,将稀疏性和非局部先验相结合建立压缩感知图像重构模型,然后通过半二次方分裂法将模型分解为3个子问题,每一个子问题的求解都在深度学习的框架下展开,最后联合建立端到端的可训练的图像重构模型。仿真实验表明,在测试的采样率与数据集下该文所提算法的峰值信噪比与当前主流的重构算法SCSNet相比平均提升了0.18 dB,与CSNet算法相比平均提升了约1.59 dB,与ISTA-Net+算法相比平均提升了约2.09 dB。     

A variational method for multiple-image blending

The main aim of this paper is to develop an algorithm for blending of multiple images in the image-stitching process. Our idea is to propose a variational method containing an energy functional to determine both a stitched image and weighting mask functions of multiple input images for image blending. The existence of the solution of the proposed energy functional is shown. We also present an alternative minimizing algorithm to solve the proposed model numerically and show the convergence of this algorithm. Experimental results show that the proposed model works effectively and efficiently and that the proposed method is competitive with the tested existing methods under noisy conditions.     

变分框架下的多尺度图像恢复与重建

变分图像分解,通过极小化能量泛函将图像分解为不同的特征分量,可以被应用到图像的恢复和重建.提出了变分框架下的多尺度图像恢复和重建的思想.基于这种思想,首先提出了一个单参数的（BV,G,E）三元变分分解模型,并且理论分析了参数与不同特征分量的尺度的关系.然后将此模型的参数选为一个二进制序列,得到多尺度的（BV,G,E）变分分解.该多尺度变分分解可以将图像分解为一序列图像结构、纹理和噪声.证明了此多尺度分解的收敛性并且基于对偶理论和交替迭代算法给出了其数值求解方法.最后将提出的多尺度的（BV,G,E）变分分解应用到图像恢复和重建,实验结果证实了理论分析的正确性,显示了将此模型进行图像多尺度恢复和重建的有效性,和与一些其他分解模型相比较的优越性.     

Unifying probabilistic and variational estimation

A maximum a posteriori (MAP) estimator using a Markov or a maximum entropy random field model for a prior distribution may be viewed as a minimizer of a variational problem.Using notions from robust statistics, a variational filter referred to as a Huber gradient descent flow is proposed. It is a result of optimizing a Huber functional subject to some noise constraints and takes a hybrid form of a total variation diffusion for large gradient magnitudes and of a linear diffusion for small gradient magnitudes. Using the gained insight, and as a further extension, we propose an information-theoretic gradient descent flow which is a result of minimizing a functional that is a hybrid between a negentropy variational integral and a total variation. Illustrating examples demonstrate a much improved performance of the approach in the presence of Gaussian and heavy tailed noise. In this article, we present a variational approach to MAP estimation with a more qualitative and tutorial emphasis. The key idea behind this approach is to use geometric insight in helping construct regularizing functionals and avoiding a subjective choice of a prior in MAP estimation. Using tools from robust statistics and information theory, we show that we can extend this strategy and develop two gradient descent flows for image denoising with a demonstrated performance.     

图像分类问题的几种深度学习策略

针对图像分类学习不够深入的问题,提出图像分类问题的几种深度学习策略研究。通过分析当前主流的主动深度学习图像、多标签图像和多尺度网络图像三种深度学习方法的工作原理和存在的优势与不足,探讨图像分类问题的优化学习策略。随后采用图像分类问题的几种深度学习策略实验的方式加以对比,实验结果表明,参数共享的深度学习图像分类方法不仅提高了预测速度,而且还能确保模型的准确性。     

基于正则化稀疏表示的图像超分辨率算法

为了从单幅低分辨率(LR)图像恢复出高分辨率(H R)图像,提出了一种应用正则化稀疏表示和基于机器学习 的超分辨率(SR)图像恢复算法。构造了一种基于稀疏表示的SR凸变模型,为了提高 恢复效果,针对模型 提出了两种稀疏正则化约束条件,一是将分类效果更好的图表拉普拉斯作为正则化约束条件 ,从而找到与 输入LR图像块在结构上最接近的学习样本;另一种是针对冗余的学习样本进行约 束,保证了图像边 缘的锐利。将输入的每一块LR图像应用正则化稀疏表示,经过学习得到与之对应的HR图像块 , 最终得到整幅HR图像。试验结果表明,算法恢复出的HR图像峰值信噪比(PSNR )值较双三次插值算法最高提升约2dB,主观目视清晰、边缘锐利。     

Variational Retinex model based on an extension of TV regularization with relative gradient and its application

In view of the shortcomings of the total variational Retinex model which use the total variation (TV) of the reflection as the regularization.An extension of TV regularization with the concept of relative gradient was introduced and finally a new variational Retinex model was proposed.Compared with variational Retinex and total variational Retinex model,the proposed model can preserve the estimated reflectance with more details as well as the more smoothed illumination.Further,a new integrated image enhancement model considering both the illumination and the reflectance was proposed.By adjusting the model parameters,the proposed model can be effectively applied to high dynamic range image tone mapping and non-uniform illumination enhancement.Compared with other algorithms,the proposed model can better handle the above image enhancement problems.     

From synthetic to natural — single natural image dehazing deep networks using synthetic dataset domain randomization

Image dehazing methods aim to solve the problem of poor visibility in images due to haze. Techniques proposed for image dehazing in literature focus on image priors, haze lines or data driven statistical models. Variations of the classical methods relying on prior model or haze line model use no-reference image quality metrics to prove their dehazing performance. Recently developed deep learning models rely on huge amounts of hazy, haze-free pairs for training, and uses PSNR and SSIM like image reconstruction metrics to show their performance. These methods perform poorly on no-reference image quality assessments and also dehazes poorly at the depths of the image. These methods though can be optimized for memory usage and are faster. This work presents a deep learning model (Feature Fusion Attention Network) trained on a domain randomized synthetic dataset generated in simulation. The proposed model achieves the highest scores on blind image assessments through the gradient rationing technique for a deep learning-based approach by a significant margin. The images were evaluated on full-reference metrics as well and obtained favorable results. This approach also yields one of the highest edge sharpness obtained after dehazing. The training procedure adopted to obtain significant gains on real-world dehazing, without using any real-world data is also detailed in this paper.     

基于非线性滤波方法的PIV计算

针对流体运动图像计算(也称为PIV),为了获得可靠的运动矢量场、散度场和旋度场,该文提出了一种基于非线性滤波思想的PIV计算方法。新方法属于变分PIV方法,其在克服传统PIV方法不足的同时避开了经典变分方法中能量范函凸性和可微性的约束,将能量函数的最小化过程转变为非线性滤波过程。该文针对实际粒子图像序列与经典方法进行了实验和比较,结果证明新方法能够在有效抑制噪声的同时可以较好地保持在多流体运动的情况下运动矢量、散度和旋度场的细节信息。     

具有聚类结构相似性的非参数贝叶斯字典学习算法

利用图像结构信息是字典学习的难点,针对传统非参数贝叶斯算法对图像结构信息利用不充分,以及算法运行效率低下的问题,该文提出一种结构相似性聚类beta过程因子分析(SSC-BPFA)字典学习算法。该算法通过Markov随机场和分层Dirichlet过程实现对图像局部结构相似性和全局聚类差异性的兼顾,利用变分贝叶斯推断完成对概率模型的高效学习,在确保算法收敛性的同时具有聚类的自适应性。实验表明,相比目前非参数贝叶斯字典学习方面的主流算法,该文算法在图像去噪和插值修复应用中具有更高的表示精度、结构相似性测度和运行效率。     

Nonlocal discrete regularization on weighted graphs: a framework for image and manifold processing

We introduce a nonlocal discrete regularization framework on weighted graphs of the arbitrary topologies for image and manifold processing. The approach considers the problem as a variational one, which consists of minimizing a weighted sum of two energy terms: a regularization one that uses a discrete weighted p-Dirichlet energy and an approximation one. This is the discrete analogue of recent continuous Euclidean nonlocal regularization functionals. The proposed formulation leads to a family of simple and fast nonlinear processing methods based on the weighted p-Laplace operator, parameterized by the degree p of regularity, the graph structure and the graph weight function. These discrete processing methods provide a graph-based version of recently proposed semi-local or nonlocal processing methods used in image and mesh processing, such as the bilateral filter, the TV digital filter or the nonlocal means filter. It works with equal ease on regular 2-D and 3-D images, manifolds or any data. We illustrate the abilities of the approach by applying it to various types of images, meshes, manifolds, and data represented as graphs.     

多层融合深度局部PCA子空间稀疏优化特征提取模型

子空间方法是主要利用全局信息的经典模式识别方法,随着深度学习思想的引入,局部自学习结构特征模型得到大家的关注.利用深度学习原理,本文提出一种多层融合的深度局部子空间稀疏优化特征自学习抽取模型解决目标识别问题.首先,对训练样本集通过最小化重构误差得到第一层的主成分（Principal Component Analysis,PCA）特征映射矩阵;然后,通过L₁范数约束对特征映射结果进行稀疏优化,提高算法鲁棒性.接着,在第二层映射层以第一层的特征输出为输入,进行同样的特征矩阵学习操作,最终将图像映射至深层PCA子空间;然后,对各个映射层的特征提取结果进行加权融合,进行二值化哈希编码和直方图分块编码,提取图像的深度子空间稀疏特征.在FERET、AR、Yale等经典人脸数据库以及MNIST、CIFAR-10等目标数据库上的实验结果表明,该算法可以取得较高的识别率以及较好的光照、表情、人脸朝向鲁棒性,并且相对于卷积神经网络等深度学习框架具有结构简洁、收敛速度快等优点.     

Impact of data set noise on distributed deep learning

The training efficiency and test accuracy are important factors in judging the scalability of distributed deep learning. In this dissertation, the impact of noise introduced in the mixed national institute of standards and technology database (MNIST) and CIFAR-10 datasets is explored, which are selected as benchmark in distributed deep learning. The noise in the training set is manually divided into cross-noise and random noise, and each type of noise has a different ratio in the dataset. Under the premise of minimizing the influence of parameter interactions in distributed deep learning, we choose a compressed model (SqueezeNet) based on the proposed flexible communication method. It is used to reduce the communication frequency and we evaluate the influence of noise on distributed deep training in the synchronous and asynchronous stochastic gradient descent algorithms. Focusing on the experimental platform TensorFlowOnSpark, we obtain the training accuracy rate at different noise ratios and the training time for different numbers of nodes. The existence of cross-noise in the training set not only decreases the test accuracy and increases the time for distributed training. The noise has positive effect on destroying the scalability of distributed deep learning.     

基于深度学习的红外与可见光决策级融合检测

提出了一种基于深度学习的红外与可见光决策级融合检测方法。首先,提出了一种介于深度学习模型之间的参数传递模型,进而从基于深度学习的可见光物体检测模型上抽取了用于红外物体检测的预训练模型,并在课题组实地采集的红外数据集上进行fine-tuning,从而得到基于深度学习的红外物体检测模型。在此基础上,提出了一种基于深度学习的红外与可见光决策级融合检测模型,并对模型设计、图像配准、决策级融合过程进行了详细地阐述。最后,进行了白天和傍晚条件下基于深度学习的单波段检测实验和双波段融合检测实验。定性分析上,由于波段之间的信息互补性,相比于单波段物体检测,双波段融合物体检测在检测结果上具有更高的置信度和更精确的物体框;定量分析上,白天时,双波段融合检测的mAP为86.0%,相比于红外检测和可见光检测分别提高了9.9%和5.3%;傍晚时,双波段融合检测的mAP为89.4%,相比于红外检测和可见光检测分别提高了3.1%和14.4%。实验结果表明:基于深度学习的双波段融合检测方法相比于单波段检测方法具有更好的检测性能和更强的鲁棒性,同时也验证了所提出方法的有效性。     

Speckle noise removal based on structural convolutional neural networks with feature fusion for medical image

Application of convolutional neural networks (CNNs) for image additive white Gaussian noise (AWGN) removal has attracted considerable attentions with the rapid development of deep learning in recent years. However, the work of image multiplicative speckle noise removal is rarely done. Moreover, most of the existing speckle noise removal algorithms are based on traditional methods with human priori knowledge, which means that the parameters of the algorithms need to be set manually. Nowadays, deep learning methods show clear advantages on image feature extraction. Multiplicative speckle noise is very common in real life images, especially in medical images. In this paper, a novel neural network structure is proposed to recover noisy images with speckle noise. Our proposed method mainly consists of three subnetworks. One network is rough clean image estimate subnetwork. Another is subnetwork of noise estimation. The last one is an information fusion network based on U-Net and several convolutional layers. Different from the existing speckle denoising model based on the statistics of images, the proposed network model can handle speckle denoising of different noise levels with an end-to-end trainable model. Extensive experimental results on several test datasets clearly demonstrate the superior performance of our proposed network over state-of-the-arts in terms of quantitative metrics and visual quality.     

Variational models for image colorization via chromaticity and brightness decomposition.

Colorization refers to an image processing task which recovers color in grayscale images when only small regions with color are given. We propose a couple of variational models using chromaticity color components to colorize black and white images. We first consider total variation minimizing (TV) colorization which is an extension from TV inpainting to color using chromaticity model. Second, we further modify our model to weighted harmonic maps for colorization. This model adds edge information from the brightness data, while it reconstructs smooth color values for each homogeneous region. We introduce penalized versions of the variational models, we analyze their convergence properties, and we present numerical results including extension to texture colorization.     

基于图像退化模型的红外与可见光图像融合方法

基于深度学习的红外与可见光图像融合算法依赖人工设计的相似度函数衡量输入与输出的相似度,这种无监督学习方式不能有效利用神经网络提取深层特征的能力,导致融合结果不理想。针对该问题,该文首先提出一种新的红外与可见光图像融合退化模型,把红外和可见光图像视为理想融合图像通过不同退化过程后产生的退化图像。其次,提出模拟图像退化的数据增强方案,采用高清数据集生成大量模拟退化图像供训练网络。最后,基于提出的退化模型设计了简单高效的端到端网络模型及其网络训练框架。实验结果表明,该文所提方法不仅拥有良好视觉效果和性能指标,还能有效地抑制光照、烟雾和噪声等干扰。     

Sequential image deep learning-based Wi-Fi human activity recognition method

For the problems existing in most of the researches,such as weak anti-noise ability,incompatible signal size and insufficient feature extraction of deep-learning-based Wi-Fi human activity recognition,a kind of sequential image deep learning-based recognition method was proposed.Based on the idea of sequential image deep learning,a series of image frames were reconstructed from time-varied Wi-Fi signal to ensure the consistency of input size.In addition,a low-rank decomposition method was innovatively designed to separate low-rank activity information merged in noises.Finally,a deep model combining temporal stream and spatial stream was proposed to automatically capture the spatiotemporal features from length-varied image sequences.The proposed method was extensively tested in WiAR dataset and self collected dataset.The experimental results show the proposed method could achieve the accuracy of 0.94 and 0.96,which indicate its high-accuracy performance and robustness in pervasive environments.