Novel cooperative neural fusion algorithms for image restoration and image fusion.

To deal with the problem of restoring degraded images with non-Gaussian noise, this paper proposes a novel cooperative neural fusion regularization (CNFR) algorithm for image restoration. Compared with conventional regularization algorithms for image restoration, the proposed CNFR algorithm can relax need of the optimal regularization parameter to be estimated. Furthermore, to enhance the quality of restored images, this paper presents a cooperative neural fusion (CNF) algorithm for image fusion. Compared with existing signal-level image fusion algorithms, the proposed CNF algorithm can greatly reduce the loss of contrast information under blind Gaussian noise environments. The performance analysis shows that the proposed two neural fusion algorithms can converge globally to the robust and optimal image estimate. Simulation results confirm that in different noise environments, the proposed two neural fusion algorithms can obtain a better image estimate than several well known image restoration and image fusion methods.     

Sequential and parallel image restoration: neural networkimplementations

Sequential and parallel image restoration algorithms and their implementations on neural networks are proposed. For images degraded by linear blur and contaminated by additive white Gaussian noise, maximum a posteriori (MAP) estimation and regularization theory lead to the same high dimension convex optimization problem. The commonly adopted strategy (in using neural networks for image restoration) is to map the objective function of the optimization problem into the energy of a predefined network, taking advantage of its energy minimization properties. Departing from this approach, we propose neural implementations of iterative minimization algorithms which are first proved to converge. The developed schemes are based on modified Hopfield (1985) networks of graded elements, with both sequential and parallel updating schedules. An algorithm supported on a fully standard Hopfield network (binary elements and zero autoconnections) is also considered. Robustness with respect to finite numerical precision is studied, and examples with real images are presented.     

交替使用小波去噪和全变差正则化的盲图像恢复算法

盲图像恢复就是在点扩散函数未知情况下从降质观测图像恢复出原图像.该文提出了一种交替使用小波去噪和全变差正则化的盲图像恢复算法.观测模型首先被分解成两个相互关联的子模型,这种分解转化盲恢复问题成为图像去噪和图像恢复两个问题,可以交替采用图像去噪和图像恢复算法求解.模糊辨识阶段,使用全变差正则化算法估计点扩散函数;图像恢复阶段,使用小波去噪和全变差正则化相结合的算法恢复图像.实验结果和与其它方法的比较表明该文算法能够获得更好的恢复效果.     

Adaptive Nonconvex Nonsmooth Regularization for Image Restoration Based on Spatial Information

Image restoration is an ill-posed problem that requires regularization to solve. Many existing regularization terms in the literature are the convex function. However, nonconvex nonsmooth regularization has advantages over convex regularization for restoring images, but its practical interest used to be limited by the difficulty of the computational stage which requires a nonconvex nonsmooth minimization. In this paper, an adaptive nonconvex nonsmooth regularization is proposed for image restoration by using the spatial information indicator. Moreover, an efficient numerical algorithm for solving the resulting minimization problem is provided by applying the variable splitting and the penalty techniques. Finally, its advantages are shown in deblurring edges and restoring fines of image simultaneously in experiments.     

A comprehensive benchmark analysis for sand dust image reconstruction

Recently, numerous sand dust removal algorithms have been proposed. To our best knowledge, however, most methods evaluated their performance in a no-reference way using few selected real-world images from the internet. It is unclear how to quantitatively analyze the performance of the algorithms in a supervised way. Moreover, due to the absence of large-scale datasets, there are no well-known sand dust reconstruction report algorithms up till now. To bridge the gaps, we presented a comprehensive perceptual study and analysis of real-world sandstorm images, then constructed a Sand-dust Image Reconstruction Benchmark(SIRB) for training Convolutional Neural Networks(CNNs) and evaluating the algorithm’s performance. We adopted the existing image transformation neural network trained on SIRB as the baseline to illustrate the generalization of SIRB for training CNNs. Finally, we conducted a comprehensive evaluation to demonstrate the performance and limitations of the sandstorm enhancement algorithms, which shed light on future research in sandstorm image reconstruction.     

A GAUSSIAN MIXTURE MODEL-BASED REGULARIZATION METHOD IN ADAPTIVE IMAGE RESTORATION

A GMM （Gaussian Mixture Model） based adaptive image restoration is proposed in this paper. The feature vectors of pixels are selected and extracted. Pixels are clustered into smooth, edge or detail texture region according to variance-sum criteria function of the feature vectors. Then parameters of GMM are calculated by using the statistical information of these feature vectors. GMM predicts the regularization parameter for each pixel adaptively. Hopfield Neural Network （Hopfield-NN） is used to optimize the objective function of image restoration, and network weight value matrix is updated by the output of GMM. Since GMM is used, the regularization parameters share properties of different kind of regions. In addition, the regularization parameters are different from pixel to pixel. GMM-based regularization method is consistent with human visual system, and it has strong generalization capability. Comparing with non-adaptive and some adaptive image restoration algorithms, experimental results show that the proposed algorithm obtains more preferable restored images.     

面向压缩图像复原的网络增强训练方法

高质量的数据是深度卷积神经网络成功的关键因素之一。在计算机视觉领域,常用图像数据集通常以JPEG格式存储。这种有损压缩技术不可避免地会导致原始数据信息的丢失,进而造成利用压缩数据训练的卷积神经网络的性能降低。因此,为了增强卷积神经网络的性能,本文提出了一种面向压缩图像复原的增强训练方法,通过复原压缩图像实现卷积神经网络的性能增强。该方法具体为一个包含复原模块和任务模块的联合增强框架。复原模块致力于恢复有损压缩技术造成的信息丢失;任务模块专注于基于任务需求增强压缩图像。两个模块联合训练,使得压缩图像的复原增强更具有目的性。本文通过图像分类任务的实验表明,与压缩图像相比,该方法能有效地复原压缩图像,增强卷积神经网络的性能。此外,该方法中两个模块间的低耦合性和可替代性保证了该方法的适用性。      

Image blind restoration based on degradation representation network

Most deep learning (DL)-based image restoration methods have exploited excellent performance by learning a non-linear mapping function from low quality images to high quality images. However, two major problems restrict the development of the image restoration methods. First, most existing methods based on fixed degradation suffer from significant performance drop when facing the unknown degradation, because of the huge gap between the fixed degradation and the unknown degradation. Second, the unknown-degradation estimation may lead to restoration task failure due to uncertain estimation errors. To handle the unknown degradation in the real application, we introduce a degradation representation network for single image blind restoration (DRN). Different from the methods of estimating pixel space, we use an encoder network to learn abstract representations for estimating different degradation kernels in the representation space. Furthermore, a degradation perception module with flexible adaptability to different degradation kernels is used to restore more structural details. In our experiments, we compare our DRN with several state-of-the-art methods for two image restoration tasks, including image super-resolution (SR) and image denoising. Quantitative results show that our degradation representation network is accurate and efficient for single image restoration.     

增广Lagrangian的快速视频复原方法

针对单幅模糊图像复原的局限性和视频应用的广泛 性,提出了一种基于时空体和增广Lagrangian的快速视频复原方法。首先对视频复原与图像 复原的特征进行比较和分析,研究视频复原的三维解卷积操作, 并对目前存在的视频复原方法的实现过程与性能进行分析和总结;然后在时空体的思想下, 通过时空联合 的各向同性全变分来控制时间误差和空间误差,并引入一种增广的Lagrangian方法完成全变 分规整化的难 题;最后通过求解Lagrangian形式的f子问题和u子问题实现全变分最小化难题,并对规整化 过程中的参 数进行研究与讨论,最终实现了视频的快速鲁棒复原。基于仿真图像和实际视频的实验结果 表明,本文方法 的性能在运行时间和视觉质量评价方面都要优于当前的其它方法,能够有效地实现图像和视 频的快速复原。     

一种生成对抗网络用于图像修复的方法

近年来基于深度学习的图像修复方法相比于传统方法,表现出明显优势,前者能更好的生成视觉上合理的图像结构和纹理.但现有的标准卷积神经网络方法,通常会造成颜色差异过大和图像纹理缺失与失真的问题.本文提出了一种新型图像修复深度网络模型,该模型由两个相互独立的生成对抗式网络模块组成.其中,图像修复网络模块旨在解决图像缺失区域的修复问题,其生成器基于部分卷积网络;图像优化网络模块旨在解决修复后图像存在局部色差的问题,其生成器基于深度残差网络.通过两个网络模块的协同作用,图像的视觉效果与图像质量得到提高.与其他先进方法进行定性和定量比较的实验结果表明,本文提出的方法在图像修复质量上表现更好.     

应用双稀疏模型和ADMM优化的图像复原

基于稀疏表示的图像复原算法大都只利用了图像整体稀疏性和局部稀疏性中的一种,未充分利用图像的先验知识,基于此,本文在稀疏表示框架下,同时引入Cosparse解析模型及平移不变小波变换两种稀疏模型,前者对每个图像块进行稀疏表示,后者对整幅图像进行稀疏表示,从而提出一种新的图像复原算法。该算法将图像复原问题归结为双稀疏正则化问题。为求解复杂的双稀疏优化问题,本文运用交替方向乘子法 (ADMM, Alternating Direction Method of Multipliers)算法将该约束优化问题分解为若干子问题,通过交替迭代求解获得复原图像。实验中对不同类型的模糊图像进行了复原,其结果表明该算法对于各类模糊图像的复原比现有复原算法效果更好,从而验证了算法的有效性。      

Multi-level fusion with deep neural networks for multimodal sentiment classification

The task of multimodal sentiment classification aims to associate multimodal information, such as images and texts with appropriate sentiment polarities. There are various levels that can affect human sentiment in visual and textual modalities. However, most existing methods treat various levels of features independently without having effective method for feature fusion. In this paper, we propose a multi-level fusion classification (MFC) model to predict the sentiment polarity based on the fusing features from different levels by exploiting the dependency among them. The proposed architecture leverages convolutional neural networks ( CNNs) with multiple layers to extract levels of features in image and text modalities. Considering the dependencies within the low-level and high-level features, a bi-directional (Bi) recurrent neural network (RNN) is adopted to integrate the learned features from different layers in CNNs. In addition, a conflict detection module is incorporated to address the conflict between modalities. Experiments on the Flickr dataset demonstrate that the MFC method achieves comparable performance compared with strong baseline methods.     

基于粒子群优化的正则化水下图像盲复原

水下图像恢复的难点在于缺少海水的点扩展函数的足够信息,而导致病态的问题.为了提高水下激光成像系统的成像质量,提出了用粒子群优化正则化参量的盲图像复原算法.该方法结合Tikhonov正则化和改进的全变分正则化的技术特点,使用一种交替迭代方法,分别估计点扩展函数和估计复原图像,同时用粒子群算法优化正则化参量.结果表明,该方法对水下图像复原具有较好的鲁棒性,算法收敛稳定.     

基于头脑风暴优化算法的BP神经网络模糊图像复原

该文提出一种基于头脑风暴智能优化算法的BP神经网络模糊图像复原方法(OBSO-BP)。该方法在聚类和变异两方面优化了头脑风暴智能算法,利用头脑风暴优化算法易于解决多峰高维函数问题的特点,自动搜寻BP神经网络更佳的初始权值和阈值,以减少BP网络对其初始权值和阈值的敏感性,避免网络陷入局部最优解,增加网络的收敛速度,减小网络误差,提高图像还原质量。该文采用20张不同的图像,对其模糊图像分别进行维纳滤波复原(Wiener)、基于头脑风暴算法的维纳滤波复原(Wiener-BSO)、BP神经网络复原以及基于头脑风暴算法的BP神经网络(BSO-BP)图像复原实验。实验结果表明,该方法能够取得更好的图像复原效果。     

基于Poisson-Markov场的超分辨力图像复原算法

图像的超分辨力复原和信噪比的提高是图像复原追求的目标.Poisson-ML图像复原方法(PML)具有很强的超分辨力复原能力,但在复原过程中会产生振荡条纹且对带噪较大的图像不能取得理想的复原效果.在Poisson和Markov分布假设的基础上,提出基于Poisson-Markov场的超分辨力图像复原算法及其正则化参数的自适应选择方法(MPML).实验表明,MPML算法不但具有很好的超分辨力复原能力,而且能有效减少和去除复原图像中的振荡条纹,对于带噪较大的图像也能取得理想的复原效果,因此其图像复原质量明显好于PML算法.正则化参数能被自动优化地选择且与图像复原的迭代运算同步进行.     

退化图像复原方法研究进展

飞行器和空间成像制导装备在大气中高速飞行时会受到湍流干扰,导致光学系统接收到的图像发生模糊降质、像素偏移、信噪比降低等问题,开展退化图像复原技术及方法研究就成为空间光学成像系统获得较高性能图像的重要途径。通过对退化图像复原技术研究进展的系统梳理和分析研究,本文首先介绍了图像退化模型,接着给出了退化图像复原方法的分类,然后比较系统地介绍了确定正则化图像复原方法、随机正则化图像复原方法、基于局部相似性的图像复原方法、基于示例学习的图像复原方法等几种新型的单幅退化图像复原方法,其后分析了视频复原的特征、介绍了新近的几种典型的视频图像复原方法,最后分析总结出了图像复原的难点所在。对于促进我国退化图像复原技术的研究和发展具有一定的参考价值。     

二阶Hopfield型神经网络的稳定性 分析及收敛速度的估计

本文讨论二阶连续Hopfield型神经网络平衡点的全局稳定性问题,利用LMI方法和Lyapunov方法得到了网络平衡点全局渐近稳定和全局指数稳定的几个充分条件,并对其指数收敛速度进行了估计.     

Speckle noise removal based on structural convolutional neural networks with feature fusion for medical image

Application of convolutional neural networks (CNNs) for image additive white Gaussian noise (AWGN) removal has attracted considerable attentions with the rapid development of deep learning in recent years. However, the work of image multiplicative speckle noise removal is rarely done. Moreover, most of the existing speckle noise removal algorithms are based on traditional methods with human priori knowledge, which means that the parameters of the algorithms need to be set manually. Nowadays, deep learning methods show clear advantages on image feature extraction. Multiplicative speckle noise is very common in real life images, especially in medical images. In this paper, a novel neural network structure is proposed to recover noisy images with speckle noise. Our proposed method mainly consists of three subnetworks. One network is rough clean image estimate subnetwork. Another is subnetwork of noise estimation. The last one is an information fusion network based on U-Net and several convolutional layers. Different from the existing speckle denoising model based on the statistics of images, the proposed network model can handle speckle denoising of different noise levels with an end-to-end trainable model. Extensive experimental results on several test datasets clearly demonstrate the superior performance of our proposed network over state-of-the-arts in terms of quantitative metrics and visual quality.     

A recursive soft-decision approach to blind image deconvolution

This paper presents a new approach to blind image deconvolution based on soft-decision blur identification and hierarchical neural networks. Traditional blind algorithms require a hard-decision on whether the blur satisfies a parametric form before their formulations. As the blurring function is usually unknown a priori, this precondition inhibits the incorporation of parametric blur knowledge domain into the restoration schemes. The new technique addresses this difficulty by providing a continual soft-decision blur adaptation with respect to the best-fit parametric structure throughout deconvolution. The approach integrates the knowledge of well-known blur models without compromising its flexibility in restoring images degraded by nonstandard blurs. An optimization scheme is developed where a new cost function is projected and minimized with respect to the image and blur domains. A nested neural network, called the hierarchical cluster model is employed to provide an adaptive, perception-based restoration. Its sparse synaptic connections are instrumental in reducing the computational cost of restoration. Conjugate gradient optimization is adopted to identify the blur due to its computational efficiency. The approach is shown experimentally to be effective in restoring images degraded by different blurs.