A Variational Framework for Exemplar-Based Image Inpainting

Non-local methods for image denoising and inpainting have gained considerable attention in recent years. This is in part due to their superior performance in textured images, a known weakness of purely local methods. Local methods on the other hand have demonstrated to be very appropriate for the recovering of geometric structures such as image edges. The synthesis of both types of methods is a trend in current research. Variational analysis in particular is an appropriate tool for a unified treatment of local and non-local methods. In this work we propose a general variational framework for non-local image inpainting, from which important and representative previous inpainting schemes can be derived, in addition to leading to novel ones. We explicitly study some of these, relating them to previous work and showing results on synthetic and real images.     

A Two-Step Regularization Framework for Non-Local Means

As an effective patch-based denoising method, non-local means （NLM） method achieves favorable denoising performance over its local counterparts and has drawn wide attention in image processing community. The in, plementation of NLM can formally be decomposed into two sequential steps, i.e., computing the weights and using the weights to compute the weighted means. In the first step, the weights can be obtained by solving a regularized optimization. And in the second step, the means can be obtained by solving a weighted least squares problem. Motivated by such observations, we establish a two-step regularization framework for NLM in this paper. Meanwhile, using the fl-amework, we reinterpret several non-local filters in the unified view. Further, taking the framework as a design platform, we develop a novel non-local median filter for removing salt-pepper noise with encouraging experimental results.     

彩色纹理图像恢复的非局部TV模型

基于局部算子不同形式的TV(total variation)模型用于彩色图像的噪声去除时往往存在边缘模糊、纹理模糊、阶梯效应、Mosaic效应等问题.因此,将传统局部的Tikhonov模型、TV模型、MTV(multi-channel total variation)模型、CTV(color total variation)模型推广到基于非局部算子概念的NL-CT(non-local color Tikhonov)模型、NL-LTV(non-local layered total variation)模型、NL-MTV(non-local multi-channel total variation)模型、NL-CTV(non-local color total variation)模型,并通过引入辅助变量和Bregman迭代参数设计了相应的快速Split Bregman算法.实验结果表明,所提出的非局部TV模型都很好地解决了局部模型中出现的问题,在纹理、边缘、光滑度等特征保持方面取得了良好特性,其中NL-CTV处理效果最好,但是计算效率较低.     

非局部降噪快速模糊C-均值聚类算法

传统的模糊C-均值聚类算法未利用图像的空间信息,在分割迭加了噪声的MR图像时分割精度较差。采用了既能有效去除噪声又能较好地保持图像边缘特征的非局部降噪方法,结合基于图像灰度直方图聚类分析的快速模糊C-均值聚类算法,得到了一种具有较高分割精度的图像快速分割算法。通过对模拟图像、仿真脑部MR图像和临床脑部MR图像的分割实验,表明提出的新算法比已有的快速模糊C-均值聚类算法有更精确的图像分割能力。     

深度卷积神经网络降噪模型的技术瓶颈与研究展望

现有的深度卷积神经网络（DCNN）图像降噪模型受其技术路线内在固有特性的制约,降噪性能仍然有待进一步改进。为了推动现有DCNN图像降噪模型技术的发展,需要正视并及时解决制约其进一步完善的瓶颈问题。本文简要概述了传统的基于自然图像非局部自相似性、稀疏性和低秩性这3种先验知识设计的图像降噪算法的技术路线特点和优缺点,从传统图像降噪算法存在的问题中引出基于DCNN构建图像降噪模型的技术优势,并梳理并总结了DCNN降噪模型未来的发展瓶颈,就相应的解决方案（研究方向）进行详细讨论。通过深入分析发现,可以从扩大卷积核的感受野、降低网络参数与训练集之间的依赖关系以及充分利用DCNN网络的建模能力这3个角度入手,突破现有基于数据驱动的DCNN降噪模型的瓶颈制约,把图像降噪算法的研究水平推向新的高度。     

基于非局部算法的序列图像超分辨率重构

提出了一种基于非局部算法的多帧图像超分辨率重构算法,该方法无需进行图像校正。为克服传统图像复原算法基于图像局部点领域的影响,指出可以使用非局部去噪模型的优越边缘保持性和去噪性来重构高分辨率图像。实验证明,该算法在低信噪比情况下,可以取得较好结果,有效解决了低信噪比条件下图像复原的一个难点。     

彩色纹理图像恢复的非局部TV模型及其特征保持

基于局部算子的不同形式的TV模型用于彩色图像噪声去除时往往存在边缘模糊、纹理模糊、阶梯效应、Mosaic效应等问题。本文基于非局部算子概念将传统的Tikhonov模型、TV模型、MTV模型、CTV模型推广到NL-CT模型、NL-LTV模型、NL-MTV模型、NL-CTV模型,并通过引入辅助变量和Bregman迭代参数设计了相应的快速Split Bregman算法。实验表明,所提出的非局部TV模型在纹理、边缘、光滑度等特征保持方面具有良好特性,且差异不大,但不同模型的计算效率存在较大差异。     

基于非局部几何信号的点模型去噪算法

与传统的基于局部几何信号去噪方法不同,提出了一种基于非局部几何信号的点模型去噪算法.该算法通过双边滤波算子,计算出每个点的微分坐标信息作为“几何灰度值”;基于模型上每个点的邻域的微分信息进行相似性匹配计算,对点模型上的“几何灰度值”进行全局加权平均,获得该点最终的微分信号;最后重建出该点的几何信息.进一步,提出了基于混合树的加速方法,对具有相似特征的邻域进行聚类,减少了匹配的空间复杂度,提高了计算效率.实验结果表明,算法简单高效,获得了满意的去噪效果.     

基于自适应权值滤波的深度图像超分辨率重建

目的 越来越多的应用都依赖于对真实场景深度图像的准确且快速的观测和分析。飞行时间相机可以实时获取场景的深度图像,但是由于硬件条件的限制,采集的深度图像分辨率较低,无法满足实际应用的需要。为此提出一种结合同场景彩色图像通过构造自适应权值滤波器对深度图像进行超分辨率重建的方法。方法 充分发掘深度图像的非局部以及局部自相似性先验约束,结合同场景的高分辨率彩色图像构造非局部及局部的自适应权值滤波算法对深度图像进行超分辨率重建。具体来说,首先利用非局部滤波算法来有效避免重建结果的振铃效应,然后利用局部滤波算法进一步提升重建的深度图像质量。结果 实验结果表明,无论在客观指标还是视觉效果上,基于自适应权值滤波的超分辨率重建算法较其他算法都可以得到更好的结果,尤其当初始的低分辨率深度图像质量较差的情况下,本文方法的优势更加明显,峰值信噪比可以得到1dB的提升。结论 结合非局部和局部自相似性先验约束,结合同场景的高分辨率彩色图像构造的自适应权值滤波算法,较其他算法可以得到更理想的结果。     

基于高阶奇异值分解和均方差迭代的图像去噪

目的 图像去噪是图像处理的难题,其难点是在尽量滤除噪声的同时对图像信息进行保持。针对该难点,本文提出了一种将非局部相似性和高阶奇异值分解(HOSVD)相融合,并利用均方差(MSE)迭代对图像进行去噪的iHOSVD算法。方法 首先利用非局部相似块聚类和高阶奇异值分解构建数据自适应的3维变换基及其变换系数;其次,对变换系数进行阈值处理后进行3维反变换,从而达到非局部协同滤波的目的;最后,由于一次去噪操作无法达到理想的去噪效果,采用一种基于均方差最优的迭代方法对图像进行去噪,并证明该迭代是一个权衡偏差和方差使得均方差达到最优的过程。结果 实验结果表明,iHOSVD算法既能够有效地去除噪声,又能够很好地保持纹理细节信息。结论 本文所提的图像去噪iHOSVD算法结合了非局部协同滤波与数据自适应去噪的思想,通过对3种高水平去噪算法BM3D、NCSR和PLOW的比较实验发现,不仅表现了较强的图像去噪能力,而且在图像纹理细节保持方面效果最好,适用于纹理信息较强的图像。     

Analysis of non-local image denoising methods

Image denoising is probably one of the most studied problems in the image processing community. Recently a new paradigm on non-local denoising was introduced. The non-local means method proposed by Buades, Morel and Coll computes the denoised image as a weighted average of pixels across the whole image. The weight between pixels is based on the similarity between neighborhoods around them. This method attracted the attention of other researchers who proposed improvements and modifications to it. In this work we analyze those methods trying to understand their properties while connecting them to segmentation based on spectral properties of the graph that represents the similarity of neighborhoods of the image. We also propose a method to automatically estimate the parameters which produce the optimal results in terms of mean square error and perceptual quality.     

Learning Explicit Smoothing Kernels for Joint Image Filtering

Smoothing noises while preserving strong edges in images is an important problem in image processing. Image smoothing filters can be either explicit (based on local weighted average) or implicit (based on global optimization). Implicit methods are usually time‐consuming and cannot be applied to joint image filtering tasks, i.e., leveraging the structural information of a guidance image to filter a target image. Previous deep learning based image smoothing filters are all implicit and unavailable for joint filtering. In this paper, we propose to learn explicit guidance feature maps as well as offset maps from the guidance image and smoothing parameter that can be utilized to smooth the input itself or to filter images in other target domains. We design a deep convolutional neural network consisting of a fully‐convolution block for guidance and offset maps extraction together with a stacked spatially varying deformable convolution block for joint image filtering. Our models can approximate several representative image smoothing filters with high accuracy comparable to state‐of‐the‐art methods, and serve as general tools for other joint image filtering tasks, such as color interpolation, depth map upsampling, saliency map upsampling, flash/non‐flash image denoising and RGB/NIR image denoising.     

基于非局部自相似性的谱聚类图像去噪算法

常见的图像去噪方法只是单独地利用了无噪图像或含噪图像的先验信息,并没有将这两种图像的先验信息有效地结合起来。针对这个问题,提出一种 联合无噪图像块的先验信息和含噪图像块的非局部自相似性进行去噪的图像去噪算法。首先,对无噪图像块进行谱聚类,通过谱聚类进行学习,图像中的相似块被聚集到同一类,并将学习得到的聚类信息用于含噪图像块的聚类;然后,向量化同一类中的含噪图像块并聚集形成一个矩阵,该矩阵中包含的原始图像数据构成一个低秩矩阵;再通过一个低秩逼近过程估计出相应的原始图像数据;最后,根据逼近得到的原始图像数据重建图像。实验结果表明,相较于已有的自适应正则化的非局部均值去噪算法以及基于主成分分析和局部像素聚类的两级图像去噪算法,提出的算法不仅可以获得较大的峰值信噪比,而且还能较好地保存图像的细节,取得了更好的去噪效果。     

Nonlocal Image and Movie Denoising

Neighborhood filters are nonlocal image and movie filters which reduce the noise by averaging similar pixels. The first object of the paper is to present a unified theory of these filters and reliable criteria to compare them to other filter classes. A CCD noise model will be presented justifying the involvement of neighborhood filters. A classification of neighborhood filters will be proposed, including classical image and movie denoising methods and discussing further a recently introduced neighborhood filter, NL-means. In order to compare denoising methods three principles will be discussed. The first principle, “method noise”, specifies that only noise must be removed from an image. A second principle will be introduced, “noise to noise”, according to which a denoising method must transform a white noise into a white noise. Contrarily to “method noise”, this principle, which characterizes artifact-free methods, eliminates any subjectivity and can be checked by mathematical arguments and Fourier analysis. “Noise to noise” will be proven to rule out most denoising methods, with the exception of neighborhood filters. This is why a third and new comparison principle, the “statistical optimality”, is needed and will be introduced to compare the performance of all neighborhood filters. The three principles will be applied to compare ten different image and movie denoising methods. It will be first shown that only wavelet thresholding methods and NL-means give an acceptable method noise. Second, that neighborhood filters are the only ones to satisfy the “noise to noise” principle. Third, that among them NL-means is closest to statistical optimality. A particular attention will be paid to the application of the statistical optimality criterion for movie denoising methods. It will be pointed out that current movie denoising methods are motion compensated neighborhood filters. This amounts to say that they are neighborhood filters and that the ideal neighborhood of a pixel is its trajectory. Unfortunately the aperture problem makes it impossible to estimate ground true trajectories. It will be demonstrated that computing trajectories and restricting the neighborhood to them is harmful for denoising purposes and that space-time NL-means preserves more movie details.     

基于非局部信息的医学图像降噪技术综述

有效的医学图像增强技术应将感兴趣目标或区域增强、背景抑制和噪声削减综合考虑,能改善图像的质量,在减少噪声的同时保持着原有的纹理特征,有助于后续得到正确的临床诊断结果,这对某些疾病的早期确诊有极大帮助。归纳了基于非局部信息的医学图像增强技术常见方法,包括非局部均值滤波算法、三维块匹配算法、形态成分分析算法等,通过介绍这几种方法原理,指出这些方法的使用范围及现状,并进行了性能对比分析,最后探索性地给出了现阶段医学图像增强技术的可能发展方向之一,即基于上下文量化的图像增强技术。     

视觉显著性驱动的全景渲染图非局部降噪

目的 传统降噪方法通常忽视人眼感知因素,对不同区域的图像块都进行同等处理。当使用传统降噪算法对全景画面滤波处理时,全景画面两极区域容易产生模糊问题,尤其是通过视口观察时,该问题更加明显。针对此问题,提出一种视觉显著性驱动的蒙特卡洛渲染生成全景图非局部均值（visual saliency driven non-local means,VSD-NLM）滤波降噪算法。方法 在VSD-NLM算法中首先使用全景图显著区域检测算法获取全景画面的显著区域;然后使用梯度幅值相似性偏差辅助的非局部均值（gradient magnitude similarity deviation assisted non-local means,GMSDA-NLM）滤波算法,降低显著区域的噪声;同时设计并行非局部均值（parallel non-local means,P-NLM）滤波算法,加快降噪处理速度,降低非显著区域噪声;最后利用改进的Canny算法提取梯度特征,同时结合各向异性扩散引导滤波来优化降噪结果。结果 采用结构相似度（structural similarity,SSIM）和FLIP作为评价指标,来对比VSDNLM算法与非局部均值滤波算法、多特征非局部均值滤波算法以及渐进式去噪算法等其他算法的性能。实验结果表明,VSD-NLM算法的降噪结果在客观评价指标上均优于对比算法,SSIM值比其他算法平均提高14.7%,FLIP值比其他算法平均降低15.2%。在视觉效果方面,VSD-NLM算法能够减轻全景画面模糊,提升视觉感知质量。本文对GMSDA-NLM和P-NLM算法的有效性进行了实验验证,相较于非局部均值滤波算法,GMSDA-NLM算法能够有效去除噪声并保持图像细节的完整性。P-NLM算法在运行速度方面相较对比算法平均提高约6倍,与串行算法生成的图像之间的SSIM值可达到0.996。结论 本文算法能够更好地用于全景图降噪,滤波效果更佳,对全景电影制作应用有重要的理论和实际意义。     

一种基于优化参数的非局部均值滤波算法

针对非局部均值滤波算法中难以找到一个全局最优的滤波参数h的问题,给出一种新的该参数的优化方法,并将其应用于传统非局部均值滤波算法的改进。首先基于SUSAN算法提取噪声图像的边缘信息,然后在大量实验的基础上,利用线性回归和非线性回归分析方法建立h与边缘信息、噪声方差之间的优化模型。最后,将基于该优化模型的非局部均值算法应用于多幅图像的去噪处理中。实验结果表明,新算法改善了传统非局部均值算法的去噪性能,取得了良好的滤波效果。     

Joint Image Denoising and Disparity Estimation via Stereo Structure PCA and Noise-Tolerant Cost

Stereo cameras are now commonly available on cars and mobile phones. However, the captured images may suffer from low image quality under noisy conditions, producing inaccurate disparity. In this paper, we aim at jointly restoring a clean image pair and estimating the corresponding disparity. To this end, we propose a new joint framework that iteratively optimizes these two different tasks in a multiscale fashion. First, structure information between the stereo pair is utilized to denoise the images using a non-local means strategy. Second, a new noise-tolerant cost function is proposed for noisy stereo matching. These two terms are integrated into a multiscale framework in which cross-scale information is leveraged to further improve both denoising and stereo matching. Extensive experiments on datasets captured from indoor, outdoor, and low-light conditions show that the proposed method achieves superior performance than the state-of-the-art image denoising and disparity estimation methods. While it outperforms multi-image denoising methods by about 2 dB on average, it achieves a 50% error reduction over radiometric-change-robust stereo matching on the challenging KITTI dataset.     

基于非局部相似联合低秩表示的高光谱图像去噪

高光谱图像(Hyperspectral Images,HSI)在采集过程中常受到多种类型的噪声干扰,会直接影响其在后续应用中的精度,因此HSI的去噪是一项十分重要的预处理过程。低秩表示(Low-Rank Representation,LRR)模型能很好地满足HSI的光谱性质,但该框架下字典的选择尤为重要,在当下仍是一个开放性的问题。同时,典型去噪方法仅考虑了图像的局部相关性,已不能满足去噪要求,非局部相似性在图像中也是不可忽略的。基于LRR,文中提出了一种新的HSI去噪算法。首先,综合考虑噪声的类型,选取具有更全面的噪声判别能力的字典;其次,在对图像分块处理的前提下,通过聚类的方式引入非局部相似信息,将相似的图像块联合起来进行低秩表示。在模拟Indian Pines数据集以及EO-1 Hyperion真实数据集上的实验结果均表明,相较于目前主流的HSI去噪方法,无论是在图像的目视效果还是在模拟数据集的定量评价指标下,所提方法均有显著提升。     

基于超像素分割的非局部均值去噪方法

针对非局部均值(NLM)去噪算法在变化丰富的纹理区域采用平移窗口的方法选择相似块的不足进行了研究,提出一种基于超像素分割的非局部均值去噪算法。该方法充分考虑非局部均值去噪算法中相似性对噪声去除的影响,利用经过超像素分割处理得到的图像块内部相邻像素间以及纹理边缘都具有一定相似性的特点,在超像素分割块基础上优化纹理区域相似窗口的选择策略,提高图像块与中心像素块之间的相似性,从而达到提升非局部均值算法的去噪水平、边缘纹理不被模糊的目的。在多幅经典自然图像上的实验结果表明,该方法能够有效的去除图像中包含的噪声信息,相比于传统的非局部均值方法,保留了更多的纹理信息。