Block-based image inpainting in the wavelet domain

This paper introduces a new model for block-based image inpainting in the wavelet domain. The proposed technique separates the inpainting process into two different and important steps, both using the energy of wavelet coefficients. First, the model explores wavelet detail coefficients to estimate the image gradient vector, weighting each vector with the energy of wavelet coefficients. Such information is then used to determine which block belonging to the inpainting region should be filled first. After that, an adapted method for texture synthesis in the wavelet domain is applied in order to successfully fill this block. These two steps are applied successively, until the inpainting region is completely filled. Experimental results indicate that the proposed algorithm can fill large inpainting regions with good visual quality, presenting results comparable to or better than other competitive approaches for image inpainting.     

基于张量扩散的小波域修复模型

在JPEG2000图像压缩标准中,有损传输过程中的小波系数的丢失将严重影响接收端图像的质量.为了修复丢失的或被损坏的小波系数,本文提出了一种基于张量扩散的小波域修复模型(TDWI),该混合模型将结构自适应各向异性正则与小波表示结合起来.同时推导该模型对应的Euler-Lagrange方程,并据此来分析它在像素域的几何正则性能.由于在正则项中采用了矩阵值的结构张量,该模型的扩散核的形状随着图像的局部结构特征(包括尖锐边缘、角点和各向同性区域)自适应地变化.与已有的小波域修复模型相比,本文所提模型能更自适应地、更准确地控制像素域的几何正则性,并对噪声有更强的鲁棒性.另外,本文采用了一个更加有效且适合的数值实现方法来进一步改善所提模型的修复性能.最后,给出了各种丢失情形下的实验结果来表明该模型在小波域修复性能和抗噪性能等方面的优越性.     

小波变换与纹理合成相结合的图像修复

目的 为了克服传统的图像修复算法在结构和纹理边界的错误修复,利用小波变换域的系数特征,探讨了一种基于小波变换与纹理合成相结合的修复算法。方法 算法先利用小波变换将待修复图像分解成具有不同分辨率的低频子图和高频子图,然后根据不同子图各自的特征分别进行修复。对代表图像结构信息的低频子图,采用FMM(fast marching method)算法进行修复;对代表图像纹理信息的高频子图,根据各子图中小波系数的特征,利用纹理合成方法进行修复。结果 分层、分类修复方法对边缘破损具有良好的修复效果,其峰值信噪比相比于传统算法提高了1~2 dB。结论 与相关算法相比,本文算法的综合修复能力较好,可以有效修复具有较强边缘和丰富纹理的破损图像,尤其对破损自然图像的修复,修复后图像质量得到较大提升,修复效果更符合人眼视觉效应。     

基于P-Laplace算子的小波域图像修补模型

本文研究如何利用不完整的小波系数来恢复原始图像. Chan, Shen 和 Zhou 已经提出了一种基于整体变分 (total variational, TV) 模型的小波域图像修补算法. TV 模型的主要优点是可以保持图像的边缘, 但该模型在平滑区容易产生阶梯效应, 使图像的修补效果不是很理想. 为了克服这个缺陷, 本文首先从局部坐标角度分析了TV模型与p-Laplace算子的物理意义, 从本质上说明了 p-Laplace 算子的扩散性能优于 TV 模型.然后给出了一种基于 p-Laplace 算子的小波域图像修补模型. 该模型不仅有效降低 TV 模型引入的阶梯效应, 而且能保持图像的边缘, 用较少的运算量得到比 TV 模型更好的修补效果. 实验结果表明, 该模型在运算时间和修补效果上都具有更好的综合性能.     

鲁棒的梯度驱动图像修复算法

数字图像形态特征的修复目前主要采用基于梯度驱动的偏微分方程(PDE)作为计算模型。虽然该类模型对较大区域的形态特征修复具有明显优势,但是修复过程中信息传播方向不确定使得它对修复对象具有选择性。在分析该类模型在图像修复中的计算本质和对应物理意义的基础上,结合典型仿真实验,认为保持信息传播方向始终指向待修复区域之外对修复结果具有决定性影响,并由此提出一种梯度驱动图像修复的新算法。实验结果表明,该算法能够保持信息传播方向的稳定,使得修复具有更强的鲁棒性。     

小波域的快速自适应图像修复算法

传统的基于偏微分方程的图像修复算法需要大量迭代,修复所耗时间较长,复杂度高。针对这一问题,提出了一种小波域的非迭代自适应图像修复算法。该算法对破损图像进行小波分解,找到待修复区域,根据待修复区域及其邻域像素值自适应选择修复模板大小,对修复模板内的像素值进行方向筛选,使修复过程严格按照等照度线方向行进,对修复后的图像进行小波重构。实验结果表明,该方法显著地缩短了修复时间,且对于图像的纹理细节、结构信息都达到了更好的修复效果。     

基于全变分小波域缺损修复算法的一种数值实现

针对基于全变分(TV)小波域图像修复算法,提出了一种新的数值计算方法。算法充分利用了图像待修复像素点周围八邻域的信息计算曲率在像素域的近似值。与传统近似求解方法不同,该数值近似计算方法不仅保证了近似结果具有更高的精确性,而且对噪声的鲁棒性好。对不同丢失率的图像用所提出的数值计算方法取得了较好的修复效果,尤其当小波系数丢失率较高时效果更为明显。所提方法可以为图像压缩导致的系数缺损修复提供解决思路。     

面向图像内补与外推问题的迭代预测统一框架

目的 图像内补与外推可看做根据已知区域绘制未知区域的问题,是计算机视觉领域研究热点。近年来,深度神经网络成为解决内补与外推问题的主流方法。然而,当前解决方法多分别对待内补与外推问题,导致二者难以统一处理;且模型多采用卷积神经网络(convolutional neural network,CNN)构建,受到视野局部性限制,较难绘制远距离内容。针对这两个问题,本文按照分而治之思想联合CNN与Transformer构建深度神经网络,提出图像内补与外推统一处理框架及模型。方法 将内补与外推问题的解决过程分解为“表征、预测、合成”3个部分,表征与合成采用CNN完成,充分利用其局部相关性进行图像到特征映射和特征到图像重建;核心预测由Transformer实现,充分发挥其强大的全局上下文关系建模能力,并提出掩膜自增策略迭代预测特征,降低Transformer同时预测大范围未知区域特征的难度;最后引入对抗学习提升绘制图像逼真度。结果 实验给出在多种数据集下内补与外推对比评测,结果显示本文方法各项性能指标均超越对比方法。通过消融实验发现,模型相比采用非分解方式具有更佳表现,说明分而治之思路功效显著。此外,对掩膜自增策略进行详细的实验分析,表明迭代预测方法可有效提升绘制能力。最后,探究了Transformer关键结构参数对模型性能的影响。结论 本文提出一种迭代预测统一框架解决图像内补与外推问题,相较对比方法性能更佳,并且各部分设计对性能提升均有贡献,显示了迭代预测统一框架及方法在图像内补与外推问题上的应用价值与潜力。     

二阶总广义变分小波修复模型

目的 针对全变分小波修复模型易导致阶梯效应的缺陷,提出一种加权的二阶总广义变分小波修复模型。方法 不同于全变分小波修复模型,假设的新模型引入二阶导数项且能够自动地调解一阶和二阶导数项。另外,为有效地利用图像的局部结构信息,新模型引入了权函数,它既能保护图像的边缘又增强光滑区域的去噪能力。 为有效地计算新模型,利用交替方向法将该模型变为两个子模型, 然后对两个子模型分别给出相应的理论和算法推导。结果 相比最近基于全变分正则小波修复模型(平均信噪比,平均绝对误差及平均结构相似性指标分别为21.884 4,6.857 8,0.827 2),新模型得到更好的修复效果(平均信噪比,平均绝对误差及平均结构相似性指标分别为22.313 8,6.626 1,0.831 8)。结论 与全变分正则相比,二阶总广义变分正则更好地减轻阶梯效应。目前, 国内外学者对该问题的研究取得一些结果。由于原始-对偶算法需要较小的参数,所以运算的速度较慢,因此更快速的算法理论有待进一步研究。另外,该正则能应用于图像去噪、分割、放大等方面。     

基于紧框架的二阶总广义变分图像修复模型

针对传统总广义变分（TGV）小波修复模型采用单一小波基变换,仅对纹理细节信息较少且结构简单的图像有很好修复能力的缺点,提出一种紧框架域下的总广义变分正则化修复模型。不同于经典小波变换,紧框架系统具有冗余、时移不变和线性相位等图像处理过程中较为重要的特性。新模型通过引入多层紧框架分解系数的低阶与高阶导数项建立正则化项,获取图像不同尺度多方向上的特征信息来对图像进行约束。模型的数值实现采用分裂技术与原始-对偶方法相结合的优化算法（PDSBA）,交替迭代求解两个易于处理的子问题,提高了图像修复过程的处理效率。相比于传统模型,所提模型不仅具有保边性能,而且对含有较多细节或纹理信息的图像也有较好的修复效果。实验结果显示,三个修复性能指标：峰值信噪比（PSNR）、平均绝对误差（MAE）和结构相似测度（SSIM）均获得显著提升。     

基于双树复小波域的马尔可夫随机场样本修补算法

为了消除大目标图像修补过程中,修补区域由于累积误差引起的马赛克和振铃效应,提出基于双树复小波域的马尔可夫随机场(MRF)样本修补算法。首先应用双树复小波变换(DTCWT)将待修补图像变换到复频域,通过合理的置信度和数据项计算待修补块的修补顺序;然后应用MRF样本修补算法在不同尺度、不同方向下修补未知区域;最后利用双树复小波逆变换重构图像。实验结果表明,与传统离散小波修补方法相比,双树复小波域MRF样本修补算法能更好地保持修补区域纹理和结构信息。     

一种轮廓波域MRF样本修补技术

提出轮廓波域MRF样本修补方法,通过轮廓波变换把待修补图像进行多尺度、多方向修补。实验结果表明,与传统的离散小波修补方法相比,轮廓波域MRF样本修补技术能更好抑制马赛克现象,保持良好的纹理和结构特征。     

Example-based 3D inpainting of point clouds using metric tensor and Christoffel symbols

In this paper, we address the problem of 3D inpainting using example-based methods for point cloud data. 3D inpainting is a process of filling holes or missing regions in the reconstructed 3D models. Typically inpainting methods addressed in the literature fill missing regions due to occlusions or inaccurate scanning of 3D models. However, we focus on scenarios involving naturally existing damaged models which are partly broken or incomplete in artifacts at cultural heritage sites. We propose two example-based inpainting techniques, namely region of interest (ROI)-based and patch-based methods, to inpaint the missing regions of the damaged model. For both the methods, we represent the 3D model as a set of Riemannian manifolds in Euclidean space, to capture the inherent geometry using metric tensor and Christoffel symbols as geometric features and decompose into basic shape (such as spherical, conical and cylindrical) regions using decomposition algorithm derived from supervised learning. In ROI-based method, instead of using single similar example for inpainting, we select the most relevant regions that best-fit the missing region from the set of basic shape regions derived from n similar examples. And in patch-based method, we not only select the most relevant regions but cluster the regions into a set of patches. The best corresponding patches that match the missing region to be inpainted are considered to be the most relevant best-fit patches that cover the complete missing region. We demonstrate the performance of proposed inpainting methods on cultural heritage artifacts with varying complexities and sizes for both synthetically generated holes and real missing regions.     

图像多尺度统计模型综述

基于小波变换的图像处理方法是目前的主流方法,而对图像特征的多尺度统计建模则是图像压缩、去噪、分割、纹理分析与合成等统计应用的关键问题。本文综述了图像的多尺度统计模型,包括边缘分布模型以及层内、层间和混合相关模型,分析了各模型的优缺点,给出了对各种相关模型捕捉系数间相关性能力的归一化量测。最后,简单介绍了基于多尺度几何分析的统计图像模型,并对多尺度统计建模的前景进行了展望。     

Optimizing predictive models for leaf chlorophyll concentration based on continuous wavelet analysis of hyperspectral data

Recent studies have demonstrated that the decomposition of hyperspectral data using wavelet analysis is able to generate wavelet coefficients that can be used for estimating leaf chlorophyll (chl) concentrations. However, there is considerable scope for refining such techniques and this study addresses this issue by identifying the optimal spectral domain for use in constructing predictive models. Leaf reflectance spectra were simulated with the PROSPECT model (a model of leaf optical properties spectra) using randomly selected values for the input parameters. From reflectance and first derivative spectra different spectral wavelength domains were extracted, ranging from 400–450 to 400–2500 nm, using increments of 50 nm for the upper wavelength limit. Using the data for each wavelength domain, continuous wavelet decomposition was applied using 53 different wavelets, in turn. The resulting wavelet coefficients, from scales 1 to 128, were used as independent factors to construct predictive models for leaf chl concentration. Wavelet coefficients (at a specific scale generated by a given wavelet) in the chl absorption region remain constant when using spectral wavelength domains of 400–900 nm and broader, but narrower domains cause variability in the coefficients. Lower scale wavelet coefficients (scales 1–32) contain little information on chl concentration and their predictive performance does not vary with the spectral wavelength domain used. The higher scale wavelet coefficients (scales 64 and 128) can capture information on chl concentration, and predictive capability increases rapidly when the spectral wavelength domains vary from 400–700 to 400–900 nm but it can decrease or fluctuate for broader domains. In terms of accuracy and computational efficiency, models derived from the spectral wavelength domain 400–900 nm which use wavelet coefficients from scale 64 are optimal and a range of wavelet functions are suitable for performing the decomposition. The importance of optimizing the spectral wavelength domain highlighted by these findings has broader significance for the use of wavelet decomposition of hyperspectral data in quantifying other vegetation biochemicals and in other remote sensing applications.     

基于双树复小波的图像修复

小波变换技术已被广泛应用于图像修复领域,但其在图像修复过程中出现的边缘部分模糊或不连接的情况成为了一个难点。针对此问题,提出了基于双树复小波变换的图像修复算法。该算法使用双树复小波变换对破损图像进行多尺度和多方向的分解,对各个高频方向子带使用全变分(Total Variation,TV)模型进行快速修复,各个低频分量使用改进了的曲率驱动扩散(Curvature-Driven-Diffusions,CCD)模型进行迭代修复,最后通过小波逆变换得到最终的修复图像。实验结果表明,该方法很好地推广了双树复小波变换在图像修复领域中的应用,并且在图像纹理的修复以及在结构部分的填充都有较好的效果。     

Range map superresolution-inpainting,and reconstruction from sparse data

Range images often suffer from issues such as low resolution (LR) (for low-cost scanners) and presence of missing regions due to poor reflectivity, and occlusions. Another common problem (with high quality scanners) is that of long acquisition times. In this work, we propose two approaches to counter these shortcomings. Our first proposal which addresses the issues of low resolution as well as missing regions, is an integrated super-resolution (SR) and inpainting approach. We use multiple relatively-shifted LR range images, where the motion between the LR images serves as a cue for super-resolution. Our imaging model also accounts for missing regions to enable inpainting. Our framework models the high resolution (HR) range as a Markov random field (MRF), and uses inhomogeneous MRF priors to constrain the solution differently for inpainting and super-resolution. Our super-resolved and inpainted outputs show significant improvements over their LR/interpolated counterparts. Our second proposal addresses the issue of long acquisition times by facilitating reconstruction of range data from very sparse measurements. Our technique exploits a cue from segmentation of an optical image of the same scene, which constrains pixels in the same color segment to have similar range values. Our approach is able to reconstruct range images with as little as 10% data. We also study the performance of both the proposed approaches in a noisy scenario as well as in the presence of alignment errors.     

基于对数函数的非局部总变分图像修复模型

针对基于总变分最小化的图像修复模型容易造成阶梯效应及假边缘的问题,提出了基于对数函数的非局部总变分图像修复模型。新的总变分能量泛函的被积函数为一个关于梯度幅度的对数函数。在总变分模型与各向异性扩散模型的偏微分方程框架下,首先,从理论上证明了对数总变分模型满足良好扩散所需的所有性质,并对其局部扩散行为进行了理论分析,证明了其在等照度方向及梯度方向扩散的良好特性。其次,为考虑图像块的相似性及避免局部模糊,采用非局部对数总变分进行数值实现。实验结果表明,与经典的总变分修复模型相比,基于对数函数的非局部总变分模型对图像修复的效果良好,避免了局部模糊,且在图像平滑区域能较好地抑制阶梯效应;与基于样例的修复模型相比,所提模型对纹理图像能获得更为自然的修复效果。实验结果表明,与三类总变分模型和基于样例的修复模型相比,所提模型的性能最优,且与各对比模型的平均结果（图2、图3、图4）相比,其结构相似性指数（SSIM）分别提高了0.065、0.022和0.051,峰值信噪比（PSNR）分别提高了5.94 dB、4.00 dB和6.22 dB。含噪图像的修复结果表明所提模型具有较好的鲁棒性,对含噪声的图像也能获得良好的修复效果。     

基于小波域HMT模型的图像超分辨率重构

小波域HMT模型采用混合高斯分布，并通过多尺度小波系数隐状态之间的Markov依赖性刻画自然图像小波系数随尺度减小呈指数衰减的特性．由于小波域HMT准确刻画了自然图像小波变换的统计特性，因此文中算法以此作为自然图像的先验模型，并把图像超分辨率问题表述为一个约束优化问题，采用Cycle-Spinning方法抑制重构出的高分辨率图像中可能存在的震铃和锯齿等失真，最后，进行了定量误差分析并给出了一些实验图像供主观评价。     

Multi-output parameter estimation of dynamic systems by output shapes

A multi-output method of parameter estimation is introduced for dynamic systems that relies on the shape attributes of model outputs. The shapes of outputs in this method are represented by the surfaces that are generated by continuous wavelet transforms (CWTs) of the outputs in the time-scale domain. Since the CWTs also enhance the delineation of outputs and their sensitivities to model parameters in the time-scale domain, regions in the time-scale plane can be identified wherein the sensitivity of the output with respect to one model parameter dominates all the others. This allows approximation of the prediction error in terms of individual model parameters in isolated regions of the time-scale domain, thus enabling parameter estimation based on a small set of wavelet coefficients. These isolated regions of the time-scale plane also reveal numerous transparencies to be exploited for parameter estimation. It is shown that by taking advantage of these transparencies, the robustness of parameter estimation can be improved. The results also indicate the potential for improved precision and faster convergence of the parameter estimates when shape attributes are used in place of the magnitude.