Exemplar-based image inpainting using adaptive two-stage structure-tensor based priority function and nonlocal filtering

For the exemplar-based image inpainting problem, the filling order and local intensity smoothness are two crucial factors that should be considered carefully. This work gives a new exemplar-based image inpainting method, preventing geometric structures from being destroyed and reconstructing textures well to obtain elegant-looking outputs. For a better filling order, we define a new adaptive two-stage structure-tensor based priority function. To promote the local intensity smoothness, we adopt a non-local way, and at the same time, propose a weighted filter based on a Gaussian-like function to generate the ideal filling patch by combining non-local patches. We compare the proposed method with some recent state-of-the-art image inpainting approaches on different tasks, such as texture and structure synthesis, object removal, and remote sensing images inpainting. Experimental results demonstrate the superiority of the proposed method, both visually and quantitatively.     

Image completion using efficient belief propagation via priority scheduling and dynamic pruning.

In this paper, a new exemplar-based framework is presented, which treats image completion, texture synthesis, and image inpainting in a unified manner. In order to be able to avoid the occurrence of visually inconsistent results, we pose all of the above image-editing tasks in the form of a discrete global optimization problem. The objective function of this problem is always well-defined, and corresponds to the energy of a discrete Markov random field (MRF). For efficiently optimizing this MRF, a novel optimization scheme, called priority belief propagation (BP), is then proposed, which carries two very important extensions over the standard BP algorithm: "priority-based message scheduling" and "dynamic label pruning." These two extensions work in cooperation to deal with the intolerable computational cost of BP, which is caused by the huge number of labels associated with our MRF. Moreover, both of our extensions are generic, since they do not rely on the use of domain-specific prior knowledge. They can, therefore, be applied to any MRF, i.e., to a very wide class of problems in image processing and computer vision, thus managing to resolve what is currently considered as one major limitation of the BP algorithm: its inefficiency in handling MRFs with very large discrete state spaces. Experimental results on a wide variety of input images are presented, which demonstrate the effectiveness of our image-completion framework for tasks such as object removal, texture synthesis, text removal, and image inpainting.     

Simultaneous structure and texture image inpainting

An algorithm for the simultaneous filling-in of texture and structure in regions of missing image information is presented in this paper. The basic idea is to first decompose the image into the sum of two functions with different basic characteristics, and then reconstruct each one of these functions separately with structure and texture filling-in algorithms. The first function used in the decomposition is of bounded variation, representing the underlying image structure, while the second function captures the texture and possible noise. The region of missing information in the bounded variation image is reconstructed using image inpainting algorithms, while the same region in the texture image is filled-in with texture synthesis techniques. The original image is then reconstructed adding back these two sub-images. The novel contribution of this paper is then in the combination of these three previously developed components, image decomposition with inpainting and texture synthesis, which permits the simultaneous use of filling-in algorithms that are suited for different image characteristics. Examples on real images show the advantages of this proposed approach.     

f+v分解与分形图像修复算法设计

提出了一种基于分形与图像分解的修复算法,将图像分解成为纹理和结构两部分,纹理部分采用以分形维度作为度量函数的修复算法,结构部分采用曲率驱动扩散CDD(Curvature-Drive Diffusions)算法进行并行修复,达到快速、高效的修复受损图像的目的。结果表明,该算法具有速度快、对大面积受损图像修复效果好的特点。     

RESEARCH ON WEIGHTED PRIORITY OF EXEMPLAR-BASED IMAGE INPAINTING

The priority of the filled patch play a key role in the exemplar-based image inpainting, and it should be determined firstly to optimize the process of image inpainting. A modified image inpainting algorithm is proposed by weighted-priority based on the Criminisi algorithm. The improved algorithm demonstrates better relationship between the data term and the confidence term for the optimization of the priority than the classical Criminisi algorithm. By comparing the effect of the inpainted images with different structure, conclusion can be drawn that the optimal priority should be chosen properly for different images with different structures.     

基于自适应模板和置信度更新的图像修复算法

去除图像背景中的部分目标,是图像修复技术应用之一。为了修补去除目标后留下的大面积空白区域,文中在基于样本图像修复方法的基础上,引入了自适应思想。首先利用Canny算子检测图像的边缘,再根据图像边缘特征自适应地改变模板块的大小;为了避免误差的累积传递,提出了新的置信度更新方法,使更新后的置信度与累积误差成反比。实验结果证明,文中方法比原方法能更好地修复图像边缘、复杂纹理,并且保留边角,使效果更自然。     

Image segmentation and inpainting using hierarchical level set and texture mapping

Image inpainting is an artistic procedure to recover a damaged painting or picture. We propose a novel approach for image inpainting by using the Mumford-Shah (MS) model and the level set method to estimate image structure of the damaged regions. This approach has been successfully used in image segmentation problem. Compared to some other inpainting methods, the MS model approach detects and preserves edges in the inpainting areas. We propose a fast and efficient algorithm that achieves both inpainting and segmentation. In previous works on the MS model, only one or two level set functions are used to segment an image. While this approach works well on simple cases, detailed edges cannot be detected in complicated image structures. Although multi-level set functions can be used to segment an image into many regions, the traditional approach causes extensive computations and the solutions depend on the location of initial curves. Our proposed approach utilizes faster hierarchical level set method and guarantees convergence independent of initial conditions. Because we detect both the main structure and the detailed edges, our approach preserves edges in the inpainting area. Also, exemplar-based approach for filling textured regions is employed. Experimental results demonstrate the advantage of our method.     

基于模板的图像修复算法优先权的改进

研究了基于块填充的图像修复算法,修复图像的质量容易受到待修复区域边界像素修复顺序的影响,通过分析待修复区域像素点所在模块的图像特征,改进了填充算法的优先权,分别是基于P_Laplace算子和Euler' s elastics模型的优先权计算方法的改进.实验结果证实了文中所介绍算法能有效提高重建图像的感知质量.     

一种基于Poisson方程的分离型图像修复方法

图像修复是一种恢复图像中损坏部分的技术,具有广泛的应用.基于采样复制的修复方法对纹理图像有较好的效果,但容易产生块效应,而且对结构信息的修复可能产生较大的偏差.本文提出了一种基于Poisson方程的分离型修复算法,首先将原图分解为结构图像和纹理图像两部分,然后根据其特件分别进行修复,叠加后得到最终的修复结果.对结构图像使用Laplacian算子强化结构信息,然后对Laplacian场进行修复并使用Poisson方程重建,可以同时保持锐利的区域边界以及平滑的区域背景.实验表明该方法可以有效改善修复的视觉效果,对大区域修复也有良好的表现.     

High efficiency depth image-based rendering with simplified inpainting-based hole filling

Hole and crack filling is the most important issue in depth-image-based rendering (DIBR) algorithms for generating virtual view images when only one view image and one depth map are available. This paper proposes a priority patch inpainting algorithm for hole filling in DIBR algorithms by generating multiple virtual views. A texture-based interpolation method is applied for crack filling. Then, an inpainting-based algorithm is applied patch by patch for hole filling. A prioritized method for selecting the critical patch is also proposed to reduce computation time. Finally, the proposed method is realized on the compute unified device architecture parallel computing platform which runs on a graphics processing unit. Simulation results show that the proposed algorithm is 51-fold faster for virtual view synthesis and achieves better virtual view quality compared to the traditional DIBR algorithm which contains depth preprocessing, warping, and hole filling.     

基于Curvelet方向特征的样本块图像修复算法

能否保持修复后图像的结构连贯性和邻域一致性决定了修复性能的优劣.为提高现有样本块修复算法性能,本文提出基于Curvelet变换的样本块图像修复算法.首先利用Curvelet变换估计待修复图像的4方向特征.然后利用颜色信息与方向信息共同衡量样本块间的相似度,在此基础上构造颜色-方向结构稀疏度函数.同时根据构造的加权颜色-方向距离寻找合适的多个匹配块,并利用多个匹配块在构造的颜色和方向空间内的邻域一致性约束下稀疏表示目标块,同时根据目标块所处区域特性自适应确定误差容限.实验结果表明提出算法较现有算法可获得更优的修复效果,尤其是在修复富含结构纹理破损类型的图像时.     

基于区域纹理合成的图像修补算法

针对传统修补技术在图像恢复中出现的纹理模糊现象,该文根据纹理合成技术,提出了一种区域纹理合成的修补算法。算法根据破损点邻域特征选择模板,去除模板内的破损点后进行匹配,在搜索范围内寻找最佳匹配点。该文采用由边缘向中心逐步推进的修补次序,提高破损区域内纹理结构延伸方向的正确性以及连续性,避免出现纹理模糊现象。算法能够适用于多种性质的纹理,在处理真实图像时使人在视觉上获得满意的效果。     

Structure and texture filling-in of missing image blocks in wireless transmission and compression applications

An approach for filling-in blocks of missing data in wireless image transmission is presented. When compression algorithms such as JPEG are used as part of the wireless transmission process, images are first tiled into blocks of 8 /spl times/ 8 pixels. When such images are transmitted over fading channels, the effects of noise can destroy entire blocks of the image. Instead of using common retransmission query protocols, we aim to reconstruct the lost data using correlation between the lost block and its neighbors. If the lost block contained structure, it is reconstructed using an image inpainting algorithm, while texture synthesis is used for the textured blocks. The switch between the two schemes is done in a fully automatic fashion based on the surrounding available blocks. The performance of this method is tested for various images and combinations of lost blocks. The viability of this method for image compression, in association with lossy JPEG, is also discussed.     

基于结构分量和信息熵的Criminisi图像修复算法

针对Criminisi图像修复算法中优先级计算易受 图像纹理影响的问题,提出了改 进的基于图像结构分量的优先级函数。首先采用变分分解模型,将待修补图像分解为结构分 量和 纹理分量;其次基于结构分量计算数据项,排除纹理的影响;然后在优先权函数中 引入度量像素块复杂度的信息熵,将像素块中除了中心点之外其它位置的结构信息 融 入到优先权的计算中,使修补次序进一步向结构丰富的像素块倾斜;最后将优先权函数 表 示为置信度、数据项和信息熵的加权和,以解决传统Criminisi算法优先权随着置信度 迅速 下降为零而造成修复次序出现偏差的不足。新的优先权函数排除了像素块中在计算数据项时 纹 理的影响,并且融合更多的结构信息,使修复次序更加准确。实验结果表明,对于 不 同的人工图像和自然图像,本文模型都能取得较为满意的修复结果。     

Unsupervised image classification, segmentation, and enhancementusing ICA mixture models

An unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent, non-Gaussian densities. The algorithm estimates the data density in each class by using parametric nonlinear functions that fit to the non-Gaussian structure of the data. This improves classification accuracy compared with standard Gaussian mixture models. When applied to images, the algorithm can learn efficient codes (basis functions) for images that capture the statistically significant structure intrinsic in the images. We apply this technique to the problem of unsupervised classification, segmentation, and denoising of images. We demonstrate that this method was effective in classifying complex image textures such as natural scenes and text. It was also useful for denoising and filling in missing pixels in images with complex structures. The advantage of this model is that image codes can be learned with increasing numbers of classes thus providing greater flexibility in modeling structure and in finding more image features than in either Gaussian mixture models or standard independent component analysis (ICA) algorithms.     

基于图像修复技术的压缩方法的研究

通过对块状缺损图像修复技术的研究,提出一种改进的纹理合成修复算法。继而依据粗糙度的分析对图像块进行分类,根据修复算法特性人为丢弃部分图像块,并结合JPEG算法进行压缩编码。在相近的视觉效果下,可以使图像压缩率得到显著的提高。     

Identification of inpainted images and natural images for digital forensics

Image forensics is a form of image analysis for finding out the condition of an image in the complete absence of any digital watermark or signature.It can be used to authenticate digital images and identify their sources.While the technology of exemplar-based inpainting provides an approach to remove objects from an image and play visual tricks.In this paper,as a first attempt,a method based on zero-connectivity feature and fuzzy membership is proposed to discriminate natural images from inpainted images.Firstly,zero-connectivity labeling is applied on block pairs to yield matching degree feature of all blocks in the region of suspicious,then the fuzzy memberships are computed and the tampered regions are identified by a cut set.Experimental results demonstrate the effectiveness of our method in detecting inpainted images.     

Hybrid error concealment method combining exemplar-based image inpainting and spatial interpolation

This paper proposes an efficient error concealment method for the reconstruction of pixels that are lost in video communication. The proposed method is developed by combining exemplar-based image inpainting for patch reconstruction and spatial interpolation for pixel reconstruction using adaptive threshold by local complexity. By exemplar-based image inpainting, regions with regular structures are reconstructed. For complex regions with irregular structures, just one pixel is reconstructed using the proposed spatial interpolation method. The proposed spatial interpolation method performs reconstruction by selecting adaptively directional interpolation or neighbor interpolation based on gradient information. Simulation results show that the proposed hybrid method performs reconstruction with significantly improved subjective quality compared with the previous spatial error concealment and image inpainting methods. The proposed method also gives substantial improvements of PSNR compared with the previous methods.     

基于样本和线性结构信息的大范围图像修复算法

基于纹理的图像修复算法对于修复破损区域比较大的图像效果较好,但该算法对于含有结构信息的图像修复效果很差.通过新的优先项的计算、平均值补偿及增加惩罚项提高传统的基于样本的图像修复算法的修复效果,结合图像中常出现的直线和曲线结构特征,提出了基于样本和结构信息的大范围图像修复算法.实验表明,该算法易于实现,修复结果能达到令人满意的效果,具有较高的实用价值.     

Image inpainting using scene constraints

Recent deep learning-based inpainting methods have shown significant improvements and generate plausible images. However, most of these methods may either synthesis unrealistic and blurry texture details or fail to capture object semantics. Furthermore, they employ huge models with inefficient mechanisms such as attention. Motivated by these observations, we propose a new end-to-end generative-based multi-stage architecture for image inpainting. Specifically, our model exploits the segmentation labels predictions to robustly reconstruct the object boundaries and avoid blurry or semantically incorrect images. Meanwhile, it employs edges predictions to recover the image structure. Different than previous approaches, we do not predict the segmentation labels/edges from the corrupted image. Instead, we employ a coarse image that contains more valuable global structure data. We conduct a set of extensive experiments to investigate the impact of merging these auxiliary pieces of information. Experiments show that our computationally efficient model achieves competitive qualitative and quantitative results compared to the state-of-the-art methods on multiple datasets.