融合边缘检测和自注意力的图像修复方法

针对修复后图像边界模糊、图像纹理不清晰、视觉效果差的问题,提出了一种融合边缘检测和自注意力机制的生成式对抗修复模型.通过边缘检测可提取出图像的轮廓信息,避免了修复后边界模糊的问题;利用自注意力机制能够捕获图像全局信息并生成图像精确细节的能力,设计出融合自注意力机制的纹理修复网络.该模型由边缘补全网络和纹理修复网络组成,首先,设计的边缘补全网络对受损图像的边缘进行补全,得到边缘补全图像;其次,利用纹理修复网络联合补全的边缘图像对缺失区域的纹理进行精确修复.在CelebA和Place2两个图像数据集上对本文所建模型进行了训练和测试.实验结果表明:本文所建模型与现有图像修复方法相比,大幅提高了图像修复的精确度,且生成的图像更加逼真.     

引入蛇模型的曲率驱动扩散图像修复方法

数字图像修复方法可以自动地修复数字图像中用户定义的污损区域。T.F.Chan等人提出的非纹理的曲率驱动扩散算法是基于偏微分方程的图像修复的代表算法之一。通过在该算法中应用蛇模型的图像增强项,提出了引入蛇模型的曲率驱动扩散图像修复方法,在热扩散过程中,拉动曲线向目标边界演化,达到修复污损图像中的断裂边缘的目的,从而较好保持图像的视觉连通性。实验结果表明,在迭代次数相同的条件下,可以比原始的曲率驱动扩散图像修复算法获得更好的修复效果。     

基于曲率驱动方向性热流的图像修复模型

提出一种曲率驱动的方向性热流图像修复模型。在该模型中,信息向待修复区域的扩散被看成是两个相互垂直的1D方向性热流的耦合,其中的扩散方向由图像的局部相关几何决定,而扩散强度是一个曲率相关的函数。实验结果表明,该模型不仅能够保证修复边缘的尖锐性还可以减少修复过程中虚假边缘的产生。     

A level set method for oil slick segmentation in SAR images

Despite much effort and significant progress in recent years, image segmentation remains a challenging problem in image processing, especially for the low contrast, noisy synthetic aperture radar (SAR) images. This paper explores the segmentation of oil slicks using a partial differential equation (PDE)‐based level set method, which represents the slick surface as an implicit propagation interface. Starting from an initial estimation with priori information, the level set method creates a set of speed functions to detect the position of the propagation interface. Specifically, the image intensity gradient and the curvature flow are utilized together to determine the speed and direction of the propagation. This allows the front interface to propagate naturally with topological changes, significant protrusions and narrow regions, giving rise to stable and smooth boundaries that discriminate oil slicks from the surrounding water. As the speckles are removed concurrently while the front interface propagates, the pre‐filtering of noise is saved. The proposed method has been illustrated by experiments on oil slick segmentation using the ERS‐2 SAR images. Its advantages over the traditional image segmentation approaches have also been demonstrated.     

Generalizing edge detection to contour detection for image segmentation

One approach to image segmentation defines a function of image partitions whose maxima correspond to perceptually salient segments. We extend previous approaches following this framework by requiring that our image model sharply decreases in its power to organize the image as a segment’s boundary is perturbed from its true position. Instead of making segment boundaries prefer image edges, we add a term to the objective function that seeks a sharp change in fitness with respect to the entire contour’s position, generalizing from edge detection’s search for sharp changes in local image brightness. We also introduce a prior on the shape of a salient contour that expresses the observed multi-scale distribution of contour curvature for physical contours. We show that our new term correlates strongly with salient structure. We apply our method to real images and verify that the new term improves performance. Comparisons with other state-of-the-art approaches validate our method’s advantages.     

基于非局部CTV-L1模型的大破损彩色纹理图像修复

针对大破损彩色纹理图像的修复问题,本文将TV-L1模型推广到非局部CTV-L1模型。该模型不仅包含非局部算子,同时还引入了CTV (Color Total Variation)规则项。前者可以修复大破损纹理图像,后者充分考虑了彩色图像层与层之间的耦合关系,在处理彩色图像时可以有效地保边缘。为提高模型的运算效率,本文通过引入辅助变量和Lagrange乘子为其设计了相应的增广Lagrangian算法。数值实验结果证实所提出的模型在处理彩色图像时可以有效地保持边缘,同时去除图像中异常的不规则点。本文的相关研究可以推广到彩色纹理图像椒盐噪声去除及彩色纹理图像分割等。     

A Linear Framework for Region-Based Image Segmentation and Inpainting Involving Curvature Penalization

We present the first method to handle curvature regularity in region-based image segmentation and inpainting that is independent of initialization.     

分区域去运动模糊

去运动模糊一直是计算机视觉领域中面向画质增强的一个热点研究方向。模糊核的估算是去运动模糊中的关键问题。提出一种新的思路,即首先将模糊图像按照模糊核的相似度进行图像分割,再对分割后的图像应用空间不变去模糊算法。本文方法主要包含以下几个步骤:分离输入图像中的光照、颜色和纹理信息;分割图像;分区域估算模糊核,计算重叠区域模糊核,并根据计算出的模糊核进行分区域单核去模糊;利用重叠区域整合拼接去模糊结果并还原光照和颜色信息。实验结果表明本文方法比基于单核的去运动模糊算法效果要好。     

利用曲率信息的图像分割改进模型

首先从理论上分析无须重新初始化的水平集方法的主动轮廓图像分割模型,该模型对一些具有不光滑尖角的图像进行分割时,捕捉这些尖角往往不精确甚至失败;然后对利用边缘检测函数的曲率信息识别出凸尖角并进行分割的方法进行研究,由于此方法没有考虑凹尖角的情形,故对含凹尖角的图像分割效果不理想。为解决该问题,提出利用图像的曲率信息识别出凹尖角,再将其与利用边缘检测函数曲率信息识别凸尖角的方法相结合,进一步修正边缘检测函数,达到准确捕捉物体的凸尖角和凹尖角的目的,保证了分割的准确性。数值实验表明,该方法的分割效果较好。     

Fast local image inpainting based on the Allen–Cahn model

In this paper, we propose a fast local image inpainting algorithm based on the Allen–Cahn model. The proposed algorithm is applied only on the inpainting domain and has two features. The first feature is that the pixel values in the inpainting domain are obtained by curvature-driven diffusions and utilizing the image information from the outside of the inpainting region. The second feature is that the pixel values outside of the inpainting region are the same as those in the original input image since we do not compute the outside of the inpainting region. Thus the proposed method is computationally efficient. We split the governing equation into one linear equation and one nonlinear equation by using an operator splitting technique. The linear equation is discretized by using a fully implicit scheme and the nonlinear equation is solved analytically. We prove the unconditional stability of the proposed scheme. To demonstrate the robustness and accuracy of the proposed method, various numerical results on real and synthetic images are presented.     

Unsupervised multi-class segmentation of SAR images using fuzzy triplet Markov fields model

Triplet Markov fields (TMF) model proposed recently is suitable for nonstationary image segmentation. For synthetic aperture radar (SAR) image segmentation, TMF model can adopt diverse statistical models for SAR data related to diverse radar backscattering sources. However, TMF model does not take into account the inherent imprecision associated with SAR images. In this paper, we propose a statistical fuzzy TMF (FTMF) model, which is a fuzzy clustering type treatment of TMF model, for unsupervised multi-class segmentation of SAR images. This paper contributes to SAR image segmentation in four aspects: (1) Nonstationarity of the statistical distribution of SAR intensity/amplitude data is taken into account to improve the spatial modeling capability of fuzzy TMF model. (2) Mean field theory is generalized to deal with planar variables to derive prior probability in fuzzy TMF model, which resolves the problem in Gibbs sampler in terms of computation cost. (3) A fuzzy objective function with regularization by Kullback–Leibler information of fuzzy TMF model is constructed for SAR image segmentation. The introduction of fuzziness for the belongingness of SAR image pixel makes fuzzy TMF model be able to retain more information from SAR image. (4) Fuzzy iterative conditional estimation (ICE) method, as an extension of the general ICE method is proposed to perform the model parameters estimation. The effectiveness of the proposed algorithm is demonstrated by application to simulated data and real SAR images.     

Multiscale Active Contours

We propose a new multiscale image segmentation model, based on the active contour/snake model and the Polyakov action. The concept of scale, general issue in physics and signal processing, is introduced in the active contour model, which is a well-known image segmentation model that consists of evolving a contour in images toward the boundaries of objects. The Polyakov action, introduced in image processing by Sochen-Kimmel-Malladi in Sochen et al. (1998), provides an efficient mathematical framework to define a multiscale segmentation model because it generalizes the concept of harmonic maps embedded in higher-dimensional Riemannian manifolds such as multiscale images. Our multiscale segmentation model, unlike classical multiscale segmentations which work scale by scale to speed up the segmentation process, uses all scales simultaneously, i.e. the whole scale space, to introduce the geometry of multiscale images in the segmentation process. The extracted multiscale structures will be useful to efficiently improve the robustness and the performance of standard shape analysis techniques such as shape recognition and shape registration. Another advantage of our method is to use not only the Gaussian scale space but also many other multiscale spaces such as the Perona-Malik scale space, the curvature scale space or the Beltrami scale space. Finally, this multiscale segmentation technique is coupled with a multiscale edge detecting function based on the gradient vector flow model, which is able to extract convex and concave object boundaries independent of the initial condition. We apply our multiscale segmentation model on a synthetic image and a medical image.     

Simultaneous inpainting for image structure and texture using anisotropic heat transfer model

We propose a PDE-based image inpainting method using anisotropic heat transfer model, which can simultaneously propagate the structure and texture information. In structure inpainting, the propagating direction and intensity are related to image contents, and the strength of propagation along gradient direction is made inversely proportional to the magnitude of gradient. In texture inpainting, the added texture term reflects periodicity along the texture and its perpendicular direction. For numerical implementation, the step size of finite difference is adaptively chosen according to the curvature, leading to fewer iteration steps and satisfactory inpainting quality. Compared with other high order PDE methods and layered methods, the proposed approach is more concise and doesn’t need image decomposition. Experiments are carried out to show effectiveness of the method.     

基于SLIC的改进GrabCut彩色图像快速分割

GrabCut算法用户交互量少且分割精度高,但它迭代使用GraphCuts的求解模式使得在处理高分辨率图像时,耗时巨大。提出了一种快速GrabCut算法,在高斯混合模型参数估计过程中,通过SLIC算法构建精简的GraphCuts模型以实现加速。通过SLIC算法将原始图像快速地预分割成具有确定边界且区域内相似度高的超像素图,并以此构建精简的网络图。以块内的RGB均值描述超像素特征进行高斯混合模型参数估计。为了提高分割精度,使用得到的GMM参数对原始图像进行分割。实验结果证明了该算法在时效和精度上都有很好的性能。     

A novel active contour model for medical images via the Hessian matrix and eigenvalues

This paper presents a new level set formulation for active contour models (ACM). We propose the idea of integrating the eigenvalue information of Hessian matrix into the level set function. By this new level set function, the principal curvature information of images is used to enhance the ability of segmenting boundary regions. The advantages of our model are as follows: firstly, the interior and exterior object boundaries can be segmented with the initial contour being anywhere in the input image. Secondly, this method can work with heterogeneous images. Thirdly, the proposed model can produce smooth and right boundaries of objects having vital importance in medical operations. Extensive experiments demonstrate that the proposed model can obtain better segmentation results.     

Dynamic Shape and Appearance Modeling via Moving and Deforming Layers

We propose a model of the shape, motion and appearance of a scene, seen through a sequence of images, that captures occlusions, scene deformations, unconstrained viewpoint variations and changes in its radiance. This model is based on a collection of overlapping layers that can move and deform, each supporting an intensity function that can change over time. We discuss the generality and limitations of this model in relation to existing ones such as traditional optical flow or motion segmentation, layers, deformable templates and deformotion. We then illustrate how this model can be used for inference of shape, motion, deformation and appearance of the scene from a collection of images. The layering structure allows for automatic inpainting of partially occluded regions. We illustrate the model on synthetic and real sequences where existing schemes fail, and show how suitable choices of constants in the model yield existing schemes, from optical flow to motion segmentation and inpainting.     

全局与局部属性一致的图像修复模型

目的 图像修复是计算机视觉领域的研究热点之一。基于深度学习的图像修复方法取得了一定成绩,但在处理全局与局部属性联系密切的图像时难以获得理想效果,尤其在修复较大面积图像缺损时,结果的语义合理性、结构连贯性和细节准确性均有待提高。针对上述问题,提出一种基于全卷积网络,结合生成式对抗网络思想的图像修复模型。方法 基于全卷积神经网络,结合跳跃连接、扩张卷积等方法,提出一种新颖的图像修复网络作为生成器修复缺损图像;引入结构相似性（structural similarity,SSIM）作为图像修复的重构损失,从人眼视觉系统的角度监督指导模型学习,提高图像修复效果;使用改进后的全局和局部上下文判别网络作为双路判别器,对修复结果进行真伪判别,同时,结合对抗式损失,提出一种联合损失用于监督模型的训练,使修复区域内容真实自然且与整幅图像具有属性一致性。结果 为验证本文图像修复模型的有效性,在CelebA-HQ数据集上,以主观感受和客观指标为依据,与目前主流的图像修复算法进行图像修复效果对比。结果表明,本文方法在修复结果的语义合理性、结构连贯性以及细节准确性等方面均取得了进步,峰值信噪比（peak signal-to-noise ratio,PSNR）和结构相似性的均值分别达到31.30 dB和90.58%。结论 本文提出的图像修复模型对图像高级语义有更好的理解,对上下文信息和细节信息把握更精准,能取得更符合人眼视觉感受的图像修复结果。     

Occlusion-Aware 3D Morphable Models and an Illumination Prior for Face Image Analysis

Faces in natural images are often occluded by a variety of objects. We propose a fully automated, probabilistic and occlusion-aware 3D morphable face model adaptation framework following an analysis-by-synthesis setup. The key idea is to segment the image into regions explained by separate models. Our framework includes a 3D morphable face model, a prototype-based beard model and a simple model for occlusions and background regions. The segmentation and all the model parameters have to be inferred from the single target image. Face model adaptation and segmentation are solved jointly using an expectation–maximization-like procedure. During the E-step, we update the segmentation and in the M-step the face model parameters are updated. For face model adaptation we apply a stochastic sampling strategy based on the Metropolis–Hastings algorithm. For segmentation, we apply loopy belief propagation for inference in a Markov random field. Illumination estimation is critical for occlusion handling. Our combined segmentation and model adaptation needs a proper initialization of the illumination parameters. We propose a RANSAC-based robust illumination estimation technique. By applying this method to a large face image database we obtain a first empirical distribution of real-world illumination conditions. The obtained empirical distribution is made publicly available and can be used as prior in probabilistic frameworks, for regularization or to synthesize data for deep learning methods.     

基于稀疏表示和结构自相似性的单幅图像盲解卷积算法

图像盲解卷积研究当模糊核未知时,如何从模糊图像复原出原始清晰图像.由于盲解卷积是一个欠定问题,现有的盲解卷积算法都直接或间接地利用各种先验知识.本文提出了一种结合稀疏表示与结构自相似性的单幅图像盲解卷积算法,该算法将图像的稀疏性先验和结构自相似性先验作为正则化约束加入到图像盲解卷积的目标函数中,并利用图像不同尺度间的结构自相似性,将观测模糊图像的降采样图像作为稀疏表示字典的训练样本,保证清晰图像在该字典下的稀疏性.最后利用交替求解的方式估计模糊核和清晰图像.模拟和真实数据上的实验表明本文算法能够准确估计模糊核,复原清晰的图像边缘,并具有很好的鲁棒性.