Sparse Seam-Carving for Structure Preserving Image Retargeting

The seam carving technique is one of the most popular techniques for content-aware image retargeting. There have been many attempts to improve existing methods of seam carving to provide more aesthetic retargeting results. However, most previous methods focus on developing an energy map for preserving important image regions while retargeting. Commonly, the existing seam carving based techniques tends to produce discontinuity artifacts when retargeting straight structures. We observe that it occurs when a bunch of seams are assigned across straight structures (i.e., lines or edges). In this paper, we propose a modified seam carving method not only preserving important parts, but also maintaining the prominent structure of the image. To this end, we put a constraint that seams are sparsely assigned each other. Experimental results show that the proposed sparse seam carving yields more plausible retargeting results than previous retargeting approaches and it is very robust to images containing prominent lines or edges.     

图像自适应中基于形变控制的细缝剪裁方法

为了降低图像自适应过程中图像内容缺失和图像扭曲变形,文章中提出了一种基于形变控制的图像自适应细缝裁剪方法,将融合局部和全局显著性的显著图作为细缝裁剪参考的能量图,对需要剪裁的细缝按照这个能量图进行排序,以更好地保护图像中感兴趣区域（ROI）,保留图像的主要内容信息;同时,利用SIFT流矢量场来衡量剪裁图像的形变程度,每移除一定数量的细缝就计算剪裁图像与原始图像之间的形变,一旦形变达到某一阈值,就停止细缝裁剪,转换为均匀缩放使图像到达目标尺寸。实验结果表明,文章中提出的方法更好地平衡了均匀缩放和非均匀剪裁,更有利于保留图像主要内容和避免主要内容的变形。      

A Distortion-Sensitive Seam Carving Algorithm for Content-Aware Image Resizing

As displays become less expensive and are incorporated into more and more devices, there has been an increased focus on image resizing techniques to fill an image to an arbitrary screen size. Traditional methods such as cropping or resampling can introduce undesirable losses in information or distortion in perception. Recently, content-aware image retargeting methods have been proposed (Avidan and Shamir, ACM Trans Graphics 26(3), 2007; Guo et al., IEEE Trans Multimedia 11(5):856–867, 2009; Shamir and Avidan, Commun ACM 52(1), 2009; Simakov et al. 2008; Wolf et al. 2007), which produce exceptional results. In particular, seam carving, proposed by Avidan and Shamir, has gained attention as an effective solution. However, there are many cases where it can fail. In this paper we propose a distortion-sensitive seam carving algorithm for content-aware image resizing that improves edge preservation and decreases aliasing artifacts. In the proposed approach, we use local gradient information along with a thresholding technique to guide the seam selection process and provide a mechanism to halt seam carving when further processing would introduce unacceptable visual distortion in the resized image. Furthermore, anti-aliasing filter is used to reduce the aliasing artifacts caused by seam removal. Experiments have demonstrated superior performance over the current seam carving methods.     

Detecting image seam carving with low scaling ratio using multi-scale spatial and spectral entropies

Seam carving is the most popular content-aware image retargeting technique. However, it may also be used to correct poor photo composition in photography competition or to remove object from image for malicious purpose. A blind detection approach is presented for seam carved image with low scaling ratio (LSR). It exploits spatial and spectral entropies (SSE) on multi-scale images (candidate image and its down-sampled versions). We observe that when a few seams are deleted from an original image, its SSE distribution is greatly changed. Forty-two features are designed to unveil the statistical properties of SSE in terms of centralized tendency, dispersion tendency and distribution tendency. They are combined with the local binary pattern (LBP)-based energy features to form ninety-six features. Finally, support vector machine (SVM) is exploited as classifier to determine whether an image is original or suffered from seam carving. Experimental results show that the proposed approach achieves superior detection accuracy over the state-of-the-art works, especially for resized image by seam carving with LSRs. Moreover, it is robust against JPEG compression and seam insertion.     

Salient edge and region aware image retargeting

The purpose of image retargeting is to automatically adapt a given image to fit the size of various displays without introducing severe visual distortions. The seam carving method can effectively achieve this task and it needs to define image importance to detect the salient context of images. In this paper we present a new image importance map and a new seam criterion for image retargeting. We first decompose an image into a cartoon and a texture part. The higher order statistics (HOS) on the cartoon part provide reliable salient edges. We construct a salient object window and a distance dependent weight to modify the HOS. The weighted HOS effectively protects salient objects from distortion by seam carving. We also propose a new seam criterion which tends to spread seam uniformly in nonsallient regions and helps to preserve large scale geometric structures. We call our method salient edge and region aware image retargeting (SERAR). We evaluate our method visually, and compare the results with related methods. Our method performs well in retargeting images with cluttered backgrounds and in preserving large scale structures.     

融合显著与深度信息的缝切割重定向方法

针对当前内容感知的重定向方法中可能出现的变形和失真问题,提出一种融合显著与深度信息的缝切割重定向方法.首先利用GBVS算法获取图像显著信息,结合图像梯度信息与通过SIFT匹配方法获取的图像深度信息构建更精确的重要度图;其次,根据重要度图的能量分布,对原始图像进行处理,得到最终的重定向结果.基于公开数据库在两个不同评价标准下与多种重定向方法的对比表明,本文方法能够最大程度的保留图像的显著部分.     

基于最小位移可视差的连续Seam Carving算法在图像缩放中的研究

Image retargeting technologies require important information preservation and less edge distortion during increasing/decreasing image size. The seam carving based algorithms, as the classic retargeting model, receive widespread attention in recent years. However, because of the discrete least energy seam searching strategy, the retargeting information can not be passed generation by generation, which causes retargeting distortions to prevail. To solve this problem, the Just Noticeable Distortion (JND) algorithm is proposed to detect the potential distribution of distortion information. Through the proposed energy weight E_w, the JND information can be passed to the following retargeting iteration for distortion reduction. According to the best knowledge, it is the first time to propose the seam carving algorithm in continuous way by the JND algorithm and energy weight, are the promising results also demonstrated compared with several new approaches at public database ‘Retarget Me’, qualitatively and quantitatively.     

基于最小位移可视差的连续Seam Carving算法在图像缩放中的研究

图像缩放技术要求对图像缩放的同时保证重要信息不丢失且物体边缘不发生扭曲。近年来,Seam Carving及其改进算法得到了广泛的关注和研究。由于采用了离散式最小能量线迭代搜索策略,缩放信息无法在迭代过程中传递导致扭曲现象普遍存在。该文针对上述问题提出最小位移可视差(JND)检测算法,能够有效地检测每一次迭代中出现的潜在扭曲信息。能量权重\begin{document}$ {E}_{w} $\end{document}能够将JND信息累加传递给后续的迭代过程,从而抑制缩放过程中的边缘扭曲现象。通过JND算法和能量权重,该文首次将离散的Seam Carving模型转变为连续缩放模型。最后,在公共数据集RetargetMe上与最新的图像缩放算法进行多组对比实验,验证了所提方法的有效性和先进性。     

基于深层特征学习的可压缩感知及缝雕刻的图像重定向

多媒体技术的飞速发展推动了图像处理与显示设备 的应用与发展,为了使图像在不同的设备上进行最佳显示,需要对图像的尺寸进行调整。因 此,本文提出一种基于深层特征学习的可压缩感知及接缝雕刻的图像重定向方法。首先从预 先训练的VGG-19网络中提取输入图像的深度特征图,从最深层开始 计算特征图像的可压缩率,根据计算的可压缩率运用接缝雕刻的方法在特征域(Feature fie lds Seam Carving,FSC)调整特征图的大小,然后依次向较浅的层传播,得到所有特征层的 重定向图像后,将输入图像对应于第一层特征图的去缝的位置处的像素去掉,得到原始图像 的重定向图像。若没有达到目标图像的大小,最后再进行均匀缩放(scaling,SCL)。在Retar getMe数据集上分别进行主观与客观评估,结果表明,与其他方法相比,本文的重定向方法 总体上实现了更好的性能。     

Optimal bi-directional seam carving for compressibility-aware image retargeting

A novel compressibility-aware image retargeting method based on seam carving is introduced in this paper. We propose a new significance detection method, with both the edge information and visual saliency taken into consideration. A wall-seam model is constructed to evaluate the image compressibility and assign the right number of seams for each direction. By repeatedly carving out or inserting seams we can retarget the image to a new size while preserving the important content. Finally, our algorithm is completed with the supplement of uniformly scaling, the stretched image is resized to the target size with the least structure distortion brought. Experimental results on images show that those improvements are effective and our approach can preserve image content better compared to several state-of-the-art image retargeting methods.     

Saliency detection in the compressed domain for adaptive image retargeting

Saliency detection plays important roles in many image processing applications, such as regions of interest extraction and image resizing. Existing saliency detection models are built in the uncompressed domain. Since most images over Internet are typically stored in the compressed domain such as joint photographic experts group (JPEG), we propose a novel saliency detection model in the compressed domain in this paper. The intensity, color, and texture features of the image are extracted from discrete cosine transform (DCT) coefficients in the JPEG bit-stream. Saliency value of each DCT block is obtained based on the Hausdorff distance calculation and feature map fusion. Based on the proposed saliency detection model, we further design an adaptive image retargeting algorithm in the compressed domain. The proposed image retargeting algorithm utilizes multioperator operation comprised of the block-based seam carving and the image scaling to resize images. A new definition of texture homogeneity is given to determine the amount of removal block-based seams. Thanks to the directly derived accurate saliency information from the compressed domain, the proposed image retargeting algorithm effectively preserves the visually important regions for images, efficiently removes the less crucial regions, and therefore significantly outperforms the relevant state-of-the-art algorithms, as demonstrated with the in-depth analysis in the extensive experiments.     

Saliency-based content-aware lifestyle image mosaics

In this paper, a novel content-aware lifestyle image mosaic technique is proposed based on image saliency. The image saliency is used in the whole process of creating mosaics. Firstly, a novel energy map for variable-size tile decomposition is proposed by combining Neighborhood Inhomogeneity Factor and Graph-Based Visual Saliency. The target image is divided into small tiles with variable sizes based on the energy map. Secondly, saliency-weighted image retrieval is introduced to choose the tile images from a certain image database. Different aspect ratio between the tile and the corresponding tile image may lead to obvious distortion. Therefore, before filled into the tile, the chosen tile image is resized by seam carving to change its aspect ratio. At last, color correction is done on the mosaic result to improve the color similarity. Compared with other mosaic methods, the proposed technique creates much better mosaics in visual aspect.     

Content-aware image resizing based on random-carving with probability

To improve the running speed of image resizing,a fast content-aware image resizing algorithm was proposed based on the threshold learning and random-carving with probability.Firstly the important map was calculated by combining the graph-based visual saliency map and gradient map.Then the image threshold value was obtained by radial basis function (RBF) neural network learning.And by the threshold,the original image was separated into the protected part and the unprotected part which was corresponding to the important part and the unimportant part of the original image individually.Finally,the two parts were allocated different resizing scales and the random-carving with probability was applied to them respectively.Experiments results show that the proposed algorithm has lower time cost comparing to the state-of-arts algorithms in MSRA image database,and has a better visual perception on image resizing.     

基于梯度矢量方向性分析的线裁剪算法

在传统线裁剪(seam carving)算法中梯度矢量的方向性是一个被忽略的因素,该文提出了一种新的基于梯度矢量方向性分析的线裁剪算法。首先利用随机纹理区域内梯度矢量方向散乱的特点,对局部区域的图像梯度矢量进行低通滤波,使算法能够提取更加合理的像素线路;接着又根据像素线路的走向不同,定义两个不同的像素能量函数,给予像素梯度矢量的x, y分量以不同的权值。实验结果图像显示,文中算法不仅可以更好地保护图像边缘等细节,还可以在整体上达到与原图像更加近似的视觉效果。定量分析结果也显示,相比其它算法该文算法在完整性距离和一致性距离两方面都取得了更好结果。     

Dynamic distortion maps for image retargeting

We present an image retargeting method that incorporates image content distortion into a mesh optimization process through the generation of dynamic distortion maps. The warping process is driven by the distortion produced by the warping process itself. We retarget the image through an iterative mesh optimization process to minimize the visual distortion. An adaptive distortion map is iteratively constructed to describe the visual distortion between the original image and the retargeted image. The mesh mapping from the source image to the retargeted image is optimized through an energy minimization process. The objective of the optimization is to allow the distortion produced by the retargeting process to be distributed to smooth and highly textured regions which cause less visual distortion while preserving the geometrical structure of the mesh at regions that may cause distortion after retargeting. Experimental evaluation of the algorithm is done both subjectively and objectively.     

基于图像分块和优化累积能量图的线裁剪算法

针对传统线裁剪方法对图像过度裁剪造成失真的问题,该文提出一种基于图像分块的线裁剪方法。该方法把分块的思想融入到线裁剪并优化累积能量图,能在一定程度上保护图像主体区域,又兼顾背景区域的裁剪效果。分块是根据显著图的平均列累加能量向量按照逐列标记的方式把图像分成保护区域和非保护区域,再根据每个区域的面积来分配裁剪线的数目。在裁剪过程中,优化了累积能量图,降低了小面积显著主体被裁剪掉的可能性。在MSRA数据库上与目前流行的线裁剪及其改进的方法进行对比,并把各种方法得到的缩放结果图在互联网上进行主观评价测试,实验结果表明该文方法具有更好的主观缩放效果,对各类图像的缩放具有普适性。     

支持可变分辨率的SPIHT编码算法

为了适应移动多媒体通信中终端设备显示尺寸多样性的要求,本文将图像缩放嵌入编码过程中,提出了一种基于线裁剪（seam carving, SC）的支持可变分辨率的多级树集合排序（set portioning in hierarchical trees,SPIHT）图像编码算法。该算法在构造新的基于根节点的空间方向树的基础上,利用SC生成基于块的线能量图来引导编码,获得空域可伸缩的码流;解码端仅需获取与终端设备显示尺寸相关的码流即可完成解码和图像缩放。实验结果表明,当编码和解码图像的分辨率一致时,本文算法的率失真（rate-distortion, R-D）性能逼近传统的SPIHT算法;当解码图像的分辨率可变时,本文算法在压缩码率与重建图像的主观质量上均优于传统的SPIHT算法。      

基于卡尔曼滤波器的焊缝识别算法

为了提高工业现场焊缝跟踪的准确性和抗干扰能力,提出了一种基于约束卡尔曼滤波器的焊缝识别算法。采用结构光测量系统获取焊缝连续图像,然后利用形态学处理定位焊缝图像角点,并通过高斯拟合法提取激光条纹中心点,将角点和中心点的位置和速度作为卡尔曼滤波器的状态变量,建立卡尔曼滤波器的状态方程和测量方程,再由角点和中心点的位置关系构建等式约束方程,通过约束方程对卡尔曼滤波后的结果进行修正。结果表明该算法能有效实现焊缝跟踪,提高了焊缝识别的精度和抗干扰能力。     

Multi-focus image fusion with Geometrical Sparse Representation

Multi-focus image fusion aims to generate an image with all objects in focus by integrating multiple partially focused images. It is challenging to find an effective focus measure to evaluate the clarity of source images. In this paper, a novel multi-focus image fusion algorithm based on Geometrical Sparse Representation (GSR) over single images is proposed. The main novelty of this work is that it shows the potential of GSR coefficients used for image fusion. Unlike the traditional sparse representation-based (SR) methods, the proposed algorithm does not need to train an overcomplete dictionary and vectorize the signal. In our algorithm, using a single dictionary image, the source images are first represented by geometrical sparse coefficients. Specifically, we employ a weighted GSR model in the sparse coding phase, ensuring the importance of the center pixel. Then, the weighted GSR coefficient is used to measure the activity level of the source image and an average pooling strategy is applied to obtain an initial decision map. Third, the decision map is refined with a simple post-processing. Finally, the fused all-in-focus image is constructed with the refined decision map. Experimental results demonstrate that the proposed method can be competitive with or even superior to the state-of-the-art fusion methods in both subjective and objective comparisons.     

图像拼接中运动目标干扰的自适应消除

运动目标在图像拼接时,易产生重影现象.在对SURF算法进行改善的基础上,提出了自适应去除重影的拼接算法.利用增强的单通道图像对其特征点进行提取、匹配,再利用帧差法确定出运动目标在图像重叠区域的位置;取运动目标区域的中心点作为参考点,采用改进的加权融合算法求取加权系数,以增大权值差异.实验表明所提方法能较好地解决运动目标对图像拼接的干扰问题,与最优拼接缝多频率融合算法相比,融合质量、清晰度以及拼接的实时性得到了提高.