Multi-operator image retargeting with automatic integration of direct and indirect seam carving

Multi-operator image resizing can preserve important objects and structure in an image by combining multiple image resizing operators. However, traditional multi-operator methods do not take both horizontal and vertical content-aware resizing potential into consideration, which essentially leads to squeeze/stretch effect in the resultant images. In this paper, we propose a new multi-operator scheme that addresses aforementioned issue by integrating direct and indirect seam carving. Compared with previous methods, the proposed scheme remarkably reduces the cost of deciding when to change operators, by employing a newly defined image artifact measure. Furthermore, we propose a novel seam carving enhancement, named ACcumulated Energy Seam Carving (ACESC), as a basic operator to improve global structure preservation. By combining horizontal and vertical seam carving, our scheme preserves the shapes of important objects well. We present typical results to demonstrate the effectiveness of our method. User study shows that our method has high user preference.     

Optimized image resizing using flow-guided seam carving and an interactive genetic algorithm

In this paper, we introduce a novel method for content-aware image resizing based on flow-guided seam carving. It extends the existing seam carving framework by replacing the conventional energy field with a “structure-aware” energy field that takes into account the feature orientations in the image. Guided by this new energy field, our approach excels in preserving (i.e., avoiding the distortion of) important structures in the image, such as shape boundaries. We also present a simple user interface to further optimize the resizing result based on the genetic selection process among multiple resizing operators such as scaling, cropping, and flow-guided seam carving. We show that such simple user interaction, coupled with the genetic algorithm, dramatically increases the chances of producing the user-desired outcome.     

Semantic‐Preserving Word Clouds by Seam Carving

Word clouds are proliferating on the Internet and have received much attention in visual analytics. Although word clouds can help users understand the major content of a document collection quickly, their ability to visually compare documents is limited. This paper introduces a new method to create semantic‐preserving word clouds by leveraging tailored seam carving, a well‐established content‐aware image resizing operator. The method can optimize a word cloud layout by removing a left‐to‐right or top‐to‐bottom seam iteratively and gracefully from the layout. Each seam is a connected path of low energy regions determined by a Gaussian‐based energy function. With seam carving, we can pack the word cloud compactly and effectively, while preserving its overall semantic structure. Furthermore, we design a set of interactive visualization techniques for the created word clouds to facilitate visual text analysis and comparison. Case studies are conducted to demonstrate the effectiveness and usefulness of our techniques.     

基于缝裁剪和变形的图像缩放方法

提出一种既能保持图像重要内容又能较好地保持重要物体形状的图像缩放算法。该方法结合传统的缝裁剪技术和变形技术来对图像进行缩放。首先利用当前公认效果良好的基于图模型的流形排序显著性检测算法得到图像的显著度图,结合图像梯度能量等信息来构造结构更为清晰的图像重要度图;其次利用之前构造的图像重要度图并按缩放尺度的大小来确定适当的缩放方法;最后根据度量比较结果来选择经典缝裁剪方法或基于能量优化的变形方法进行图像缩放。对比实验结果表明,该方法在图像缩放时能保持重要内容和显著物体形状结构。     

结合美学原则的内容感知图像缩放算法

针对目前内容感知的图像缩放算法存在美学效果考虑不足的问题,提出一种结合美学原则的缩放算法。算法首先采用图像协同分割的思想并结合视觉显著性检测获取图像的重要度图,以此作为图像重要内容的依据,在后续处理中防止其变形;其次针对移动终端常见的两类图像分别选择相适应的美学原则并建立其量化公式;借鉴经典的Seam Carving算法思想,利用美学构图原则和重要度图来指导裁剪线的复制与删除,达到图像缩放的效果。实验结果表明,与同类算法相比,该算法的缩放结果在保留原图重要信息的条件下,更具美感。     

基于边缘检测的保特征的图像缩放算法

针对Seam Carving算法在实现图像缩放的过程中,对图像边缘几何特征造成破坏的现象,提出一种基于边缘检测的改进Seam Carving算法。首先在Seam Carving算法计算低能量线的同时引入图像边缘检测,其次确定低能量线与边缘的交点,进而提升交点邻域的能量,分散过于集中的低能量线。仿真实验结果表明,该算法能够有效减少Seam Carving算法缩放图像后产生的畸变,很好地保持了图像的重要几何特征。     

Nonhomogeneous scaling optimization for realtime image resizing

We present a novel approach for interactive content-aware image resizing. The resizing is performed on warping a triangular mesh over the image, which captures the image saliency information as well as the underlying image features. The warped triangular mesh and the horizontal and vertical scales of all triangles are simultaneously obtained by a quadratic optimization which can be achieved by solving a sparse linear system. Our approach can preserve the shapes of curved features in the resized images. The resizing operation can be performed in an interactive rate which makes the proposed approach practically useful for realtime image resizing. To guarantee a foldover free resizing result, we modify the optimization to a standard quadratic programming. A number of experimental results have shown that our approach has obtained pleasing results and outperforms the previous approaches.     

Real-time content-aware image resizing

Content-aware image resizing is a kind of new and effective approach for image resizing, which preserves image content well and does not cause obvious distortion when changing the aspect ratio of images. Recently, a seam based approach for content-aware image resizing was proposed by Avidan and Shamir. Their results are impressive, but because the method uses dynamic programming many times, it is slow. In this paper, we present a more efficient algorithm for seam based content-aware image resizing, which searches seams through establishing the matching relation between adjacent rows or columns. We give a linear algorithm to find the optimal matches within a weighted bipartite graph composed of the pixels in adjacent rows or columns. Therefore, our method is fast (e.g. our method needs only about 100 ms to reduce a 768 × 1024 image’s width to 1/3 while Avidan and Shamir’s method needs 12 s). This supports immediate image resizing whereas Avidan and Shamir’s method requires a more costly pre-processing step to enable subsequent real-time processing. A fast method such as the one proposed will be also needed for future real-time video resizing applications.     

Image resizing via non-homogeneous warping

Image resizing aims to adapt images to displays with different sizes and aspect ratios. In this paper, we provide a new image resizing approach for efficiently determining the non-homogeneous warp that better preserves the global image configuration and concentrates the distortion in regions of the image where they are least-likely to be noticed. Considering the different properties of large displays and small displays, we design different strategies for upsizing and downsizing. We define a variety of quadratic metrics to measure image distortion. We introduce a patch-linking scheme that can better preserve the global image configuration. We formulate image resizing as a quadratic minimization problem, which can be efficiently solved. We experiment with our method on a variety category of images and compare our results to the state of the art.     

基于主体区域保持的图像缩放算法

基于插值运算的缩放算法和经典的缝裁剪算法是两种常用的图像缩放算法,传统的 缩放算法在缩放比例不一致的情况下其效果不佳,而缝裁剪算法在主体区域较大或者图像背景较 为复杂时对图像的主体区域会造成一定破坏。针对以上问题,提出了一种基于主体区域保持的图 像缩放算法,使用高斯差分对图像进行角点检测,利用角点产生凸包,根据凸包对图像进行主体 区域检测,计算能量图并对位于主体区域像素点的能量给予相应的权重,根据权重的不同对主体 区域进行不同程度的保护。实验结果表明,该算法能更好地保持图像主体区域。     

内容感知图像缩放技术综述

随着不同分辨率和纵横比的显示设备的迅猛增长,内容感知图像缩放技术逐渐成为图像处理领域新的研究热点之一。内容感知图像缩放的目标是在任意改变图像大小时保持图像中的主体特征不变。围绕其关键步骤:图像重要度识别和基于重要度的缩放,先概述重要度识别的相关方法,然后重点综述基于重要度的缩放技术。根据缩放是在像素级上操作还是亚像素级上操作,或者两者兼有,将其分为基于线裁剪缩放、基于图像变形缩放和多操作缩放3类,并比较各类方法的优缺点,同时给出各类方法所适合处理的图像类型。最后在分析各类研究方法的基础上,给出了内容感知缩放技术的可能发展方向。     

基于全局图像结构信息的Seam Carving算法*

本文提出了一种基于全局图像结构信息的Seam Carving算法,它根据像素的重要性修改图像尺寸和比例。通过从图像提取特定方向的边缘结构信息,再利用每个像素的梯度信息,从全局和局部两方面定义新的像素能量计算函数,以此来阻止Seam通路与特定方向图像边缘的交叉,避免边缘像素的不一致位移,以此保持图像的边缘结构。实验结果证明,本文算法减少了处理后图像的结构形变,有效改进了Seam Carving算法的处理效果。     

Geometry seam carving

We present a novel approach to feature-aware mesh deformation. Previous mesh editing methods are based on an elastic deformation model and thus tend to uniformly distribute the distortion in a least-squares sense over the entire deformation region. Recent results from image resizing, however, show that discrete local modifications such as deleting or adding connected seams of image pixels in regions with low saliency lead to far superior preservation of local features compared to uniform scaling — the image retargeting analog to least-squares mesh deformation. Hence, we propose a discrete mesh editing scheme that combines elastic as well as plastic deformation (in regions with little geometric detail) by transferring the concept of seam carving from image retargeting to the mesh deformation scenario. A geometry seam consists of a connected strip of triangles within the mesh’s deformation region. By collapsing or splitting the interior edges of this strip, we perform a deletion or insertion operation that is equivalent to image seam carving and can be interpreted as a local plastic deformation. We use a feature measure to rate the geometric saliency of each triangle in the mesh and a well-adjusted distortion measure to determine where the current mesh distortion asks for plastic deformations, i.e., for deletion or insertion of geometry seams. Precomputing a fixed set of low-saliency seams in the deformation region allows us to perform fast seam deletion and insertion operations in a predetermined order such that the local mesh modifications are properly restored when a mesh editing operation is (partially) undone. Geometry seam carving hence enables the deformation of a given mesh in a way that causes stronger distortion in homogeneous mesh regions while salient features are preserved much better.     

基于SIFT相似性度量的图像缩放

针对传统图像缩放算法存在的图像内容破坏与失真等问题,提出一种结合接缝雕刻和标准缩放的图像缩放算法。利用接缝雕刻算法对原始图像进行尺寸调整,分别从原始图像和缩放图像上提取尺度不变特征变换(SIFT)特征,并对2幅图像的SIFT特征进行匹配计算,得到两者的相似性距离。当原始图像与缩放图像之间的相似性距离达到某阈值时,即在图像重要目标或内容将要被破坏之前,停止使用接缝雕刻算法,采用标准缩放算法进行整体尺寸的调整操作。实验结果表明,该算法不仅可以有效避免图像内容的破坏与失真,而且可以较好地保护图像局部结构和全局视觉效果。     

基于DCT域视觉显著性检测的图像缩放算法*

为适应不同终端显示多样化的要求,需对接收到的图像进行缩放调整。针对现有的基于内容感知(content-aware)的图像缩放方法中视觉内容的连贯性易被破环而出现失真的问题,提出了一个基于离散余弦变换(discrete cosine transform, DCT)域的视觉显著性检测的图像缩放算法。该算法利用DCT域的视觉显著性检测模型获取视觉显著图,然后结合视觉显著图和能量分布图进行线裁剪(Seam Carving),实现了图像的缩放。实验结果表明,该算法与现有的基于内容感知的图像缩放方法相比,不仅保护了视觉显著内容,还保证了图像内容的连贯性,算法质量指数也获得明显的提高。     

An efficient semantic – Related image retrieval method

Many previous techniques were designed to retrieve semantic images in a certain neighborhood of the query image and thus bypassing the semantically related images in the whole feature space. Several recently methods were designed to retrieve semantically related images in the entire feature space but with low precision. In this paper, we propose a Semantic – Related Image Retrieval method (SRIR), which can retrieve semantic images spread in the entire feature space with high precision. Our method takes advantage of the user feedback to determine the semantic importance of each query and the importance of each feature. In addition, the retrieval time of our method does not increase with the number of user feedback. We also provide experimental results to demonstrate the effectiveness of our method.     

Image re-targeting with balanced energy map and foreground constraint

This paper presents a content-aware image re-targeting method based on seam carving. It first combines the image gradient and the visual saliency to measure the cost of the seams. Then proposes a method to evaluate the diagonal artifacts in addition to the previous horizontal and vertical artifacts for the forward seam carving method. At last, it develops a simple high-level saliency detection method to constrain the seam carving procedure for protecting the foreground contents. The experimental results showed that the proposed method can improve the visual quality of the re-targeted image and the robustness of the seam carving method. Moreover, the improved method is simple to implement, and can be easily applied to many existing seam carving based image resizing methods.     

基于切缝对齐和变形校正的缝切割图像尺寸调整算法

针对图像尺寸调整的缝切割方法中存在的结构断裂和变形扭曲等问题,提出了一种基于切缝对齐和变形校正的切缝优化算法以获得更好的图像尺寸缩放效果.算法首先利用基于图割的切缝优化方法获得当前需要移除的一条切缝,然后采取动态规划算法,由粗到细地优化得到切缝上下像素之间的高精度匹配对应关系,最后通过逆向映射来对图像进行变形以校正发生扭曲甚至断裂的结构信息.实验结果表明,该方法能够很好地弥补缝切割所产生的扭曲变形,达到结构连续的视觉效果.基于切缝对齐和变形校正的切缝优化算法可用于调整图像尺寸使之与相应的显示设备相符,具有巨大的应用潜力.     

基于罗伯茨梯度的彩色图像缩放新法

为了缩放图像而不改变图像中重要特征物的大小和比率,提出一种叫做缝络删增的缩放方法.一个缝络就是从顶到底或从左到右8-连接且仅连接一个像素的路径.最佳路径由图像的罗伯茨梯度函数来定义.通过在行列一个或二个方向重复删除或插入缝络就能改变图像的尺寸和比率.算法定义的缝络绕过了特征物边界,从而保护了图像的内容.理论分析和实验结果表明,该算法较其它算法能更有效地保护图像特征物.