Sparse Seam-Carving for Structure Preserving Image Retargeting

The seam carving technique is one of the most popular techniques for content-aware image retargeting. There have been many attempts to improve existing methods of seam carving to provide more aesthetic retargeting results. However, most previous methods focus on developing an energy map for preserving important image regions while retargeting. Commonly, the existing seam carving based techniques tends to produce discontinuity artifacts when retargeting straight structures. We observe that it occurs when a bunch of seams are assigned across straight structures (i.e., lines or edges). In this paper, we propose a modified seam carving method not only preserving important parts, but also maintaining the prominent structure of the image. To this end, we put a constraint that seams are sparsely assigned each other. Experimental results show that the proposed sparse seam carving yields more plausible retargeting results than previous retargeting approaches and it is very robust to images containing prominent lines or edges.     

Optimal bi-directional seam carving for compressibility-aware image retargeting

A novel compressibility-aware image retargeting method based on seam carving is introduced in this paper. We propose a new significance detection method, with both the edge information and visual saliency taken into consideration. A wall-seam model is constructed to evaluate the image compressibility and assign the right number of seams for each direction. By repeatedly carving out or inserting seams we can retarget the image to a new size while preserving the important content. Finally, our algorithm is completed with the supplement of uniformly scaling, the stretched image is resized to the target size with the least structure distortion brought. Experimental results on images show that those improvements are effective and our approach can preserve image content better compared to several state-of-the-art image retargeting methods.     

Cluster-Based Saliency-Guided Content-Aware Image Retargeting

Straightforward image resizing operators without considering image contents (e.g., uniform scaling) cannot usually produce satisfactory results, while content-aware image retargeting aims to arbitrarily change image size while preserving visually prominent features. In this paper, a cluster-based saliency-guided seam carving algorithm for content-aware image retargeting is proposed. To cope with the main drawback of the original seam carving algorithm relying on only gradient-based image importance map, we integrate a gradient-based map and a cluster-based saliency map to generate a more reliable importance map, resulting in better single image retargeting results. Experimental results have demonstrated the efficacy of the proposed algorithm.     

图像自适应中基于形变控制的细缝剪裁方法

为了降低图像自适应过程中图像内容缺失和图像扭曲变形,文章中提出了一种基于形变控制的图像自适应细缝裁剪方法,将融合局部和全局显著性的显著图作为细缝裁剪参考的能量图,对需要剪裁的细缝按照这个能量图进行排序,以更好地保护图像中感兴趣区域（ROI）,保留图像的主要内容信息;同时,利用SIFT流矢量场来衡量剪裁图像的形变程度,每移除一定数量的细缝就计算剪裁图像与原始图像之间的形变,一旦形变达到某一阈值,就停止细缝裁剪,转换为均匀缩放使图像到达目标尺寸。实验结果表明,文章中提出的方法更好地平衡了均匀缩放和非均匀剪裁,更有利于保留图像主要内容和避免主要内容的变形。      

Saliency detection in the compressed domain for adaptive image retargeting

Saliency detection plays important roles in many image processing applications, such as regions of interest extraction and image resizing. Existing saliency detection models are built in the uncompressed domain. Since most images over Internet are typically stored in the compressed domain such as joint photographic experts group (JPEG), we propose a novel saliency detection model in the compressed domain in this paper. The intensity, color, and texture features of the image are extracted from discrete cosine transform (DCT) coefficients in the JPEG bit-stream. Saliency value of each DCT block is obtained based on the Hausdorff distance calculation and feature map fusion. Based on the proposed saliency detection model, we further design an adaptive image retargeting algorithm in the compressed domain. The proposed image retargeting algorithm utilizes multioperator operation comprised of the block-based seam carving and the image scaling to resize images. A new definition of texture homogeneity is given to determine the amount of removal block-based seams. Thanks to the directly derived accurate saliency information from the compressed domain, the proposed image retargeting algorithm effectively preserves the visually important regions for images, efficiently removes the less crucial regions, and therefore significantly outperforms the relevant state-of-the-art algorithms, as demonstrated with the in-depth analysis in the extensive experiments.     

基于最小位移可视差的连续Seam Carving算法在图像缩放中的研究

图像缩放技术要求对图像缩放的同时保证重要信息不丢失且物体边缘不发生扭曲.近年来,SeamCarving及其改进算法得到了广泛的关注和研究.由于采用了离散式最小能量线迭代搜索策略,缩放信息无法在迭代过程中传递导致扭曲现象普遍存在.该文针对上述问题提出最小位移可视差(JND)检测算法,能够有效地检测每一次迭代中出现的潜在扭...     

基于最小位移可视差的连续Seam Carving算法在图像缩放中的研究

图像缩放技术要求对图像缩放的同时保证重要信息不丢失且物体边缘不发生扭曲。近年来,Seam Carving及其改进算法得到了广泛的关注和研究。由于采用了离散式最小能量线迭代搜索策略,缩放信息无法在迭代过程中传递导致扭曲现象普遍存在。该文针对上述问题提出最小位移可视差(JND)检测算法,能够有效地检测每一次迭代中出现的潜在扭曲信息。能量权重\begin{document}$ {E}_{w} $\end{document}能够将JND信息累加传递给后续的迭代过程,从而抑制缩放过程中的边缘扭曲现象。通过JND算法和能量权重,该文首次将离散的Seam Carving模型转变为连续缩放模型。最后,在公共数据集RetargetMe上与最新的图像缩放算法进行多组对比实验,验证了所提方法的有效性和先进性。     

基于深层特征学习的可压缩感知及缝雕刻的图像重定向

多媒体技术的飞速发展推动了图像处理与显示设备 的应用与发展,为了使图像在不同的设备上进行最佳显示,需要对图像的尺寸进行调整。因 此,本文提出一种基于深层特征学习的可压缩感知及接缝雕刻的图像重定向方法。首先从预 先训练的VGG-19网络中提取输入图像的深度特征图,从最深层开始 计算特征图像的可压缩率,根据计算的可压缩率运用接缝雕刻的方法在特征域(Feature fie lds Seam Carving,FSC)调整特征图的大小,然后依次向较浅的层传播,得到所有特征层的 重定向图像后,将输入图像对应于第一层特征图的去缝的位置处的像素去掉,得到原始图像 的重定向图像。若没有达到目标图像的大小,最后再进行均匀缩放(scaling,SCL)。在Retar getMe数据集上分别进行主观与客观评估,结果表明,与其他方法相比,本文的重定向方法 总体上实现了更好的性能。     

Detecting image seam carving with low scaling ratio using multi-scale spatial and spectral entropies

Seam carving is the most popular content-aware image retargeting technique. However, it may also be used to correct poor photo composition in photography competition or to remove object from image for malicious purpose. A blind detection approach is presented for seam carved image with low scaling ratio (LSR). It exploits spatial and spectral entropies (SSE) on multi-scale images (candidate image and its down-sampled versions). We observe that when a few seams are deleted from an original image, its SSE distribution is greatly changed. Forty-two features are designed to unveil the statistical properties of SSE in terms of centralized tendency, dispersion tendency and distribution tendency. They are combined with the local binary pattern (LBP)-based energy features to form ninety-six features. Finally, support vector machine (SVM) is exploited as classifier to determine whether an image is original or suffered from seam carving. Experimental results show that the proposed approach achieves superior detection accuracy over the state-of-the-art works, especially for resized image by seam carving with LSRs. Moreover, it is robust against JPEG compression and seam insertion.     

Quality assessment of retargeted images by salient region deformity analysis

Displaying images on different devices, requires resizing of the media. Traditional image resizing methods result in quality degradation. Content-aware retargeting algorithms aim to resize images for displaying them on a new device with the goal of preserving important contents of the image. Quality assessment of retargeted images can be employed to choose among outputs of different retargeting methods or help the optimization of such methods. In this paper we propose a learning based quality assessment method for retargeted images. An optical flow algorithm is used to find the correspondence between regions in the scaled and retargeted images. Three groups of features are defined to cover different aspects of distortions that are important to human observers. Area related features are used to detect how the areas of salient regions are retained and how much geometrical deformities are produced in the image. Also, to better assess the retargeted image we introduce features to show how well the aspect ratios of objects are retained. More importantly, we introduce the concept of measuring the homogeneity of distribution of deformities throughout the image. Experimental results demonstrate that our quality estimation method has better correlation with subjective scores and outperforms existing methods.     

A Distortion-Sensitive Seam Carving Algorithm for Content-Aware Image Resizing

As displays become less expensive and are incorporated into more and more devices, there has been an increased focus on image resizing techniques to fill an image to an arbitrary screen size. Traditional methods such as cropping or resampling can introduce undesirable losses in information or distortion in perception. Recently, content-aware image retargeting methods have been proposed (Avidan and Shamir, ACM Trans Graphics 26(3), 2007; Guo et al., IEEE Trans Multimedia 11(5):856–867, 2009; Shamir and Avidan, Commun ACM 52(1), 2009; Simakov et al. 2008; Wolf et al. 2007), which produce exceptional results. In particular, seam carving, proposed by Avidan and Shamir, has gained attention as an effective solution. However, there are many cases where it can fail. In this paper we propose a distortion-sensitive seam carving algorithm for content-aware image resizing that improves edge preservation and decreases aliasing artifacts. In the proposed approach, we use local gradient information along with a thresholding technique to guide the seam selection process and provide a mechanism to halt seam carving when further processing would introduce unacceptable visual distortion in the resized image. Furthermore, anti-aliasing filter is used to reduce the aliasing artifacts caused by seam removal. Experiments have demonstrated superior performance over the current seam carving methods.     

基于梯度矢量方向性分析的线裁剪算法

在传统线裁剪(seam carving)算法中梯度矢量的方向性是一个被忽略的因素,该文提出了一种新的基于梯度矢量方向性分析的线裁剪算法。首先利用随机纹理区域内梯度矢量方向散乱的特点,对局部区域的图像梯度矢量进行低通滤波,使算法能够提取更加合理的像素线路;接着又根据像素线路的走向不同,定义两个不同的像素能量函数,给予像素梯度矢量的x, y分量以不同的权值。实验结果图像显示,文中算法不仅可以更好地保护图像边缘等细节,还可以在整体上达到与原图像更加近似的视觉效果。定量分析结果也显示,相比其它算法该文算法在完整性距离和一致性距离两方面都取得了更好结果。     

Image reproduction with compensation of luminance adaptation

We introduce an image reproduction model that retargets colors for printing purposes to ensure similar luminance perception under photopic and scotopic vision. Our model is based on the physiological functioning of the rod and cone cells in the retina in varying lighting conditions, so that the human visual system exhibits responses akin to a printed output of the model for different illumination levels. Prior to retargeting, digital color images are converted to spectral representations and their photopic and scotopic luminance responses are obtained. The color retargeting is realized by optimizing our compensation function over the color space. In addition, we present a spatially varying operator to enhance the color coherence over salient regions. Reproduction results demonstrate substantially decreased difference between the two luminance responses. Further, it is validated through psychophysical evaluation that our model on average provides superior recognition rates in dark environments, while keeping the noticeable differences in aesthetic appeal acceptable in well-lit environments.     

Using hidden scale for salient object detection.

This paper describes a method for detecting salient regions in remote-sensed images, based on scale and contrast interaction. We consider the focus on salient structures as the first stage of an object detection/recognition algorithm, where the salient regions are those likely to contain objects of interest. Salient objects are modeled as spatially localized and contrasted structures with any kind of shape or size. Their detection exploits a probabilistic mixture model that takes two series of multiscale features as input, one that is more sensitive to contrast information, and one that is able to select scale. The model combines them to classify each pixel in salient/nonsalient class, giving a binary segmentation of the image. The few parameters are learned with an EM-type algorithm.     

视觉注意驱动的基于混沌分析的运动检测方法

提出了视觉注意驱动的基于混沌分析的运动检测方法(MDSA)。MDSA首先基于视觉注意机制提取图像的显著区域,而后对显著区域进行混沌分析以检测运动目标。算法技术路线为:首先根据场景图像提取多种视觉敏感的底层图像特征;然后根据特征综合理论将这些特征融合起来得到一幅反映场景图像中各个位置视觉显著性的显著图;而后对显著性水平最高的图像位置所在的显著区域运用混沌分析的方法进行运动检测;根据邻近优先和返回抑制原则提取下一最显著区域并进行运动检测,直至遍历所有的显著区域。本文对传统的显著区域提取方法进行了改进以减少计算量:以邻域标准差代替center-surround算子评估图像各位置的局部显著度,采用显著点聚类的方法代替尺度显著性准则提取显著区域;混沌分析首先判断各显著区域的联合直方图（JH）是否呈现混沌特征,而后依据分维数以一固定阈值对存在混沌的JH中各散点进行分类,最后将分类结果对应到显著区域从而实现运动分割。MDSA具有较好的运动分割效果和抗噪性能,对比实验和算法开销分析证明MDSA优于基于马塞克的运动检测方法（MDM）。      

Seam carving based aesthetics enhancement for photos

Recently people are becoming more and more interested in the quality of photographs with the growing interest of image aesthetics. Many previous works start to focus on aesthetically enhancing the quality of images. In this paper, we come up with a novel approach to enhance image aesthetics. An aesthetically beautiful image usually has a clever composition of objects, the optimal positions of which have been deeply discussed by previous methods and reached good performance. After getting the optimal position of the object in images, we try to rearrange all the objects. Instead of picking the object out and pasting it on the suggested place, we propose an improved seam carving approach to change the relative positions of the objects in the image, which is able to move the object to a better place. We adopt the energy function to measure the saliency of each pixel and then find out the seams that should be cut off and inserted. After cutting off unimportant seams by pixel-removing and inserting seams by inpainting, we are able to maintain the resolution of the image as well as enhance the aesthetics in composition. In order to test the effectiveness of our method, we compare the performance of our approach with other state-of-the-art techniques, which well illustrates the satisfying performance of our method.     

Image denoising by supervised adaptive fusion of decomposed images restored using wave atom,curvelet and wavelet transform

This paper presents an efficient image denoising method that adaptively combines the features of wavelets, wave atoms and curvelets. Wavelet shrinkage is used to denoise the smooth regions in the image while wave atoms are employed to denoise the textures, and the edges will take advantage of curvelet denoising. The received noisy image is firstly decomposed into a homogenous (smooth/cartoon) part and a textural part. The cartoon part of the noisy image is denoised using wavelet transform, and the texture part of the noisy image is denoised using wave atoms. The two denoised images are then fused adaptively. For adaptive fusion, different weights are chosen from the variance map of the denoised texture image. Further improvement in denoising results is achieved by denoising the edges through curvelet transform. The information about edge location is gathered from the variance map of denoised cartoon image. The denoised image results in perfect presentation of the smooth regions and efficient preservation of textures and edges in the image.     

A Hybrid Method for Multi-sensor Remote Sensing Image Registration Based on Salience Region

In order to align the remote sensing images, we propose a novel hybrid method that combines image segmentation and salient region detection, which is inspired by human vision system. First of all, we present a novel superpixel-based method for dividing the image into sub-areas. Second, we propose a novel method based on color and image textures for detecting salient regions composed by superpixels. Then, we extract a new feature based on difference of Gaussian and local binary pattern from the salient regions. Finally, the sensed image is transformed by thin-plate spline. The proposed algorithm was tested on 30 pairs of remote sensing images and compared to other three state of the art methods. Experimental results show our approach is fast and robust, while still being efficient, which is better than other three methods.     

Image information and visual quality.

Measurement of visual quality is of fundamental importance to numerous image and video processing applications. The goal of quality assessment (QA) research is to design algorithms that can automatically assess the quality of images or videos in a perceptually consistent manner. Image QA algorithms generally interpret image quality as fidelity or similarity with a "reference" or "perfect" image in some perceptual space. Such "full-reference" QA methods attempt to achieve consistency in quality prediction by modeling salient physiological and psychovisual features of the human visual system (HVS), or by signal fidelity measures. In this paper, we approach the image QA problem as an information fidelity problem. Specifically, we propose to quantify the loss of image information to the distortion process and explore the relationship between image information and visual quality. QA systems are invariably involved with judging the visual quality of "natural" images and videos that are meant for "human consumption." Researchers have developed sophisticated models to capture the statistics of such natural signals. Using these models, we previously presented an information fidelity criterion for image QA that related image quality with the amount of information shared between a reference and a distorted image. In this paper, we propose an image information measure that quantifies the information that is present in the reference image and how much of this reference information can be extracted from the distorted image. Combining these two quantities, we propose a visual information fidelity measure for image QA. We validate the performance of our algorithm with an extensive subjective study involving 779 images and show that our method outperforms recent state-of-the-art image QA algorithms by a sizeable margin in our simulations. The code and the data from the subjective study are available at the LIVE website.     

A cartoon-plus-texture image decomposition model for blind deconvolution

In this paper, we study a blind deconvolution problem by using an image decomposition technique. Our idea is to make use of a cartoon-plus-texture image decomposition procedure into the deconvolution problem. Because cartoon and texture components can be represented differently in images, we can adapt suitable regularization methods to restore their components. In particular, the total variational regularization is used to describe the cartoon component, and Meyer’s G-norm is employed to model the texture component. In order to obtain the restored image automatically, we also use the generalized cross validation method efficiently and effectively to estimate their corresponding regularization parameters. Experimental results are reported to demonstrate that the visual quality of restored images by using the proposed method is very good, and is competitive with the other testing methods.