Saliency-directed color image interpolation using artificial neural network and particle swarm optimization

In this study, a saliency-directed color image interpolation approach using artificial neural network (ANN) and particle swarm optimization (PSO) is proposed. First, a high-quality saliency map of a color image to be interpolated is generated by a modified block-based visual attention model in an effective manner. Then, based on the saliency map, bilinear interpolation and ANN-PSO interpolation are employed for non-saliency (non-ROI) and saliency (ROI) blocks, respectively, to obtain the final color interpolation results. In the proposed ANN-PSO interpolation scheme, ANN is used to determine the orientation of each 5 × 5 image pattern (block), whereas PSO is employed to determine the weights in 5 × 5 interpolation filtering masks. The proposed approach is applicable to image interpolation with arbitrary magnification factors (MFs). Based on the experimental results obtained in this study, the color interpolation results by the proposed approach are better than those by five comparison approaches.     

Saliency detection using joint spatial-color constraint and multi-scale segmentation

In this paper, a novel method is proposed to detect salient regions in images. To measure pixel-level saliency, joint spatial-color constraint is defined, i.e., spatial constraint (SC), color double-opponent (CD) constraint and similarity distribution (SD) constraint. The SC constraint is designed to produce global contrast with ability to distinguish the difference between “center and surround”. The CD constraint is introduced to extract intensive contrast of red-green and blue-yellow double opponency. The SD constraint is developed to detect the salient object and its background. A two-layer structure is adopted to merge the SC, CD and SD saliency into a saliency map. In order to obtain a consistent saliency map, the region-based saliency detection is performed by incorporating a multi-scale segmentation technique. The proposed method is evaluated on two image datasets. Experimental results show that the proposed method outperforms the state-of-the-art methods on salient region detection as well as human fixation prediction.     

基于双引导滤波的红外和可见光图像融合算法

图像融合是将多幅图像中有用或互补信息整合成一幅图像的过程。本文提出了一种基于引导滤波多尺度分解的红外和可见光图像融合算法。在传统的引导滤波图像融合算法的基础之上,利用双引导滤波器代替均值滤波器将源图像分解为小尺度纹理细节、大尺度边缘和基础图像;直接利用纹理细节及边缘层图像构建显著性映射图,用其代替额外的特征提取操作,可很好地突出源图像显著性信息的同时大大降低算法复杂度;利用显著性映射图、Sigmoid函数构造权重图,将源图像中具有视觉意义的信息注入到融合图像中;利用色彩模型转换融合方式,可更好保留图像的色彩信息。定性和定量实验结果证明,相比于传统的基于引导滤波的图像融合算法,本文算法的融合效果得到进一步提升。     

基于多特征扩散方法的显著性物体检测

现有的大部分基于扩散理论的显著性物体检测方法只用了图像的底层特征来构造图和扩散矩阵,并且忽视了显著性物体在图像边缘的可能性。针对此,该文提出一种基于图像的多层特征的扩散方法进行显著性物体检测。首先,采用由背景先验、颜色先验、位置先验组成的高层先验方法选取种子节点。其次,将选取的种子节点的显著性信息通过由图像的底层特征构建的扩散矩阵传播到每个节点得到初始显著图,并将其作为图像的中层特征。然后结合图像的高层特征分别构建扩散矩阵,再次运用扩散方法分别获得中层显著图、高层显著图。最后,非线性融合中层显著图和高层显著图得到最终显著图。该算法在3个数据集MSRA10K,DUT-OMRON和ECSSD上,用3种量化评价指标与现有4种流行算法进行实验结果对比,均取得最好的效果。     

融合相位一致性与二维主成分分析的视觉显著性预测

为了更加有效地预测图像中吸引视觉注意的关键区域,该文提出一种融合相位一致性与2维主成分分析(2DPCA)的显著性方法。该方法不同于传统的利用相位谱的方式,而是提出采用相位一致性(PC)获取图像中重要的特征点和边缘信息,经快速漂移超像素优化后,融合局部和全局颜色对比度,生成低层特征显著图。接着提出利用2DPCA提取图像块的主成分后,计算主成分空间中图像块的局部和全局可区分性,得到模式显著图。最后,通过空间离散度度量分配合适的权重,使两者融合,提取显著性区域。在两种人眼跟踪数据库上与5种经典算法的实验对比结果表明,该算法能更加准确地预测人眼视觉关注点。     

Patchwise scaling method for content-aware image resizing

Image resizing becomes more and more important in content-aware image displaying. This paper proposes a patchwise scaling method to resize an image to emphasize the important areas and preserve the globally visual effect (smoothness, coherence and integrity). This method for resizing image is based on optimizing the image distance presented in this paper. The image distance is defined based on so-called local bidirectional similarity measurement and smoothness measurement to quantify the quality of resizing outputs. The original image is divided into small important patches and unimportant patches based on an important map. The important map is generated automatically using a novel combination of image edge and saliency measurement. A scaling factor is computed for each small patch. The resized image is produced by iteratively optimizing, which is based on our image distance, the scaling factor for each small patch. Experiments of different type images demonstrate that our method can be effectively used in image processing applications to locally shrink and enlarge important areas while preserving image quality.     

融合颜色特征和对比度特征的图像显著性检测

图像显著性检测能够获取一幅图像的视觉显著性区域,是计算机视觉的研究热点之一。提出一种结合颜色特征和对比度特征的图像显著性检测方法。首先构造图像在HSV空间的颜色函数以获取图像颜色特征;然后使用SLIC超像素分割算法对图像进行预处理,基于超像素块的对比度特征计算图像显著性;最后将融合颜色特征和对比度特征的显著图经过导向滤波优化形成最终的显著图。使用本文算法在公开数据集MSRA-1000上进行图像显著性检测,并与其他6种算法进行比较。实验结果表明本文算法结合了图像像素点和像素块的信息,检测的图像显著性区域轮廓更加完整,优于其他方法。     

Stretchability-aware block scaling for image retargeting

This paper proposes an efficient approach to retarget images based on stretchability-aware block scaling. The image stretchability is first evaluated based on gradient, saliency and color features, and is used to generate the stretchable space. Then the optimal size of the stretched image is determined under the constraint of stretchable space and the same aspect ratio as the target image. Based on the analysis of image stretchability measures, the original image is partitioned into non-stretchable blocks and stretchable blocks, and their scaling factors are calculated based on their stretchability measures and the stretched image size, in order to possibly preserve non-stretchable blocks without distortion and reasonably resize stretchable blocks. Finally, the stretched image is uniformly scaled to generate the target image. Experimental results on a variety of images and the user study demonstrate that our approach achieves an overall better retargeting performance compared to the state-of-the-art image retargeting approaches.     

A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency

With the emerging development of three-dimensional (3D) related technologies, 3D visual saliency modeling is becoming particularly important and challenging. This paper presents a new depth perception and visual comfort guided saliency computational model for stereoscopic 3D images. The prominent advantage of the proposed model is that we incorporate the influence of depth perception and visual comfort on 3D visual saliency computation. The proposed saliency model is composed of three components: 2D image saliency, depth saliency and visual comfort based saliency. In the model, color saliency, texture saliency and spatial compactness are computed respectively and fused to derive 2D image saliency. Global disparity contrast is considered to compute depth saliency. Particularly, we train a visual comfort prediction function to distinguish stereoscopic image pair as high comfortable stereo viewing (HCSV) or low comfortable stereo viewing (LCSV), and devise different computational rules to generate a visual comfort based saliency map. The final 3D saliency map is obtained by using a linear combination and enhanced by a “saliency-center bias” model. Experimental results show that the proposed 3D saliency model outperforms the state-of-the-art models on predicting human eye fixations and visual comfort assessment.     

局部和全局特征融合的红外图像显著性检测

为了更好地凸显复杂环境的红外目标特征,提出 一种融合局部和全局特征的红外图像 显著性检测方法。在获取图像超像素的基础上,提取每个区域空间距离加权的邻域对比度特 征,并考虑区域大小和位置的影响,构建局部显著图;然后提取每个区域空间距离加权的全 局灰度特征,构建全局显著图;最后融合局部和全局显著图,实现图像显著性检测。实验结 果 表明,本文方法的显著图结果目标区域一致高亮且边缘清晰,同时背景杂波抑制效果好。无 论 主观评价还是客观指标,本文方法都优于当前流行的图像显著性检测方法。     

Residential area extraction based on saliency analysis for high spatial resolution remote sensing images

Traditional residential area extraction methods for remote sensing image depend on classification, segmentation and prior knowledge which are time-consuming and difficult to build. In this paper, an efficient, saliency analysis-based residential area extraction method is proposed. In the proposed model, an adaptive directional prediction-based lifting wavelet transform (ADP-LWT) is introduced to obtain the orientation feature. A logarithm co-occurrence histogram is employed to compute the intensity feature. The color opponency and diagram objection based on the information are proposed to extract color feature from the contrast in the red–green opponent channel. The saliency map is obtained through a weighted combination based on the feature competition and the residential area is extracted by saliency map threshold segmentation. The experimental results reveal that the residential area extracted by our model has more demarcated boundaries and better performance in background subtraction.     

图像自适应中基于形变控制的细缝剪裁方法

为了降低图像自适应过程中图像内容缺失和图像扭曲变形,文章中提出了一种基于形变控制的图像自适应细缝裁剪方法,将融合局部和全局显著性的显著图作为细缝裁剪参考的能量图,对需要剪裁的细缝按照这个能量图进行排序,以更好地保护图像中感兴趣区域（ROI）,保留图像的主要内容信息;同时,利用SIFT流矢量场来衡量剪裁图像的形变程度,每移除一定数量的细缝就计算剪裁图像与原始图像之间的形变,一旦形变达到某一阈值,就停止细缝裁剪,转换为均匀缩放使图像到达目标尺寸。实验结果表明,文章中提出的方法更好地平衡了均匀缩放和非均匀剪裁,更有利于保留图像主要内容和避免主要内容的变形。      

Saliency model-based face segmentation and tracking in head-and-shoulder video sequences

In this paper, a novel face segmentation algorithm is proposed based on facial saliency map (FSM) for head-and-shoulder type video application. This method consists of three stages. The first stage is to generate the saliency map of input video image by our proposed facial attention model. In the second stage, a geometric model and an eye-map built from chrominance components are employed to localize the face region according to the saliency map. The third stage involves the adaptive boundary correction and the final face contour extraction. Based on the segmented result, an effective boundary saliency map (BSM) is then constructed, and applied for the tracking based segmentation of the successive frames. Experimental evaluation on test sequences shows that the proposed method is capable of segmenting the face area quite effectively.     

Generative image inpainting via edge structure and color aware fusion

Very recently, with the widespread research of deep learning, its achievements are increasingly evident in image inpainting tasks. However, many existing methods fail to effectively reconstruct vivid contents and refine structures. In order to solve this issue, in this paper, a novel two-stage generative adversarial network based on the fusion of edge structures and color aware maps is proposed. In the first-stage network, edges with missing regions are employed to train an edge structure generator. Meanwhile, the input image with missing regions is transformed into a global color feature map after the content aware fill algorithm and a large kernel size Gaussian filtering. In the second-stage network, the image fused from the edge map and the color map is used as a label to guide the network to reconstruct the refined image. Qualitative and quantitative experiments conducted on multiple public datasets demonstrate that the method proposed in this paper has superior performance.     

Saliency detection in the compressed domain for adaptive image retargeting

Saliency detection plays important roles in many image processing applications, such as regions of interest extraction and image resizing. Existing saliency detection models are built in the uncompressed domain. Since most images over Internet are typically stored in the compressed domain such as joint photographic experts group (JPEG), we propose a novel saliency detection model in the compressed domain in this paper. The intensity, color, and texture features of the image are extracted from discrete cosine transform (DCT) coefficients in the JPEG bit-stream. Saliency value of each DCT block is obtained based on the Hausdorff distance calculation and feature map fusion. Based on the proposed saliency detection model, we further design an adaptive image retargeting algorithm in the compressed domain. The proposed image retargeting algorithm utilizes multioperator operation comprised of the block-based seam carving and the image scaling to resize images. A new definition of texture homogeneity is given to determine the amount of removal block-based seams. Thanks to the directly derived accurate saliency information from the compressed domain, the proposed image retargeting algorithm effectively preserves the visually important regions for images, efficiently removes the less crucial regions, and therefore significantly outperforms the relevant state-of-the-art algorithms, as demonstrated with the in-depth analysis in the extensive experiments.     

Combining multi-layer integration algorithm with background prior and label propagation for saliency detection

In this paper, we propose a novel approach to automatically detect salient regions in an image. Firstly, some corner superpixels serve as the background labels and the saliency of other superpixels are determined by ranking their similarities to the background labels based on ranking algorithm. Subsequently, we further employ an objectness measure to pick out and propagate foreground labels. Furthermore, an integration algorithm is devised to fuse both background-based saliency map and foreground-based saliency map, meanwhile an original energy function is acted as refinement before integration. Finally, results from multiscale saliency maps are integrated to further improve the detection performance. Our experimental results on five benchmark datasets demonstrate the effectiveness of the proposed method. Our method produces more accurate saliency maps with better precision-recall curve, higher F-measure and lower mean absolute error than other 13 state-of-the-arts approaches on ASD, SED, ECSSD, iCoSeg and PASCAL-S datasets.     

3D saliency detection based on background detection

Unlike 2D saliency detection, 3D saliency detection can consider the effects of depth and binocular parallax. In this paper, we propose a 3D saliency detection approach based on background detection via depth information. With the aid of the synergism between a color image and the corresponding depth map, our approach can detect the distant background and surfaces with gradual changes in depth. We then use the detected background to predict the potential characteristics of the background regions that are occluded by foreground objects through polynomial fitting; this step imitates the human imagination/envisioning process. Finally, a saliency map is obtained based on the contrast between the foreground objects and the potential background. We compare our approach with 14 state-of-the-art saliency detection methods on three publicly available databases. The proposed model demonstrates good performance and succeeds in detecting and removing backgrounds and surfaces of gradually varying depth on all tested databases.     

结合改进显著性检测与NSST的红外与可见光图像融合方法

针对当前基于显著性检测的红外与可见光图像融合方法存在目标不够突出、对比度低等问题,本文提出了一种结合改进显著性检测与非下采样剪切波变换（non-subsampled shearlet transform, NSST）的融合方法。首先,使用改进最大对称环绕（maximum symmetric surround, MSS）算法提取出红外图像的显著性图,并进一步通过改进伽马校正进行增强,同时应用同态滤波增强可见光图像。然后,对红外图像与增强的可见光图像进行NSST分解,利用显著性图指导低频部分进行融合;同时设定区域能量取大规则指导高频部分融合。最后,通过NSST逆变换重构融合图像。实验结果表明,本文方法在平均梯度、信息熵、空间频率和标准差上远优于其他7种融合方法,可以有效突出红外目标,提高融合图像的对比度和清晰度,并保留可见光图像的丰富背景信息。