Image saliency detection using Gabor texture cues

Image saliency analysis plays an important role in various applications such as object detection, image compression, and image retrieval. Traditional methods for saliency detection ignore texture cues. In this paper, we propose a novel method that combines color and texture cues to robustly detect image saliency. Superpixel segmentation and the mean-shift algorithm are adopted to segment an original image into small regions. Then, based on the responses of a Gabor filter, color and texture features are extracted to produce color and texture sub-saliency maps. Finally, the color and texture sub-saliency maps are combined in a nonlinear manner to obtain the final saliency map for detecting salient objects in the image. Experimental results show that the proposed method outperforms other state-of-the-art algorithms for images with complex textures.     

Capture and Synthesis of 3D Surface Texture

We present and compare five approaches for capturing, synthesising and relighting real 3D surface textures. Unlike 2D texture synthesis techniques they allow the captured textures to be relit using illumination conditions that differ from those of the original. We adapted a texture quilting method due to Efros and combined this with five different relighting representations, comprising: a set of three photometric images; surface gradient and albedo maps; polynomial texture maps; and two eigen based representations using 3 and 6 base images.We used twelve real textures to perform quantitative tests on the relighting methods in isolation. We developed a qualitative test for the assessment of the complete synthesis systems. Ten observers were asked to rank the images obtained from the five methods using five real textures. Statistical tests were applied to the rankings.The six-base-image eigen method produced the best quantitative relighting results and in particular was better able to cope with specular surfaces. However, in the qualitative tests there were no significant performance differences detected between it and the other two top performers. Our conclusion is therefore that the cheaper gradient and three-base-image eigen methods should be used in preference, especially where the surfaces are Lambertian or near Lambertian.     

Wavelet Flow: Optical Flow Guided Wavelet Facial Image Fusion

Estimating the correspondence between the images using optical flow is the key component for image fusion, however, computing optical flow between a pair of facial images including backgrounds is challenging due to large differences in illumination, texture, color and background in the images. To improve optical flow results for image fusion, we propose a novel flow estimation method, wavelet flow, which can handle both the face and background in the input images. The key idea is that instead of computing flow directly between the input image pair, we estimate the image flow by incorporating multi‐scale image transfer and optical flow guided wavelet fusion. Multi‐scale image transfer helps to preserve the background and lighting detail of input, while optical flow guided wavelet fusion produces a series of intermediate images for further fusion quality optimizing. Our approach can significantly improve the performance of the optical flow algorithm and provide more natural fusion results for both faces and backgrounds in the images. We evaluate our method on a variety of datasets to show its high outperformance.     

各向异性导向滤波的红外与可见光图像融合

目的 针对红外与可见光图像融合时易产生边缘细节信息丢失、融合结果有光晕伪影等问题，同时为充分获取多源图像的重要特征，将各向异性导向滤波和相位一致性结合，提出一种红外与可见光图像融合算法。方法 首先，采用各向异性导向滤波从源图像获得包含大尺度变化的基础图和包含小尺度细节的系列细节图；其次，利用相位一致性和高斯滤波计算显著图，进而通过对比像素显著性得到初始权重二值图，再利用各向异性导向滤波优化权重图，达到去除噪声和抑制光晕伪影；最后，通过图像重构得到融合结果。结果 从主客观两个方面，将所提方法与卷积神经网络（convolutional neural network，CNN）、双树复小波变换（dual-tree complex wavelet transform，DTCWT）、导向滤波（guided filtering，GFF）和各向异性扩散（anisotropic diffusion，ADF）等4种经典红外与可见光融合方法在TNO公开数据集上进行实验对比。主观分析上，所提算法结果在边缘细节、背景保存和目标完整度等方面均优于其他4种方法；客观分析上，选取互信息（mutual information，MI）、边缘信息保持度（degree of edge information，Q^AB/F）、熵（entropy，EN）和基于梯度的特征互信息（gradient based feature mutual information，FMI_gradient）等4种图像质量评价指数进行综合评价。相较于其他4种方法，本文算法的各项指标均有一定幅度的提高，MI平均值较GFF提高了21.67%，Q^AB/F平均值较CNN提高了20.21%，EN平均值较CNN提高了5.69%，FMI_gradient平均值较GFF提高了3.14%。结论 本文基于各向异性导向滤波融合算法可解决原始导向滤波存在的细节"光晕"问题，有效抑制融合结果中伪影的产生，同时具有尺度感知特性，能更好保留源图像的边缘细节信息和背景信息，提高了融合结果的准确性。     

一种保留特征的网格简化和压缩递进传输方法

针对数字博物馆中三维藏品网络传输及传输过程中藏品特征保留的需要,提出了一种保留拓扑及纹理特征的网格简化方法,在三角形折叠简化算法的基础之上,通过引入边界三角形和色异三角形等概念,对误差矩阵的计算和误差控制方法进行了改进,保留了原始模型的几何边界和纹理属性等特征信息;并结合递进网格和压缩编码,构造了基于八叉树编码的递进网格文件,从而实现了基于网络的三维模型递进传输系统.     

Selecting texture resolution using a task‐specific visibility metric

In real‐time rendering, the appearance of scenes is greatly affected by the quality and resolution of the textures used for image synthesis. At the same time, the size of textures determines the performance and the memory requirements of rendering. As a result, finding the optimal texture resolution is critical, but also a non‐trivial task since the visibility of texture imperfections depends on underlying geometry, illumination, interactions between several texture maps, and viewing positions. Ideally, we would like to automate the task with a visibility metric, which could predict the optimal texture resolution. To maximize the performance of such a metric, it should be trained on a given task. This, however, requires sufficient user data which is often difficult to obtain. To address this problem, we develop a procedure for training an image visibility metric for a specific task while reducing the effort required to collect new data. The procedure involves generating a large dataset using an existing visibility metric followed by refining that dataset with the help of an efficient perceptual experiment. Then, such a refined dataset is used to retune the metric. This way, we augment sparse perceptual data to a large number of per‐pixel annotated visibility maps which serve as the training data for application‐specific visibility metrics. While our approach is general and can be potentially applied for different image distortions, we demonstrate an application in a game‐engine where we optimize the resolution of various textures, such as albedo and normal maps.     

Adaptive saliency fusion based on quality assessment

A variety of saliency models based on different schemes and methods have been proposed in the recent years, and the performance of these models often vary with images and complement each other. Therefore it is a natural idea whether we can elevate saliency detection performance by fusing different saliency models. This paper proposes a novel and general framework to adaptively fuse saliency maps generated using various saliency models based on quality assessment of these saliency maps. Given an input image and its multiple saliency maps, the quality features based on the input image and saliency maps are extracted. Then, a quality assessment model, which is learned using the boosting algorithm with multiple kernels, indicates the quality score of each saliency map. Next, a linear summation method with power-law transformation is exploited to fuse these saliency maps adaptively according to their quality scores. Finally, a graph cut based refinement method is exploited to enhance the spatial coherence of saliency and generate the high-quality final saliency map. Experimental results on three public benchmark datasets with state-of-the-art saliency models demonstrate that our saliency fusion framework consistently outperforms all saliency models and other fusion methods, and effectively elevates saliency detection performance.     

Rendering deformable surface reflectance fields

Animation of photorealistic computer graphics models is an important goal for many applications. Image-based modeling has emerged as a promising approach to capture and visualize real-world objects. Animating image-based models, however, is still a largely unsolved problem. In this paper, we extend a popular image-based representation called surface reflectance field to animate and render deformable real-world objects under arbitrary illumination. Deforming the surface reflectance field is achieved by modifying the underlying impostor geometry. We augment the impostor by a local parameterization that allows the correct evaluation of acquired reflectance images, preserving the original light model on the deformed surface. We present a deferred shading scheme to handle the increased amount of data involved in shading the deformable surface reflectance field. We show animations of various objects that were acquired with 3D photography.     

Saliency detection based on self-adaptive multiple feature fusion for remote sensing images

ABSTRACT

Saliency detection has been revealed an effective and reliable approach to extract the region of interest (ROI) in remote sensing images. However, most existing saliency detection methods employing multiple saliency cues ignore the intrinsic relationship between different cues and do not distinguish the diverse contributions of different cues to the final saliency map. In this paper, we propose a novel self-adaptively multiple feature fusion model for saliency detection in remote sensing images to take advantage of this relationship to improve the accuracy of ROI extraction. First, we take multiple feature channels, namely colour, intensity, texture and global contrast into consideration to produce primary feature maps. Particularly, we design a novel method based on dual-tree complex wavelet transform for remote sensing images to generate texture feature pyramids. Then, we introduce a novel self-adaptive multiple feature fusion method based on low-rank matrix recovery, in which the significances of feature maps are ranked by the low rank constraint recovery, and subsequently multiple features’ contributions are allocated adaptively to produce the final saliency map. Experimental results demonstrate that our proposal outperforms the state-of-the-art methods.     

Estimating Facial Reflectance Properties Using Shape-from-Shading

In this paper we show how to estimate facial surface reflectance properties (a slice of the BRDF and the albedo) in conjunction with the facial shape from a single image. The key idea underpinning our approach is to iteratively interleave the two processes of estimating reflectance properties based on the current shape estimate and updating the shape estimate based on the current estimate of the reflectance function. For frontally illuminated faces, the reflectance properties can be described by a function of one variable which we estimate by fitting a curve to the scattered and noisy reflectance samples provided by the input image and estimated shape. For non-frontal illumination, we fit a smooth surface to the scattered 2D reflectance samples. We make use of a novel statistical face shape constraint which we term ‘model-based integrability’ which we use to regularise the shape estimation. We show that the method is capable of recovering accurate shape and reflectance information from single grayscale or colour images using both synthetic and real world imagery. We use the estimated reflectance measurements to render synthetic images of the face in varying poses. To synthesise images under novel illumination, we show how to fit a parametric model of reflectance to the estimated reflectance function.     

Surface Texture Using Photometric Stereo Data: Classification and Direction of Illumination Detection

This paper generalizes the recently proposed sinusoidal model used for modeling the variation of texture features under changes in illumination direction, so that it can handle surfaces which are very rough and of variable albedo. It deals with the problem of identifying the direction of illumination of a rough surface from a single image, using information from a photometric stereo set of images. In addition, it presents methodology for classifying the texture of a rough surface, using generalized normals that capture both shape and albedo information. It assumes that the surface is Lambertian and is presented to the camera in a fronto-parallel view.     

动静态联合滤波器在多聚焦图像融合中的应用

动静态联合滤波器具有良好的边缘平滑特性,对梯度反转和全局强度迁移等伪影具有很强的鲁棒性。为了保留源图像的结构信息,提出了基于动静态联合滤波器的多聚焦图像融合方法。首先采用动静态联合滤波器将源图像分解为结构分量和纹理分量,以视觉显著度加权法对结构分量进行融合,综合相位一致性和清晰度信息对纹理分量进行融合;将两分量叠加获得初始融合图像,并通过计算源图像与初始融合图像间的结构相似度作为决策矩阵,获得最终的融合图像。通过对比多组融合图像主、客观评价结果发现,该方法能有效保留边缘信息。     

融合深度模型和传统模型的显著性检测

目的 显著性检测是图像和视觉领域一个基础问题,传统模型对于显著性物体的边界保留较好,但是对显著性目标的自信度不够高,召回率低,而深度学习模型对于显著性物体的自信度高,但是其结果边界粗糙,准确率较低。针对这两种模型各自的优缺点,提出一种显著性模型以综合利用两种方法的优点并抑制各自的不足。方法 首先改进最新的密集卷积网络,训练了一个基于该网络的全卷积网络（FCN）显著性模型,同时选取一个现有的基于超像素的显著性回归模型,在得到两种模型的显著性结果图后,提出一种融合算法,融合两种方法的结果以得到最终优化结果,该算法通过显著性结果Hadamard积和像素间显著性值的一对一非线性映射,将FCN结果与传统模型的结果相融合。结果 实验在4个数据集上与最新的10种方法进行了比较,在HKU-IS数据集中,相比于性能第2的模型,F值提高了2.6%;在MSRA数据集中,相比于性能第2的模型,F值提高了2.2%,MAE降低了5.6%;在DUT-OMRON数据集中,相比于性能第2的模型,F值提高了5.6%,MAE降低了17.4%。同时也在MSRA数据集中进行了对比实验以验证融合算法的有效性,对比实验结果表明提出的融合算法改善了显著性检测的效果。结论 本文所提出的显著性模型,综合了传统模型和深度学习模型的优点,使显著性检测结果更加准确。     

Variational Reflectance Estimation from Multi-view Images

We tackle the problem of reflectance estimation from a set of multi-view images, assuming known geometry. The approach we put forward turns the input images into reflectance maps, through a robust variational method. The variational model comprises an image-driven fidelity term and a term which enforces consistency of the reflectance estimates with respect to each view. If illumination is fixed across the views, then reflectance estimation remains under-constrained: A regularization term, which ensures piecewise-smoothness of the reflectance, is thus used. Reflectance is parameterized in the image domain, rather than on the surface, which makes the numerical solution much easier, by resorting to an alternating majorization–minimization approach. Experiments on both synthetic and real-world datasets are carried out to validate the proposed strategy.     

Texture image retrieval using new rotated complex wavelet filters.

A new set of two-dimensional (2-D) rotated complex wavelet filters (RCWFs) are designed with complex wavelet filter coefficients, which gives texture information strongly oriented in six different directions (45 degrees apart from complex wavelet transform). The 2-D RCWFs are nonseparable and oriented, which improves characterization of oriented textures. Most texture image retrieval systems are still incapable of providing retrieval result with high retrieval accuracy and less computational complexity. To address this problem, we propose a novel approach for texture image retrieval by using a set of dual-tree rotated complex wavelet filter (DT-RCWF) and dual-tree-complex wavelet transform (DT-CWT) jointly, which obtains texture features in 12 different directions. The information provided by DT-RCWF complements the information generated by DT-CWT. Features are obtained by computing the energy and standard deviation on each subband of the decomposed image. To check the retrieval performance, texture database D1 of 1856 textures from Brodatz album and database D2 of 640 texture images from VisTex image database is created. Experimental results indicates that the proposed method improves retrieval rate from 69.61% to 77.75% on database D1, and from 64.83% to 82.81% on database D2, in comparing with traditional discrete wavelet transform based approach. The proposed method also retains comparable levels of computational complexity.     

Master-CAM: Multi-scale fusion guided by Master map for high-quality class activation maps

Class Activation Map (CAM) is one of the most popular approaches to visually explain the convolutional neural networks (CNNs). To obtain fine-grained saliency maps, some works fuse saliency signals of the same image at larger scales. However, existing methods based on multi-scale fusion cannot effectively remove the noise from larger-scale images. In this paper, we propose Master-CAM, which uses Master map to guide multi-scale fusion process to obtain a high-quality class activation map. Master-CAM utilizes the general localization ability of the Master map to reduce the noise of the maps. We call the one with the general localization ability among the saliency maps from the same image as Master map, which is the saliency map of the original-scale input in the multi-scale scenario. In addition, we also present a simple yet effective fusion strategy, Master-Fusion, which is derived from the fusion operation in Master-CAM. Master-Fusion strategy can be easily attached to some saliency methods to improve the performance of these methods. We show through qualitative and quantitative experiments that the proposed Master-CAM outperforms the state-of-the-art methods in different CNN frameworks and datasets.     

改进残差密集生成对抗网络的红外与可见光图像融合

基于深度学习的红外与可见光图像融合算法通常无法感知源图像显著性区域,导致融合结果没有突出红外与可见光图像各自的典型特征,无法达到理想的融合效果.针对上述问题,设计一种适用于红外与可见光图像融合任务的改进残差密集生成对抗网络结构.首先,使用改进残差密集模块作为基础网络分别构建生成器与判别器,并引入基于注意力机制的挤压激励网络来捕获通道维度下的显著特征,充分保留红外图像的热辐射信息和可见光图像的纹理细节信息;其次,使用相对平均判别器,分别衡量融合图像与红外图像、可见光图像之间的相对差异,并根据差异指导生成器保留缺少的源图像信息;最后,在TNO等多个图像融合数据集上进行实验,结果表明所提方法能够生成目标清晰、细节丰富的融合图像,相比基于残差网络的融合方法,边缘强度和平均梯度分别提升了64.56%和64.94%.     

A Randomized Approach for Patch-based Texture Synthesis using Wavelets

We present a wavelet‐based approach for selecting patches in patch‐based texture synthesis. We randomly select the first block that satisfies a minimum error criterion, computed from the wavelet coefficients (using 1D or 2D wavelets) for the overlapping region. We show that our wavelet‐based approach improves texture synthesis for samples where previous work fails, mainly textures with prominent aligned features. Also, it generates similar quality textures when compared against texture synthesis using feature maps with the advantage that our proposed method uses implicit edge information (since it is embedded in the wavelet coefficients) whereas feature maps rely explicitly on edge features. In previous work, the best patches are selected among all possible using a L2 norm on the RGB or grayscale pixel values of boundary zones. The L2 metric provides the raw pixel‐to‐pixel difference, disregarding relevant image structures — such as edges — that are relevant in the human visual system and therefore on synthesis of new textures.     

Building Illumination Coherent 3D Models of Large-Scale Outdoor Scenes

Systems for the creation of photorealistic models using range scans and digital photographs are becoming increasingly popular in a wide range of fields, from reverse engineering to cultural heritage preservation. These systems employ a range finder to acquire the geometry information and a digital camera to measure color detail. But bringing together a set of range scans and color images to produce an accurate and usable model is still an area of research with many unsolved problems. In this paper we address the problem of how to build illumination coherent integrated texture maps from images that were taken under different illumination conditions. To achieve this we present two different solutions. The first one is to align all the images to the same illumination, for which we have developed a technique that computes a relighting operator over the area of overlap of a pair of images that we then use to relight the entire image. Our proposed method can handle images with shadows and can effectively remove the shadows from the image, if required. The second technique uses the ratio of two images to factor out the diffuse reflectance of an image from its illumination. We do this without any light measuring device. By computing the actual reflectance we remove from the images any effects of the illumination, allowing us to create new renderings under novel illumination conditions.     

Intrinsic Decompositions for Image Editing

Intrinsic images are a mid‐level representation of an image that decompose the image into reflectance and illumination layers. The reflectance layer captures the color/texture of surfaces in the scene, while the illumination layer captures shading effects caused by interactions between scene illumination and surface geometry. Intrinsic images have a long history in computer vision and recently in computer graphics, and have been shown to be a useful representation for tasks ranging from scene understanding and reconstruction to image editing. In this report, we review and evaluate past work on this problem. Specifically, we discuss each work in terms of the priors they impose on the intrinsic image problem. We introduce a new synthetic ground‐truth dataset that we use to evaluate the validity of these priors and the performance of the methods. Finally, we evaluate the performance of the different methods in the context of image‐editing applications.