High-level background prior based salient object detection

Salient object detection is a fundamental problem in computer vision. Existing methods using only low-level features failed to uniformly highlight the salient object regions. In order to combine high-level saliency priors and low-level appearance cues, we propose a novel Background Prior based Salient detection method (BPS) for high-quality salient object detection.Different from other background prior based methods, a background estimation is added before performing saliency detection. We utilize the distribution of bounding boxes generated by a generic object proposal method to obtain background information. Three background priors are mainly considered to model the saliency, namely background connectivity prior, background contrast prior and spatial distribution prior, allowing the proposed method to highlight the salient object as a whole and suppress background clutters.Experiments conducted on two benchmark datasets validate that our method outperforms 11 state-of-the-art methods, while being more efficient than most leading methods.     

基于深度学习的图像显著对象检测

显著区域检测可应用在对象识别、图像分割、视 频/图像压缩中,是计算机视觉领域的重要研究主题。然而,基于不 同视觉显著特征的显著区域检测法常常不能准确地探测出显著对象且计算费时。近来,卷积 神经网络模型在图像分析和处理 领域取得了极大成功。为提高图像显著区域检测性能,本文提出了一种基于监督式生成对抗 网络的图像显著性检测方法。它 利用深度卷积神经网络构建监督式生成对抗网络,经生成器网络与鉴别器网络的不断相互对 抗训练,使卷积网络准确学习到 图像显著区域的特征,进而使生成器输出精确的显著对象分布图。同时,本文将网络自身误 差和生成器输出与真值图间的 L1距离相结合,来定义监督式生成对抗网络的损失函数,提升了显著区域检测精度。在MSRA 10K与ECSSD数据库上的实 验结果表明,本文方法 分别获得了94.19%与96.24%的准确率和93.99%与90.13%的召回率,F -Measure值也高达94.15%与94.76%,优于先 前常用的显著性检测模型。     

Global and local information aggregation network for edge-aware salient object detection

Aggregation of local and global contextual information by exploiting multi-level features in a fully convolutional network is a challenge for the pixel-wise salient object detection task. Most existing methods still suffer from inaccurate salient regions and blurry boundaries. In this paper, we propose a novel edge-aware global and local information aggregation network (GLNet) to fully exploit the integration of side-output local features and global contextual information and utilization of contour information of salient objects. The global guidance module (GGM) is proposed to learn discriminative multi-level information with the direct guidance of global semantic knowledge for more accurate saliency prediction. Specifically, the GGM consists of two key components, where the global feature discrimination module exploits the inter-channel relationship of global semantic features to boost representation power, and the local feature discrimination module enables different side-output local features to selectively learn informative locations by fusing with global attentive features. Besides, we propose an edge-aware aggregation module (EAM) to employ the correlation between salient edge information and salient object information for generating estimated saliency maps with explicit boundaries. We evaluate our proposed GLNet on six widely-used saliency detection benchmark datasets by comparing with 17 state-of-the-art methods. Experimental results show the effectiveness and superiority of our proposed method on all the six benchmark datasets.     

GCENet: Global contextual exploration network for RGB-D salient object detection

Representing contextual features at multiple scales is important for RGB-D SOD. Recently, due to advances in backbone convolutional neural networks (CNNs) revealing stronger multi-scale representation ability, many methods achieved comprising performance. However, most of them represent multi-scale features in a layer-wise manner, which ignores the fine-grained global contextual cues in a single layer. In this paper, we propose a novel global contextual exploration network (GCENet) to explore the performance gain of multi-scale contextual features in a fine-grained manner. Concretely, a cross-modal contextual feature module (CCFM) is proposed to represent the multi-scale contextual features at a single fine-grained level, which can enlarge the range of receptive fields for each network layer. Furthermore, we design a multi-scale feature decoder (MFD) that integrates fused features from CCFM in a top-down way. Extensive experiments on five benchmark datasets demonstrate that the proposed GCENet outperforms the other state-of-the-art (SOTA) RGB-D SOD methods.     

Depth cue enhancement and guidance network for RGB-D salient object detection

Depth maps have been proven profitable to provide supplements for salient object detection in recent years. However, most RGB-D salient object detection approaches ignore that there are usually low-quality depth maps, which will inevitably result in unsatisfactory results. In this paper, we propose a depth cue enhancement and guidance network (DEGNet) for RGB-D salient object detection by exploring the depth quality enhancement and utilizing the depth cue guidance to generate predictions with highlighted objects and suppressed backgrounds. Specifically, a depth cue enhancement module is designed to generate high-quality depth maps by enhancing the contrast between the foreground and the background. Then considering the different characteristics of unimodal RGB and depth features, we use different feature enhancement strategies to strengthen the representation capability of side-output unimodal features. Moreover, we propose a depth-guided feature fusion module to excavate depth cues provided by the depth stream to guide the fusion of multi-modal features by fully making use of different modal properties, thus generating discriminative cross-modal features. Besides, we aggregate cross-modal features at different levels to obtain the final prediction by adopting a pyramid feature shrinking structure. Experimental results on six benchmark datasets demonstrate that the proposed network DEGNet outperforms 17 state-of-the-art methods.     

AFLNet: Adversarial focal loss network for RGB-D salient object detection

Because salient objects usually have fewer data in a scene, the problem of class imbalance is often encountered in salient object detection (SOD). In order to address this issue and achieve the consistent salient objects, we propose an adversarial focal loss network with improving generative adversarial networks for RGB-D SOD (called AFLNet), in which color and depth branches constitute the generator to achieve the saliency map, and adversarial branch with high-order potentials, instead of pixel-wise loss function, refines the output of the generator to obtain contextual information of objects. We infer the adversarial focal loss function to solve the problem of foreground–background class imbalance. To sufficiently fuse the high-level features of color and depth cues, an inception model is adopted in deep layers. We conduct a large number of experiments using our proposed model and its variants, and compare them with state-of-the-art methods. Quantitative and qualitative experimental results exhibit that our proposed approach can improve the accuracy of salient object detection and achieve the consistent objects.     

Bayesian salient object detection based on saliency driven clustering

Salient object detection is essential for applications, such as image classification, object recognition and image retrieval. In this paper, we design a new approach to detect salient objects from an image by describing what does salient objects and backgrounds look like using statistic of the image. First, we introduce a saliency driven clustering method to reveal distinct visual patterns of images by generating image clusters. The Gaussian Mixture Model (GMM) is applied to represent the statistic of each cluster, which is used to compute the color spatial distribution. Second, three kinds of regional saliency measures, i.e, regional color contrast saliency, regional boundary prior saliency and regional color spatial distribution, are computed and combined. Then, a region selection strategy integrating color contrast prior, boundary prior and visual patterns information of images is presented. The pixels of an image are divided into either potential salient region or background region adaptively based on the combined regional saliency measures. Finally, a Bayesian framework is employed to compute the saliency value for each pixel taking the regional saliency values as priority. Our approach has been extensively evaluated on two popular image databases. Experimental results show that our approach can achieve considerable performance improvement in terms of commonly adopted performance measures in salient object detection.     

Global guidance-based integration network for salient object detection in low-light images

Most of current salient object detection (SOD) methods focus on well-lit scenes, and their performance drops when generalized into low-light scenes due to limitations such as blurred boundaries and low contrast. To solve this problem, we propose a global guidance-based integration network (G2INet) customized for low-light SOD. First, we propose a Global Information Flow (GIF) to extract comprehensive global information, for guiding the fusion of multi-level features. To facilitate information integration, we design a Multi-level features Cross Integration (MCI) module, which progressively fuses low-level details, high-level semantics, and global information by interweaving. Furthermore, a U-shaped Attention Refinement (UAR) module is proposed to further refine edges and details for accurate saliency predictions. In terms of five metrics, extensive experimental results demonstrate that our method outperforms the existing twelve state-of-the-art models.     

基于区域特征融合的RGBD显著目标检测

为了对各类自然场景中的显著目标进行检测,本文提出了一种将图像的深度信息引入区域显著性计算的方法,用于目标检测。首先对图像进行多尺度分割得到若干区域,然后对区域多类特征学习构建回归随机森林,采用监督学习的方法赋予每个区域特征显著值,最后采用最小二乘法对多尺度的显著值融合,得到最终的显著图。实验结果表明,本文算法能较准确地定位RGBD图像库中每幅图的显著目标。     

SRI-Net: Similarity retrieval-based inference network for light field salient object detection

The cutting-edge RGB saliency models are prone to fail for some complex scenes, while RGB-D saliency models are often affected by inaccurate depth maps. Fortunately, light field images can provide a sufficient spatial layout depiction of 3D scenes. Therefore, this paper focuses on salient object detection of light field images, where a Similarity Retrieval-based Inference Network (SRI-Net) is proposed. Due to various focus points, not all focal slices extracted from light field images are beneficial for salient object detection, thus, the key point of our model lies in that we attempt to select the most valuable focal slice, which can contribute more complementary information for the RGB image. Specifically, firstly, we design a focal slice retrieval module (FSRM) to choose an appropriate focal slice by measuring the foreground similarity between the focal slice and RGB image. Secondly, in order to combine the original RGB image and the selected focal slice, we design a U-shaped saliency inference module (SIM), where the two-stream encoder is used to extract multi-level features, and the decoder is employed to aggregate multi-level deep features. Extensive experiments are conducted on two widely used light field datasets, and the results firmly demonstrate the superiority and effectiveness of the proposed SRI-Net.     

A salient object detection framework using linear quadratic regulator controller

In this paper, a novel salient object detection framework based on Linear Quadratic Regulator (LQR) controller is proposed. The major goal of this research is to take advantage of optimal control theory for improving the performance of detecting salient objects in images. In this regard, for the sake of detection of salient and non-salient regions, two LQR-based control systems are employed. In the proposed framework, for the initialization of the control systems, background and foreground estimations have been done with two different strategies. Doing so, we would ultimately have more effective distinction between those regions. After the initialization step, the control systems refine both estimations in parallel until reaching a steady state for each of them. Within the mentioned process, by using optimal control concept, specifically LQR controller (for the first time in the field), control signals which are in charge of determining saliency values, would be constantly optimized. At the end, the raw saliency map will be generated by combination of background and foreground optimized initial maps. Finally, the integrated saliency map will be refined by using angular embedding method. The experimental evaluations on three benchmark datasets shows that the proposed framework performs well and introduces comparable results with some deep learning based methods.     

Superpixel based color contrast and color distribution driven salient object detection

Color is the most informative low-level feature and might convey tremendous saliency information of a given image. Unfortunately, color feature is seldom fully exploited in the previous saliency models. Motivated by the three basic disciplines of a salient object which are respectively center distribution prior, high color contrast to surroundings and compact color distribution, in this paper, we design a comprehensive salient object detection system which takes the advantages of color contrast together with color distribution and outputs high quality saliency maps. The overall procedure flow of our unified framework contains superpixel pre-segmentation, color contrast and color distribution computation, combination, and final refinement.In color contrast saliency computation, we calculate center-surrounded color contrast and then employ the distribution prior in order to select correct color components. A global saliency smoothing procedure that is based on superpixel regions is introduced as well. This processing step preferably alleviates the saliency distortion problem, leading to the entire object being highlighted uniformly. Finally, a saliency refinement approach is adopted to eliminate artifacts and recover unconnected parts within the combined saliency maps.In visual comparison, our method produces higher quality saliency maps which stress out the total object meanwhile suppress background clutter. Both qualitative and quantitative experiments show our approach outperforms 8 state-of-the-art methods, achieving the highest precision rate 96% (3% improvement from the current highest), when evaluated via one of the most popular data sets. Excellent content-aware image resizing also could be achieved using our saliency maps.     

基于多尺度信息处理和Gabor初始化的图像显著性对象检测

基于全卷积网络的图像显著性检测获得了广泛的关 注,并取得了令人瞩目的检测性能 。然而,该类型神经网络依然存在许多问题,如高复杂网络导致难以训练、显著性对象边缘 结果不准确等。针对这些问题,本文提出基于Gabor初始化的卷积神经网络。该网络主要特 点包括:1) 利用Gabor特征初始化卷积神经网络,提高神经网络训练效率; 2) 构建多尺 度 桥接模块,有效衔接编码和解码阶段,进而提高显著性检测结果; 3) 提出加权交叉熵损失 函数,提高训练效果。实验结果表明,本文提出的神经网络在三个不同的数据集上均显示出 优异的显著性对象检测性能。     

Video object segmentation and its salient motion detection using adaptive background generation

Video object segmentation often fails when the background and foreground contain a similar distribution of colours. Proposed is a novel image segmentation algorithm to detect salient motion under a complex environment by combining temporal difference and background generation. Experimental results show that the proposed algorithm provides a twice higher matching ratio than the conventional Gaussian mixture-based approaches under various conditions.     

基于多尺度整体嵌套池化语义的装甲目标检测

针对现有目标检测网络很难适应复杂战场环境下的超视距、遮挡、多视角变化等干扰的影响,提出了一种基于多金字塔池化模型的整体嵌套卷积网络,该网络通过引入空洞卷积思想,在保证卷积特征分辨率不变的基础上提高弱小目标的检测精度;同时,本文所提的模型也将通过多孔空间金字塔池化将多尺度上下文特征进行融合,然后在整体嵌套卷积基础上利用装...     

Global salient information maximization for saliency detection

In this paper, a new method for saliency detection is proposed. Based on the defined features of the salient object, we solve the problem of saliency detection from three aspects. Firstly, from the view of global information, we partition the image into two clusters, namely, salient component and background component by employing Principal Component Analysis (PCA) and k-means clustering. Secondly, the maximal salient information is applied to find the position of saliency and eliminate the noise. Thirdly, we enhance the saliency for the salient regions while weaken the background regions. Finally, the saliency map is obtained based on these aspects. Experimental results show that the proposed method achieves better results than the state of the art methods. And this method can be applied for graph based salient object segmentation.     

Extracting salient region for pornographic image detection

Content-based pornographic image detection, in which region-of-interest (ROI) plays an important role, is effective to filter pornography. Traditionally, skin-color regions are extracted as ROI. However, skin-color regions are always larger than the subareas containing pornographic parts, and the approach is difficult to differentiate between human skins and other objects with the skin-colors. In this paper, a novel approach of extracting salient region is presented for pornographic image detection. At first, a novel saliency map model is constructed. Then it is integrated with a skin-color model and a face detection model to capture ROI in pornographic images. Next, a ROI-based codebook algorithm is proposed to enhance the representative power of visual-words. Taking into account both the speed and the accuracy, we fuse speed up robust features (SURF) with color moments (CM). Experimental results show that the precision of our ROI extraction method averagely achieves 91.33%, more precisely than that of using the skin-color model alone. Besides, the comparison with the state-of-the-art methods of pornographic image detection shows that our approach is able to remarkably improve the performance.     

Spatiotemporal saliency detection and salient region determination for H.264 videos

In this study, a spatiotemporal saliency detection and salient region determination approach for H.264 videos is proposed. After Gaussian filtering in Lab color space, the phase spectrum of Fourier transform is used to generate the spatial saliency map of each video frame. On the other hand, the motion vector fields from each H.264 compressed video bitstream are backward accumulated. After normalization and global motion compensation, the phase spectrum of Fourier transform for the moving parts is used to generate the temporal saliency map of each video frame. Then, the spatial and temporal saliency maps of each video frame are combined to obtain its spatiotemporal saliency map using adaptive fusion. Finally, a modified salient region determination scheme is used to determine salient regions (SRs) of each video frame. Based on the experimental results obtained in this study, the performance of the proposed approach is better than those of two comparison approaches.     

Infinite hidden Markov models for unusual-event detection in video.

We address the problem of unusual-event detection in a video sequence. Invariant subspace analysis (ISA) is used to extract features from the video, and the time-evolving properties of these features are modeled via an infinite hidden Markov model (iHMM), which is trained using "normal"/"typical" video. The iHMM retains a full posterior density function on all model parameters, including the number of underlying HMM states. Anomalies (unusual events) are detected subsequently if a low likelihood is observed when associated sequential features are submitted to the trained iHMM. A hierarchical Dirichlet process framework is employed in the formulation of the iHMM. The evaluation of posterior distributions for the iHMM is achieved in two ways: via Markov chain Monte Carlo and using a variational Bayes formulation. Comparisons are made to modeling based on conventional maximum-likelihood-based HMMs, as well as to Dirichlet-process-based Gaussian-mixture models.     

Unsupervised image segmentation based on analysis of binary partition tree for salient object extraction

This paper proposes an unsupervised image segmentation approach aimed at salient object extraction. Starting from an over-segmentation result of a color image, region merging is performed using a novel dissimilarity measure considering the impact of color difference, area factor and adjacency degree, and a binary partition tree (BPT) is generated to record the whole merging sequence. Then based on a systematic analysis of the evaluated BPT, an appropriate subset of nodes is selected from the BPT to represent a meaningful segmentation result with a small number of segmented regions. Experimental results demonstrate that the proposed approach can obtain a better segmentation performance from the perspective of salient object extraction.