AFLNet: Adversarial focal loss network for RGB-D salient object detection

Because salient objects usually have fewer data in a scene, the problem of class imbalance is often encountered in salient object detection (SOD). In order to address this issue and achieve the consistent salient objects, we propose an adversarial focal loss network with improving generative adversarial networks for RGB-D SOD (called AFLNet), in which color and depth branches constitute the generator to achieve the saliency map, and adversarial branch with high-order potentials, instead of pixel-wise loss function, refines the output of the generator to obtain contextual information of objects. We infer the adversarial focal loss function to solve the problem of foreground–background class imbalance. To sufficiently fuse the high-level features of color and depth cues, an inception model is adopted in deep layers. We conduct a large number of experiments using our proposed model and its variants, and compare them with state-of-the-art methods. Quantitative and qualitative experimental results exhibit that our proposed approach can improve the accuracy of salient object detection and achieve the consistent objects.     

GCENet: Global contextual exploration network for RGB-D salient object detection

Representing contextual features at multiple scales is important for RGB-D SOD. Recently, due to advances in backbone convolutional neural networks (CNNs) revealing stronger multi-scale representation ability, many methods achieved comprising performance. However, most of them represent multi-scale features in a layer-wise manner, which ignores the fine-grained global contextual cues in a single layer. In this paper, we propose a novel global contextual exploration network (GCENet) to explore the performance gain of multi-scale contextual features in a fine-grained manner. Concretely, a cross-modal contextual feature module (CCFM) is proposed to represent the multi-scale contextual features at a single fine-grained level, which can enlarge the range of receptive fields for each network layer. Furthermore, we design a multi-scale feature decoder (MFD) that integrates fused features from CCFM in a top-down way. Extensive experiments on five benchmark datasets demonstrate that the proposed GCENet outperforms the other state-of-the-art (SOTA) RGB-D SOD methods.     

Global and local information aggregation network for edge-aware salient object detection

Aggregation of local and global contextual information by exploiting multi-level features in a fully convolutional network is a challenge for the pixel-wise salient object detection task. Most existing methods still suffer from inaccurate salient regions and blurry boundaries. In this paper, we propose a novel edge-aware global and local information aggregation network (GLNet) to fully exploit the integration of side-output local features and global contextual information and utilization of contour information of salient objects. The global guidance module (GGM) is proposed to learn discriminative multi-level information with the direct guidance of global semantic knowledge for more accurate saliency prediction. Specifically, the GGM consists of two key components, where the global feature discrimination module exploits the inter-channel relationship of global semantic features to boost representation power, and the local feature discrimination module enables different side-output local features to selectively learn informative locations by fusing with global attentive features. Besides, we propose an edge-aware aggregation module (EAM) to employ the correlation between salient edge information and salient object information for generating estimated saliency maps with explicit boundaries. We evaluate our proposed GLNet on six widely-used saliency detection benchmark datasets by comparing with 17 state-of-the-art methods. Experimental results show the effectiveness and superiority of our proposed method on all the six benchmark datasets.     

基于多尺度信息处理和Gabor初始化的图像显著性对象检测

基于全卷积网络的图像显著性检测获得了广泛的关 注,并取得了令人瞩目的检测性能 。然而,该类型神经网络依然存在许多问题,如高复杂网络导致难以训练、显著性对象边缘 结果不准确等。针对这些问题,本文提出基于Gabor初始化的卷积神经网络。该网络主要特 点包括:1) 利用Gabor特征初始化卷积神经网络,提高神经网络训练效率; 2) 构建多尺 度 桥接模块,有效衔接编码和解码阶段,进而提高显著性检测结果; 3) 提出加权交叉熵损失 函数,提高训练效果。实验结果表明,本文提出的神经网络在三个不同的数据集上均显示出 优异的显著性对象检测性能。     

CCFNet: Cross-Complementary fusion network for RGB-D scene parsing of clothing images

Schemes to complement context relationships by cross-scale feature fusion have appeared in many RGB-D scene parsing algorithms; however, most of these works conduct multi-scale information interaction after multi-modal feature fusion, which ignores the information loss of the two modes in the original coding. Therefore, a cross-complementary fusion network (CCFNet) is designed in this paper to calibrate the multi-modal information before feature fusion, so as to improve the feature quality of each mode and the information complementarity ability of RGB and the depth map. First, we divided the features into low, middle, and high levels, among which the low-level features contain the global details of the image and the main learning features include texture, edge, and other features. The middle layer features contain not only some global detail features but also some local semantic features. Additionally, the high-level features contain rich local semantic features. Then, the feature information lost in the coding process of low and middle level features is supplemented and extracted through the designed cross feature enhancement module, and the high-level features are extracted through the feature enhancement module. In addition, the cross-modal fusion module is designed to integrate multi-modal features of different levels. The experimental results verify that the proposed CCFNet achieves excellent performance on the RGB-D scene parsing dataset containing clothing images, and the generalization ability of the model is verified by the dataset NYU Depth V2.     

基于提取双选紧密特征的RGB-D显著性检测网络

针对现有算法对不同来源特征之间的交互选择关注度欠缺以及对跨模态特征提取不充分的问题,提出了一种基于提取双选紧密特征的RGB-D显著性检测网络。首先,为了筛选出能够同时增强RGB图像显著区域和深度图像显著区域的特征,引入双向选择模块(bi-directional selection module, BSM);为了解决跨模态特征提取不充分,导致算法计算冗余且精度低的问题,引入紧密提取模块(dense extraction module, DEM);最后,通过特征聚合模块(feature aggregation module, FAM)对密集特征进行级联融合,并将循环残差优化模块(recurrent residual refinement aggregation module, RAM)配合深度监督实现粗显著图的持续优化,最终得到精确的显著图。在4个广泛使用的数据集上进行的综合实验表明,本文提出的算法在4个关键指标方面优于7种现有方法。     

基于区域特征融合的RGBD显著目标检测

为了对各类自然场景中的显著目标进行检测,本文提出了一种将图像的深度信息引入区域显著性计算的方法,用于目标检测。首先对图像进行多尺度分割得到若干区域,然后对区域多类特征学习构建回归随机森林,采用监督学习的方法赋予每个区域特征显著值,最后采用最小二乘法对多尺度的显著值融合,得到最终的显著图。实验结果表明,本文算法能较准确地定位RGBD图像库中每幅图的显著目标。     

SRI-Net: Similarity retrieval-based inference network for light field salient object detection

The cutting-edge RGB saliency models are prone to fail for some complex scenes, while RGB-D saliency models are often affected by inaccurate depth maps. Fortunately, light field images can provide a sufficient spatial layout depiction of 3D scenes. Therefore, this paper focuses on salient object detection of light field images, where a Similarity Retrieval-based Inference Network (SRI-Net) is proposed. Due to various focus points, not all focal slices extracted from light field images are beneficial for salient object detection, thus, the key point of our model lies in that we attempt to select the most valuable focal slice, which can contribute more complementary information for the RGB image. Specifically, firstly, we design a focal slice retrieval module (FSRM) to choose an appropriate focal slice by measuring the foreground similarity between the focal slice and RGB image. Secondly, in order to combine the original RGB image and the selected focal slice, we design a U-shaped saliency inference module (SIM), where the two-stream encoder is used to extract multi-level features, and the decoder is employed to aggregate multi-level deep features. Extensive experiments are conducted on two widely used light field datasets, and the results firmly demonstrate the superiority and effectiveness of the proposed SRI-Net.     

基于优化的极限学习机和深度层次的RGB-D显著检测

目前,相当多的显著目标检测方法均聚焦于2D的图像上,而RGB-D图像所需要的显著检测方法与单纯的2D图像相去甚远,这就需要新的适用于RGB-D的显著检测方法。该文在经典的RGB显著检测方法,即极限学习机的应用的基础上,提出融合了特征提取、前景增强、深度层次检测等多种思路的新的RGB-D显著性检测方法。该文的方法是：第一,运用特征提取的方法,提取RGB图4个超像素尺度的4096维特征;第二,依据特征提取中产生的4个尺度的超像素数量,分别提取RGB图的RGB, LAB, LBP特征以及深度图的LBE特征;第三,根据LBE和暗通道特征两种特征求出粗显著图,并在4个尺度的遍历中不断强化前景、削弱背景;第四,根据粗显著图选取前景与背景种子,放入极限学习机中进行分类,得到第1阶段显著图;第五,运用深度层次检测、图割等方法对第1阶段显著图进行再次优化,得到第2阶段显著图,即最终显著图。     

Global guidance-based integration network for salient object detection in low-light images

Most of current salient object detection (SOD) methods focus on well-lit scenes, and their performance drops when generalized into low-light scenes due to limitations such as blurred boundaries and low contrast. To solve this problem, we propose a global guidance-based integration network (G2INet) customized for low-light SOD. First, we propose a Global Information Flow (GIF) to extract comprehensive global information, for guiding the fusion of multi-level features. To facilitate information integration, we design a Multi-level features Cross Integration (MCI) module, which progressively fuses low-level details, high-level semantics, and global information by interweaving. Furthermore, a U-shaped Attention Refinement (UAR) module is proposed to further refine edges and details for accurate saliency predictions. In terms of five metrics, extensive experimental results demonstrate that our method outperforms the existing twelve state-of-the-art models.     

特征增强和双线性特征向量融合的移动端工业货箱文本检测

在实际工业环境下,光线昏暗、文本不规整、设备有限等因素,使得文本检测成为一项具有挑战性的任务。针对此问题,设计了一种基于双线性操作的特征向量融合模块,并联合特征增强与半卷积组成轻量级文本检测网络RGFFD(ResNet18+GhostModule+特征金字塔增强模块(feature pyramid enhancement module,FPEM)+特征融合模块(feature fusion module,FFM)+可微分二值化(differenttiable binarization,DB))。其中,Ghost模块内嵌特征增强模块,提升特征提取能力,双线性特征向量融合模块融合多尺度信息,添加自适应阈值分割算法提高DB模块分割能力。在实际工厂环境下,采用嵌入式设备UP2 board对货箱编号进行文本检测,RGFFD检测速度达到6.5 f/s。同时在公共数据集ICDAR2015、Total-text上检测速度分别达到39.6 f/s和49.6 f/s,在自定义数据集上准确率达到88.9%,检测速度为30.7 f/s。     

High-level background prior based salient object detection

Salient object detection is a fundamental problem in computer vision. Existing methods using only low-level features failed to uniformly highlight the salient object regions. In order to combine high-level saliency priors and low-level appearance cues, we propose a novel Background Prior based Salient detection method (BPS) for high-quality salient object detection.Different from other background prior based methods, a background estimation is added before performing saliency detection. We utilize the distribution of bounding boxes generated by a generic object proposal method to obtain background information. Three background priors are mainly considered to model the saliency, namely background connectivity prior, background contrast prior and spatial distribution prior, allowing the proposed method to highlight the salient object as a whole and suppress background clutters.Experiments conducted on two benchmark datasets validate that our method outperforms 11 state-of-the-art methods, while being more efficient than most leading methods.     

SCPNet: Self-constrained parallelism network for keypoint-based lightweight object detection

Keypoint-based object detection achieves better performance without positioning calculations and extensive prediction. However, they have heavy backbone, and high-resolution is restored using upsampling that obtain unreliable features. We propose a self-constrained parallelism keypoint-based lightweight object detection network (SCPNet), which speeds inference, drops parameters, widens receptive fields, and makes prediction accurate. Specifically, the parallel multi-scale fusion module (PMFM) with parallel shuffle blocks (PSB) adopts parallel structure to obtain reliable features and reduce depth, adopts repeated multi-scale fusion to avoid too many parallel branches. The self-constrained detection module (SCDM) has a two-branch structure, with one branch predicting corners, and employing entad offset to match high-quality corner pairs, and the other branch predicting center keypoints. The distances between the paired corners’ geometric centers and the center keypoints are used for self-constrained detection. On MS-COCO 2017 and PASCAL VOC, SCPNet’s results are competitive with the state-of-the-art lightweight object detection. https://github.com/mengdie-wang/SCPNet.git.     

基于显著性检测的星载嵌入式跟踪系统

随着航天科技的不断发展,计算机视觉算法在卫星上的应用方兴未艾,为了实现更多的功能需求和应对可能的威胁,视觉目标跟踪作为其中基础但具有挑战性的任务更是至关重要。然而,目前已有的目标跟踪算法大多数算法只限于对图像序列进行跟踪。另一方面,受到硬件条件制约,很多优秀的算法因为复杂度较高很少被应用到星载嵌入式系统中。这些目标跟踪算法运行时,通常需要人为地给出目标的边界框。为了自动得到边界框,需要目标检测算法对输入图像进行运动目标检测。本文提出了一种基于显著性检测和相关滤波的单目标检测与跟踪一体化算法,并与嵌入式系统相结合,在搭载的TMS320C6678芯片上达到了2 048 pixel×2 048 pixel分辨率下24 fps的帧率。具体地,检测算法负责对图像进行预处理并获得边界框,然后目标跟踪算法给出目标在后续帧中的位置。为了验证算法在实际跟踪中的有效性,本研究搭建了一个由相机、DSP和云台组成的光学平台并进行了实验验证。在该系统中,DSP自动完成检测、跟踪、驱动云台和再检测任务,达到了很好的检测跟踪效果。     

融合多尺度信息和跨维特征引导的轻量行人检测

针对复杂道路场景下行人检测精度与速度难以提升的问题,提出一种融合多尺度信息和跨维特征引导的轻量级行人检测算法。首先以高性能检测器YOLOX为基础框架,构建多尺度轻量卷积并嵌入主干网络中,以获取多尺度特征信息。然后设计了一种端到端的轻量特征引导注意力模块,采用跨维通道加权的方式将空间信息与通道信息融合,引导模型关注行人的可视区域。最后为减少模型在轻量化过程中特征信息的损失,使用增大感受野的深度可分离卷积构建特征融合网络。实验结果表明,相比于其他主流检测算法,所提算法在KITTI数据集上达到了71.03%的检测精度和80 FPS的检测速度,在背景复杂、密集遮挡、尺度不一等场景中都具有较好的鲁棒性和实时性。     

FD-SSD: An improved SSD object detection algorithm based on feature fusion and dilated convolution

Objects that occupy a small portion of an image or a frame contain fewer pixels and contains less information. This makes small object detection a challenging task in computer vision. In this paper, an improved Single Shot multi-box Detector based on feature fusion and dilated convolution (FD-SSD) is proposed to solve the problem that small objects are difficult to detect. The proposed network uses VGG-16 as the backbone network, which mainly includes a multi-layer feature fusion module and a multi-branch residual dilated convolution module. In the multi-layer feature fusion module, the last two layers of the feature map are up-sampled, and then they are concatenated at the channel level with the shallow feature map to enhance the semantic information of the shallow feature map. In the multi-branch residual dilated convolution module, three dilated convolutions with different dilated ratios based on the residual network are combined to obtain the multi-scale context information of the feature without losing the original resolution of the feature map. In addition, deformable convolution is added to each detection layer to better adapt to the shape of small objects. The proposed FD-SSD achieved 79.1% mAP and 29.7% mAP on PASCAL VOC2007 dataset and MS COCO dataset respectively. Experimental results show that FD-SSD can effectively improve the utilization of multi-scale information of small objects, thus significantly improve the effect of the small object detection.     

Human object interaction detection based on feature optimization and key human-object enhancement

Aiming at the problem of unclear or missing human object interaction behavior objects in complex background, we propose a human object interaction detection algorithm based on feature optimization and key human-object enhancement. In order to solve the problem of missing human behavior objects, we propose Feature Optimized Faster Region Convolutional Neural Network (FOFR-CNN). FOFR-CNN is an object detection network optimized by multi-scale feature optimization algorithm, taking into account both image semantics and image structure. In order to reduce the interference of complex background, we propose a Key Human-Object Enhancement Network. The network uses an instance-based method to enhance the features of interactive objects. In order to enrich the interaction information, we use the graph convolutional network. Experimental results on HICO-DET, V-COCO and HOI-A datasets show that the proposed algorithm has significantly improved accuracy and multi-scale object detection ability compared with other human object interaction algorithms.     

Bayesian salient object detection based on saliency driven clustering

Salient object detection is essential for applications, such as image classification, object recognition and image retrieval. In this paper, we design a new approach to detect salient objects from an image by describing what does salient objects and backgrounds look like using statistic of the image. First, we introduce a saliency driven clustering method to reveal distinct visual patterns of images by generating image clusters. The Gaussian Mixture Model (GMM) is applied to represent the statistic of each cluster, which is used to compute the color spatial distribution. Second, three kinds of regional saliency measures, i.e, regional color contrast saliency, regional boundary prior saliency and regional color spatial distribution, are computed and combined. Then, a region selection strategy integrating color contrast prior, boundary prior and visual patterns information of images is presented. The pixels of an image are divided into either potential salient region or background region adaptively based on the combined regional saliency measures. Finally, a Bayesian framework is employed to compute the saliency value for each pixel taking the regional saliency values as priority. Our approach has been extensively evaluated on two popular image databases. Experimental results show that our approach can achieve considerable performance improvement in terms of commonly adopted performance measures in salient object detection.     

基于结构标签学习的显著性目标检测

提出了一种基于结构标签学习的显著性目标检测算法,将结构化学习方法应用到显著性目标检测中。首先从含有标记的图像中随机采集固定大小的矩形区域,并记录其结构标签;然后使用含结构标签的区域特征构建决策树集合;最后采用监督学习的方法对图像进行优化预测,得到最终的显著图。实验结果表明,本文方法能较准确地检测出图像库中图像的显著性区域,在数据库MSRA5000和BSD300的AUC值分别为0.891 8、0.705 2,说明本文方法具有较好的显著性检测效果。