Exploiting multigranular salient features with hierarchical multi-mode attention network for pedestrian re-IDentification

In this paper, we propose an end-to-end hierarchical-based multi-mode attention network and adaptive fusion (HMAN-HAF) strategy to learn different-level salient features for re-ID tasks. First, according to each layer’s characteristics, a hierarchical multi-mode attention network (HMAN) is designed to adopt different attention models for different-level salient feature learning. Specifically, refined channel-wise attention (CA) is adopted to capture high-level valuable semantic information, an attentive region model (AR) is used to detect salient regions in the low layer, and fused attention (FA) is designed to capture the salient regions of valuable channels in the middle layer. Second, a hierarchical adaptive fusion (HAF) is constructed to fulfill the complementary strengths of different-level salient features. Experimental results demonstrate that the proposed method outperforms the state-of-the-art methods on the following challenging benchmarks: Market-1501, DukeMTMC-reID and CUHK03.     

视觉注意驱动的基于混沌分析的运动检测方法

提出了视觉注意驱动的基于混沌分析的运动检测方法(MDSA)。MDSA首先基于视觉注意机制提取图像的显著区域,而后对显著区域进行混沌分析以检测运动目标。算法技术路线为:首先根据场景图像提取多种视觉敏感的底层图像特征;然后根据特征综合理论将这些特征融合起来得到一幅反映场景图像中各个位置视觉显著性的显著图;而后对显著性水平最高的图像位置所在的显著区域运用混沌分析的方法进行运动检测;根据邻近优先和返回抑制原则提取下一最显著区域并进行运动检测,直至遍历所有的显著区域。本文对传统的显著区域提取方法进行了改进以减少计算量:以邻域标准差代替center-surround算子评估图像各位置的局部显著度,采用显著点聚类的方法代替尺度显著性准则提取显著区域;混沌分析首先判断各显著区域的联合直方图（JH）是否呈现混沌特征,而后依据分维数以一固定阈值对存在混沌的JH中各散点进行分类,最后将分类结果对应到显著区域从而实现运动分割。MDSA具有较好的运动分割效果和抗噪性能,对比实验和算法开销分析证明MDSA优于基于马塞克的运动检测方法（MDM）。      

Global and local information aggregation network for edge-aware salient object detection

Aggregation of local and global contextual information by exploiting multi-level features in a fully convolutional network is a challenge for the pixel-wise salient object detection task. Most existing methods still suffer from inaccurate salient regions and blurry boundaries. In this paper, we propose a novel edge-aware global and local information aggregation network (GLNet) to fully exploit the integration of side-output local features and global contextual information and utilization of contour information of salient objects. The global guidance module (GGM) is proposed to learn discriminative multi-level information with the direct guidance of global semantic knowledge for more accurate saliency prediction. Specifically, the GGM consists of two key components, where the global feature discrimination module exploits the inter-channel relationship of global semantic features to boost representation power, and the local feature discrimination module enables different side-output local features to selectively learn informative locations by fusing with global attentive features. Besides, we propose an edge-aware aggregation module (EAM) to employ the correlation between salient edge information and salient object information for generating estimated saliency maps with explicit boundaries. We evaluate our proposed GLNet on six widely-used saliency detection benchmark datasets by comparing with 17 state-of-the-art methods. Experimental results show the effectiveness and superiority of our proposed method on all the six benchmark datasets.     

一种新的自适应图像增强融合算法

提出了一种新的自适应图像增强融合算法,算法将输入图像分解为反映图像平均照度特征的低频分量和反映图像对比度特征的高频分量图像,基于局部灰度方差增强因子和非线性投影变换因子进行自适应图像增强处理,基于局部能量、局部归一化互相关融合测度和加权融合算子对增强后的图像进行特征提取和重构,得到融合图像.有效地将输入图像细节特征传递组合到融合图像中.     

A heuristic framework for perceptual saliency prediction

Saliency prediction can be regarded as the human spontaneous activity. The most effective saliency model should highly approximate the response of viewers to the perceived information. In the paper, we exploit the perception response for saliency detection and propose a heuristic framework to predict salient region. First, to find the perceptually meaningful salient regions, an orientation selectivity based local feature and a visual Acuity based global feature are proposed to jointly predict candidate salient regions. Subsequently, to further boost the accuracy of saliency map, we introduce a visual error sensitivity based operator to activate the meaningful salient regions from a local and global perspective. In addition, an adaptive fusion method based on free energy principle is designed to combine the sub-saliency maps from each image channel to obtain the final saliency map. Experimental results on five natural and emotional datasets demonstrate the superiority of the proposed method compared to twelve state-of-the-art algorithms.     

基于多尺度池化和范数注意力机制的遥感图像检索

遥感图像内容丰富,一般的深度模型提取遥感图像特征时容易受复杂背景干扰,对关键特征的提取效果不佳,并且难以表达图像的空间信息,该文提出一种基于多尺度池化和范数注意力机制的深度卷积神经网络,在通道层面与空间层面自适应地给显著特征加权.首先,在多尺度池化通道注意力模块中,结合空间金字塔池化的思想,对每个通道上的特征图进行不同...     

A new deep spatial transformer convolutional neural network for image saliency detection

In this paper we propose a novel deep spatial transformer convolutional neural network (Spatial Net) framework for the detection of salient and abnormal areas in images. The proposed method is general and has three main parts: (1) context information in the image is captured by using convolutional neural networks (CNN) to automatically learn high-level features; (2) to better adapt the CNN model to the saliency task, we redesign the feature sub-network structure to output a 6-dimensional transformation matrix for affine transformation based on the spatial transformer network. Several local features are extracted, which can effectively capture edge pixels in the salient area, meanwhile embedded into the above model to reduce the impact of highlighting background regions; (3) finally, areas of interest are detected by means of the linear combination of global and local feature information. Experimental results demonstrate that Spatial Nets obtain superior detection performance over state-of-the-art algorithms on two popular datasets, requiring less memory and computation to achieve high performance.     

Salient object detection via boosting object-level distinctiveness and saliency refinement

Many salient object detection approaches share the common drawback that they cannot uniformly highlight heterogeneous regions of salient objects, and thus, parts of the salient objects are not discriminated from background regions in a saliency map. In this paper, we focus on this drawback and accordingly propose a novel algorithm that more uniformly highlights the entire salient object as compared to many approaches. Our method consists of two stages: boosting the object-level distinctiveness and saliency refinement. In the first stage, a coarse object-level saliency map is generated based on boosting the distinctiveness of the object proposals in the test images, using a set of object-level features and the Modest AdaBoost algorithm. In the second stage, several saliency refinement steps are executed to obtain a final saliency map in which the boundaries of salient objects are preserved. Quantitative and qualitative comparisons with state-of-the-art approaches demonstrate the superior performance of our approach.     

基于阶梯型特征空间分割与局部注意力机制的行人重识别

为了让网络捕捉到更有效的内容来进行行人的判别,该文提出一种基于阶梯型特征空间分割与局部分支注意力网络(SLANet)机制的多分支网络来关注局部图像的显著信息。首先,在网络中引入阶梯型分支注意力模块,该模块以阶梯型对特征图进行水平分块,并且使用了分支注意力给每个分支分配不同的权重。其次,在网络中引入多尺度自适应注意力模块,该模块对局部特征进行处理,自适应调整感受野尺寸来适应不同尺度图像,同时融合了通道注意力和空间注意力筛选出图像重要特征。在网络的设计上,使用多粒度网络将全局特征和局部特征进行结合。最后,该方法在3个被广泛使用的行人重识别数据集Market-1501,DukeMTMC-reID和CUHK03上进行验证。其中在Market-1501数据集上的mAP和Rank-1分别达到了88.1%和95.6%。实验结果表明,该文所提出的网络模型能够提高行人重识别准确率。     

Global-local feature attention network with reranking strategy for image caption generation

In this paper, a novel framework, named as global-local feature attention network with reranking strategy (GLAN-RS), is presented for image captioning task. Rather than only adopting unitary visual information in the classical models, GLAN-RS explores the attention mechanism to capture local convolutional salient image maps. Furthermore, we adopt reranking strategy to adjust the priority of the candidate captions and select the best one. The proposed model is verified using the Microsoft Common Objects in Context (MSCOCO) benchmark dataset across seven standard evaluation metrics. Experimental results show that GLAN-RS significantly outperforms the state-of-the-art approaches, such as multimodal recurrent neural network (MRNN) and Google NIC, which gets an improvement of 20% in terms of BLEU4 score and 13 points in terms of CIDER score.     

基于双模态特征增强的目标检测算法研究与应用北大核心CSCD

为提升目标检测算法在复杂环境下的精确性和实用性,将多源信息和深度学习技术相结合,提出了一种基于双模态特征增强的目标检测方法。该方法以红外和可见光图像作为输入,利用颜色空间转换、边缘提取、直方图均衡化等传统图像处理方法丰富图像信息,达到数据增强效果;特征提取部分采用卷积神经网络结构分别提取目标红外及可见光信息,并设计混合注意力机制分别从通道和空间位置角度提升有效特征权重;同时,针对目标双模态信息,引入了自适应交叉融合结构,提高特征多样性;最后,利用交替上下采样将目标全局和局部特征充分融合,并以自主选择方式提取目标相关特征实现检测。通过在标准数据集以及实际场景数据集上的实验结果表明,所提方法有效融合并增强了目标多模态特征,提升了目标检测效果,并能较好的应用于电网场景中,辅助机器人完成目标设备检测。     

Hot spot detection based on feature space representation of visual search

This paper presents a new framework for capturing intrinsic visual search behavior of different observers in image understanding by analysing saccadic eye movements in feature space. The method is based on the information theory for identifying salient image features based on which visual search is performed. We demonstrate how to obtain feature space fixation density functions that are normalized to the image content along the scan paths. This allows a reliable identification of salient image features that can be mapped back to spatial space for highlighting regions of interest and attention selection. A two-color conjunction search experiment has been implemented to illustrate the theoretical framework of the proposed method including feature selection, hot spot detection, and back-projection. The practical value of the method is demonstrated with computed tomography image of centrilobular emphysema, and we discuss how the proposed framework can be used as a basis for decision support in medical image understanding.     

先验引导的特征金字塔阴影检测网络

阴影检测向来是计算机视觉领域的一个基础性挑战。它需要网络理解图像的全局语义和局部细节信息。本文提出了一种检测阴影区域的先验特征金字塔网络结构。该网络搭建了先验加权模块来提取图像中蕴含的阴影先验信息,通过使用阴影先验信息加权卷积特征,引导网络学习到阴影区域。同时,该网络还应用了特征融合模块来融合粗略的语义信息和自上而下路径中的精细特征,并且加入了后处理,进一步优化网络的预测结果。本文在两个公开的阴影检测基准数据集上进行了实验来评估其网络性能。实验表明,本文的方法能够更准确地检测到阴影,和过去最先进的方法相比也表现出色,在SBU数据集上正确率达到了96.6%,平衡检测错误因子为6.22。      

可见光与红外图像增强融合算法研究

提出了自适应图像增强算法，用于可见光和红外图像的融合。首先对输入的可见光图像和红外图像进行自适应增强，然后采用基于图像空间能量窗及归一化互相关测度构造融合图像，利用图像的信息熵评估算法的融合效果，最后给出了一组可见光和红外图像融合的试验结果，表明该算法十分有效，融合图像有丰富的互补信息，有利于人眼观察和目标识别。     

基于双重注意力机制的遥感图像场景分类特征表示方法

针对遥感图像场景分类面临的类内差异性大、类间相似性高导致的部分场景出现分类混淆的问题,该文提出了一种基于双重注意力机制的强鉴别性特征表示方法。针对不同通道所代表特征的重要性程度以及不同局部区域的显著性程度不同,在卷积神经网络提取的高层特征基础上,分别设计了一个通道维和空间维注意力模块,利用循环神经网络的上下文信息提取能力,依次学习、输出不同通道和不同局部区域的重要性权重,更加关注图像中的显著性特征和显著性区域,而忽略非显著性特征和区域,以提高特征表示的鉴别能力。所提双重注意力模块可以与任意卷积神经网络相连,整个网络结构可以端到端训练。通过在两个公开数据集AID和NWPU45上进行大量的对比实验,验证了所提方法的有效性,与现有方法对比,分类准确率取得了明显的提升。     

Using hidden scale for salient object detection.

This paper describes a method for detecting salient regions in remote-sensed images, based on scale and contrast interaction. We consider the focus on salient structures as the first stage of an object detection/recognition algorithm, where the salient regions are those likely to contain objects of interest. Salient objects are modeled as spatially localized and contrasted structures with any kind of shape or size. Their detection exploits a probabilistic mixture model that takes two series of multiscale features as input, one that is more sensitive to contrast information, and one that is able to select scale. The model combines them to classify each pixel in salient/nonsalient class, giving a binary segmentation of the image. The few parameters are learned with an EM-type algorithm.     

基于边缘检测的显著区域提取方法

多年以来,众多专家学者建立了诸多模型来模拟人的视觉选择性注意机制,其中最具影响力的当属Itti模型,但其存在着显著区域漏检测及显著区域范围是固定形状的问题。文章基于人眼对物体的轮廓形状信息的感知能力,提出了一种改进型的显著区域提取方法,在原Itti模型基础上加入轮廓特征。本方法与原Itti模型相比较,能够改善其显著性区域的提取效果并且能够较准确的实现显著区域的分割。     

A hybrid algorithm for automatic segmentation of slowly moving objects

Segmentation of moving objects in video sequences is a basic task in many applications. However, it is still challenging due to the semantic gap between the low-level visual features and the high-level human interpretation of video semantics. Compared with segmentation of fast moving objects, accurate and perceptually consistent segmentation of slowly moving objects is more difficult. In this paper, a novel hybrid algorithm is proposed for segmentation of slowly moving objects in video sequence aiming to acquire perceptually consistent results. Firstly, the temporal information of the differences among multiple frames is employed to detect initial moving regions. Then, the Gaussian mixture model (GMM) is employed and an improved expectation maximization (EM) algorithm is introduced to segment a spatial image into homogeneous regions. Finally, the results of motion detection and spatial segmentation are fused to extract final moving objects. Experiments are conducted and provide convincing results.