Fast Non-Local Attention network for light super-resolution

Although convolutional neural network-based methods have achieved significant performance improvement for Single Image Super-Resolution (SISR), their vast computational cost hinders real-world environment application. Thus, the interest in light networks for SISR is rising. Since existing SISR light models mainly focus on extracting fine local features using convolution operation, they have a limitation in that networks hardly capture global information. To capture the long-range dependency, Non-Local (NL) attention and Transformers have been explored in the SISR task. However, they are still suffering from a balancing problem between performance and computational cost. In this paper, we propose Fast Non-Local attention NETwork (FNLNET) for a super light SISR, which can capture the global representation. To acquire global information, we propose The Fast Non-Local Attention (FNLA) module that has low computational complexity while capturing global representation that reflects long-distance relationships between patches. Then, FNLA requires only 16 times lower computational cost than conventional NL networks while improving performance. In addition, we propose a powerful module called Global Self-Intension Mining (GSIM) that fuses the multi-information resources such as local, and global representation. Our FNLNET shows outstanding performance with fewer parameters and computational costs in the experiments on the benchmark datasets against state-of-the-art light SISR models.     

基于阶梯型特征空间分割与局部注意力机制的行人重识别

为了让网络捕捉到更有效的内容来进行行人的判别,该文提出一种基于阶梯型特征空间分割与局部分支注意力网络(SLANet)机制的多分支网络来关注局部图像的显著信息。首先,在网络中引入阶梯型分支注意力模块,该模块以阶梯型对特征图进行水平分块,并且使用了分支注意力给每个分支分配不同的权重。其次,在网络中引入多尺度自适应注意力模块,该模块对局部特征进行处理,自适应调整感受野尺寸来适应不同尺度图像,同时融合了通道注意力和空间注意力筛选出图像重要特征。在网络的设计上,使用多粒度网络将全局特征和局部特征进行结合。最后,该方法在3个被广泛使用的行人重识别数据集Market-1501,DukeMTMC-reID和CUHK03上进行验证。其中在Market-1501数据集上的mAP和Rank-1分别达到了88.1%和95.6%。实验结果表明,该文所提出的网络模型能够提高行人重识别准确率。     

融合局部语义与全局信息的人脸表情识别

人脸表情识别在人机交互等人工智能领域发挥着 重要作用,当前研究忽略了人脸的语 义信息。本 文提出了一种融合局部语义与全局信息的人脸表情识别网络,由两个分支组成:局部语义区 域提取分支 和局部-全局特征融合分支。首先利用人脸解析数据集训练语义分割网络得到人脸语义解析 ,通过迁移训 练的方法得到人脸表情数据集的语义解析。在语义解析中获取对表情识别有意义的区域及其 语义特征, 并将局部语义特征与全局特征融合,构造语义局部特征。最后,融合语义局部特征与全局特 征构成人脸 表情的全局语义复合特征,并通过分类器分为7种基础表情之一。本文同时提出了解冻部分 层训练策略, 该训练策略使语义特征更适用于表情识别,减 少语义信息冗余性。在两个公开数据集JAFFE 和KDEF上 的平均识别准确率分别达到了93.81%和88.78% ,表现优于目前的深度学习方法和传统方法。实验结果证 明了本文提出的融合局部语义和全局信息的网络能够很好地描述表情信息。     

结合一阶和二阶空间信息的行人重识别

针对行人重识别中行人检测误差引起的空间错位,基于局部的深度网络模型仅学习相邻局部关系,导致远距离局部相关性缺失,因此,提出了一种结合一阶和二阶空间信息的行人重识别算法。在主干网络上,学习一阶空间掩模对输入图像的空间权值进行微调,以减少背景干扰;通过二阶空间掩模对远距离的依赖关系进行建模,并将局部特征集成到依赖模型中,以获取全局特征表示。局部分支引入DropBlock对抽取的行人特征进行正则化,避免了网络模型过于依赖特定部位特征。训练阶段用标签平滑分类损失和引入正样本中心的三元组损失联合优化整个网络。在Market-1501和DukeMTMC-reID数据集上的实验结果表明,相比其他主流算法,本算法的行人重识别精度更高,且提取的行人特征判别性和鲁棒性更好。     

Denoising image by matrix factorization in U-shaped convolutional neural network

Image denoising requires both spatial details and global contextualized information to recover a clean version from the deteriorative one. Previous deep convolution networks usually focus on modeling the local feature and stacked convolution blocks to expand the receptive field, which can catch the long-distance dependencies. However, contrary to the expectation, the extracted local feature incapacity recovers the global details by traditional convolution while the stacked blocks hinder the information flow. To tackle these issues, we introduce the Matrix Factorization Denoising Module (MD) to model the interrelationship between the global context aggregating process and the reconstructed process to attain the context details. Besides, we redesign a new basic block to ease the information flow and maintain the network performance. In addition, we conceive the Feature Fusion Module (FFU) to fuse the information from the different sources. Inspired by the multi-stage progressive restoration architecture, we adopt two-stage convolution branches progressively reconstructing the denoised image. In this paper, we propose an original and efficient neural convolution network dubbed MFU. Experimental results on various image denoising datasets: SIDD, DND, and synthetic Gaussian noise datasets show that our MFU can produce comparable visual quality and accuracy results with state-of-the-art methods.     

Transformer-based global–local feature learning model for occluded person re-identification

Most recent occluded person re-identification (re-ID) methods usually learn global features directly from pedestrian images, or use additional pose estimation and semantic analysis model to learn local features, while ignoring the relationship between global and local features, thus incorrectly retrieving different pedestrians with similar attributes as the same pedestrian. Moreover, learning local features using auxiliary models brings additional computational cost. In this work, we propose a Transformer-based dual-branch feature learning model for occluded person re-ID. Firstly, we propose a global–local feature interaction module to learn the relationship between global and local features, thus enhancing the richness of information in pedestrian features. Secondly, we randomly erase local areas in the input image to simulate the real occlusion situation, thereby improving the model’s adaptability to the occlusion scene. Finally, a spilt group module is introduced to explore the local distinguishing features of pedestrian. Numerous experiments validate the effectiveness of our proposed method.     

Fine-grained-based multi-feature fusion for occluded person re-identification

Many previous occluded person re-identification(re-ID) methods try to use additional clues (pose estimation or semantic parsing models) to focus on non-occluded regions. However, these methods extremely rely on the performance of additional clues and often capture pedestrian features by designing complex modules. In this work, we propose a simple Fine-Grained Multi-Feature Fusion Network (FGMFN) to extract discriminative features, which is a dual-branch structure consisting of global feature branch and partial feature branch. Firstly, we utilize a chunking strategy to extract multi-granularity features to make the pedestrian information contained in it more comprehensive. Secondly, a spatial transformer network is introduced to localize the pedestrian’s upper body, and then introduce a relation-aware attention module to explore the fine-grained information. Finally, we fuse the features obtained from the two branches to obtain a more robust pedestrian representation. Extensive experiments verify the effectiveness of our method under the occlusion scenario.     

Robust online tracking via sparse gradient convolution networks

Convolution networks trained offline have recently exhibited promising performance in object tracking tasks. However, offline training is time-consuming and their performance heavily rely on the category of auxiliary training sets. In this paper, we propose a sparse gradient convolution network without pretraining for object tracking. This approach combines shallow convolutional networks and traditional methods (gradient features and sparse representations) to avoid the offline training. In the first frame, we utilize the sparse representation method to learn a series of gradient-based local patches served as fixed filters, and they are used to convolving the input image in the subsequent frames to encode local structural information. Then, we stack all the local structure features to construct global spatial structure features, and the inner geometric layout information is preserved. Moreover, sparse coding and online updating are used to overcome issues related to target appearance variations. Qualitative and quantitative evaluations based on a challenging benchmark dataset demonstrate the effectiveness of the proposed algorithm against several state-of-the-art tracking methods.     

红外与可见光图像注意力生成对抗融合方法研究

目前,基于深度学习的融合方法依赖卷积核提取局部特征,而单尺度网络、卷积核大小以及网络深度的限制无法满足图像的多尺度与全局特性.为此,本文提出了红外与可见光图像注意力生成对抗融合方法.该方法采用编码器和解码器构成的生成器以及两个判别器.在编码器中设计了多尺度模块与通道自注意力机制,可以有效提取多尺度特征,并建立特征通道长...     

基于多样化局部注意力网络的行人重识别

针对现实场景中行人图像被遮挡以及行人姿态或视角变化造成的未对齐问题,该文提出一种基于多样化局部注意力网络(DLAN)的行人重识别(Re-ID)方法。首先,在骨干网络后分别设计了全局网络和多分支局部注意力网络,一方面学习全局的人体空间结构特征,另一方面自适应地获取人体不同部位的显著性局部特征;然后,构造了一致性激活惩罚函数引导各局部分支学习不同身体区域的互补特征,从而获取行人的多样化特征表示;最后,将全局特征与局部特征集成到分类识别网络中,通过联合学习形成更全面的行人描述。在Market1501, DukeMTMC-reID和CUHK03行人重识别数据集上,DLAN模型的mAP值分别达到了88.4%, 79.5%和74.3%,Rank-1值分别达到了95.1%, 88.7%和76.3%,明显优于大多数现有方法,实验结果充分验证了所提方法的鲁棒性和判别能力。     

Uncertainty-guided joint attention and contextual relation network for person re-identification

Due to the influence of factors such as camera angle and pose changes, some salient local features are often suppressed in person re-identification tasks. Moreover, many existing person re-identification methods do not consider the relation between features. To address these issues, this paper proposes two novel approaches: (1) To solve the problem of being confused and misidentified when local features of different individuals have similar attributes, we design a contextual relation network that focuses on establishing the relationship between local features and contextual features, so that all local features of the same person both contain contextual information. (2) To fully and correctly express key local features, we propose an uncertainty-guided joint attention module. The module focuses on the joint representation of individual pixels and local spatial features to enhance the credibility of local features. Finally, our method achieves competitive performance on four widely recognized datasets compared with state-of-the-art methods.     

MTNet: Mutual tri-training network for unsupervised domain adaptation on person re-identification

The existing unsupervised domain adaptation (UDA) methods on person re-identification (re-ID) often employ clustering to assign pseudo labels for unlabeled target domain samples. However, it is difficult to give accurate pseudo labels to unlabeled samples in the clustering process. To solve this problem, we propose a novel mutual tri-training network, termed MTNet, for UDA person re-ID. The MTNet method can avoid noisy labels and enhance the complementarity of multiple branches by collaboratively training the three different branch networks. Specifically, the high-confidence pseudo labels are used to update each network branch according to the joint decisions of the other two branches. Moreover, inspired by self-paced learning, we employ a sample filtering scheme to feed unlabeled samples into the network from easy to hard, so as to avoid trapping in the local optimal solution. Extensive experiments show that the proposed method can achieve competitive performance compared with the state-of-the-art person re-ID methods.     

特征图知识蒸馏引导的轻量化任意方向SAR舰船目标检测器

基于有向边界框的合成孔径雷达(SAR)舰船目标检测器能输出精准的边界框,但仍存在模型计算复杂度高、推理速度慢、存储消耗大等问题,导致其难以在星载平台上部署。基于此该文提出了结合特征图和检测头分支知识蒸馏的无锚框轻量化旋转检测方法。首先,结合目标的长宽比和方向角信息提出改进高斯核,使生成的热度图能更好地刻画目标形状。然后在检测器预测头部引入前景区域增强分支,使网络更关注前景特征且抑制背景杂波的干扰。在训练轻量化网络时,将像素点间的相似度构建为热度图蒸馏知识。为解决特征蒸馏中正负样本不平衡问题,将前景注意力区域作为掩模引导网络蒸馏与目标相关的特征。另外,该文提出全局语义模块对像素进行上下文信息建模,能够结合背景知识加强目标精确表征。基于HRSID数据集的实验结果表明所提方法在模型参数仅有9.07 M的轻量化条件下,mAP能达到80.71%,且检测帧率满足实时应用需求。      

Research on image feature extraction and retrieval algorithms based on convolutional neural network

With the rapid development of mobile Internet and digital technology, people are more and more keen to share pictures on social networks, and online pictures have exploded. How to retrieve similar images from large-scale images has always been a hot issue in the field of image retrieval, and the selection of image features largely affects the performance of image retrieval. The Convolutional Neural Networks (CNN), which contains more hidden layers, has more complex network structure and stronger ability of feature learning and expression compared with traditional feature extraction methods. By analyzing the disadvantage that global CNN features cannot effectively describe local details when they act on image retrieval tasks, a strategy of aggregating low-level CNN feature maps to generate local features is proposed. The high-level features of CNN model pay more attention to semantic information, but the low-level features pay more attention to local details. Using the increasingly abstract characteristics of CNN model from low to high. This paper presents a probabilistic semantic retrieval algorithm, proposes a probabilistic semantic hash retrieval method based on CNN, and designs a new end-to-end supervised learning framework, which can simultaneously learn semantic features and hash features to achieve fast image retrieval. Using convolution network, the error rate is reduced to 14.41% in this test set. In three open image libraries, namely Oxford, Holidays and ImageNet, the performance of traditional SIFT-based retrieval algorithms and other CNN-based image retrieval algorithms in tasks are compared and analyzed. The experimental results show that the proposed algorithm is superior to other contrast algorithms in terms of comprehensive retrieval effect and retrieval time.     

X光安检图像多尺度违禁品检测

安检是保障人民生命财产安全的第一道防线,智能安检是安检行业未来发展的必然趋势。X光安检图像存在背景复杂、违禁品尺度多样以及相互遮挡现象,导致传统的目标检测算法无法获得满意的效果。本文在一阶段目标检测网络SSD框架的基础上,提出了适用于X光安检图像多尺度违禁品检测网络——非对称卷积多视野神经网络ACMNet(Asymmetrical Convolution Multi-View Neural Network)。检测网络增加了三个模块:小卷积非对称模块(Asymmetrical Tiny Convolution Module,ATM)、空洞多视野卷积模块(Dilated Convolution Multi-View Module,DCM)、多尺度特征图融合策略(Fusion strategy of multi-scale feature map,MF)。 ATM学习到的细节特征有助于小尺度违禁品的识别;DCM通过提供局部与全局之间的上下文特征信息来解决遮挡问题;MF则是通过融合高、低层特征图以提高模型在背景干扰情况下违禁品的检测精度。在仿真实验中,采用X光安检领域公开的数据集与自建的数据集,ACMNet在精确度上取得了令人满意的效果。      

A new deep spatial transformer convolutional neural network for image saliency detection

In this paper we propose a novel deep spatial transformer convolutional neural network (Spatial Net) framework for the detection of salient and abnormal areas in images. The proposed method is general and has three main parts: (1) context information in the image is captured by using convolutional neural networks (CNN) to automatically learn high-level features; (2) to better adapt the CNN model to the saliency task, we redesign the feature sub-network structure to output a 6-dimensional transformation matrix for affine transformation based on the spatial transformer network. Several local features are extracted, which can effectively capture edge pixels in the salient area, meanwhile embedded into the above model to reduce the impact of highlighting background regions; (3) finally, areas of interest are detected by means of the linear combination of global and local feature information. Experimental results demonstrate that Spatial Nets obtain superior detection performance over state-of-the-art algorithms on two popular datasets, requiring less memory and computation to achieve high performance.     

基于空间特征融合的双路径图像去噪网络

深度卷积神经网络(Convolutional Neural Network, CNN)在图像去噪领域受到广泛关注。然而,随着网络深度的增加,大多数深度CNN会出现性能饱和、学习能力下降等问题。提出了一种结合局部和全局特征的双路径去噪网络,将两个不同结构的网络组合后构成一个双路径模型,增加网络的宽度,从而获得更多不同的特征。通过长路径连接融合全局和局部特征,增强层间相关性。注意力机制利用当前阶段引导前一阶段的输入,获得更多的特征。实验结果表明,我们提出的网络模型在Set12和BSD68两个数据集中的峰值信噪比(Peak Signal-to-Noise Ratio, PSNR)分别达到了32.95 dB和31.74dB。同时,主观视觉效果(如图像的边缘等细节)恢复得更好、更清晰。     

A comparative study of local matching approach for face recognition.

In contrast to holistic methods, local matching methods extract facial features from different levels of locality and quantify them precisely. To determine how they can be best used for face recognition, we conducted a comprehensive comparative study at each step of the local matching process. The conclusions from our experiments include: (1) additional evidence that Gabor features are effective local feature representations and are robust to illumination changes; (2) discrimination based only on a small portion of the face area is surprisingly good; (3) the configuration of facial components does contain rich discriminating information and comparing corresponding local regions utilizes shape features more effectively than comparing corresponding facial components; (4) spatial multiresolution analysis leads to better classification performance; (5) combining local regions with Borda count classifier combination method alleviates the curse of dimensionality. We implemented a complete face recognition system by integrating the best option of each step. Without training, illumination compensation and without any parameter tuning, it achieves superior performance on every category of the FERET test: near perfect classification accuracy (99.5%) on pictures taken on the same day regardless of indoor illumination variations, and significantly better than any other reported performance on pictures taken several days to more than a year apart. The most significant experiments were repeated on the AR database, with similar results.     

引入全局上下文特征模块的DenseNet孪生网络目标跟踪

近年来,采用孪生网络提取深度特征的方法由于其较好的跟踪精度和速度,成为目标跟踪领域的研究热点之一,但传统的孪生网络并未提取目标较深层特征来保持泛化性能,并且大多数孪生网络只提取局部领域特征,这使得模型对于外观变化是非鲁棒和局部的。针对此,该文提出一种引入全局上下文特征模块的DenseNet孪生网络目标跟踪算法。该文创新性地将DenseNet网络作为孪生网络骨干,采用一种新的密集型特征重用连接网络设计方案,在构建更深层网络的同时减少了层之间的参数量,提高了算法的性能,此外,为应对目标跟踪过程中的外观变化,该文将全局上下文特征模块(GC-Model)嵌入孪生网络分支,提升算法跟踪精度。在VOT2017和OTB50数据集上的实验结果表明,与当前较为主流的算法相比,该文算法在跟踪精度和鲁棒性上有明显优势,在尺度变化、低分辨率、遮挡等情况下具有良好的跟踪效果,且达到实时跟踪要求。