RAFNet: RGB-D attention feature fusion network for indoor semantic segmentation

Semantic segmentation based on the complementary information from RGB and depth images has recently gained great popularity, but due to the difference between RGB and depth maps, how to effectively use RGB-D information is still a problem. In this paper, we propose a novel RGB-D semantic segmentation network named RAFNet, which can selectively gather features from the RGB and depth information. Specifically, we construct an architecture with three parallel branches and propose several complementary attention modules. This structure enables a fusion branch and we add the Bi-directional Multi-step Propagation (BMP) strategy to it, which can not only retain the feature streams of the original RGB and depth branches but also fully utilize the feature flow of the fusion branch. There are three kinds of complementary attention modules that we have constructed. The RGB-D fusion module can effectively extract important features from the RGB and depth branch streams. The refinement module can reduce the loss of semantic information and the context aggregation module can help propagate and integrate information better. We train and evaluate our model on NYUDv2 and SUN-RGBD datasets, and prove that our model achieves state-of-the-art performances.     

基于卷积块注意力模块的图像描述生成模型

图像描述生成模型是使用自然语言描述图片的内容及其属性之间关系的算法模型.对现有模型描述质量不高、图片重要部分特征提取不足和模型过于复杂的问题进行了研究,提出了一种基于卷积块注意力机制模块(CBAM)的图像描述生成模型.该模型采用编码器-解码器结构,在特征提取网络Inception-v4中加入CBAM,并作为编码器提取图片的重要特征信息,将其送入解码器长短期记忆网络(LSTM)中,生成对应图片的描述语句.采用MSCOCO2014数据集中训练集和验证集进行训练和测试,使用多个评价准则评估模型的准确性.实验结果表明,改进后模型的评价准则得分优于其他模型,其中Model2实验能够更好地提取到图像特征,生成更加准确的描述.     

A novel deep learning prediction model for concrete dam displacements using interpretable mixed attention mechanism

Dam displacements can effectively reflect its operational status, and thus establishing a reliable displacement prediction model is important for dam health monitoring. The majority of the existing data-driven models, however, focus on static regression relationships, which cannot capture the long-term temporal dependencies and adaptively select the most relevant influencing factors to perform predictions. Moreover, the emerging modeling tools such as machine learning (ML) and deep learning (DL) are mostly black-box models, which makes their physical interpretation challenging and greatly limits their practical engineering applications. To address these issues, this paper proposes an interpretable mixed attention mechanism long short-term memory (MAM-LSTM) model based on an encoder-decoder architecture, which is formulated in two stages. In the encoder stage, a factor attention mechanism is developed to adaptively select the highly influential factors at each time step by referring to the previous hidden state. In the decoder stage, a temporal attention mechanism is introduced to properly extract the key time segments by identifying the relevant hidden states across all the time steps. For interpretation purpose, our emphasis is placed on the quantification and visualization of factor and temporal attention weights. Finally, the effectiveness of the proposed model is verified using monitoring data collected from a real-world dam, where its accuracy is compared to a classical statistical model, conventional ML models, and homogeneous DL models. The comparison demonstrates that the MAM-LSTM model outperforms the other models in most cases. Furthermore, the interpretation of global attention weights confirms the physical rationality of our attention-based model. This work addresses the research gap in interpretable artificial intelligence for dam displacement prediction and delivers a model with both high-accuracy and interpretability.     

基于注意力的热点块和显著像素卷积神经网络的人脸防伪方法

人脸防伪用于验证被测试者是否为真实活体,是计算机视觉领域的一个研究热点。攻击手段的多样性以及人脸识别主要在嵌入式、移动式等不具备高计算能力的设备上应用,使得快速有效的人脸防伪计算成为具有挑战性的任务。针对该问题,文中提出了一种基于注意力的热点块和显著像素卷积神经网络的方法。其中,热点块机制以对5个热点块的判别来取代对整张人脸的判别,显著降低了计算量,迫使网络模型集中关注更具有鉴别信息的热点块,提高了网络模型的准确率;显著像素方法对输入的人脸图像进行显著像素预测,通过判断显著预测图是否符合人脸的深度特性来鉴别活体与攻击。该方法将热点块与显著像素的结果进行融合,充分发挥了局部特征和全局特征的作用,进一步提升了人脸防伪的效果。与现有方法相比,所提方法在CASIA-MFSD、Replay-Attack以及SiW数据集上都达到了很好的效果。     

基于多模态特征学习的人体行为识别方法

由于从单一行为模态中获取的特征难以准确地表达复杂的人体动作,本文提出基于多模态特征学习的人体行为识别算法.首先采用两条通道分别提取行为视频的RGB特征和3D骨骼特征,第1条通道C3DP-LA网络由两部分组成:(1)包含时空金字塔池化(Spatial Temporal Pyramid Pooling,STPP)的改进3D CNN;(2)基于时空注意力机制的LSTM,第2条通道为时空图卷积网络(ST-GCN),然后,本文将提取到的两种特征融合使其优势互补,最后用Softmax分类器对融合特征进行分类,并在公开数据集UCF101和NTU RGB+D上验证.实验表明,本文提出的方法与现有行为识别算法相比具有较高的识别准确度.     

中职学校计算机教学中值得重视的四个问题

随着当今社会的飞速发展,计算机的发展愈来愈迅速,应用也愈来愈普及,中职学校的计算机专业的很多课程已渗透到其他专业,计算机教学已越来越普及、重要。本文根据作者多年计算机教学的体会,简单总结和分析了在中职学校计算机教学中值得重视的强化学生学习计算机知识的意识、合理安排计算机课程比例、培养学生对计算机知识的自学能力、考核计算机教学中的实践能力等四个问题。     

Multimodal focus attention and stress detection and feedback in an augmented driver simulator

This paper presents a driver simulator, which takes into account the information about the user’s state of mind (level of attention, fatigue state, stress state). The user’s state of mind analysis is based on video data and biological signals. Facial movements such as eyes blinking, yawning, head rotations, etc., are detected on video data: they are used in order to evaluate the fatigue and the attention level of the driver. The user’s electrocardiogram and galvanic skin response are recorded and analyzed in order to evaluate the stress level of the driver. A driver simulator software is modified so that the system is able to appropriately react to these critical situations of fatigue and stress: some audio and visual messages are sent to the driver, wheel vibrations are generated and the driver is supposed to react to the alert messages. A multi-threaded system is proposed to support multi-messages sent by the different modalities. Strategies for data fusion and fission are also provided. Some of these components are integrated within the first prototype of OpenInterface: the multimodal similar platform.     

Few-shot object segmentation with a new feature aggregation module

The success of convolutional neural network for object segmentation depends on a large amount of training data and high-quality samples. But annotating such high-quality training data for pixel-wise segmentation is labor-intensive. To reduce the massive labor work, few-shot learning has been introduced to segment objects, which uses a few samples for training without compromising the performance. However, the current few-shot models are biased towards the seen classes rather than being class-irrelevant due to lack of global context prior attention. Therefore, this study aims at proposing a few-shot object segmentation model with a new feature aggregation module. Specifically, the proposed work develops a detail-aware module to enhance the discrimination of details with diversified attributes. To enhance the semantics of each pixel, we propose a global attention module to aggregate detailed features containing semantic information. Furthermore, to improve the performance of the proposed model, the model uses support samples that represents class-specific prototype obtained by respective category prototype block. Next, the proposed model predicts label of each pixel of query sample by estimating the distance between the pixel and prototypes. Experiments on standard datasets demonstrate significance of the proposed model over SOTA in terms of segmentation with a few training samples.