RAFNet: RGB-D attention feature fusion network for indoor semantic segmentation

Semantic segmentation based on the complementary information from RGB and depth images has recently gained great popularity, but due to the difference between RGB and depth maps, how to effectively use RGB-D information is still a problem. In this paper, we propose a novel RGB-D semantic segmentation network named RAFNet, which can selectively gather features from the RGB and depth information. Specifically, we construct an architecture with three parallel branches and propose several complementary attention modules. This structure enables a fusion branch and we add the Bi-directional Multi-step Propagation (BMP) strategy to it, which can not only retain the feature streams of the original RGB and depth branches but also fully utilize the feature flow of the fusion branch. There are three kinds of complementary attention modules that we have constructed. The RGB-D fusion module can effectively extract important features from the RGB and depth branch streams. The refinement module can reduce the loss of semantic information and the context aggregation module can help propagate and integrate information better. We train and evaluate our model on NYUDv2 and SUN-RGBD datasets, and prove that our model achieves state-of-the-art performances.     

A novel deep learning prediction model for concrete dam displacements using interpretable mixed attention mechanism

Dam displacements can effectively reflect its operational status, and thus establishing a reliable displacement prediction model is important for dam health monitoring. The majority of the existing data-driven models, however, focus on static regression relationships, which cannot capture the long-term temporal dependencies and adaptively select the most relevant influencing factors to perform predictions. Moreover, the emerging modeling tools such as machine learning (ML) and deep learning (DL) are mostly black-box models, which makes their physical interpretation challenging and greatly limits their practical engineering applications. To address these issues, this paper proposes an interpretable mixed attention mechanism long short-term memory (MAM-LSTM) model based on an encoder-decoder architecture, which is formulated in two stages. In the encoder stage, a factor attention mechanism is developed to adaptively select the highly influential factors at each time step by referring to the previous hidden state. In the decoder stage, a temporal attention mechanism is introduced to properly extract the key time segments by identifying the relevant hidden states across all the time steps. For interpretation purpose, our emphasis is placed on the quantification and visualization of factor and temporal attention weights. Finally, the effectiveness of the proposed model is verified using monitoring data collected from a real-world dam, where its accuracy is compared to a classical statistical model, conventional ML models, and homogeneous DL models. The comparison demonstrates that the MAM-LSTM model outperforms the other models in most cases. Furthermore, the interpretation of global attention weights confirms the physical rationality of our attention-based model. This work addresses the research gap in interpretable artificial intelligence for dam displacement prediction and delivers a model with both high-accuracy and interpretability.     

中职学校计算机教学中值得重视的四个问题

随着当今社会的飞速发展,计算机的发展愈来愈迅速,应用也愈来愈普及,中职学校的计算机专业的很多课程已渗透到其他专业,计算机教学已越来越普及、重要。本文根据作者多年计算机教学的体会,简单总结和分析了在中职学校计算机教学中值得重视的强化学生学习计算机知识的意识、合理安排计算机课程比例、培养学生对计算机知识的自学能力、考核计算机教学中的实践能力等四个问题。     

Few-shot object segmentation with a new feature aggregation module

The success of convolutional neural network for object segmentation depends on a large amount of training data and high-quality samples. But annotating such high-quality training data for pixel-wise segmentation is labor-intensive. To reduce the massive labor work, few-shot learning has been introduced to segment objects, which uses a few samples for training without compromising the performance. However, the current few-shot models are biased towards the seen classes rather than being class-irrelevant due to lack of global context prior attention. Therefore, this study aims at proposing a few-shot object segmentation model with a new feature aggregation module. Specifically, the proposed work develops a detail-aware module to enhance the discrimination of details with diversified attributes. To enhance the semantics of each pixel, we propose a global attention module to aggregate detailed features containing semantic information. Furthermore, to improve the performance of the proposed model, the model uses support samples that represents class-specific prototype obtained by respective category prototype block. Next, the proposed model predicts label of each pixel of query sample by estimating the distance between the pixel and prototypes. Experiments on standard datasets demonstrate significance of the proposed model over SOTA in terms of segmentation with a few training samples.     

Ship detection based on multi-scale weighted fusion

Ship surveillance plays an important role in ensuring the safety of maritime transportation and navigation. Due to the influence of factors such as waves and special weather, the existing detection methods still cannot balance the accuracy, speed and the parameters of the model in the changeable and complex marine environment. To solve this problem, this paper proposes an improved real-time method based on YOLOv5, which has few parameters and achieves high detection accuracy with little memory and computation cost. Collaborative Attention (CA) mechanism is added to the network structure, which enables the model to more accurately locate and identify target regions. We also design a Spatial Pyramid Pooling module (SPP) and a weighted pyramid network called Bidirectional Feature Pyramid Network (BiFPN) based on the characteristics of the ships to better fuse feature information. Transformer encoder is introduced to capture long-distance dependencies and preserve global and local features to the greatest extent. Furthermore, the ability of our proposed structure to localize objects at each stage is improved through integrating the output of multiple modules. The experimental results show that, the comprehensive performance of this method is better than the existing technology in ship detection on different evaluation criteria.     

Relational-Convergent Transformer for image captioning

Image captioning describes the visual content of a given image by using natural language sentences, and plays a key role in the fusion and utilization of the image features. However, in the existing image captioning models, the decoder sometimes fails to efficiently capture the relationships between image features because of their lack of sequential dependencies. In this paper, we propose a Relational-Convergent Transformer (RCT) network to obtain complex intramodality representations in image captioning. In RCT, a Relational Fusion Module (RFM) is designed for capturing the local and global information of an image by a recursive fusion. Then, a Relational-Convergent Attention (RCA) is proposed, which is composed of a self-attention and a hierarchical fusion module for aggregating global relational information to extract a more comprehensive intramodal contextual representation. To validate the effectiveness of the proposed model, extensive experiments are conducted on the MSCOCO dataset. The experimental results show that the proposed method outperforms some of the state-of-the-art methods.     

Efficient graph attentional network for 3D object detection from Frustum-based LiDAR point clouds

LiDAR-based 3D object detection is important for autonomous driving scene perception, but point clouds produced by LiDAR are irregular and unstructured in nature, and cannot be adopted by the conventional Convolutional Neural Networks (CNN). Recently, Graph Convolutional Networks (GCN) has been proved as an ideal way to handle non-Euclidean structure data, as well as for point cloud processing. However, GCN involves massive computation for searching adjacent nodes, and the heavy computational cost limits its applications in processing large-scale LiDAR point cloud in autonomous driving. In this work, we adopt a frustum-based point cloud-image fusion scheme to reduce the amount of LiDAR point clouds, thus making the GCN-based large-scale LiDAR point clouds feature learning feasible. On this basis, we propose an efficient graph attentional network to accomplish the goal of 3D object detection in autonomous driving, which can learn features from raw LiDAR point cloud directly without any conversions. We evaluate the model on the public KITTI benchmark dataset, the 3D detection mAP is 63.72% on KITTI Cars, Pedestrian and Cyclists, and the inference speed achieves 7.9 fps on a single GPU, which is faster than other methods of the same type.     

Comparative study of three training methods for enhancing process control performance: Emphasis shift training,situation awareness training,and drill and practice

Three training methods to improve attention management skills in process control were compared. Forty students from technical disciplines participated in a five-hour module of emphasis shift training (EST), EST combined with situation awareness training (EST/SA), and drill and practice (D&P) on a simulated process control task. Participants were then tested three times for 45 min each (immediately after training, two weeks after training, and six weeks after training) for system control performance and diagnostic performance on familiar and nonfamiliar fault states. D&P led to superior diagnostic performance on familiar system faults. EST/SA training supported the diagnosis of novel system faults. EST was less effective than expected for system control performance. Implications for training design in process control are discussed.     

Change detection in desktop virtual environments: An eye-tracking study

It is common knowledge that attention is important for learning. We need to utilize attention in order to learn something efficiently and effectively. Similarly, we may also need to acquire familiarity with (i.e., learn) our surroundings in order to utilize our attention. In this study, learning is defined as a product of one’s exposure to natural visual stimuli. Using a virtual model of a natural scene, we investigate both attention and its relationship to learning, according to this definition. Specifically, our focus is the effect of environment familiarity on gaze direction. Our findings reveal that the factor of familiarity with one’s surroundings in virtual reality environments exerts a significant influence on peoples’ ability to detect a variety of specific changes that occur within scenes under their observation.