基于BiLSTM与注意力机制的剩余使用寿命预测研究EI北大核心CSCD

剩余使用寿命(remaining useful life,RUL)预测在现代工业中占有重要地位,如何提高剩余使用寿命预测的准确性已经成为当今研究的热点。传统的剩余使用寿命预测方式是采用基于模型的方法进行预测,需要人工提取特征,不能自动地学习特征信息,无法获得原始数据与剩余使用寿命之间的复杂映射关系。该研究提出一种基于双向长短期记忆网络(bi-directional long short term memory,BiLSTM)与注意力机制的剩余使用寿命预测模型,与已有的剩余使用寿命预测方法不同之处在于:直接将获取的原始时间序列输入到BiLSTM神经网络中,通过BiLSTM自动地提取设备状态特征信息;然后利用注意力机制对特征分配不同的权重,这样可以更准确地提取设备的健康状态信息。进行了发动机和轴承剩余使用寿命预测试验,并与长短期记忆网络(long short-term memory,LSTM)模型和BiLSTM剩余使用寿命预测模型进行比较,试验结果表明提出的BiLSTM与注意力机制相结合的模型能够更准确地进行剩余使用寿命预测,具有应用价值。     

基于全局注意力机制和LSTM的连续手语识别算法

目的 为提高连续手语识别准确率,缓解听障人群与非听障人群的沟通障碍。方法 提出了基于全局注意力机制和LSTM的连续手语识别算法。通过帧间差分法对视频数据进行预处理,消除视频冗余帧,借助ResNet网络提取特征序列。通过注意力机制加权,获得全局手语状态特征,并利用LSTM进行时序分析,形成一种基于全局注意力机制和LSTM的连续手语识别算法,实现连续手语识别。结果 实验结果表明,该算法在中文连续手语数据集CSL上的平均识别率为90.08%,平均词错误率为41.2%,与5种算法相比,该方法在识别准确率与翻译性能上具有优势。结论 基于全局注意力机制和LSTM的连续手语识别算法实现了连续手语识别,并且具有较好的识别效果及翻译性能,对促进听障人群无障碍融入社会方面具有积极的意义。     

中文电子病历命名实体标注语料库构建

针对中文电子病历命名实体语料标注空白的现状,研究了中文电子病历命名实体标注语料库的构建。参考2010年美国国家集成生物与临床信息学研究中心(1282)给出的电子病历命名实体类型及修饰类型的定义,在专业医生的指导下制定了详尽的中文电子病历标注规范;通过对大量中文电子病历的分析,提出了一套完整的中文电子病历命名实体标注方案,而且采用预标注和正式标注的方法,建立了一定规模的中文电子病历命名实体标注语料库,其标注语料的一致性达到了92%以上。该工作对中文电子病历的命名实体识别及信息抽取研究提供了可靠的数据支持,对医疗知识挖掘也有重要意义。     

基于条件随机场的中文命名实体识别研究

条件随机场模型是文本信息抽取的重要方法之一,在命名实体识别方面CRF性能要明显优于隐马尔科夫模型和最大熵模型。本文以基于字一级的条件随机场模型实现了中文命名实体识别,取得了较好的识别效果。     

注意力机制及其在医学视觉任务中的作用研究

注意力机制模仿人类对特定信息的认知机制，放大关键细节，更多地关注数据本身，其本质就是定位感兴趣的信息，抑制无用信息。注意力也是一种对输入分配偏好的资源分配机制，计算一系列的注意力系数或权重参数。在计算机视觉领域中，注意力基于原有数据觅求信息间的关联，突出待处理对象的重要特征并捕捉图像感受野。本文基于注意力机制一般原理，阐释注意力之于神经网络的重要性，提升了神经网络近似复杂函数的能力，一定程度上缓解了深度神经网络“黑匣子”的不可解释问题。本文重点分析了基于自注意机制的Transformer：模型容量大、能更好地学习信息表征、可提取更高维的信息，以“长程依赖”的优势建模，克服CNN不能捕获图像全局信息的缺憾，是一种较强大的视觉识别工具。鉴于局部和全局特征对于医学图像分割这种密集的预测任务而言缺一不可，未来结合Transformer的整体捕获和CNN的局部提取工作将推动医学视觉任务的进一步突破。     

基于通道分组注意力的无监督图像风格转换模型

为了解决无监督图像风格转换模型输出结果的局部伪影和局部特征丢失问题,提出了一种基于通道分组注意力机制的图像风格转换模型.生成器部分采用通道分组注意力残差块,以增强生成器部分对于图像特征的提取以及有效特征的利用;鉴别器部分采用双鉴别器结构,利用增加的局部鉴别器增强对于生成图像细节的鉴别,利用多分辨率尺度的全局鉴别器增强生成图像的内容合理性与结构连贯性.实验结果表明:本模型比起BicycleGAN、MUNIT等模型不但体积更小,而且可以获得更高的NIMA美观度得分以及LPIPS多样性得分;在包装类产品的平面设计迁移应用任务中,本模型同样表现良好.     

改进注意力机制的航空发动机试验转子系统智能故障诊断

考虑到航空发动机的工作环境十分恶劣,其故障的振动信号特征隐蔽且噪声干扰严重,为了加强网络对振动信号中关键特征的提取能力,提出了改进注意力机制的航空发动机转子系统智能故障诊断方法对航空发动机转子系统的不平衡和碰摩等故障进行诊断。提出局部池化改进的通道注意力机制,能够通过预提取局部极值解决现有通道注意力机制对航空发动机转子故障通道信息提取能力不足的问题;提出多评分机制改进的空间注意力机制,能够通过不同尺度的卷积评分解决现有空间注意力机制对航空发动机转子故障空间信息提取能力不足的问题;将二者结合构建改进的通道空间注意力机制模块,再导入一维卷积神经网络中构建改进注意力机制的一维卷积神经网络完成智能故障诊断,并且通过航空发动机转子系统故障数据集对比分析试验证明了该网络优秀的检测性能、抗噪性能和泛化性能等综合性能以及注意力机制改进方法的可行性。     

基于卷积神经网络和Transformer网络的鸟声识别

针对传统鸟声识别算法中特征提取方式单一、分类识别准确率低等问题,提出一种结合卷积神经网络和Transformer网络的鸟声识别方法。该方法综合考虑网络局部特征学习和全局上下文依赖性构造,从原始鸟声音频信号中提取短时傅里叶变换(Short Time Fourier Transform,STFT)语谱图特征,将其输入到卷积神经网络(ConvolutionalNeural Network,CNN)中提取局部频谱特征信息,同时提取鸟声信号的对数梅尔特征及一阶差分、二阶差分特征用于合成梅尔频率倒谱系数(Mel Frequency Cepstrum Coefficient,MFCC)混合特征向量,将其输入到Transformer网络中获取全局序列特征信息,最后融合所提取的特征可得到更丰富的鸟声特征参数,通过Softmax分类器得到鸟声识别结果。在Birdsdata和xeno-canto鸟声数据集上进行实验,平均识别准确率分别达到了97.81%和89.47%。实验结果表明该方法相较于其他现有的鸟声识别模型具有更高的识别准确率。     

基于自身注意力时空特征的语音情感识别算法

针对语音情感识别中无法对关键的时空依赖关系进行建模，导致识别率低的问题，提出一种基于自身注意力（self-attention）时空特征的语音情感识别算法，利用双线性卷积神经网络、长短期记忆网络和多组注意力（multi-headattention）机制去自动学习语音信号的最佳时空表征。首先提取语音信号的对数梅尔（log-Mel）特征、一阶差分和二阶差分特征合成3D log-Mel特征集作为卷积神经网络的输入；然后综合考虑空间特征和时间依赖性关系，将双线性池化和双向长短期记忆网络的输出融合得到空间-时间特征表征，利用多组注意力机制捕获判别性强的特征；最后利用softmax函数进行分类。在IEMOCAP和EMO-DB数据库上进行实验，结果表明两种数据库的识别率分别为63.12%和87.09%，证明了此方法的有效性。     

集成 KPCA 与 t‑SNE 的滚动轴承故障特征提取方法

针对滚动轴承原始数据集包含高维非敏感特征的问题,提出一种集成核主成分分析（Kernel Principal Component Analysis,KPCA）与 t?分布随机邻域嵌入（t?distributed Stochastic Neighbor Embedding,t?SNE）的滚动轴承故障低维敏感特征提取方法。该方法先计算滚动轴承原始振动信号的时域、频域以及时频域特征,构建初始高维特征数据集。利用 KPCA 降低高维数据集的相关性,在最大化高维数据全局特征方差的目标下,提取出非线性特征子集。通过 t?SNE 充分挖掘故障特征数据集的局部结构信息,进一步获取具有高判别性的低维敏感特征子集。将低维特征子集输入到 k?近邻分类器（k?nearest Neighbor Classifier,KNNC）进行分类,以分类准确率和聚类结果作为度量指标,对特征提取结果的优劣予以评价。上述过程综合考虑了数据集的全局和局部结构特征,充分利用了数据自身的结构信息,从而可准确提取其低维敏感特征。将该方法用于滚动轴承故障诊断实验中,通过与其他典型特征提取方法进行对比,及其对含噪情况下轴承故障特征的准确提取,验证了方法的有效性。     

Multi-Level Feature-Based Ensemble Model for Target-Related Stance Detection

Stance detection is the task of attitude identification toward a standpoint. Previous work of stance detection has focused on feature extraction but ignored the fact that irrelevant features exist as noise during higher-level abstracting. Moreover, because the target is not always mentioned in the text, most methods have ignored target information. In order to solve these problems, we propose a neural network ensemble method that combines the timing dependence bases on long short-term memory (LSTM) and the excellent extracting performance of convolutional neural networks (CNNs). The method can obtain multi-level features that consider both local and global features. We also introduce attention mechanisms to magnify target information-related features. Furthermore, we employ sparse coding to remove noise to obtain characteristic features. Performance was improved by using sparse coding on the basis of attention employment and feature extraction. We evaluate our approach on the SemEval-2016Task 6-A public dataset, achieving a performance that exceeds the benchmark and those of participating teams.     

Accurate Multi-Scale Feature Fusion CNN for Time Series Classification in Smart Factory

Time series classification (TSC) has attracted various attention in the community of machine learning and data mining and has many successful applications such as fault detection and product identification in the process of building a smart factory. However, it is still challenging for the efficiency and accuracy of classification due to complexity, multi-dimension of time series. This paper presents a new approach for time series classification based on convolutional neural networks (CNN). The proposed method contains three parts: short-time gap feature extraction, multi-scale local feature learning, and global feature learning. In the process of short-time gap feature extraction, large kernel filters are employed to extract the features within the short-time gap from the raw time series. Then, a multi-scale feature extraction technique is applied in the process of multi-scale local feature learning to obtain detailed representations. The global convolution operation with giant stride is to obtain a robust and global feature representation. The comprehension features used for classifying are a fusion of short time gap feature representations, local multi-scale feature representations, and global feature representations. To test the efficiency of the proposed method named multi-scale feature fusion convolutional neural networks (MSFFCNN), we designed, trained MSFFCNN on some public sensors, device, and simulated control time series data sets. The comparative studies indicate our proposed MSFFCNN outperforms other alternatives, and we also provided a detailed analysis of the proposed MSFFCNN.     

区域渐近增强与感受野结合的多核近似学习网

为充分提取图像中可辨识信息、提高分类正确率,提出多核近似学习网,该网络主要由2部分构成。在特征提取部分,利用二维高斯分布对原始图像进行区域渐进增强,局部感受野和全局感受野被用于充分提取原始图像和区域渐进增强图像中的局部和全局特征,并将其串联以组成代表图像的特征向量。在分类部分,提出多核近似算法,将近似核映射编码出的低秩特征矩阵作为网络的隐藏层,以求解网络的输出权重。为验证该网络的有效性,利用USPS、MNIST和NORB数据集进行实验,实验证明所提出的多核近似学习网能够在局部感受野极端学习机的基础上进一步提取出特征信息,有效提高了分类正确率。     

Adversarial Active Learning for Named Entity Recognition in Cybersecurity

Owing to the continuous barrage of cyber threats, there is a massive amount of cyber threat intelligence. However, a great deal of cyber threat intelligence come from textual sources. For analysis of cyber threat intelligence, many security analysts rely on cumbersome and time-consuming manual efforts. Cybersecurity knowledge graph plays a significant role in automatics analysis of cyber threat intelligence. As the foundation for constructing cybersecurity knowledge graph, named entity recognition (NER) is required for identifying critical threat-related elements from textual cyber threat intelligence. Recently, deep neural network-based models have attained very good results in NER. However, the performance of these models relies heavily on the amount of labeled data. Since labeled data in cybersecurity is scarce, in this paper, we propose an adversarial active learning framework to effectively select the informative samples for further annotation. In addition, leveraging the long short-term memory (LSTM) network and the bidirectional LSTM (BiLSTM) network, we propose a novel NER model by introducing a dynamic attention mechanism into the BiLSTM-LSTM encoderdecoder. With the selected informative samples annotated, the proposed NER model is retrained. As a result, the performance of the NER model is incrementally enhanced with low labeling cost. Experimental results show the effectiveness of the proposed method.     

基于孪生并行注意力网络的包装印刷品商标真伪 鉴别研究

为了借助手机精确快速地鉴别微小篡改的包装印刷品商标真伪,提出孪生并行注意力卷积神经网络判别模型。通过孪生网络的共享权重机制降低网络系统的表征偏差,通过并行注意力机制提高对微小篡改变化特征的提取能力,最大程度降低打印拍照引入的噪声对篡改特征提取的影响。在2种印刷纸张、2种拍摄光源和2种拍摄手机组合的8种开放场景中,拍摄多组真伪商标的印刷图像,建立篡改面积0.4%~0.7%的商标真伪数据集。模型在该打印拍照数据集上鉴别准确率为94%以上,在真实山寨商标上的鉴别准确率为100%。本文提出的孪生并行注意力卷积神经网络模型,具有较高的细粒度鉴别精度和较强的泛化能力,能够在开放场景下有效地实现基于图像微小篡改的包装印刷品商标真伪的鉴别。     

A Multi-Feature Learning Model with Enhanced Local Attention for Vehicle Re-Identification

Vehicle re-identification (ReID) aims to retrieve the target vehicle in an extensive image gallery through its appearances from various views in the cross-camera scenario. It has gradually become a core technology of intelligent transportation system. Most existing vehicle re-identification models adopt the joint learning of global and local features. However, they directly use the extracted global features, resulting in insufficient feature expression. Moreover, local features are primarily obtained through advanced annotation and complex attention mechanisms, which require additional costs. To solve this issue, a multi-feature learning model with enhanced local attention for vehicle re-identification (MFELA) is proposed in this paper. The model consists of global and local branches. The global branch utilizes both middle and high-level semantic features of ResNet50 to enhance the global representation capability. In addition, multi-scale pooling operations are used to obtain multi-scale information. While the local branch utilizes the proposed Region Batch Dropblock (RBD), which encourages the model to learn discriminative features for different local regions and simultaneously drops corresponding same areas randomly in a batch during training to enhance the attention to local regions. Then features from both branches are combined to provide a more comprehensive and distinctive feature representation. Extensive experiments on VeRi-776 and VehicleID datasets prove that our method has excellent performance.     

基于改进YOLOv5s的腌制蔬菜真空包装缺陷检测

目的 针对传统的基于人工的腌制蔬菜真空缺陷包装剔除效率低、漏检率高等问题,提出一种基于改进YOLOv5s的腌制蔬菜真空包装缺陷检测方法。方法 首先,使用Ghost卷积替换CSP模块中的卷积,在提高模型特征提取能力的同时降低网络的参数量;其次,利用空间换深度(Space-to-Depth, SPD)和深度可分离卷积(Depthwise-Separable Convolution, DSConv)组合操作SPD–DSConv进行下采样,减少下采样造成的特征信息损耗;最后,在网络中引入SE注意力机制,提高算法的精确率。结果 在自制的腌制蔬菜真空包装数据集上,改进后的网络平均精度(man Average Precision, A_mAP)为93.88%,模型尺寸为3.91MB,相比原网络精度提高了2.05%,模型尺寸缩减了44.38%。结论 文中方法能够实现腌制蔬菜真空缺陷包装的分类和定位,为基于机器人的缺陷包装剔除奠定了基础。     

CLGA Net: Cross Layer Gated Attention Network for Image Dehazing

In this paper, we propose an end-to-end cross-layer gated attention network (CLGA-Net) to directly restore fog-free images. Compared with the previous dehazing network, the dehazing model presented in this paper uses the smooth cavity convolution and local residual module as the feature extractor, combined with the channel attention mechanism, to better extract the restored features. A large amount of experimental data proves that the defogging model proposed in this paper is superior to previous defogging technologies in terms of structure similarity index (SSIM), peak signal to noise ratio (PSNR) and subjective visual quality. In order to improve the efficiency of decoding and encoding, we also describe a fusion residual module and conduct ablation experiments, which prove that the fusion residual is suitable for the dehazing problem. Therefore, we use fusion residual as a fixed module for encoding and decoding. In addition, we found that the traditional defogging model based on the U-net network may cause some information losses in space. We have achieved effective maintenance of low-level feature information through the cross-layer gating structure that better takes into account global and subtle features. We also present the application of our CLGA-Net in challenging scenarios where the best results in both quantity and quality can be obtained. Experimental results indicate that the present cross-layer gating module can be widely used in the same type of network.