A new supervised multi-head self-attention autoencoder for health indicator construction and similarity-based machinery RUL prediction

Remaining useful life (RUL) prediction plays a significant role in the prognostic and health management (PHM) of rotating machineries. A good health indicator (HI) can ensure the accuracy and reliability of RUL prediction. However, numerous existing deep learning-based HI construction approaches rely heavily on the prior knowledge, and they are difficult to capture the key information in the process of machinery degradation from raw signals, thereby affecting the performance of RUL prediction. To tackle the aforementioned problem, a new supervised multi-head self-attention autoencoder (SMSAE) is proposed for extracting the HI that effectively reflects the degraded state of rotating machinery. By embedding the multi-head self-attention (MS) module into autoencoder and imposing the constraint of power function-type labels on the hidden variable, SMSAE can directly extract the HIs from raw vibration signals. As the current HI evaluation indexes don’t consider the global monotonicity and variation law of HI, two improved monotonicity and robustness indexes are designed for the better evaluation of HI. With the proposed HI, a two-stage residual life prediction framework based on similarity is developed. Extensive experiments have been performed on an actual wind turbine gearbox bearing dataset and a well-known open commercial modular aero-propulsion system simulation (C-MAPSS) dataset. The comparative results verify that the constructed SMSAE HI has better comprehensive performance than the typical HIs, and the proposed prediction method is competitive with the state-of-the-art methods.     

小世界回声状态网络的风机变桨系统预警模型研究

提出一种基于自编码器的小世界回声状态网络的故障预测方法,以此来应对风电机组变桨系统可能产生的故障,提高故障响应速度。通过分析系统中的监测参数和数据,建立基于自编码器的小世界回声状态网络系统模型。实例表明该方法可在故障发生前提前作出预警,验证了方法的可行性,有效降低了机组的运维成本。     

Unsupervised Electric Motor Fault Detection by Using Deep Autoencoders

Fault diagnosis of electric motors is a fundamental task for production line testing, and it is usually performed by experienced human operators. In the recent years, several methods have been proposed in the literature for detecting faults automatically. Deep neural networks have been successfully employed for this task, but, up to the authors' knowledge, they have never been used in an unsupervised scenario. This paper proposes an unsupervised method for diagnosing faults of electric motors by using a novelty detection approach based on deep autoencoders. In the proposed method, vibration signals are acquired by using accelerometers and processed to extract Log-Mel coefficients as features. Autoencoders are trained by using normal data only, i.e., data that do not contain faults. Three different autoencoders architectures have been evaluated: the multi-layer perceptron (MLP) autoencoder, the convolutional neural network autoencoder, and the recurrent autoencoder composed of long short-term memory (LSTM) units. The experiments have been conducted by using a dataset created by the authors, and the proposed approaches have been compared to the one-class support vector machine (OC-SVM) algorithm. The performance has been evaluated in terms area under curve (AUC) of the receiver operating characteristic curve, and the results showed that all the autoencoder-based approaches outperform the OC-SVM algorithm. Moreover, the MLP autoencoder is the most performing architecture, achieving an AUC equal to 99.11%      

基于改进LSTM的航空发动机寿命预测方法研究

发动机剩余寿命(RUL)预测时,进行数据特征提取易导致预测效率低下。为解决此问题,提出一种改进的长短期记忆(LSTM)算法模型。通过引入深度稀疏自动编码器(SDAE)完成时序数据的处理与特征提取,优化LSTM模型,改善航空发动机RUL预测效果。利用SDAE进行特征提取,构建健康因子(HI)曲线;同时考虑运行工况、故障模式和传感器3个因素,并分别训练其权重。利用LSTM模型进行发动机剩余寿命预测。利用涡扇发动机退化过程数据集C-MAPSS开展实验,并与DNN、BiLSTM、单层LSTM进行对比分析。结果表明：与上述3种算法相比,改进后算法的均方根误差和〖JP2〗评分函数值至少分别降低6.6%和39.1%;该方法寿命预测结果和实际寿命曲线拟合度高,验证了该方法的可行性和有效性。     

堆叠去噪自编码器在垃圾邮件过滤中的应用

针对垃圾邮件数量日益攀升的问题,提出了将堆叠去噪自编码器应用到垃圾邮件分类中.首先,在无标签数据集上,使用无监督学习方法最小化重构误差,对堆叠去噪自编码器进行贪心逐层预训练,从而获得原始数据更加抽象和健壮的特征表示; 然后,在堆叠去噪自编码器的最上层添加一个分类器后,在有标签数据集上,利用有监督学习方法最小化分类误差,对预训练获得的网络参数进行微调,获得最优化的模型; 最后, 利用训练完成的堆叠去噪编码器在6个不同的公开数据集上进行测试.将准确率、召回率、更具有平衡性的马修斯相关系数作为实验性能评价标准,实验结果表明,相比支持向量机算法、贝叶斯方法和深度置信网络的分类效果,基于堆叠去噪自编码器的垃圾邮件分类器的准确率都高于95%,马修斯相关系数都大于0.88,在应用中具有更高的准确率和更好的健壮性.     

A novel intrusion detection model for the CAN bus packet of in-vehicle network based on attention mechanism and autoencoder

The attacks on in-vehicle Controller Area Network (CAN) bus messages severely disrupt normal communication between vehicles. Therefore, researches on intrusion detection models for CAN have positive business value for vehicle security, and the intrusion detection technology for CAN bus messages can effectively protect the in-vehicle network from unlawful attacks. Previous machine learning-based models are unable to effectively identify intrusive abnormal messages due to their inherent shortcomings. Hence, to address the shortcomings of the previous machine learning-based intrusion detection technique, we propose a novel method using Attention Mechanism and AutoEncoder for Intrusion Detection (AMAEID). The AMAEID model first converts the raw hexadecimal message data into binary format to obtain better input. Then the AMAEID model encodes and decodes the binary message data using a multi-layer denoising autoencoder model to obtain a hidden feature representation that can represent the potential features behind the message data at a deeper level. Finally, the AMAEID model uses the attention mechanism and the fully connected layer network to infer whether the message is an abnormal message or not. The experimental results with three evaluation metrics on a real in-vehicle CAN bus message dataset outperform some traditional machine learning algorithms, demonstrating the effectiveness of the AMAEID model.     

基于Autoencoder网络的数据降维和重构

在机器学习,模式识别以及数据挖掘等诸多研究领域中,往往会面临着维数灾难问题。因此,特征数据的降维方法,即将高维的特征数据如何进行简化投射到低维空间中再进行处理,成为当前数据驱动的计算方法研究热点之一。该文引入一种特殊的非线性降维方法,称为自编码(Autoencoder)神经网络,该方法采用CRBM(Continuous Restricted Boltzmann Machine)的网络结构,通过训练具有多个中间层的双向深层神经网络将高维数据转换成低维嵌套并继而重构高维数据。特别地,自编码网络提供了高维数据空间和低维嵌套结构的双向映射,有效解决了大多数非线性降维方法所不具备的逆向映射问题。将Autoencoder用于人工数据和真实图像数据的实验表明,Autoencoder不仅能发现嵌入在高维数据中的非线性低维结构,也能有效地从低维结构中恢复原始高维数据。     

基于SDAE与CART联合智能算法的通信网络用户满意度分析方法

论文提出一种基于栈式降噪自编码器(Stacked Denoising Autoencoder,SDAE)与分类和回归决策树(Classification and Regression Tree,CART)的移动互联网满意度预测方法,此模型能挖掘出用户的满意度与用户的特征和网络特征的关联规则,通过这种规则能更精准及时地预测到用户满意度的变化,以便运营商针对这种变化提前作出决策。论文所提方法能够挖掘特征间的深层关系,通过SDAE编码样本可以获得影响用户体验的隐含特征,及时发现用户对于网络贬损的真正痛点,为运营商网络建设和运行维护部门制定提升用户的网络感知策略提供依据,从而提升用户体验。     

基于改进自编码器的文本分类算法

词的向量化表达是文本挖掘应用的必要前提。为了改善自编码器在词嵌入中的效果,提高文本分类的准确性,提出了一种改进的自编码器并将其用于文本分类。在传统自编码器的基础上,在隐藏层加入了一个全局调整函数,其将绝对值小的特征值调整到绝对值大的特征值上,实现了隐藏层特征向量的稀疏化。得到调整后的特征向量之后,采用全连接神经网络进行文本分类。在20news数据集上的实验结果表明,所提方法具有更好的词向量嵌入式效果,并且在文本分类中也具有更好的效果。     

基于深度自编码器的单样本人脸识别*

由于每个目标仅有一幅已知样本,无法描述目标的类内变化,诸多人脸识别算法在解决单样本人脸识别问题时识别性能较低.因此文中提出基于深度自编码器的单样本人脸识别算法.算法首先采用所有已知样本训练深度自编码器,得到广义深度自编码器,然后使用每个单样本目标的单个样本微调广义深度自编码器,得到特定类别的深度自编码器.识别时,将识别图像输入每个特定类别的深度自编码器,得到包含与测试图像相同类内变化的该类别的重构图像,使用重构图像训练Softmax回归模型,分类测试图像.在公共测试库上进行测试,并与其它算法在相同环境下进行对比,结果表明文中算法在获得更优识别率的同时,识别一幅图像所需平均时间更少.