基于并行双向时间卷积网络和双向长短期记忆网络的轴承剩余使用寿命预测方法

在基于深度学习的轴承剩余使用寿命(RUL)预测方法中,时间卷积网络(TCN)忽略了振动数据中未来时间信息的重要性,长短期记忆网络(LSTM)难以有效地学习振动数据的长时间序列特征.针对以上问题,提出一种基于并行双向时间卷积网络(Bi-TCN)和双向长短期记忆网络(Bi-LSTM)的轴承RUL预测方法.首先,对多传感器数据进行归一化处理,并将每个传感器数据进行通道合并,实现多传感器数据的高效融合;然后,采用Bi-TCN和Bi-LSTM构建并行的双分支特征学习网络,其中Bi-TCN提取数据的双向长时间序列特征, Bi-LSTM提取数据的时间相关特征;同时,设计一种特征融合注意力机制,该机制分别计算Bi-TCN和Bi-LSTM的输出权重,以实现两种网络输出特征的自适应加权融合;最后,融合特征通过全连接层并输出轴承RUL的预测结果.利用西安交通大学轴承数据集和PHM 2012轴承数据集进行RUL预测实验,实验结果表明,与其他先进的预测方法相比,所提出方法可以准确预测更多类型轴承的RUL,同时具有更低的预测误差.     

基于多注意力和变分编码时序网络的发动机剩余使用寿命预测方法

准确可靠的剩余使用寿命(RUL)预测结果可为决策者提供有价值的信息,以采取适当的维护策略,最大限度地利用设备,避免昂贵的故障维修费用。为了从高噪声的真实工况中对发动机故障进行有效诊断,提出了一种融合多注意力机制和变分编码的时序预测模型(MA-VBLSTM),首先通过嵌入多注意力机制获得所有特征在空间维度和通道维度的不同权重,以提高退化特征的提取能力；然后,采用变分编码器进行退化信息编码并学习数据间深度隐藏的信息；最后,利用双向长短时记忆网络的长短期时序数据双向处理能力实现发动机RUL的预测。实验结果表明,该模型在发动机CMAPSS数据集的FD001、FD002、FD003、FD004子数据集上RMSE 和Score值相比现有方法分别平均降低5.27%和10.70%、1.37%和1.68%、6.37%和26.94%、3.02%和2.06%。     

基于双路注意力机制的习题推荐模型研究

习题推荐是利用推荐算法将习题推荐给学生的任务，点击率(CTR)预测则是推荐领域的主流研究方向之一，现有的大部分习题推荐模型没有重视注意力机制的创新，因而落后于CTR预测领域。为了研究CTR预测模型中注意力机制在教育领域的应用前景，该文提出一种分层次学习注意力权重的双路注意力推荐模型(SEFM)。该模型通过因子分解机(FM)与压缩激励注意力网络(SENET)两个注意力机制的并行运行，实现学习特征之间的关系以及特征本身的权重，从而完成推荐。在两个CTR广告数据集与一个教育数据集上的实验表明，SEFM能准确地学习特征在多种维度上的权重，在两个评价指标上的表现均优于现有的先进基准模型。     

注意力机制综述

现在注意力机制已广泛地应用在深度学习的诸多领域.基于注意力机制的结构模型不仅能够记录信息间的位置关系,还能依据信息的权重去度量不同信息特征的重要性.通过对信息特征进行相关与不相关的抉择建立动态权重参数,以加强关键信息弱化无用信息,从而提高深度学习算法效率同时也改进了传统深度学习的一些缺陷.从图像处理、自然语言处理、数据预测等不同应用方面介绍了一些与注意力机制结合的算法结构,并对近几年大火的基于注意力机制的transformer和reformer算法进行了综述.鉴于注意力机制的重要性,综述了注意力机制的研究发展,分析了注意力机制目前的发展现状并探讨了该机制未来可行的研究方向.     

融合用户兴趣表征与注意力机制的推荐算法

为利用用户行为挖掘用户的兴趣,提出一种融合用户兴趣表征与注意力机制的推荐算法.利用CVR算法将传统的用户-项目表征转换为用户-兴趣表征;构建一种应用于用户兴趣预测的深度森林模型,引入兴趣簇重要性作为特征选择权重,融合时间注意力机制进行兴趣预测,将用户-兴趣模型结合基于用户的协同过滤算法预测推荐结果.两个数据集上的实验结果表明,该算法能够提高用户兴趣预测准确率,提升推荐效果.     

基于注意力机制优化EDM机场客流量预测

针对标准编码解码模型(Encoder-Decoder Model,EDM)对于时间序列数据提取能力弱的问题,提出一种融合双向长短时记忆网络(Bi-directional Long Short-Term Memory,Bi-LSTM)和注意力机制(Attention)的编码解码模型.通过Bi-LSTM对输入数据从正反两个方向进行特征提取,基于注意力机制将所得到的特征根据不同时刻分配不同权重,根据解码阶段的不同时刻生成相应背景变量,进而实现对机场客流量的预测.选取上海虹桥机场为例用该算法进行实验仿真,实验结果表明,本文所提方法与RNN、LSTM相比,平均标准误差降低了57.9％以上,为机场客流量预测提供了一种新的思路.     

基于高效通道注意力机制和特征融合网络的冠心病诊断算法研究

针对冠心病重要特征不确定、诊断模型预测性能低等因素而导致冠心病早期诊断精度低的问题,提出一种基于高效通道注意力机制和特征融合的网络。通过XGBoost(eXtreme Gradient Boosting)来确定冠心病重要特征,设计数据生成图片的特征组合算法以适用该模型;为提高诊断模型预测性能,采用可以提升模型学习能力和特征利用率的高效通道注意力机制模块和特征融合模块。实验结果表明,在UCI克利夫兰心脏病数据集上,与其他诊断算法相比,该算法优于传统机器学习方法,预测精度可达100%且稳定性好。     

基于深度学习的伺服系统状态预测算法

针对传统状态预测方法难以从伺服系统历史数据中有效提取特征的问题,提出一种基于深度学习的伺服系统状态预测算法。该算法利用长短时记忆网络LSTM(Long Short-Term Memory)从时序和特征参数两个维度在系统状态参数中提取数据特征。并在多任务学习MTL(Multi-task Learning)框架下将具有相同特征参数的预测任务整合到同一个模型当中,所有预测任务共享LSTM网络权重。在每一状态参数预测阶段,独立地引入注意力机制,以调节不同时刻、不同特征对所预测状态的影响。针对应用中预测参数的重要性不同,构建加权损失函数,以减小重要参数的预测误差。实验结果表明,该算法与传统LSTM模型、单任务模型STL-LSTM相比,预测误差平均降低40.9%、19.8%。     

基于双重注意力机制的异步优势行动者评论家算法

深度强化学习是目前机器学习领域发展最快的技术之一.传统的深度强化学习方法在处理高维度大状态的空间任务时,庞大的计算量导致其训练时间过长.虽然异步深度强化学习利用异步方法极大缩短了训练时间,但会忽略某些更具价值的图像区域和图像特征.针对上述问题,本文提出了一种基于双重注意力机制的异步优势行动者评论家算法.新算法利用特征注意力机制和视觉注意力机制来改进传统的异步深度强化学习模型.其中,特征注意力机制为卷积神经网络卷积后的所有特征图设置不同的权重,使得智能体聚焦于重要的图像特征;同时,视觉注意力机制为图像不同区域设置权重参数,权重高的区域表示该区域信息对智能体后续的策略学习有重要价值,帮助智能体更高效地学习到最优策略.新算法引入双重注意力机制,从表层和深层两个角度对图像进行编码表征,帮助智能体将聚焦点集中在重要的图像区域和图像特征上.最后,通过Atari 2600部分经典实验验证了基于双重注意力机制的异步优势行动者评论家算法的有效性.     

基于优化混合模型的航空发动机剩余寿命预测方法

针对航空发动机剩余使用寿命（RUL）预测方法没有同时加权不同时间步下的数据,包括原始数据和所提取的特征,导致RUL预测准确性较低的问题,提出了一种基于优化混合模型的RUL预测方法。首先,选用三种不同的路径提取特征：1）将原始数据的均值和趋势系数输入至全连接网络;2）将原始数据输入双向长短期记忆（Bi-LSTM）网络,并采用注意力机制处理得到的特征;3）使用注意力机制处理原始数据,并将加权特征输入至卷积神经网络（CNN）和Bi-LSTM网络中。然后,采用融合多路径特征预测的思想,将上述提取到的特征融合后输入至全连接网络获得RUL预测结果。最后,使用商用模块化航空推进系统仿真（C-MAPSS）数据集验证方法的有效性。实验结果显示,所提方法在4个数据集上均有较好的表现。以FD001数据集为例,所提方法的均方根误差（RMSE）比Bi-LSTM网络降低了9.01%。     

A Risk-Averse Remaining Useful Life Estimation for Predictive Maintenance

Remaining useful life (RUL) prediction is an advanced technique for system maintenance scheduling. Most of existing RUL prediction methods are only interested in the precision of RUL estimation; the adverse impact of over-estimated RUL on maintenance scheduling is not of concern. In this work, an RUL estimation method with risk-averse adaptation is developed which can reduce the over-estimation rate while maintaining a reasonable under-estimation level. The proposed method includes a module of degradation feature selection to obtain crucial features which reflect system degradation trends. Then, the latent structure between the degradation features and the RUL labels is modeled by a support vector regression (SVR) model and a long short-term memory (LSTM) network, respectively. To enhance the prediction robustness and increase its marginal utility, the SVR model and the LSTM model are integrated to generate a hybrid model via three connection parameters. By designing a cost function with penalty mechanism, the three parameters are determined using a modified grey wolf optimization algorithm. In addition, a cost metric is proposed to measure the benefit of such a risk-averse predictive maintenance method. Verification is done using an aero-engine data set from NASA. The results show the feasibility and effectiveness of the proposed RUL estimation method and the predictive maintenance strategy.      

Remaining useful life prediction of bearing based on stacked autoencoder and recurrent neural network

Remaining Useful Life (RUL) prediction play a crucial part in bearing maintenance, which directly affects the production efficiency and safety of equipment. Moreover, the accuracy of the prediction model is constrained by the feature extraction process and full life data of bearings. In this paper, the life prediction method of faulty rolling bearing with limited data is presented including degradation state model and RUL prediction model. In order to obtain health indication (HI) without human interference in the degradation state modeling stage, the bottleneck structure of Stacked Autoencoder (SAE) is utilized to fuse the four selected features into one HI using Intelligent Maintenance Systems (IMS) bearing dataset as training sample. In RUL prediction model, the Long Short-Term Memory (LSTM) neural network is carried out to establish the model with Standard deviation (Std) input and HI training label. In order to solve the problem of large training error caused by insufficient data in the failure stage of bearing acceleration test, the third-order spline curve interpolation is utilized to enhance the data points. Through parameter analysis, the RMSE and MAE of the test set on the prediction model are 0.032582 and 0.024038, respectively. Furthermore, the effectiveness of the proposed method is further validated by dataset from Case Western Reserve University (CWRU) with different bearing fault degrees. The analysis indicates that the RUL prediction of bearing fault data is consistent with the size of artificial added faults, that is,the more severe the fault the shorter the time of remaining life. The results validate that the proposed method can effectively extract the bearing health state by incorporating feature fusion and establish accurately prediction model for bearing remaining life.     

加速冲击损伤退化系统剩余寿命预测及预测维修决策

随着检测传感技术的发展,诸如风力发电机叶片等可对其状态进行检测,并依据检测结果进行剩余寿命预测.但此类系统在运行中受环境冲击影响较大,如何对冲击影响下的系统剩余寿命进行预测,并结合预测结果进行经济可靠的维修决策是一个值得研究的问题.对此,针对状态可检测的连续退化系统,研究考虑加速冲击损伤特性下的系统剩余寿命预测及基于预测的维修决策.首先,考虑自然退化和与退化相关的冲击损伤,构建加速冲击损伤退化模型和剩余寿命预测模型;其次,制定基于周期检测的状态维修与预测维修相结合的混合维修策略,并推导不同维修活动的发生概率;然后,构建以长期平均费用率最小为目标,以检测间隔和故障率阈值为决策变量的决策模型,并给出了优化解法;最后,以风力发电机叶片为案例验证模型的适用性和有效性,对系统的参数进行灵敏度分析,并与未考虑加速冲击损伤和未考虑预测的维修决策结果进行对比分析.     

Remaining useful life prediction of bearings by a new reinforced memory GRU network

The remaining useful life (RUL) prediction of bearings has great significance in the predictive maintenance of mechanical equipment. Owing to the difficulty of collecting abundant lifecycle datasets with correct labels, it is quite necessary to explore a prediction method with high precision and robustness in the case of small samples. It follows that a novel RUL prediction approach is put forward to overcome this problem. First, for reducing the man-made interference and the demand for expert knowledge, an unsupervised health indicator (HI) is constructed by Gaussian mixture model (GMM) and Kullback-Leibler divergence (KLD), which is named as KLD-based HI. Then because of the rapid forgetting of historical trend information in the current RNN-based prediction models, a novel reinforced memory gated recurrent unit (RMGRU) network is proposed by reusing the state information at the previous moment. According to the constructed KLD-based HI vector, the unknown HIs are successively predicted by RMGRU until the predicted HI value exceeds the failure threshold, and then RUL is calculated. The contrast experiment on IEEE 2012PHM bearing datasets shows the superiority of the bearing RUL prediction approach based on RMGRU over the classical time series forecasting methods. It can be concluded that this method has great application potential in bearing RUL prediction.     

Combining empirical mode decomposition and deep recurrent neural networks for predictive maintenance of lithium-ion battery

Predictive maintenance of lithium-ion batteries has been one of the popular research subjects in recent years. Lithium-ion batteries can be used as the energy supply for industrial equipment, such as automated guided vehicles and battery electric vehicles. Predictive maintenance plays an important role in the application of smart manufacturing. This mechanism can provide different levels of pre-diagnosis for machines or components. Remaining useful life (RUL) prediction is crucial for the implementation of predictive maintenance strategies. RUL refers to the estimated useful life remaining before the machine cannot operate after a certain period of operation. This study develops a hybrid data science model based on empirical mode decomposition (EMD), grey relational analysis (GRA), and deep recurrent neural networks (RNN) for the RUL prediction of lithium-ion batteries. The EMD and GRA methods are first adopted to extract the characteristics of time series data. Then, various deep RNNs, including vanilla RNN, gated recurrent unit, long short-term memory network (LSTM), and bidirectional LSTM, are established to forecast state of health (SOH) and the RUL of lithium-ion batteries. Bayesian optimization is also used to find the best hyperparameters of deep RNNs. Experimental results with the lithium-ion batteries data of NASA Ames Prognostics Data Repository show that the proposed hybrid data science model can accurately predict the SOH and RUL of lithium-ion batteries. The LSTM network has the optimal results. The proposed hybrid data science model with multiple artificial intelligence-based technologies also demonstrates critical digital-technology enablers for digital transformation of smart manufacturing and transportation.     

Degradation prediction and rolling predictive maintenance policy for multi-sensor systems based on two-dimensional self-attention

Traditional preventive maintenance policy gradually failed to guarantee the security and economy of current mechanical systems. This paper proposed a highly efficient rolling predictive maintenance (RPdM) policy for multi-sensor system, to make maintenance decisions. In this policy, to cope with the uncertainty of remaining useful life (RUL) prediction, the degradation process of the system is first divided into four intervals according to the inspection interval and spare parts lead time. Then, the two-dimensional self-attention (TDSA) method, which extract time dimensional and feature dimensional features by parallel computation, is developed to predict the probabilities of system RUL in the four intervals instead of the point of RUL. In addition, the output probabilities of the TDSA method are utilized to calculate the maintenance cost rate of the current inspection point and future point. The maintenance decision including spare parts ordering time and maintenance time is determined by comparing the maintenance cost rate of each inspection point, and the decision is updated at the next inspection point. To verify the effectiveness of the proposed RPdM policy, the C-MAPSS dataset provided by NASA is employed to implement degradation prediction and maintenance decision. Experiment results show that the cost rate of RPdM policy is lower than preventive maintenance policy, and only 27.7% higher than ideal maintenance policy which is impossible in real engineering. Moreover, the impact of different out-of-stock costs and corrective costs are explored and shows the good robustness of the RPdM policy.     

Multi-task spatio-temporal augmented net for industry equipment remaining useful life prediction

Accurate estimating the machine health indicator is an essential part of industrial intelligence. Despite having considerable progress, remaining useful life (RUL) prediction based on deep learning still confronts the following two challenges. Firstly, the length of condition monitoring data obtained from sensors is inconsistent, and the existing fixed window data processing method cannot fully adapt to all individual samples. Secondly, it is challenging to extract local and global features for long-series prediction tasks. To address these issues, this paper proposes a Multi-task Spatio-Temporal Augmented Net(MTSTAN) for industrial RUL prediction, which enhances the local features of different sensors data through channel attention mechanism, and proposes a skip connected causal augmented convolution network to enhance the global feature extraction in time series. For the industrial scenario of inconsistent data lengths, a multi-window multi-task sharing mechanism is set up to capture various time dependencies among different time scales. The robustness and universality of the model are increased by sharing information among tasks and multi-task window mechanism. Finally, a large number of experiments were carried out on the turbofan aircraft engine run-to-failure prognostic benchmark dataset (C-MAPSS) to evaluate the proposed model, and compared with the existing 14 state-of-the-art approaches. The results show that the enhancement of local and global time series features can effectively improve the prediction accuracy. The Multi-task learning strategy has excellent applicability in dealing with the problem of inconsistent data length.     

基于并联CNN-SE-Bi-LSTM的轴承剩余使用寿命预测

滚动轴承作为一种机械标准件,广泛应用于各类旋转机械设备,其健康状况对机器设备的正常运行至关重要,掌握其剩余使用寿命(RUL)可以更好地保证生产活动安全有效的进行.针对目前基于深度学习的机器RUL预测方法普遍存在:a)预测性能很大程度依赖手工特征设计;b)模型不能够充分提取数据中的有用特征;c)学习过程中没有明确考虑多传感器数据等缺点,提出了一种新的深度预测网络——并联多个带有压缩激励机制的卷积神经网络和双向长短期记忆网络集成网络(CNN-SE-Bi-LSTM),用于设备的RUL预测.在该预测网络中,不同传感器采集的监测数据直接作为预测网络的输入.然后,在改进的压缩激励卷积网络(CNN-SE-Net)提取空间特征的基础上进一步通过双向长短期记忆网络(Bi-LSTM)提取时序特征,建立起多个独立的可以自动从输入数据中学习高级表示的RU L预测模型分支.最后,将各独立分支学习到的特征通过全连接层并联获得最终的RU L预测模型.通过滚动轴承加速退化实验的数据,验证了所提网络的有效性并与现有的一些改进算法进行了对比实验.结果表明,面对原始多传感器数据,该算法能够自适应地提供准确的RU L预测结果,且预测表现优于现有一些预测方法.     

基于LSTM网络的移动云计算多元负载预测模型

针对移动云主机负载变化大、难以精准预测的问题,提出一种联合特征选择下基于长短期记忆网络的AR-LSTM-ED负载预测模型,能够对云主机负载进行单步和长时间多步预测。首先采用联合特征选择的方法得到与目标预测负载序列相关的其他负载序列,并且利用适用于在线预测的无抽取的小波变换方法将目标预测特征分解成更加易于预测的子序列。最后将这些序列和目标预测序列一起输入AR-LSTM-ED模型中,AR-LSTM-ED模型利用长短期记忆编-解码网络对目标负载进行预测,具有能够捕捉负载中的长期依赖关系的优点,且进一步结合了自回归模型（AR）以预测负载中的线性数据。在真实的Google云计算数据集上验证算法,对比实验结果表明,本文提出的方法取得了更好的性能。