Transformer-based hierarchical latent space VAE for interpretable remaining useful life prediction

Data-driven prediction of remaining useful life (RUL) has emerged as one of the most sought-after research in prognostics and health management (PHM). Nevertheless, most RUL prediction methods based on deep learning are black-box models that lack a visual interpretation to understand the RUL degradation process. To remedy the deficiency, we propose an intrinsically interpretable RUL prediction method based on three main modules: a temporal fusion separable convolutional network (TF-SCN), a hierarchical latent space variational auto-encoder (HLS-VAE), and a regressor. TF-SCN is used to extract the local feature information of the temporal signal. HLS-VAE is based on a transformer backbone that mines long-term temporal dependencies and compresses features into a hierarchical latent space. To enhance the streaming representation of the latent space, the temporal degradation information, i.e., health indicators (HI), is incorporated into the latent space in the form of inductive bias by using intermediate latent variables. The latent space can be used as a visual representation with self-interpretation to evaluate RUL degradation patterns visually. Experiments based on turbine engines show that the proposed approach achieves the same high-quality RUL prediction as black-box models while providing a latent space in which degradation rate can be captured to provide the interpretable evaluation.     

A multi-head neural network with unsymmetrical constraints for remaining useful life prediction

This paper proposes a multi-head neural network (MHNN) model with unsymmetrical constraints for remaining useful life (RUL) prediction of industrial equipment. Generally, the existing deep learning methods proposed for RUL prediction utilize symmetrical constraint loss functions such as the mean squared error function to calculate training errors. However, if the predicted RUL is much larger than the actual value in some safety–critical applications, severe damage may occur. To address this issue, an unsymmetrical constraint function is proposed as the loss function in this work that penalizes the late predictions (i.e., the predicted RUL is larger than the actual RUL) more strongly. In addition, an adjustable parameter is added to this function to adjust the model’s attention to the late predictions. In MHNN model, the bidirectional gated recurrent units (BGRU) and self-attention mechanism are employed to extract temporal features from the condition monitoring data. In addition, the structure of the multi-head neural network is adopted in the proposed model, helping to capture more degradation information by means of multiple identical and parallel networks. The proposed method is validated against a commonly used turbofan engine dataset. Compared with other latest methods on the same dataset, the proposed method is proven to be superior. Taking the FD004 dataset as an example, the score obtained by MHNN is 24.09% lower than that obtained by the best existing method.     

Hybrid scheme through read-first-LSTM encoder-decoder and broad learning system for bearings degradation monitoring and remaining useful life estimation

This paper proposes a novel hybrid scheme through read-first-LSTM (RLSTM) encoder-decoder and broad learning system (BLS) for bearings degradation monitoring and remaining useful life (RUL) estimation, which aims to describe the nonlinear characteristics of the degradation process. Firstly, the raw signals are processed premier by complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and a novel dimensionality reduction method composed of t-distribution stochastic neighbor embedding (t-SNE) and density-based spatial clustering of application with noise algorithm (DBSCAN). Then, the health indicator is constructed with the Hilbert-Huang transform (HHT) corresponding to the bearings’ natural fault frequency, which can be employed as the hybrid scheme training label. Linear rectification technology (LRT) and exponentially weighted moving average (EWMA) control chart are adapted to define the exact process of the degradation. Secondly, a novel RLSTM is proposed. And simultaneously, an encoder-decoder model, where RLSTM is utilized as an encoder, and LSTM is adopted as a decoder, is designed for degradation monitoring. Finally, a broad learning system (BLS), which differs from deep learning with a deeper structure, is established in a flat network to estimate the RUL of bearings. Compared with the state-of-the-art techniques, the better efficacy of the proposed hybrid scheme is illustrated using the PRONOSTIA platform dataset.     

Multi-task spatio-temporal augmented net for industry equipment remaining useful life prediction

Accurate estimating the machine health indicator is an essential part of industrial intelligence. Despite having considerable progress, remaining useful life (RUL) prediction based on deep learning still confronts the following two challenges. Firstly, the length of condition monitoring data obtained from sensors is inconsistent, and the existing fixed window data processing method cannot fully adapt to all individual samples. Secondly, it is challenging to extract local and global features for long-series prediction tasks. To address these issues, this paper proposes a Multi-task Spatio-Temporal Augmented Net(MTSTAN) for industrial RUL prediction, which enhances the local features of different sensors data through channel attention mechanism, and proposes a skip connected causal augmented convolution network to enhance the global feature extraction in time series. For the industrial scenario of inconsistent data lengths, a multi-window multi-task sharing mechanism is set up to capture various time dependencies among different time scales. The robustness and universality of the model are increased by sharing information among tasks and multi-task window mechanism. Finally, a large number of experiments were carried out on the turbofan aircraft engine run-to-failure prognostic benchmark dataset (C-MAPSS) to evaluate the proposed model, and compared with the existing 14 state-of-the-art approaches. The results show that the enhancement of local and global time series features can effectively improve the prediction accuracy. The Multi-task learning strategy has excellent applicability in dealing with the problem of inconsistent data length.     

Utilizing uncertainty information in remaining useful life estimation via Bayesian neural networks and Hamiltonian Monte Carlo

The estimation of remaining useful life (RUL) of machinery is a major task in prognostics and health management (PHM). Recently, prognostic performance has been enhanced significantly due to the application of deep learning (DL) models. However, only few authors assess the uncertainty of the applied DL models and therefore can state how certain the model is about the predicted RUL values. This is especially critical in applications, in which unplanned failures lead to high costs or even to human harm. Therefore, the determination of the uncertainty associated with the RUL estimate is important for the applicability of DL models in practice. In this article, Bayesian DL models, that naturally quantify uncertainty, were applied to the task of RUL estimation of simulated turbo fan engines. Inference is carried out via Hamiltonian Monte Carlo (HMC) and variational inference (VI). The experiments show, that the performance of Bayesian DL models is similar and in many cases even beneficial compared to classical DL models. Furthermore, an approach for utilizing the uncertainty information generated by Bayesian DL models is presented. The approach was applied and showed how to further enhance the predictive performance.     

A new convolutional dual-channel Transformer network with time window concatenation for remaining useful life prediction of rolling bearings

Deep learning has achieved numerous breakthroughs in bearing predicting remaining useful life (RUL). However, the current mainstream deep learning framework inevitably has flaws, including the disadvantage of the small receptive field, the difficulty of learning long-term dependencies and the singularity of feature extraction domains, etc. Given the challenges mentioned above, we propose a new convolutional dual-channel Transformer network (CDCT) for remaining useful life prediction of rolling bearings. In the CDCT, the causal convolution operation is applied to extract local features from the time and frequency domains and add positional encoding to the input signal, while the transformer block is utilized for extracting bidirectional features and fusing them. The CDCT not only has a global receptive field but also can learn long-term dependencies regardless of sequence length. Besides, the time window concatenation is adopted to ameliorate the problem of large amounts of trainable parameters of the Transformer-based models. In the experiments, we conduct a detailed analysis of each crucial element and hyperparameter of the CDCT and compare it to multiple basic and advanced methods. The experimental results highlight the superiority of the CDCT in bearing RUL prediction and demonstrate the effectiveness of crucial elements in the CDCT.     

航空发动机的健康指标构建与剩余寿命预测

预测与健康管理技术能够有效的评估系统健康状态、预测系统剩余使用寿命,是提高复杂系统安全性、经济性的重要保障.为全面评估系统健康状态,本文提出了一种基于深度置信网络(DBN)的无监督健康指标构建方法,并结合隐马尔可夫模型(HMM)进行系统剩余寿命预测.首先,通过无监督训练深度置信网络实现历史数据的特征提取,进而构建健康指标;其次,利用健康指标集训练隐马尔可夫模型,实现设备健康状态的自动识别;最后,通过DBN-HMM混合模型来计算系统剩余寿命.采用商用模块化航空推进系统仿真软件(C-MAPSS)给出的航空发动机数据集,验证了上述方法的有效性.     

A novel deep learning scheme for multi-condition remaining useful life prediction of rolling element bearings

The remaining useful life (RUL) prediction of a rolling element bearing is important for more reasonable maintenance of machinery and equipment. Generally, the information of a failure can hardly be acquired in advance while running and the degradation process varies in terms of different faults. Thus, fault identification is indispensable for a multi-condition RUL prediction, where, however, the fault identification and RUL prediction are separated in most studies. A new hybrid scheme is proposed in this paper for the multi-condition RUL prediction of rolling element bearings. The proposed scheme contains both classification and regression, where the 2D-DCNN based classifier and predictors are built concerning typical fault conditions of a bearing. For the online prediction, the raw signals are spanned in the time-frequency domain and then transferred into images as the input of the scheme. The classifier is used to monitor the vibration of rolling bearings for online fault recognition and excite the corresponding predictor for RUL prediction once a fault is detected. The output from the predictor is amended by the proposed adaptive delay correction method as the final prediction results. A demonstration is performed based on the XJTU-SY datasets and the results are compared with those from the state-of-the-art methods, which proves the superiority of the proposed scheme in improving the accuracy and linearity of RUL prediction. The time cost of the proposed online prediction scheme is also investigated and the results indicate high time effectiveness.     

Remaining useful life prediction of bearing based on stacked autoencoder and recurrent neural network

Remaining Useful Life (RUL) prediction play a crucial part in bearing maintenance, which directly affects the production efficiency and safety of equipment. Moreover, the accuracy of the prediction model is constrained by the feature extraction process and full life data of bearings. In this paper, the life prediction method of faulty rolling bearing with limited data is presented including degradation state model and RUL prediction model. In order to obtain health indication (HI) without human interference in the degradation state modeling stage, the bottleneck structure of Stacked Autoencoder (SAE) is utilized to fuse the four selected features into one HI using Intelligent Maintenance Systems (IMS) bearing dataset as training sample. In RUL prediction model, the Long Short-Term Memory (LSTM) neural network is carried out to establish the model with Standard deviation (Std) input and HI training label. In order to solve the problem of large training error caused by insufficient data in the failure stage of bearing acceleration test, the third-order spline curve interpolation is utilized to enhance the data points. Through parameter analysis, the RMSE and MAE of the test set on the prediction model are 0.032582 and 0.024038, respectively. Furthermore, the effectiveness of the proposed method is further validated by dataset from Case Western Reserve University (CWRU) with different bearing fault degrees. The analysis indicates that the RUL prediction of bearing fault data is consistent with the size of artificial added faults, that is,the more severe the fault the shorter the time of remaining life. The results validate that the proposed method can effectively extract the bearing health state by incorporating feature fusion and establish accurately prediction model for bearing remaining life.     

锂离子电池剩余寿命预测方法研究综述

电池故障预测和健康管理(Prediction and Health Management, PHM)评价的主要方法是确定电池的健康状态和剩余使用寿命(Remaining Useful Life, RUL),以此保证锂离子电池安全可靠地工作和实现寿命优化。锂电池RUL预测不仅是PHM中的热点问题和挑战问题,其预测方法的准确性也会直接影响电池管理系统(Battery Management System, BMS)的整体性能。介绍了单体电芯测评标准,对影响锂电池循环寿命的主要因素进行详细分析。简述电池日历寿命和循环寿命。概括和总结了近几年锂离子电池剩余寿命预测方法,比较不同方法的优缺点。提出了当前实际应用中预测锂电池RUL仍存在的关键问题并进行探讨。     

基于BA-WPHM的滚动轴承两阶段剩余寿命预测方法

为了提高滚动轴承剩余寿命预测的准确性,根据滚动轴承运行过程的两阶段性特点,提出了一种基于蝙蝠算法(BA)和威布尔比例风险模型(WPHM)的滚动轴承两阶段剩余寿命预测方法。首先,构建基于WPHM的剩余寿命预测模型;其次,提出了两阶段极大似然估计法,建立新的似然函数,并利用BA算法进行求解,以提高参数估计的准确性;最后,建立BA-WPHM模型对滚动轴承进行剩余寿命预测。案例分析表明,相比于Newton-Raphson算法、自组织分层猴群算法(SHMA)和独特的自适应粒子群算法(UAPSO),提出的方法参数估计的准确性更高,剩余寿命的预测精度优于支持向量回归(SVR)方法,验证了所提方法的有效性,为滚动轴承维修决策的可行性提供了依据。     

Remaining useful life prediction for degrading systems with random shocks considering measurement uncertainty

Degradation data have been widely used for the remaining useful life (RUL) prediction of systems. Most existing works apply a preset model to capture the degradation process and focus on the degradation process without shocks or constant shock effects. More generally, the actual degradation path is unobservable due to the existence of measurement uncertainty, which interferes with the determination of the degradation model. Besides, the effect of random shocks is usually fluctuating. Given these problems, a general degradation model with the random shock fluctuant effects considering the measurement uncertainty is first developed to describe the degradation process, and a two-step approach combining the arithmetic average filter and the Bayesian information criterion is adopted to identify the degradation path. Subsequently, the transfer processes of the actual degradation state and the abrupt change caused by shocks are depicted using a two-dimensional state-space model, and an expectation-maximization algorithm combined with the particle filtering is developed for parameter estimation. Furthermore, the explicit solution of RUL distribution is obtained when only considering harmful shocks, while a simulation method of RUL distribution is provided when both harmful and beneficial shocks exist. Finally, the effectiveness of the proposed method is verified by a numerical example and two practical case studies.     

基于并联CNN-SE-Bi-LSTM的轴承剩余使用寿命预测

滚动轴承作为一种机械标准件,广泛应用于各类旋转机械设备,其健康状况对机器设备的正常运行至关重要,掌握其剩余使用寿命(RUL)可以更好地保证生产活动安全有效的进行.针对目前基于深度学习的机器RUL预测方法普遍存在:a)预测性能很大程度依赖手工特征设计;b)模型不能够充分提取数据中的有用特征;c)学习过程中没有明确考虑多传感器数据等缺点,提出了一种新的深度预测网络——并联多个带有压缩激励机制的卷积神经网络和双向长短期记忆网络集成网络(CNN-SE-Bi-LSTM),用于设备的RUL预测.在该预测网络中,不同传感器采集的监测数据直接作为预测网络的输入.然后,在改进的压缩激励卷积网络(CNN-SE-Net)提取空间特征的基础上进一步通过双向长短期记忆网络(Bi-LSTM)提取时序特征,建立起多个独立的可以自动从输入数据中学习高级表示的RU L预测模型分支.最后,将各独立分支学习到的特征通过全连接层并联获得最终的RU L预测模型.通过滚动轴承加速退化实验的数据,验证了所提网络的有效性并与现有的一些改进算法进行了对比实验.结果表明,面对原始多传感器数据,该算法能够自适应地提供准确的RU L预测结果,且预测表现优于现有一些预测方法.     

基于多注意力和变分编码时序网络的发动机剩余使用寿命预测方法

准确可靠的剩余使用寿命(RUL)预测结果可为决策者提供有价值的信息,以采取适当的维护策略,最大限度地利用设备,避免昂贵的故障维修费用。为了从高噪声的真实工况中对发动机故障进行有效诊断,提出了一种融合多注意力机制和变分编码的时序预测模型(MA-VBLSTM),首先通过嵌入多注意力机制获得所有特征在空间维度和通道维度的不同权重,以提高退化特征的提取能力;然后,采用变分编码器进行退化信息编码并学习数据间深度隐藏的信息;最后,利用双向长短时记忆网络的长短期时序数据双向处理能力实现发动机RUL的预测。实验结果表明,该模型在发动机CMAPSS数据集的FD001、FD002、FD003、FD004子数据集上RMSE 和Score值相比现有方法分别平均降低5.27%和10.70%、1.37%和1.68%、6.37%和26.94%、3.02%和2.06%。     

A data-driven two-stage maintenance framework for degradation prediction in semiconductor manufacturing industries

To reduce the production costs and breakdown risks in industrial manufacturing systems, condition-based maintenance has been actively pursued for prediction of equipment degradation and optimization of maintenance schedules. In this paper, a two-stage maintenance framework using data-driven techniques under two training types will be developed to predict the degradation status in industrial applications. The proposed framework consists of three main blocks, namely, Primary Maintenance Block (PMB), Secondary Maintenance Block (SMB), and degradation status determination block. As the popular methods with deterministic training, back-propagation Neural Network (NN) and evolvable NN are employed in PMB for the degradation prediction. Another two data-driven methods with probabilistic training, namely, restricted Boltzmann machine and deep belief network are applied in SMB as the backup of PMB to model non-stationary processes with the complicated underlying characteristics. Finally, the multiple regression forecasting is adopted in both blocks to check prediction accuracies. The effectiveness of our proposed two-stage maintenance framework is testified with extensive computation and experimental studies on an industrial case of the wafer fabrication plant in semiconductor manufactories, achieving up to 74.1% in testing accuracies for equipment degradation prediction.     

Remaining useful life estimation for micro switches of railway vehicles

Micro switches are widely used in modern control systems, and the reliability of each micro switch may be of great significance to the whole system. Remaining useful life of micro switches is an essential index evaluating their reliability in operation, and the real-time estimation can prevent system failure in a more controllable manner. In this paper, Bayesian updating and expectation maximization are combined to achieve the remaining useful life estimation. Additionally, strong tracking filtering technique is employed to improve the adaptive update capability. The effectiveness of the method is illustrated from an experiment of a micro switch for rail vehicles.     

A novel diagnostic and prognostic framework for incipient fault detection and remaining service life prediction with application to industrial rotating machines

Data-driven machine health monitoring systems (MHMS) have been widely investigated and applied in the field of machine diagnostics and prognostics with the aim of realizing predictive maintenance. It involves using data to identify early warnings that indicate potential system malfunctioning, predict when system failure might occur, and pre-emptively service equipment to avoid unscheduled downtime. One of the most critical aspects of data-driven MHMS is the provision of incipient fault diagnosis and prognosis regarding the system’s future working conditions. In this work, a novel diagnostic and prognostic framework is proposed to detect incipient faults and estimate remaining service life (RSL) of rotating machinery. In the proposed framework, a novel canonical variate analysis (CVA)-based monitoring index, which takes into account the distinctions between past and future canonical variables, is employed for carrying out incipient fault diagnosis. By incorporating the exponentially weighted moving average (EWMA) technique, a novel fault identification approach based on Pearson correlation analysis is presented and utilized to identify the influential variables that are most likely associated with the fault. Moreover, an enhanced metabolism grey forecasting model (MGFM) approach is developed for RSL prediction. Particle filter (PF) is employed to modify the traditional grey forecasting model for improving its prediction performance. The enhanced MGFM approach is designed to address two generic issues namely dealing with scarce data and quantifying the uncertainty of RSL in a probabilistic form, which are often encountered in the prognostics of safety-critical and complex assets. The proposed CVA-based index is validated on slowly evolving faults in a continuous stirred tank reactor (CSTR) system, and the effectiveness of the proposed integrated diagnostic and prognostic method for the monitoring of rotating machinery is demonstrated for slow involving faults in two case studies of an operational industrial centrifugal pump and one case study of an operational centrifugal compressor.     

基于特征增强与时空信息嵌入的涡扇发动机剩余寿命预测

针对现有的剩余寿命预测方法对原始数据利用率不高以及多维数据特征提取能力不足的问题,提出了一种基于特征增强和时空信息嵌入的卷积神经模型。首先,通过特征增强模块在原始数据基础上进一步提取工况特征与手工特征作为辅助特征;其次,提出了时空嵌入模块,对原始数据进行时空信息编码以嵌入时间序列信息和空间特征信息;最后,拼接上述特征并通过回归预测模块捕获数据内在关系得到回归预测结果。在通用的涡扇发动机模拟数据集(C-MAPSS)上对该模型预测效果进行了测试。实验结果表明,与现有主流深度学习方法相比,该模型在四个子集上的均方根误差平均减少了8.8%,且在多工况的运行条件和故障类型下,其预测精度均优于现有先进算法,充分证明了该模型在涡扇发动机剩余使用寿命预测方面的有效性和准确性。     

基于可自动扩展的LSTM模型的航空发动机剩余寿命预测方法

对航空发动机进行实时状态监测和健康管理可以有效降低发动机故障风险,确保飞机飞行安全。准确预测航空发动机的剩余寿命是有效监测发动机运行状态的一种重要手段,其中长短期记忆（long-short term memory,LSTM）网络常被使用。但由于航空发动机复杂的机械结构与运行模式,使用传统的LSTM网络对航空发动机的剩余寿命进行单次预测后,所得预测结果的准确率不足以满足其寿命预测的精度要求。基于LSTM网络的广泛使用以及它对时间序列数据的有效预测能力,考虑到采用多级预测的方法能够有效降低预测误差,提出了一种新型的可自动扩展的长短期记忆（automatically expandable LSTM,AELSTM）预测模型。AELSTM模型依托多个子模块逐级连接的网络结构,不断地提取前一级模块的输出误差作为后一级模块的训练值,形成了误差的多级预测机制,有效降低了模型的预测误差,提升了预测结果的准确性。基于美国国家航空航天局发布的C-MAPSS数据集的四个子集对AELSTM模型的预测效果进行了测试,实验结果表明,与传统的LSTM网络相比,AELSTM模型在四个子集上的均方根误差平均减少了95.44%,同时它的预测效果也优于现有的一些先进算法。实验充分验证了AELSTM模型在提升航空发动机剩余寿命预测准确度方面的有效性及优势。     

基于TransMP模型的Web系统剩余寿命预测方法

针对当前软件剩余使用寿命预测方法忽略了多性能指标间所蕴涵寿命信息的问题,提出一种融合多性能指标Transformer（TransMP）模型的Web系统剩余寿命预测方法。首先,搭建内存故障型Web系统加速老化实验平台,创建包含内存使用量、响应时间和吞吐率性能指标的数据集;其次,考虑不同性能指标蕴涵老化特征信息的差异性,构造由多编码器-解码器组成的TransMP模型,将性能指标数据分别输入内存指标编码器、响应时间编码器和吞吐率编码器提取老化特征信息,再引入特征融合层进行信息融合;最后,将融合信息输入由掩码注意力-多头注意力结构构成的解码器,预测得到系统状态达到老化阈值的剩余寿命。实验结果表明,该Web系统剩余寿命预测方法与最优的SALSTM方法相比,均方根误差分别降低了12.0%、17.3%和13.2%,平均绝对误差分别降低了13.3%、21.0%和10.4%,证明了该方法的有效性。