FD-LSTM:基于大规模系统日志的故障分析模型

可靠性研究是高性能计算领域的经典问题,随着制程技术与集成工艺的不断发展,当前全系统规模呈指数级快速增长,给可靠性研究尤其是故障分析带来巨大挑战.收集了自主高性能计算系统投产后工作故障日志信息203510247条,时间自2016年1月28日至2016年12月6日.首先使用K-M eans聚类方法对故障进行分类,并分析故障...

FD-LSTM: A fault analysis model based on large-scale system logs

Reliability research is a classic problem in the field of high-performance computing. With the continuous development of process technology and integrated technology, the current scale of the entire system has grown exponentially, which has brought great challenges to reliability research, especially failure analysis. This paper collects 203510247 pieces of work failure log information after the operation of the independent high-performance computing system, from January 28, 2016 to December 6, 2016. Firstly, the K-Means clustering method is used to classify the faults and analyze the fault distribution characteristics. Secondly, based on the clustering results, a time-based fault analysis model FD-LSTM is designed. After training with structured logs, the occurrence time and space of different fault types are predicted. The results show that the accuracy of the proposed FD-LSTM prediction model can reach 80.56%. The research in this paper shows that, compared with the traditional fault analysis mo- del, in terms of time prediction and spatial prediction, the time series model FD-LSTM based on log information have practical guiding significance in improving the accuracy of fault analysis, enhancing the efficiency of machine operation and maintenance, improving the rationalization of collaborative whole system design, and other aspects.