ML-Parser:一种高效的在线日志解析方法

各种Web服务器和大数据框架每天都会生成大量日志,在服务管理中,会将原始日志转换为结构化格式,然后应用数据挖掘模型来分析服务状态,其中最为关键的步骤之一是日志解析.细粒度的解析和LCS可以提供更好的日志解析质量,而粗粒度的解析和简单的相似性度量可以达到更好的解析性能.对此,提出一个基于两层框架的在线日志解析方法(ML-...

ML-PARSER:AN EFFICIENT AND ACCURATE ONLINE LOG PARSER

Various Web servers and big data frameworks are widely used,so vast amounts of logs are generated every day.In service management,the raw logs are transformed into the structured format,and then data mining models are applied to analyze the status of the service.One of the most critical steps is log parsing.Fine-grained parsing and LCS lead to better parsing quality,while coarse-grained parsing and simple similarity measure can achieve better parsing performance.This paper proposes a ML-Parser based on two-layer framework,which can obtain better parsing quality without sacrificing performance.The experimental results show that this method can effectively improve the quality of log parsing,and its performance can meet the need of large-scale applications.