基于归一化特征判别的日志模板挖掘算法

针对传统日志模板挖掘时需要日志聚类数目作为先验信息的问题，提出了一种基于归一化特征判别的日志模板挖掘算法.首先，对日志数据进行压缩，以提高后续处理效率；其次，进行日志聚类过程，使用归一化的日志统计特征判断聚类是否满足要求，若满足，则聚类成功；若不满足，则采用二分搜索的方式调整日志聚类的数目，重新进行聚类；最后，从聚类结果中提取日志模板，设计了一种衡量模板挖掘效果的评价指标.在真实数据集上的实验结果表明，算法的模板挖掘匹配度优于基准方法，并且具有良好的泛化性能.

Log Template Extraction Algorithm Based on Normalized Feature Discrimination

A log template extraction algorithm based on normalized feature discrimination is proposed, aiming at the problem that the number of clusters needs to be provided as a priori information in traditional log template extraction. First, log data is initially compressed to reduce data redundancy. Then, a log clustering process is implemented, and the normalized feature is used to discriminate whether the clustering result meets requirement:if so, the clustering process is successfully completed; if not, the number of log clusters is adjusted by using binary search and redo clustering. Finally, the log template is extracted via clustering results. In addition, an evaluation metric that measures the effectiveness of template extraction is designed. Experiments on real data indicated that the algorithm can achieve more stable and accurate template extraction performance than the benchmark method, and it had good generalization performance.