基于Isolation Forest的并行化异常探测设计

异常探测具有广泛的应用,受到了工业界和学术界的共同关注。在众多异常探测方法中,Isolation Forest算法具有执行效率高、探测准确度好的特点,获得了众多应用。但是,传统Isolation Forest算法难以处理大规模数据。为解决此问题,设计了一种基于云计算平台的算法。具体地,使用Hadoop分布式存储系统和MapReduce分布式计算框架设计并实现了基于Isolation Forest的并行化异常探测算法PIFH。通过将探测模型构建和数据异常评价的过程并行化,提升了PIFH算法探测异常的执行效率,扩展了其应用范围。利用真实世界数据集验证了所提算法的执行效率和可扩展性。

Parallel anomaly detection based on Isolation Forest

Anomaly detection, which is used in a variety of applications, attracts attention both in industry and academia. Among numerous methods for anomaly detection, the Isolation Forest algorithm, whose characteristics include high efficiency, sound detection accuracy, has wide real world applications. However, the conventional Isolation forest algorithm can hardly deal with large scale data sets. To break this limitation, we propose a cloud computing platform based algorithm. Specifically, we design and implement a parallel algorithm for anomaly detection based on Isolation Forest, named PIFH,using the Hadoop distributed storage system and the MapReduce distributed computational framework. By parallelizing the processes of detection model construction and anomaly evaluation, its efficiency is improved, and the application range is also extended. Experiments using real world data sets demonstrate that the proposed algorithm is efficient and scalable.