基于数据流方法的大规模网络异常发现

随着网络规模和速度的增加,大规模网络异常发现要求检测算法能够在无保留状态或者少保留状态下对G比特级的海量网络业务量数据进行实时在线分析。针对在高速骨干网上进行大规模网络异常发现的特点和要求,提出了一种基于数据流的大规模网络异常发现的方法,第一次将数据流模型用于大规模网络的异常发现。主要包括以下创新点:设计了一种面向异常发现的网络流量概要数据结构和突发高频事件检测算法;提出了一种基于安全监测策略定制的预查询方法来进行多数据流的关联监测并且对数据流查询进行了优化;在真实数据分析的基础上,对网络业务量进行了数据约减,使得监测部分特殊类型的数据流能最大程度地获得整体网络业务量的变化特征以提高异常发现的效率。通过真实网络环境下的实验和性能评价验证了数据流方法的有效性。

Anomaly detection of large scale network based on data streams

The anomaly detection algorithms of the large scale network(LSN) were required to analysis the vast network traffic of G bit level in real-time and on-the-fly.A novel monitoring mechanism of LSN anomaly detection based on the data stream approach was proposed.The main contributions included: the sketch data structure and the frequent sketch algorithm of data streams were designed for anomaly detection of LSN.Optimized query methods were designed for customizing the security monitoring and detection policy with the correlations of multi data streams.The data reduction was proposed to make it possible that the whole network traffic character could be got using a few of special data streams.The experiments of the real networking environments validate the effectivity of LSN anomaly detection methods.