一种分布式的僵尸网络实时检测算法

僵尸网络通过控制的主机实现多类恶意行为,使得当前的检测方法失效,其中窃取敏感数据已经成为主流。鉴于僵尸网络实现的恶意行为,检测和减轻方法的研究已经势在必行。提出了一种新颖的分布式实时僵尸网络检测方法,该方法通过将Netflow组织成主机Netflow图谱和主机关系链,并提取隐含的C&C通信特征来检测僵尸网络。同时,基于Spark Streaming分布式实时流处理引擎,使用该算法实现了BotScanner分布式检测系统。为了验证该系统的有效性,采用5个主流的僵尸网络家族进行训练,并分别使用模拟网络流量和真实网络流量进行测试。实验结果表明,在无需深度包解析的情况下,BotScanner分布式检测系统能够实时检测指定的僵尸网络,并获得了较高的检测率和较低的误报率。而且,在真实的网络环境中,BotScanner分布式检测系统能够进行实时检测,加速比接近线性,验证了Spark Streaming引擎在分布式流处理方面的优势,以及用于僵尸网络检测方面的可行性。

Distributed Real-time Botnet Detection Algorithm

Compared with other types of malware,botnets have recently been adopted by hackers for their resiliency against take-down efforts.Besides being harder to take down,modern botnets tend to be stealthier in the way they perform malicious activities by using the infected computer,making current detection approaches ineffective.Given the malicious activities botnets can realize,detection and mitigation of botnet threats are imperative.In this paper,we presented a novel approach for botnet detection,called distributed real-time botnet detection algorithm.It uses Spark engine,where Netflow related data are correlated as the host Netflow graph structure and the host access chain structure,and a feature extraction method based on the Spark Streaming is leveraged for exacting implicit characteristics.Meanwhile,this paper established distributed BotScanner detection system based on the Spark Streaming,which is a distributed real-time steam processing engine.We trained BotScanner system on the five representative bot families and evaluated BotScanner on simulated network traffic and real-world network traffic.The experimental results show that the BotScanner is able to detect bots in network traffic without the need of deep packet inspection,and achieves high detection rates with very few false positives.When the traffic data from the Internet service provider are very large,the BotScanner is able to detect botnets in real-time by adding the compute nodes,and BotScanner has approximate linear speedup.It proves the feasibility of Applying Spark Streaming engine to distributed botnet detection.