Detecting P2P botnets by discovering flow dependency in C&C traffic

Botnets are widely used by attackers and they have evolved from centralized structures to distributed structures. Most of the modern P2P bots launch attacks in a stealthy way and the detection approaches based on the malicious traffic of bots are inefficient. In this paper, an approach that aims to detect Peer-to-Peer (P2P) botnets is proposed. Unlike previous works, the approach is independent of any malicious traffic generated by bots and does not require bots’ information provided by external systems. It detects P2P bots by focusing on the instinct characteristics of their Command and Control (C&C) communications, which are identified by discovering flow dependencies in C&C traffic. After discovering the flow dependencies, our approach distinguishes P2P bots and normal hosts by clustering technique. Experimental results on real-world network traces merged with synthetic P2P botnet traces indicate that 1) flow dependency can be used to detect P2P botnets, and 2) the proposed approach can detect P2P botnets with a high detection rate and a low false positive rate.     

Resource monitoring for the detection of parasite P2P botnets

Detecting botnet behaviors in networks is a popular topic in the current research literature. The problem of detection of P2P botnets has been denounced as one of the most difficult ones, and this is even sounder when botnets use existing P2P networks infrastructure (parasite P2P botnets). The majority of the detection proposals available at present are based on monitoring network traffic to determine the potential existence of command-and-control communications (C&C) between the bots and the botmaster. As a different and novel approach, this paper introduces a detection scheme which is based on modeling the evolution of the number of peers sharing a resource in a P2P network over time. This allows to detect abnormal behaviors associated to parasite P2P botnet resources in this kind of environments. We perform extensive experiments on Mainline network, from which promising detection results are obtained while patterns of parasite botnets are tentatively discovered.     

New facets of mobile botnet: architecture and evaluation

It is without a doubt that botnets pose a growing threat to the Internet, with DDoS attacks of any kind carried out by botnets to be on the rise. Nowadays, botmasters rely on advanced Command and Control (C&C) infrastructures to achieve their goals and most importantly to remain undetected. This work introduces two novel botnet architectures that consist only of mobile devices and evaluates both their impact in terms of DNS amplification and TCP flooding attacks, and their cost pertaining to the maintenance of the C&C channel. The first one puts forward the idea of using a continually changing mobile HTTP proxy in front of the botherder, while the other capitalizes on DNS protocol as a covert channel for coordinating the botnet. That is, for the latter, the messages exchanged among the bots and the herder appear as legitimate DNS transactions. Also, a third architecture is described and assessed, which is basically an optimized variation of the first one. Namely, it utilizes a mixed layout where all the attacking bots are mobile, but the proxy machines are typical PCs not involved in the actual attack. For the DNS amplification attack, which is by nature more powerful, we report an amplification factor that fluctuates between 32.7 and 34.1. Also, regarding the imposed C&C cost, we assert that it is minimal (about 0.25 Mbps) per bot in the worst case happening momentarily when the bot learns about the parameters of the attack.     

Scalable P2P bot detection system based on network data stream

P2P Botnets are one of the most malevolent threats to the Internet users due to their resiliency against takedown efforts. In this paper, we propose a bot detection system that is capable of detecting stealthy bots in a network. This system treats network traffic as a data stream, segregating the traffic into two parallel streams. The detection is based on failure traffic and communication traffic. The traffic is analyzed during small time window, and the infected hosts are reported immediately. The network administrator can monitor the status of hosts in the network and can take the necessary action before the infected hosts harm the system or can involve in the attacks. Experiments and evaluation of the proposed system on a variety of P2P data transfer applications and P2P botnets have demonstrated high accuracy of detection. The scalability of the proposed system is exhibited through its implementation on Hadoop MapReduce.     

双层架构的P2P Botnet研究

Botnet(僵尸网络)是由bot(僵尸主机)组成的可通信、可被攻击者控制的网络,而P2P botnet是一种利用P2P技术构建控制信道的僵尸网络.对比于以往具有明显追踪特征的P2P botnet而言,一种基于双层架构的P2P botnet在控制感染主机方面采用了更加隐蔽和灵活的方式,使追踪难度增大.通过模拟实验对基于这种通信控制模型的僵尸网络进行了功能和性能方面的研究,并提出了相应的防御与追踪方案.     

Automatic protocol reverse-engineering: Message format extraction and field semantics inference

Understanding the command-and-control (C&C) protocol used by a botnet is crucial for anticipating its repertoire of nefarious activity. However, the C&C protocols of botnets, similar to many other application layer protocols, are undocumented. Automatic protocol reverse-engineering techniques enable understanding undocumented protocols and are important for many security applications, including the analysis and defense against botnets. For example, they enable active botnet infiltration, where a security analyst rewrites messages sent and received by a bot in order to contain malicious activity and to provide the botmaster with an illusion of successful and unhampered operation.In this work, we propose a novel approach to automatic protocol reverse engineering based on dynamic program binary analysis. Compared to previous work that examines the network traffic, we leverage the availability of a program that implements the protocol. Our approach extracts more accurate and complete protocol information and enables the analysis of encrypted protocols. Our automatic protocol reverse-engineering techniques extract the message format and field semantics of protocol messages sent and received by an application that implements an unknown protocol specification. We implement our techniques into a tool called Dispatcher and use it to analyze the previously undocumented C&C protocol of MegaD, a spam botnet that at its peak produced one third of the spam on the Internet.     

Efficient botnet herding within the Tor network

During 2013 the Tor network had a massive spike in new users as a botnet started using Tor hidden services to hide its C&C (Command and Control) servers. This resulted in network congestion and reduced performance for all users. Tor hidden services are attractive to botnet herders because they provide anonymity for both the C&C servers and the bots. The aim of this paper is to present a superior way that Tor hidden services can be used for botnet C&C which minimises harm to the Tor network while retaining all security benefits.     

A novel method of mining network flow to detect P2P botnets

Botnets are a serious threat to cyber-security. As a consequence, botnet detection has become an important research topic in network protection and cyber-crime prevention. P2P botnets are one of the most malicious zombie networks, as their architecture imitates P2P software. Characteristics of P2P botnets include (1) the use of multiple controllers to avoid single-point failure; (2) the use of encryption to evade misuse detection technologies; and (3) the capacity to evade anomaly detection, usually by initiating numerous sessions without consuming substantial bandwidth. To overcome these difficulties, we propose a novel data mining method. First, we identify the differences between P2P botnet behavior and normal network behavior. Then, we use these differences to tune the data-mining parameters to cluster and distinguish normal Internet behavior from that lurking P2P botnets. This method can identify a P2P botnet without breaking the encryption. Furthermore, the detection system can be deployed without altering the existing network architecture, and it can detect the existence of botnets in a complex traffic mix before they attack. The experimental results reveal that the method is effective in recognizing the existence of botnets. Accordingly, the results of this study will be of value to information security academics and practitioners.     

P2P僵尸网络研究

P2P僵尸网络是一种新型网络攻击方式,因其稳定可靠、安全隐蔽的特性被越来越多地用于实施网络攻击,给网络安全带来严峻挑战.为深入理解P2P僵尸网络工作机理和发展趋势,促进检测技术研究,首先分析了P2P僵尸程序功能结构,然后对P2P僵尸网络结构进行了分类,并分析了各类网络结构的特点;在介绍了P2P僵尸网络生命周期的基础上,着重阐述了P2P僵尸网络在各个生命周期的工作机制;针对当前P2P僵尸网络检测研究现状,对检测方法进行了分类并介绍了各类检测方法的检测原理;最后对P2P僵尸网络的发展趋势进行了展望,并提出一种改进的P2P僵尸网络结构.     

结构化对等网络中P2P 僵尸网络传播模型

依赖结构化对等网传播的P2P僵尸是未来互联网面临的重要威胁.详细分析了两种典型的结构化P2P协议Chord和Kademlia的工作原理,在此基础上,使用数学建模的方法建立了结构化P2P僵尸网络的传播模型.该模型将Kademlia,Chord协议与双因子免疫机制、主机在线率等因素相结合,较为全面地研究了两种典型的结构化P2P网络中僵尸的传播机理,并使用软件仿真的方法模拟了节点超过百万时,结构化P2P网络中僵尸的传播行为,通过软件仿真得出的数据与理论数据进行对比,验证了模型的正确性.从实验结果可以看出:对于Kademlia和Chord两种结构化P2P网络,僵尸传播无论是双因子免疫模型还是结合双因子与主机在线率的模型,理论模型与仿真结果都非常吻合,体现了模型的准确性,为僵尸的检测与防御提供了理论依据.     

基于主机行为异常的P2P僵尸网络在线检测方法

僵尸网络已经成为当前最为严重的网络威胁之一,其中P2P僵尸网络得到迅速发展,其自身的通信特征给检测带来巨大的挑战.针对P2P僵尸网络检测技术的研究已经引起研究人员的广泛关注.提出一种P2P僵尸网络在线检测方法,首先采用信息熵技术发现网络流量中的异常点,然后通过分析P2P僵尸网络中主机的行为异常,利用统计学中的假设检验技术,从正常的网络流量数据中识别出可疑P2P僵尸主机,同时根据僵尸主机通信模式的相似性进行最终确认.实验结果表明该方法能够有效实现P2P僵尸网络的在线检测.     

一种分布式的僵尸网络实时检测算法

僵尸网络通过控制的主机实现多类恶意行为,使得当前的检测方法失效,其中窃取敏感数据已经成为主流。鉴于僵尸网络实现的恶意行为,检测和减轻方法的研究已经势在必行。提出了一种新颖的分布式实时僵尸网络检测方法,该方法通过将Netflow组织成主机Netflow图谱和主机关系链,并提取隐含的C&C通信特征来检测僵尸网络。同时,基于Spark Streaming分布式实时流处理引擎,使用该算法实现了BotScanner分布式检测系统。为了验证该系统的有效性,采用5个主流的僵尸网络家族进行训练,并分别使用模拟网络流量和真实网络流量进行测试。实验结果表明,在无需深度包解析的情况下,BotScanner分布式检测系统能够实时检测指定的僵尸网络,并获得了较高的检测率和较低的误报率。而且,在真实的网络环境中,BotScanner分布式检测系统能够进行实时检测,加速比接近线性,验证了Spark Streaming引擎在分布式流处理方面的优势,以及用于僵尸网络检测方面的可行性。     

A short visit to the bot zoo [malicious bots software]

The past year (2004-5) has seen, a new attack trend emerge: bots. After a successful compromise, the attacker installs a bot (also called a zombie or drone) on the system; this small program enables a remote control mechanism to then command the victim. Attackers use this technique repeatedly to form networks of compromised machines (botnets) to further enhance the effectiveness of their attacks. In recent years, malicious bots have become commonplace, with botnets in particular posing a severe threat to the Internet community. Attackers primarily use them for distributed denial-of-service (DDoS) attacks, mass identity theft, or sending spam.     

MRRbot: 基于冗余机制的多角色P2P僵尸网络模型

对现有僵尸网络的防御已取得很大成效,但僵尸网络不断演变进化,尤其在三网融合不断推进的背景下,这给防御者带来新的挑战.因此,预测未来僵尸网络以及时应对,非常必要.提出了一种基于冗余机制的多角色P2P僵尸网络模型(MRRbot),该模型引入虚壳僵尸终端,能够很大程度上验证僵尸终端的软硬件环境,增强其可信度和针对性;采用信息冗余机制和服务终端遴选算法,使僵尸终端能够均衡、高效地访问服务终端,提高命令控制信道的健壮性和抗毁性.对MRRbot的可控性、时效性和抗毁性进行了理论分析和实验评估,并就其健壮性与前人工作进行了比较.结果表明,MRRbot能够高效下发指令,有效对抗防御,更具威胁.探讨了可能的防御策略,提出基于志愿者网络的防御体系.     

CoCoSpot: Clustering and recognizing botnet command and control channels using traffic analysis

We present CoCoSpot, a novel approach to recognize botnet command and control channels solely based on traffic analysis features, namely carrier protocol distinction, message length sequences and encoding differences. Thus, CoCoSpot can deal with obfuscated and encrypted C&C protocols and complements current methods to fingerprint and recognize botnet C&C channels. Using average-linkage hierarchical clustering of labeled C&C flows, we show that for more than 20 recent botnets and over 87,000 C&C flows, CoCoSpot can recognize more than 88% of the C&C flows at a false positive rate below 0.1%.     

Clustering botnet communication traffic based on n-gram feature selection

Recognized as one the most serious security threats on current Internet infrastructure, botnets can not only be implemented by existing well known applications, e.g. IRC, HTTP, or Peer-to-Peer, but also can be constructed by unknown or creative applications, which makes the botnet detection a challenging problem. Previous attempts for detecting botnets are mostly to examine traffic content for bot command on selected network links or by setting up honeypots. Traffic content, however, can be encrypted with the evolution of botnet, and as a result leading to a fail of content based detection approaches. In this paper, we address this issue and propose a new approach for detecting and clustering botnet traffic on large-scale network application communities, in which we first classify the network traffic into different applications by using traffic payload signatures, and then a novel decision tree model is used to classify those traffic to be unknown by the payload content (e.g. encrypted traffic) into known application communities where network traffic is clustered based on n-gram features selected and extracted from the content of network flows in order to differentiate the malicious botnet traffic created by bots from normal traffic generated by human beings on each specific application. We evaluate our approach with seven different traffic trace collected on three different network links and results show the proposed approach successfully detects two IRC botnet traffic traces with a high detection rate and an acceptable low false alarm rate.     

Effective bot host detection based on network failure models

Botnet is one of the most notorious threats to Internet users. Attackers intrude into a large group of computers, install remote-controllable software, and then ask the compromised computers to launch large-scale Internet attacks, including sending spam and DDoS attacks. From the perspective of network administrators, it is important to identify bots in local networks. Bots residing in a local network could increase the difficulty to manage the network. Compared with bots outside of a local network, inside bots can easily bypass access controls applied to outsiders and access resources restricted to local users.In this paper, we propose an effective solution to detect bot hosts within a monitored local network. Based on our observations, a bot often has a differentiable failure pattern because of the botnet-distributed design and implementation. Hence, by monitoring failures generated by a single host for a short period, it is possible to determine whether the host is a bot or not by using a well-trained model. The proposed solution does not rely on aggregated network information, and therefore, works independent of network size. Our experiments show that the failure patterns among normal traffic, peer-to-peer traffic, and botnet traffic can be classified accurately. In addition to the ability to detect bot variants, the classification model can be retrained systematically to improve the detection ability for new bots. The evaluation results show that the proposed solution can detect bot hosts with more than 99% accuracy, whereas the false positive rate is lower than 0.5%.     

EFFORT: A new host–network cooperated framework for efficient and effective bot malware detection

Bots are still a serious threat to Internet security. Although a lot of approaches have been proposed to detect bots at host or network level, they still have shortcomings. Host-level approaches can detect bots with high accuracy. However they usually pose too much overhead on the host. While network-level approaches can detect bots with less overhead, they have problems in detecting bots with encrypted, evasive communication C&C channels. In this paper, we propose EFFORT, a new host–network cooperated detection framework attempting to overcome shortcomings of both approaches while still keeping both advantages, i.e., effectiveness and efficiency. Based on intrinsic characteristics of bots, we propose a multi-module approach to correlate information from different host- and network-level aspects and design a multi-layered architecture to efficiently coordinate modules to perform heavy monitoring only when necessary. We have implemented our proposed system and evaluated on real-world benign and malicious programs running on several diverse real-life office and home machines for several days. The final results show that our system can detect all 17 real-world bots (e.g., Waledac, Storm) with low false positives (0.68%) and with minimal overhead. We believe EFFORT raises a higher bar and this host–network cooperated design represents a timely effort and a right direction in the malware battle.     

基于统计特征的隐匿P2P主机实时检测系统

针对当前隐匿恶意程序多转为使用分布式架构来应对检测和反制的问题,为快速精确地检测出处于隐匿阶段的对等网络(P2P)僵尸主机,最大限度地降低其危害,提出了一种基于统计特征的隐匿P2P主机实时检测系统。首先,基于3个P2P主机统计特征采用机器学习方法检测出监控网络内的所有P2P主机;然后,再基于两个P2P僵尸主机统计特征,进一步检测出P2P僵尸主机。实验结果证明,所提系统能在5 min内检测出监控网内所有隐匿的P2P僵尸主机,准确率高达到99.7%,而误报率仅为0.3%。相比现有检测方法,所提系统检测所需统计特征少,且时间窗口较小,具备实时检测的能力。     

一种新颖的P2P僵尸网络检测技术

针对当前僵尸网络向P2P方向发展的趋势,在对P2P僵尸网络本质的理解和把握的基础上,提出了一种新颖的P2P僵尸网络检测技术。对于某个被监视的网络,关注其内部每台主机的通信行为和网络恶意活动。把这些通信行为和网络恶意活动分类,找出具有相似或相关通信和网络恶意行为的主机。根据我们对定义的理解,这些主机就属于某个P2P僵尸网络。