Detecting zero-day attacks using context-aware anomaly detection at the application-layer

Anomaly detection allows for the identification of unknown and novel attacks in network traffic. However, current approaches for anomaly detection of network packet payloads are limited to the analysis of plain byte sequences. Experiments have shown that application-layer attacks become difficult to detect in the presence of attack obfuscation using payload customization. The ability to incorporate syntactic context into anomaly detection provides valuable information and increases detection accuracy. In this contribution, we address the issue of incorporating protocol context into payload-based anomaly detection. We present a new data representation, called \({c}_n\)-grams, that allows to integrate syntactic and sequential features of payloads in an unified feature space and provides the basis for context-aware detection of network intrusions. We conduct experiments on both text-based and binary application-layer protocols which demonstrate superior accuracy on the detection of various types of attacks over regular anomaly detection methods. Furthermore, we show how \({c}_n\)-grams can be used to interpret detected anomalies and thus, provide explainable decisions in practice.     

N-gram analysis for computer virus detection

Generic computer virus detection is the need of the hour as most commercial antivirus software fail to detect unknown and new viruses. Motivated by the success of datamining/machine learning techniques in intrusion detection systems, recent research in detecting malicious executables is directed towards devising efficient non-signature-based techniques that can profile the program characteristics from a set of training examples. Byte sequences and byte n-grams are considered to be basis of feature extraction. But as the number of n-grams is going to be very large, several methods of feature selections were proposed in literature. A recent report on use of information gain based feature selection has yielded the best-known result in classifying malicious executables from benign ones. We observe that information gain models the presence of n-gram in one class and its absence in the other. Through a simple example we show that this may lead to erroneous results. In this paper, we describe a new feature selection measure, class-wise document frequency of byte n-grams. We empirically demonstrate that the proposed method is a better method for feature selection. For detection, we combine several classifiers using Dempster Shafer Theory for better classification accuracy instead of using any single classifier. Our experimental results show that such a scheme detects virus program far more efficiently than the earlier known methods.     

SDN环境下的LDoS攻击检测与防御技术

低速率拒绝服务(LDoS)攻击是一种新型的网络攻击方式,其特点是攻击成本低,隐蔽性强。作为一种新型的网络架构,软件定义网络(SDN)同样面临着LDoS攻击的威胁。但SDN网络的控制与转发分离、网络行为可编程等特点又为LDoS攻击的检测和防御提供了新的思路。提出了一种基于OpenFlow协议的LDoS攻击检测和防御方法。通过对每条OpenFlow数据流的速率单独进行统计,并利用信号检测中的双滑动窗口法实现对攻击流量的检测,一旦检测到攻击流量,控制器便可以通过下发流表的方式实现对攻击行为的实时防御。实验表明,该方法能够有效检测出LDoS攻击,并能够在较短时间内实现对攻击行为的防御。     

基于混合深度学习的多类型低速率DDoS攻击检测方法

低速率分布式拒绝服务攻击针对网络协议自适应机制中的漏洞实施攻击,对网络服务质量造成了巨大威胁,具有隐蔽性强、攻击速率低和周期性的特点.现有检测方法存在检测类型单一和识别精度低的问题,因此提出了一种基于混合深度学习的多类型低速率DDoS攻击检测方法.模拟不同类型的低速率DDoS攻击和5G环境下不同场景的正常流量,在网络入...     

From zero-shot machine learning to zero-day attack detection

Machine learning (ML) models have proved efficient in classifying data samples into their respective categories. The standard ML evaluation methodology assumes that test data samples are derived from pre-observed classes used in the training phase. However, in applications such as Network Intrusion Detection Systems (NIDSs), obtaining data samples of all attack classes to be observed is challenging. ML-based NIDSs face new attack traffic known as zero-day attacks that are not used in training due to their non-existence at the time. Therefore, this paper proposes a novel zero-shot learning methodology to evaluate the performance of ML-based NIDSs in recognising zero-day attack scenarios. In the attribute learning stage, the learning models map network data features to semantic attributes that distinguish between known attacks and benign behaviour. In the inference stage, the models construct the relationships between known and zero-day attacks to detect them as malicious. A new evaluation metric is defined as Zero-day Detection Rate (Z-DR) to measure the effectiveness of the learning model in detecting unknown attacks. The proposed framework is evaluated using two key ML models and two modern NIDS data sets. The results demonstrate that for certain zero-day attack groups discovered in this paper, ML-based NIDSs are ineffective in detecting them as malicious. Further analysis shows that attacks with a low Z-DR have a significantly distinct feature distribution and a higher Wasserstein Distance range than the other attack classes.

     

Detecting attacks in high-speed networks: Issues and solutions

ABSTRACT

Intrusion detection systems are one of the necessities of networks to identify the problem of network attacks. Organizations striving to protect their data from intruders are often challenged by attackers, who find new ways to attack and compromise the security of the network. The detection process becomes quite difficult while dealing with high-speed and distributed attacks that are performed using botnets. These attacks threat both the confidentiality of legitimate users and the infrastructure of the network and to protect them, early discovery of network attacks is important. In this paper, an open source Intrusion Detection System (IDS), Snort is presented as a solution to detect DoS and Port Scan network attacks in a high-speed network. A set of custom rules has been proposed for Snort to detect DoS and Port Scan attacks in high-speed network. The rules are compared and tested using different attack generators like Scapy, Hping3, LOIC and Nmap. Snort’s efficiency in detecting the DoS and Port Scan attacks using the new rules is experimentally proved to be around 99% for all the attacks except for Ping of Death. The proposed system works well for different attack generators in a high-speed network.     

一种基于小波分析的DDoS攻击检测方法

通过对网络流量的分形特性和分布式拒绝服务（DDoS）的特点进行研究,提出了一种基于小波分析的DDoS攻击检测方法,并设计了该方法检测攻击的模型。对网络流量的分形特性进行判断,然后对具有自相似特性和多重分形特性的网络流量,分别采用基于小波分析的Hurst指数方差法和基于多窗口小波分析的Holder指数法检测DDoS攻击。通过对DARPA 2000年数据的实验表明,该方法能够有效地检测到攻击,对大流量背景攻击、低速率攻击、反射式攻击也都达到了较高的检测率,比传统方法有效。     

Pattern recognition for detecting distributed node exhaustion attacks in wireless sensor networks

Malicious attacks when launched by the adversary-class against sensor nodes of a wireless sensor network, can disrupt routine operations of the network. The mission-critical nature of these networks signifies the need to protect sensory resources against all such attacks. Distributed node exhaustion attacks are such attacks that may be launched by the adversarial class from multiple ends of a wireless sensor network against a set of target sensor nodes. The intention of such attacks is the exhaustion of the victim’s limited energy resources. As a result of the attack, the incapacitated data-generating legitimate sensor nodes are replaced with malicious nodes that will involve in further malicious activity against sensory resources. One such activity is the generation of fictitious sensory data to misguide emergency response systems to mobilize unwanted contingency activity. In this paper, a model is proposed for such an attack based on network traffic flow. In addition, a distributed mechanism for detecting such attacks is also defined. Specific network topology-based patterns are defined to model normal network traffic flow, and to facilitate differentiation between legitimate traffic packets and anomalous attack traffic packets. The performance of the proposed attack detection scheme is evaluated through simulation experiments, in terms of the size of the sensor resource set required for participation in the detection process for achieving a desired level of attack detection accuracy. The results signify the need for distributed pattern recognition for detecting distributed node exhaustion attacks in a timely and accurate manner.     

基于改进双重深度Q网络的入侵检测模型

入侵检测技术作为网络安全有效的防御手段,是网络安全体系中的重要组成部分。随着互联网的快速发展,网络数据量快速增加,网络攻击更加趋于复杂化和多元化,目前主流的入侵检测技术无法有效识别各种攻击。针对实际网络环境中正常流量和攻击流量数据不平衡,且对攻击类流量检测率低的问题,基于深度强化学习提出一种基于改进双重深度Q网络的CBL_DDQN网络入侵检测模型。该模型将一维卷积神经网络和双向长短期记忆网络的混合网络模型引入深度强化学习的DDQN框架,并使用深度强化学习中的反馈学习和策略生成机制训练智能体来对不同类别的攻击样本进行分类,在一定程度上减弱了训练模型过程中对数据标签的依赖性。采用Borderline-SMOTE算法降低数据的不平衡度,从而提高稀有攻击的检测率。通过NSL_KDD和UNSW_NB15数据集对模型的性能进行评估,结果表明：该模型在准确率、精确率、召回率这三项指标上均取得了良好的结果,检测效果远优于Adam-BNDNN、KNN、SVM等检测方法,是一种高效的网络入侵检测模型。     

Monitoring the macroscopic effect of DDoS flooding attacks

Creating defenses against flooding-based, distributed denial-of-service (DDoS) attacks requires real-time monitoring of network-wide traffic to obtain timely and significant information. Unfortunately, continuously monitoring network-wide traffic for suspicious activities presents difficult challenges because attacks may arise anywhere at any time and because attackers constantly modify attack dynamics to evade detection. In this paper, we propose a method for early attack detection. Using only a few observation points, our proposed method can monitor the macroscopic effect of DDoS flooding attacks. We show that such macroscopic-level monitoring might be used to capture shifts in spatial-temporal traffic patterns caused by various DDoS attacks and then to inform more detailed detection systems about where and when a DDoS attack possibly arises in transit or source networks. We also show that such monitoring enables DDoS attack detection without any traffic observation in the victim network.     

基于随机森林的网络入侵检测方法

为了提高网络安全水平,及时对网络攻击进行主动检测,提出了一种基于随机森林的网络入侵检测模型。该模型能够对大流量攻击进行分布式检测,且检测算法在引入了两个随机性后,即可降低网络流量内不同属性特征字段的噪声,并消除关联性,以便更为便捷、迅速地对攻击进行主动检测。将经典的Adaboost组合多分类器方法与提出的算法在检测率、正确率、精确率三个方面进行对比,体现了该算法的优越性,为大数据时代下网络安全提供了更好的保护。     

An autonomous defense against SYN flooding attacks: Detect and throttle attacks at the victim side independently

Distributed denial of service (DDoS) attacks seriously threaten Internet services yet there is currently no defence against such attacks that provides both early detection, allowing time for counteraction, and an accurate response. Traditional detection methods rely on passively sniffing an attacking signature and are inaccurate in the early stages of an attack. Current counteractions such as traffic filter or rate-limit methods do not accurately distinguish between legitimate and illegitimate traffic and are difficult to deploy. This work seeks to provide a method that detects SYN flooding attacks in a timely fashion and that responds accurately and independently on the victim side. We use the knowledge of network traffic delay distribution and apply an active probing technique (DARB) to identify half-open connections that, suspiciously, may not arise from normal network congestion. This method is suitable for large network areas and is capable of handling bursts of traffic flowing into a victim server. Accurate filtering is ensured by a counteraction method using IP address and time-to-live(TTL) fields. Simulation results show that our active detection method can detect SYN flooding attacks accurately and promptly and that the proposed rate-limit counteraction scheme can efficiently minimize the damage caused by DDoS attacks and guarantee constant services to legitimate users.     

Network traffic fusion and analysis against DDoS flooding attacks with a novel reversible sketch

Distributed Denial of Service (DDoS) flooding attacks are one of the typical attacks over the Internet. They aim to prevent normal users from accessing specific network resources. How to detect DDoS flooding attacks arises a significant and timely research topic. However, with the continuous increase of network scale, the continuous growth of network traffic brings great challenges to the detection of DDoS flooding attacks. Incomplete network traffic collection or non-real-time processing of big-volume network traffic will seriously affect the accuracy and efficiency of attack detection. Recently, sketch data structures are widely applied in high-speed networks to compress and fuse network traffic. But sketches suffer from a reversibility problem that it is difficult to reconstruct a set of keys that exhibit abnormal behavior due to the irreversibility of hash functions. In order to address the above challenges, in this paper, we first design a novel Chinese Remainder Theorem based Reversible Sketch (CRT-RS). CRT-RS is not only capable of compressing and fusing big-volume network traffic but also has the ability of reversely discovering the anomalous keys (e.g., the sources of malicious or unwanted traffic). Then, based on traffic records generated by CRT-RS, we propose a Modified Multi-chart Cumulative Sum (MM-CUSUM) algorithm that supports self-adaptive and protocol independent detection to detect DDoS flooding attacks. The performance of the proposed detection method is experimentally examined by two open source datasets. The experimental results show that the method can detect DDoS flooding attacks with efficiency, accuracy, adaptability, and protocol independability. Moreover, by comparing with other attack detection methods using sketch techniques, our method has quantifiable lower computation complexity when recovering the anomalous source addresses, which is the most important merit of the developed method.     

网络未知攻击检测的深度学习方法

为了实现入侵检测系统对未知攻击类型的检测,提出基于深度学习的网络异常检测方法。利用置信度神经网络,对已知类型流量和未知攻击流量进行自适应判别。基于深度神经网络,制定置信度估计方法评估模型分类结果,训练模型面向已知类型流量时输出高置信度值,识别到未知攻击流量时输出低置信度值,从而实现对未知攻击网络流量的检测,并设计自适应损失平衡策略和基于学习自动机的动态正则化策略优化异常检测模型。在网络异常检测UNSW-NB15和CICIDS 2017数据集上进行仿真实验,评估模型效果。结果表明,该方法实现了未知攻击流量的有效检测,并提高了已知类型流量的分类效果,从而增强了入侵检测系统的综合性能。     

基于S-Kohonen的DoS攻击检测算法研究

【】：针对现在DoS攻击检测算法过程中检测效率较低且检测时间比较长的问题,提出了基于S-Kohonen的DoS攻击检测算法。使用此算法实现并量化网络流量数据包的分割,并有效提取累积量特征,在DoS攻击检测过程中使用累积量。对现代入侵检测数据集进行全面的分析,此算法能够实现DoS攻击的全面检测。与传统以网络流量熵值为基础的异常检测算法相比,此算法可有效提高检测的精准度,缩短检测时间。     

基于可编程协议无关报文处理的分布式拒绝服务攻击检测

传统软件定义网络（SDN）中的分布式拒绝服务（DDoS）攻击检测方法需要控制平面与数据平面进行频繁通信,这会导致显著的开销和延迟,而目前可编程数据平面由于语法无法实现复杂检测算法,难以保证较高检测效率。针对上述问题,提出了一种基于可编程协议无关报文处理（P4）可编程数据平面的DDoS攻击检测方法。首先,利用基于P4改进的信息熵进行初检,判断是否有可疑流量发生;然后再利用P4提取特征只需微秒级时长的优势,提取可疑流量的六元组特征导入数据标准化—深度神经网络（data standardization-deep neural network,DS-DNN）复检模块,判断其是否为DDoS攻击流量;最后,模拟真实环境对该方法的各项评估指标进行测试。实验结果表明,该方法能够较好地检测SDN环境下的DDoS攻击,在保证较高检测率与准确率的同时,有效降低了误报率,并将检测时长缩短至毫秒级别。     

基于循环神经网络的Web应用防火墙加固方案

Web应用防火墙（WAF）基于一组规则检测和过滤进出Web应用程序的HTTP流量,鉴于恶意流量的复杂性,需要对WAF规则进行不断更新以抵御最新的攻击。然而,现有的WAF规则更新方法都需要专业知识来人工设计关于某种攻击的恶意测试流量,并针对该恶意流量生成防护规则,这种方法十分耗时且不能扩展到其他类型的攻击。提出一种基于循环神经网络（RNN）的Web应用防火墙加固方案,在不依赖任何专业知识的情况下自动化加固WAF。使用RNN模型生成恶意攻击样本,从中找到能够绕过WAF的恶意攻击,发现WAF规则存在的安全风险。在此基础上,通过设计评分函数找到恶意攻击样本的重要字符串来生成加固签名,阻止后续类似的攻击,并设计简化的正则表达式作为加固签名的表达形式。在4款WAF上针对SQL注入、跨站脚本攻击和命令注入这3种攻击进行测试,结果显示,该方案成功生成了大量绕过WAF的恶意样本,WAF针对这些样本的平均拦截率仅为52%,与传统突变方案和SQLMap工具相比能够生成更多绕过恶意攻击,在应用加固签名后,WAF的恶意攻击拦截率提升至90%以上且误报率维持为0,表明加固签名成功阻止了这些绕过攻击,验证了所提方案的有效性。     

Detection and defense of application-layer DDoS attacks in backbone web traffic

Web servers are usually located in a well-organized data center where these servers connect with the outside Internet directly through backbones. Meanwhile, the application-layer distributed denials of service (AL-DDoS) attacks are critical threats to the Internet, particularly to those business web servers. Currently, there are some methods designed to handle the AL-DDoS attacks, but most of them cannot be used in heavy backbones. In this paper, we propose a new method to detect AL-DDoS attacks. Our work distinguishes itself from previous methods by considering AL-DDoS attack detection in heavy backbone traffic. Besides, the detection of AL-DDoS attacks is easily misled by flash crowd traffic. In order to overcome this problem, our proposed method constructs a Real-time Frequency Vector (RFV) and real-timely characterizes the traffic as a set of models. By examining the entropy of AL-DDoS attacks and flash crowds, these models can be used to recognize the real AL-DDoS attacks. We integrate the above detection principles into a modularized defense architecture, which consists of a head-end sensor, a detection module and a traffic filter. With a swift AL-DDoS detection speed, the filter is capable of letting the legitimate requests through but the attack traffic is stopped. In the experiment, we adopt certain episodes of real traffic from Sina and Taobao to evaluate our AL-DDoS detection method and architecture. Compared with previous methods, the results show that our approach is very effective in defending AL-DDoS attacks at backbones.     

基于SDN的DDoS攻击防御系统

软件定义网络（SDN）是一种新兴网络架构,通过将转发层和控制层分离,实现网络的集中管控。控制器作为SDN网络的核心,容易成为被攻击的目标,分布式拒绝服务（DDoS）攻击是SDN网络面临的最具威胁的攻击之一。针对这一问题,本文提出一种基于机器学习的DDoS攻击检测模型。首先基于信息熵监控交换机端口流量来判断是否存在异常流量,检测到异常后提取流量特征,使用SVM+K-Means的复合算法检测DDoS攻击,最后控制器下发丢弃流表处理攻击流量。实验结果表明,本文算法在误报率、检测率和准确率指标上均优于SVM算法和K-Means算法。