Dynamic emulation based modeling and detection of polymorphic shellcode at the network level

It is a promising way to detect polymorphic shellcode using emulation method. However, previous emulation-based approaches are limited in their performance and resilience against evasions. A new enhanced emulation-based detection approach is proposed, including an automaton-based model of the dynamic behavior of polymorphic shellcode and a detection algorithm, the detection criterion of which is derived from that model and ensures high detection accuracy. The algorithm also contains several optimization techniques, highly improving the running performance and the resilience against detection evasion shellcode. We have implemented a prototype system for our approach. The advantages of our algorithm are validated by the experiments with real network data, polymorphic shellcode samples generated by available polymorphic engines and hand-crafted detection evasion shellcode.     

基于人工免疫理论的Shellcode检测方法

Shellcode是缓冲区溢出漏洞攻击的核心代码部分,往往嵌入到文件和网络流量载体中。针对特征码匹配等检测手段存在时间滞后、准确率低等问题,结合人工免疫理论,提出一种采用实值编码的shellcode检测方法。收集shellcode样本并进行反汇编,利用n-gram模型对汇编指令序列提取特征生成抗原,作为免疫系统未成熟检测器来源,之后经历阴性选择算法的免疫耐受过程,生成成熟检测器。对检测器进行克隆和变异,繁衍出更加优良的后代,提高检测器的多样性和亲和度。实验结果表明,该方法对非编码shellcode和多态shellcode均具有较高的检测准确率。     

K-DDoS-SDN: A distributed DDoS attacks detection approach for protecting SDN environment

Software-defined networking (SDN) is an advanced networking paradigm that decouples forwarding control logic from the data plane. Therefore, it provides a loosely-coupled architecture between the control and data plane. This separation provides flexibility in the SDN environment for addressing any transformations. Further, it delivers a centralized way of managing networks due to control logic embedded in the SDN controller. However, this advanced networking paradigm has been facing several security issues, such as topology spoofing, exhausting bandwidth, flow table updating, and distributed denial of service (DDoS) attacks. A DDoS attack is one of the most powerful menaces to the SDN environment. Further, the central data controller of SDN becomes the primary target of DDoS attacks. In this article, we propose a Kafka-based distributed DDoS attacks detection approach for protecting the SDN environment named K-DDoS-SDN. The K-DDoS-SDN consists of two modules: (i) Network traffic classification (NTClassification) module and (ii) Network traffic storage (NTStorage) module. The NTClassification module is the detection approach designed using scalable H2O ML techniques in a distributed manner and deployed an efficient model on the two-nodes Kafka Streams cluster to classify incoming network traces in real-time. The NTStorage module collects raw packets, network flows, and 21 essential attributes and then systematically stores them in the HDFS to re-train existing models. The proposed K-DDoS-SDN designed and evaluated using the recent and publically available CICDDoS2019 dataset. The average classification accuracy of the proposed distributed K-DDoS-SDN for classifying network traces into legitimate and one of the most popular attacks, such as DDoS_UDP is 99.22%. Further, the outcomes demonstrate that proposed distributed K-DDoS-SDN classifies traffic traces into five categories with at least 81% classification accuracy.     

网络DDoS攻击流的小波分析与检测

将小波分析中的小波变换模极大方法用于检测分布式拒绝服务攻击引起的突发流量。在探讨如何运用小波模极大对突发流量进行判定的基础上,设计了一个检测突发攻击流量的方法,并对实际采集到的网络流量和仿真攻击流量的混合流作了计算机模拟验证。结果表明,当攻击流的突变幅度为正常流量的2倍￣3倍时,检测漏判率不超过5%;当攻击流的突变幅度提升为正常流量均值的3倍￣5倍时,检测漏判率不超过1%。攻击越强,检测漏判率越小。     

Robust and efficient detection of DDoS attacks for large-scale internet

In recent years, distributed denial of service (DDoS) attacks have become a major security threat to Internet services. How to detect and defend against DDoS attacks is currently a hot topic in both industry and academia. In this paper, we propose a novel framework to robustly and efficiently detect DDoS attacks and identify attack packets. The key idea of our framework is to exploit spatial and temporal correlation of DDoS attack traffic. In this framework, we design a perimeter-based anti-DDoS system, in which traffic is analyzed only at the edge routers of an internet service provider (ISP) network. Our framework is able to detect any source-address-spoofed DDoS attack, no matter whether it is a low-volume attack or a high-volume attack. The novelties of our framework are (1) temporal-correlation based feature extraction and (2) spatial-correlation based detection. With these techniques, our scheme can accurately detect DDoS attacks and identify attack packets without modifying existing IP forwarding mechanisms at routers. Our simulation results show that the proposed framework can detect DDoS attacks even if the volume of attack traffic on each link is extremely small. Especially, for the same false alarm probability, our scheme has a detection probability of 0.97, while the existing scheme has a detection probability of 0.17, which demonstrates the superior performance of our scheme.     

A queueing analysis for the denial of service (DoS) attacks in computer networks

In most network security analysis, researchers mainly focus on qualitative studies on security schemes and possible attacks, and there are few papers on quantitative analysis in the current literature. In this paper, we propose one queueing model for the evaluation of the denial of service (DoS) attacks in computer networks. The network under DoS attacks is characterized by a two-dimensional embedded Markov chain model. With this model, we can develop a memory-efficient algorithm for finding the stationary probability distribution which can be used to find other interesting performance metrics such as the connection loss probability and buffer occupancy percentages of half-open connections for regular traffic and attack traffic. Different from previous works in the literature, this paper gives a more general analytical approach to the study of security measures of a computer network under DoS attacks. We hope that our approach opens a new avenue to the quantitative evaluation of more complicated security schemes in computer networks.     

Semi-supervised machine learning approach for DDoS detection

Even though advanced Machine Learning (ML) techniques have been adopted for DDoS detection, the attack remains a major threat of the Internet. Most of the existing ML-based DDoS detection approaches are under two categories: supervised and unsupervised. Supervised ML approaches for DDoS detection rely on availability of labeled network traffic datasets. Whereas, unsupervised ML approaches detect attacks by analyzing the incoming network traffic. Both approaches are challenged by large amount of network traffic data, low detection accuracy and high false positive rates. In this paper we present an online sequential semi-supervised ML approach for DDoS detection based on network Entropy estimation, Co-clustering, Information Gain Ratio and Exra-Trees algorithm. The unsupervised part of the approach allows to reduce the irrelevant normal traffic data for DDoS detection which allows to reduce false positive rates and increase accuracy. Whereas, the supervised part allows to reduce the false positive rates of the unsupervised part and to accurately classify the DDoS traffic. Various experiments were performed to evaluate the proposed approach using three public datasets namely NSL-KDD, UNB ISCX 12 and UNSW-NB15. An accuracy of 98.23%, 99.88% and 93.71% is achieved for respectively NSL-KDD, UNB ISCX 12 and UNSW-NB15 datasets, with respectively the false positive rates 0.33%, 0.35% and 0.46%.     

Deciding optimal entropic thresholds to calibrate the detection mechanism for variable rate DDoS attacks in ISP domain: honeypot based approach

High bandwidth DDoS attacks consume more resources and have direct impact at ISP level in contrast to low rate DDoS attacks which lead to graceful degradation of network and are mostly undetectable. Although an array of detection schemes have been proposed, current requirement is a real time DDoS detection mechanism that adapts itself to varying network conditions to give minimum false alarms. DDoS attacks that disturb the distribution of traffic features in ISP domain are reflected by entropic variations on in stream samples. We propose honeypot detection for attack traffic having statistically similar distribution features as legitimate traffic. Next we propose to calibrate the detection mechanism for minimum false alarm rate by varying tolerance factor in real time. Simulations are carried out in ns-2 at different attack strengths. We also report our experimental results over MIT Lincoln lab dataset and its subset KDD 99 dataset. Results show that the proposed approach is comparable to previously reported approaches with an advantage of variable rate attack detection with minimum false positives and negatives.     

Data preprocessing for anomaly based network intrusion detection: A review

Data preprocessing is widely recognized as an important stage in anomaly detection. This paper reviews the data preprocessing techniques used by anomaly-based network intrusion detection systems (NIDS), concentrating on which aspects of the network traffic are analyzed, and what feature construction and selection methods have been used. Motivation for the paper comes from the large impact data preprocessing has on the accuracy and capability of anomaly-based NIDS. The review finds that many NIDS limit their view of network traffic to the TCP/IP packet headers. Time-based statistics can be derived from these headers to detect network scans, network worm behavior, and denial of service attacks. A number of other NIDS perform deeper inspection of request packets to detect attacks against network services and network applications. More recent approaches analyze full service responses to detect attacks targeting clients. The review covers a wide range of NIDS, highlighting which classes of attack are detectable by each of these approaches.Data preprocessing is found to predominantly rely on expert domain knowledge for identifying the most relevant parts of network traffic and for constructing the initial candidate set of traffic features. On the other hand, automated methods have been widely used for feature extraction to reduce data dimensionality, and feature selection to find the most relevant subset of features from this candidate set. The review shows a trend toward deeper packet inspection to construct more relevant features through targeted content parsing. These context sensitive features are required to detect current attacks.     

Ant-based IP traceback

The denial-of-service (DoS) attacks with the source IP address spoofing techniques has become a major threat to the Internet. An intrusion detection system is often used to detect DoS attacks and to coordinate with the firewall to block them. However, DoS attack packets consume and may exhaust all the resources, causing degrading network performance or, even worse, network breakdown. A proactive approach to DoS attacks is allocating the original attack host(s) issuing the attacks and stopping the malicious traffic, instead of wasting resources on the attack traffic.

In this paper, an ant-based traceback approach is proposed to identify the DoS attack origin. Instead of creating a new type or function or processing a high volume of fine-grained data used by previous research, the proposed traceback approach uses flow level information to identify the origin of a DoS attack.

Two characteristics of ant algorithm, quick convergence and heuristic, are adopted in the proposed approach on finding the DoS attack path. Quick convergence efficiently finds out the origin of a DoS attack; heuristic gives the solution even though partial flow information is provided by the network.

The proposed method is evaluated through simulation on various network environments and two simulated real networks, NSFNET and DFN. The simulation results show that the proposed method can successfully and efficiently find the DoS attack path in various simulated network environments, with full and partial flow information provided by the networks.     

Detection and defense of application-layer DDoS attacks in backbone web traffic

Web servers are usually located in a well-organized data center where these servers connect with the outside Internet directly through backbones. Meanwhile, the application-layer distributed denials of service (AL-DDoS) attacks are critical threats to the Internet, particularly to those business web servers. Currently, there are some methods designed to handle the AL-DDoS attacks, but most of them cannot be used in heavy backbones. In this paper, we propose a new method to detect AL-DDoS attacks. Our work distinguishes itself from previous methods by considering AL-DDoS attack detection in heavy backbone traffic. Besides, the detection of AL-DDoS attacks is easily misled by flash crowd traffic. In order to overcome this problem, our proposed method constructs a Real-time Frequency Vector (RFV) and real-timely characterizes the traffic as a set of models. By examining the entropy of AL-DDoS attacks and flash crowds, these models can be used to recognize the real AL-DDoS attacks. We integrate the above detection principles into a modularized defense architecture, which consists of a head-end sensor, a detection module and a traffic filter. With a swift AL-DDoS detection speed, the filter is capable of letting the legitimate requests through but the attack traffic is stopped. In the experiment, we adopt certain episodes of real traffic from Sina and Taobao to evaluate our AL-DDoS detection method and architecture. Compared with previous methods, the results show that our approach is very effective in defending AL-DDoS attacks at backbones.     

Detection of DDoS attacks using optimized traffic matrix

Distributed Denial of Service (DDoS) attacks have been increasing with the growth of computer and network infrastructures in Ubiquitous computing. DDoS attacks generating mass traffic deplete network bandwidth and/or system resources. It is therefore significant to detect DDoS attacks in their early stage. Our previous approach used a traffic matrix to detect DDoS attacks quickly and accurately. However, it could not find out to tune up parameters of the traffic matrix including (i) size of traffic matrix, (ii) time based window size, and (iii) a threshold value of variance from packets information with respect to various monitored environments and DDoS attacks. Moreover, the time based window size led to computational overheads when DDoS attacks did not occur. To cope with it, we propose an enhanced DDoS attacks detection approach by optimizing the parameters of the traffic matrix using a Genetic Algorithm (GA) to maximize the detection rates. Furthermore, we improve the traffic matrix building operation by (i) reforming the hash function to decrease hash collisions and (ii) replacing the time based window size with a packet based window size to reduce the computational overheads. We perform experiments with DARPA 2000 LLDOS 1.0, LBL-PKT-4 of Lawrence Berkeley Laboratory and generated attack datasets. The experimental results show the feasibility of our approach in terms of detection accuracy and speed.     

Network traffic fusion and analysis against DDoS flooding attacks with a novel reversible sketch

Distributed Denial of Service (DDoS) flooding attacks are one of the typical attacks over the Internet. They aim to prevent normal users from accessing specific network resources. How to detect DDoS flooding attacks arises a significant and timely research topic. However, with the continuous increase of network scale, the continuous growth of network traffic brings great challenges to the detection of DDoS flooding attacks. Incomplete network traffic collection or non-real-time processing of big-volume network traffic will seriously affect the accuracy and efficiency of attack detection. Recently, sketch data structures are widely applied in high-speed networks to compress and fuse network traffic. But sketches suffer from a reversibility problem that it is difficult to reconstruct a set of keys that exhibit abnormal behavior due to the irreversibility of hash functions. In order to address the above challenges, in this paper, we first design a novel Chinese Remainder Theorem based Reversible Sketch (CRT-RS). CRT-RS is not only capable of compressing and fusing big-volume network traffic but also has the ability of reversely discovering the anomalous keys (e.g., the sources of malicious or unwanted traffic). Then, based on traffic records generated by CRT-RS, we propose a Modified Multi-chart Cumulative Sum (MM-CUSUM) algorithm that supports self-adaptive and protocol independent detection to detect DDoS flooding attacks. The performance of the proposed detection method is experimentally examined by two open source datasets. The experimental results show that the method can detect DDoS flooding attacks with efficiency, accuracy, adaptability, and protocol independability. Moreover, by comparing with other attack detection methods using sketch techniques, our method has quantifiable lower computation complexity when recovering the anomalous source addresses, which is the most important merit of the developed method.     

Monitoring the macroscopic effect of DDoS flooding attacks

Creating defenses against flooding-based, distributed denial-of-service (DDoS) attacks requires real-time monitoring of network-wide traffic to obtain timely and significant information. Unfortunately, continuously monitoring network-wide traffic for suspicious activities presents difficult challenges because attacks may arise anywhere at any time and because attackers constantly modify attack dynamics to evade detection. In this paper, we propose a method for early attack detection. Using only a few observation points, our proposed method can monitor the macroscopic effect of DDoS flooding attacks. We show that such macroscopic-level monitoring might be used to capture shifts in spatial-temporal traffic patterns caused by various DDoS attacks and then to inform more detailed detection systems about where and when a DDoS attack possibly arises in transit or source networks. We also show that such monitoring enables DDoS attack detection without any traffic observation in the victim network.     

An artificial intelligence membrane to detect network intrusion

We propose an artificial intelligence membrane to detect network intrusion, which is analogous to a biological membrane that prevents viruses from entering cells. This artificial membrane is designed to monitor incoming packets and to prevent a malicious program code (e.g., a shellcode) from breaking into a stack or heap in a memory. While monitoring incoming TCP packets, the artificial membrane constructs a TCP segment of incoming packets, and derives the byte frequency of the TCP segment (from 0 to 255 bytes) as well as the entropy and size of the segment. These features of the segment can be classified by a data-mining technique such as a decision tree or neural network. If the data-mining method finds a suspicious byte sequence, the sequence is emulated to ensure that it is just a shellcode. If the byte sequence is a shellcode, the sequence is dropped. At the same time, an alert is communicated to the system administrator. Our experiments examined seven data-mining methods for normal and malicious network traffic. The malicious traffic included 114 shellcodes, provided by the Metasploit framework, and including 10 types of metamorphic or polymorphic shellcodes. In addition, real network traffic involving shellcodes was examined. We found that a random forest method outperformed all the other datamining methods and had a very high detection accuracy, including a true-positive rate of 99.6% and a false-positive rate of 0.4%.     

面向不平衡样本的物联网入侵检测方法

随着设备的迭代,网络流量呈现指数级别的增长,针对各种应用的攻击行为越来越多,从流量层面识别并对这些攻击流量进行分类具有重要意义。同时,随着物联网设备的激增,针对这些设备的攻击行为也逐渐增多,造成的危害也越来越大。物联网入侵检测方法可以从这些海量的流量中识别出攻击流量,从流量层面保护物联网设备,阻断攻击行为。针对现阶段各类攻击流量检测准确率低以及样本不平衡问题,提出了基于重采样随机森林（RF,random forest）的入侵检测模型——Resample-RF,共包含3种具体算法：最优样本选择算法、基于信息熵的特征归并算法、多分类贪心转化算法。在物联网环境中,针对不平衡样本问题,提出最优样本选择算法,增加小样本所占权重,从而提高模型准确率;针对随机森林特征分裂效率不高的问题,提出基于信息熵的特征归并算法,提高模型运行效率;针对随机森林多分类精度不高的问题,提出多分类贪心转化算法,进一步提高准确率。在两个公开数据集上进行模型的检验,在 IoT-23 数据集上 F1 达到0.99,在Kaggle数据集上F1达到1.0,均具有显著效果。从实验结果中可知,提出的模型具有非常好的效果,能从海量流量中有效识别出攻击流量,较好地防范黑客对应用的攻击,保护物联网设备,从而保护用户。     

Pure-Call Oriented Programming (PCOP): chaining the gadgets using call instructions

Return-oriented programming (ROP) and jump-oriented programming (JOP) are two well-known code-reuse attacks in which short code sequences ending in ret or jmp instructions are located and chained in a specific order to execute the attacker’s desired payload. JOP, comparing to ROP, is even more effective because it can be invoked without any reliance on the ret instruction and therefore it can bypass new defense mechanisms against ROP. In this paper, we continue this line of work by proposing Pure-Call Oriented Programming (PCOP). In PCOP, we drive the control flow by proposing special gadgets that all end in a call instruction rather than ret or jmp. We then propose techniques for chaining gadgets that removes the side-effects arise from the call-ending gadgets. The idea of having call-ending gadgets with the term Call Oriented Programming has been noted in some previous work but using call gadgets in these works, due to side-effects of the call instruction, was limited to one or two call-ending gadgets between other ret/jmp gadgets. Our work is the first that shows real code-reuse attacks solely based on call gadgets. We also show that our proposed approach is Turing-complete, meaning that any functionality can be driven by PCOP. We have successfully identified some call-oriented gadgets inside GNU libc library. Our experiments with the example shellcode show the practicality of the proposed approach. Finally, we propose a variant of PCOP named TinyCOP which resists detection by recent code-reuse defense mechanisms.     

基于协议分析技术的网络入侵检测系统中DDoS攻击的方法研究

随着网络技术的发展,网络环境变得越来越复杂,对网络安全来说,单纯的防火墙技术暴露出明显的不足和弱点,包括无法解决安全后门问题,不能阻止网络内部攻击等问题。在众多的网络安全威胁中,DDoS攻击以其实施容易,破坏力度大,检测困难等特点而成为网络攻击检测与防御的重中之重。近年来,针对网络流量相关性的DDoS攻击检测方法层出不穷,文章在分析DDoS攻击检测方法的基础上,利用基于协议分析技术的网络入侵检测系统对DDoS进行研究。     

Mathematical Model for Low-Rate DoS Attacks Against Application Servers

In recent years, variants of denial of service (DoS) attacks that use low-rate traffic have been proposed, including the Shrew attack, reduction of quality attacks, and low-rate DoS attacks against application servers (LoRDAS). All of these are flooding attacks that take advantage of vulnerability in the victims for reducing the rate of the traffic. Although their implications and impact have been comprehensively studied, mainly by means of simulation, there is a need for mathematical models by which the behaviour of these sometimes complex processes can be described. In this paper, we propose a mathematical model for the LoRDAS attack. This model allows us to evaluate its performance by relating it to the configuration parameters of the attack and the dynamics of network and victim. The model is validated by comparing the performance values given against those obtained from a simulated environment. In addition, some applicability issues for the model are contributed, together with interpretation guidelines to the model's behaviour. Finally, experience of the model enables us to make some recommendations for the challenging task of building defense techniques against this attack.      

A router-based technique to mitigate reduction of quality (RoQ) attacks

We propose a router-based technique to mitigate the stealthy reduction of quality (RoQ) attacks at the routers in the Internet. The RoQ attacks have been shown to impair the QoS sensitive VoIP and the TCP traffic in the Internet. It is difficult to detect these attacks because of their low average rates. We also show that our generalized approach can detect these attacks even if they employ the source IP address spoofing, the destination IP address spoofing, and undefined periodicity to evade several router-based detection systems. The detection system operates in two phases: in phase 1, the presence of the RoQ attack is detected from the readily available per flow information at the routers, and in phase 2, the attack filtering algorithm drops the RoQ attack packets. Assuming that the attacker uses the source IP address and the destination IP address spoofing, we propose to detect the sudden increase in the traffic load of all the expired flows within a short period. In a network without RoQ attacks, we show that the traffic load of all the expired flows is less than certain thresholds, which are derived from real Internet traffic analysis. We further propose a simple filtering solution to drop the attack packets. The filtering scheme treats the long-lived flows in the Internet preferentially, and drops the attack traffic by monitoring the queue length if the queue length exceeds a threshold percent of the queue limit. Our results show that we can successfully detect and mitigate RoQ attacks even with the source and destination IP addresses spoofed. The detection system is implemented in the ns2 simulator. In the simulations, we use the flowid field available in ns2 to implement per-flow logic, which is a combination of the source IP address, the destination IP address, the source port, and the destination port. We also discuss the real implementation of the proposed detection system.