基于网络流量小波分析的异常检测研究*

网络流量是局域网和广域网的重要特征之一,小波分析能将复杂的非线性网络流量时间序列分解成不同频率的子序列.基于小波分解的思想,利用网络流量的自相似特性来对网络的异常行为进行检测,给出了根据网络流量自相似特征参数的偏差来检测攻击的方法,对不同分辨率下Hurst参数的变化进行了比较分析.在DARPA上的测试结果表明,该方法不仅能够发现网络中存在的突发性流量攻击,还能够确定异常发生的位置.     

A hybrid machine learning approach to network anomaly detection

Zero-day cyber attacks such as worms and spy-ware are becoming increasingly widespread and dangerous. The existing signature-based intrusion detection mechanisms are often not sufficient in detecting these types of attacks. As a result, anomaly intrusion detection methods have been developed to cope with such attacks. Among the variety of anomaly detection approaches, the Support Vector Machine (SVM) is known to be one of the best machine learning algorithms to classify abnormal behaviors. The soft-margin SVM is one of the well-known basic SVM methods using supervised learning. However, it is not appropriate to use the soft-margin SVM method for detecting novel attacks in Internet traffic since it requires pre-acquired learning information for supervised learning procedure. Such pre-acquired learning information is divided into normal and attack traffic with labels separately. Furthermore, we apply the one-class SVM approach using unsupervised learning for detecting anomalies. This means one-class SVM does not require the labeled information. However, there is downside to using one-class SVM: it is difficult to use the one-class SVM in the real world, due to its high false positive rate. In this paper, we propose a new SVM approach, named Enhanced SVM, which combines these two methods in order to provide unsupervised learning and low false alarm capability, similar to that of a supervised SVM approach.We use the following additional techniques to improve the performance of the proposed approach (referred to as Anomaly Detector using Enhanced SVM): First, we create a profile of normal packets using Self-Organized Feature Map (SOFM), for SVM learning without pre-existing knowledge. Second, we use a packet filtering scheme based on Passive TCP/IP Fingerprinting (PTF), in order to reject incomplete network traffic that either violates the TCP/IP standard or generation policy inside of well-known platforms. Third, a feature selection technique using a Genetic Algorithm (GA) is used for extracting optimized information from raw internet packets. Fourth, we use the flow of packets based on temporal relationships during data preprocessing, for considering the temporal relationships among the inputs used in SVM learning. Lastly, we demonstrate the effectiveness of the Enhanced SVM approach using the above-mentioned techniques, such as SOFM, PTF, and GA on MIT Lincoln Lab datasets, and a live dataset captured from a real network. The experimental results are verified by m-fold cross validation, and the proposed approach is compared with real world Network Intrusion Detection Systems (NIDS).     

Bloccess: Enabling Fine-Grained Access Control Based on Blockchain

Botnets pose significant threats to cybersecurity. The infected Internet of Things (IoT) devices are used to launch unsupported malicious activities on target entities to disrupt their operations and services. To address this danger, we propose a machine learning-based method, for detecting botnets by analyzing network traffic data flow including various types of botnet attacks. Our method uses a hybrid model where a Variational AutoEncoder (VAE) is trained in an unsupervised manner to learn latent representations that describe the benign traffic data, and one-class classifier (OCC) for detecting anomaly (also called novelty detection). The main aim of this research is to learn the discriminating representations of the normal data in low dimensional latent space generated by VAE, and thus improve the predictive power of the OCC to detect malicious traffic. We have evaluated the performance of our model, and compared it against baseline models using a real network based dataset, containing popular IoT devices, and presenting a wide variety of attacks from two recent botnet families Mirai and Bashlite. Tests showed that our model can detect botnets with a satisfactory performance.

     

An adaptive method for anomaly detection in symmetric network traffic

Symmetry is an obvious phenomenon in two-way communications. In this paper, we present an adaptive nonparametric method that can be used for anomaly detection in symmetric network traffic. Two important features are emphasized in this method: (i) automatic adjustment of the detection threshold according to the traffic conditions; and (ii) timely detection of the end of an anomalous event. Source-end defense against SYN flooding attacks is used to illustrate the efficacy of this method. Experiments on real traffic traces show that this method has high detection accuracy and low detection delays, and excels at detecting low intensity attacks.     

Intrusion detection in computer networks by a modular ensemble of one-class classifiers

Since the early days of research on intrusion detection, anomaly-based approaches have been proposed to detect intrusion attempts. Attacks are detected as anomalies when compared to a model of normal (legitimate) events. Anomaly-based approaches typically produce a relatively large number of false alarms compared to signature-based IDS. However, anomaly-based IDS are able to detect never-before-seen attacks. As new types of attacks are generated at an increasing pace and the process of signature generation is slow, it turns out that signature-based IDS can be easily evaded by new attacks. The ability of anomaly-based IDS to detect attacks never observed in the wild has stirred up a renewed interest in anomaly detection. In particular, recent work focused on unsupervised or unlabeled anomaly detection, due to the fact that it is very hard and expensive to obtain a labeled dataset containing only pure normal events.The unlabeled approaches proposed so far for network IDS focused on modeling the normal network traffic considered as a whole. As network traffic related to different protocols or services exhibits different characteristics, this paper proposes an unlabeled Network Anomaly IDS based on a modular Multiple Classifier System (MCS). Each module is designed to model a particular group of similar protocols or network services. The use of a modular MCS allows the designer to choose a different model and decision threshold for different (groups of) network services. This also allows the designer to tune the false alarm rate and detection rate produced by each module to optimize the overall performance of the ensemble. Experimental results on the KDD-Cup 1999 dataset show that the proposed anomaly IDS achieves high attack detection rate and low false alarm rate at the same time.     

Systematic performance modeling and characterization of heterogeneous IP networks

Accurate measurement and modeling of IP networks is essential for network design, planning, and management. Efforts are being made to detect the state of the network from end-to-end measurements using different techniques and paradigms. In this paper we propose a novel concept to use in the modeling of real network scenarios under measurement and analysis. We called this new concept Service Condition. We explain our proposal's motivations and we use some simple examples to show how to apply the Service Condition concept to the study of real heterogeneous network scenarios. To show the real applicability of our proposal, preliminary results from a performance evaluation study over real heterogeneous networks (where the integration of LAN, WLAN, ADSL, UMTS, and GPRS is present) are given.     

A Traffic Generator for Testing Communication Systems: Presentation,Implementation and Performance

This paper presents a multi-application traffic generator (MTG), aimed at the generation of packets over a LAN. The generated traffic simulates the one produced by a number of both isochronous and anisochronous applications, thus allowing the measurement of a number of parameters relevant to the communication network. From a test point of view, data generated by the MTG system is equivalent to data generated by real applications spread over a LAN. The MTG system is presented, its implementation is described, some figures relevant to the MTG performance are shown, and the statistical analysis which can be performed on the recorded data is briefly introduced. The user manual of the MTG system is referenced in (Celandroni, Ferro, and Potorti 1995).     

应用层洪泛攻击的异常检测

从近年的发展趋势看, 分布式拒绝服务攻击已经从原来的低层逐渐向应用层发展, 它比传统的攻击更加有效且更具隐蔽性. 为检测利用合法应用层HTTP请求发动的洪泛攻击, 本文把应用层洪泛攻击视为一种异常的用户访问行为, 从用户浏览行为的角度实现攻击检测. 基于实际网络流的试验表明,该模型可以有效测量Web用户的访问行为正常度并实现应用层的DDoS洪泛攻击检测.     

Detecting zero-day attacks using context-aware anomaly detection at the application-layer

Anomaly detection allows for the identification of unknown and novel attacks in network traffic. However, current approaches for anomaly detection of network packet payloads are limited to the analysis of plain byte sequences. Experiments have shown that application-layer attacks become difficult to detect in the presence of attack obfuscation using payload customization. The ability to incorporate syntactic context into anomaly detection provides valuable information and increases detection accuracy. In this contribution, we address the issue of incorporating protocol context into payload-based anomaly detection. We present a new data representation, called \({c}_n\)-grams, that allows to integrate syntactic and sequential features of payloads in an unified feature space and provides the basis for context-aware detection of network intrusions. We conduct experiments on both text-based and binary application-layer protocols which demonstrate superior accuracy on the detection of various types of attacks over regular anomaly detection methods. Furthermore, we show how \({c}_n\)-grams can be used to interpret detected anomalies and thus, provide explainable decisions in practice.     

A monitoring system for detecting repeated packets with applications to computer worms

We present a monitoring system which detects repeated packets in network traffic, and has applications including detecting computer worms. It uses Bloom filters with counters. The system analyzes traffic in routers of a network. Our preliminary evaluation of the system involved traffic from our internal lab and a well known historical data set. After appropriate configuration, no false alarms are obtained under these data sets and we expect low false alarm rates are possible in many network environments. We also conduct simulations using real Internet Service Provider topologies with realistic link delays and simulated traffic. These simulations confirm that this approach can detect worms at early stages of propagation. We believe our approach, with minor adaptations, is of independent interest for use in a number of network applications which benefit from detecting repeated packets, beyond detecting worm propagation. These include detecting network anomalies such as dangerous traffic fluctuations, abusive use of certain services, and some distributed denial-of-service attacks. P. van Oorschot(Ph.D. Waterloo, 1988) is a Professor in the School of Computer Science at Carleton University, and Canada Research Chair in Network and Software Security. He is the founding director of Carleton's Digital Security Group. He has worked in research and development in cryptography and network security, including at Bell-Northern Research (Ottawa), and Entrust Technologies (Ottawa) as VP and Chief Scientist. He is coauthor of the standard reference Handbook of Applied Cryptography. His current research interests include authentication and identity management, network security, software protection, and security infrastructures. J.-M. Robertis a Principal Security Researcher at Alcatel in Ottawa, Ontario. His research interests are network and telecom infrastructure security, focusing mainly on denial-of-service attacks and worm propagation. Previously, Dr. Robert worked as Security Director for the North American Development Center of Gemplus International as well as Professor at the Université du Québec à Chicoutimi. Dr. Robert received a Ph.D. in Computer Science from McGill University. M. Vargas Martinis an Assistant Professor at the University of Ontario Institute of Technology (Oshawa, Canada), with faculty appointments in Business and Information Technology, as well as Engineering and Applied Science. He was previously a post-doctoral researcher at Carleton University supported in part by Alcatel Canada. He holds a Ph.D. in Computer Science (Carleton University, 2002), a Masters degree in Electrical Engineering (Cinvestav, Mexico, 1998), and a Bachelor of Computer Science (Universidad Autónoma de Aguascalientes, Mexico, 1996). His current research interests include network and host-based intrusion detection and reaction, mitigation of denial-of-service (DoS) and distributed DoS attacks, Web modeling and optimization, Internet connectivity, and interconnection protocols.     

A hybrid intrusion detection system design for computer network security

Intrusions detection systems (IDSs) are systems that try to detect attacks as they occur or after the attacks took place. IDSs collect network traffic information from some point on the network or computer system and then use this information to secure the network. Intrusion detection systems can be misuse-detection or anomaly detection based. Misuse-detection based IDSs can only detect known attacks whereas anomaly detection based IDSs can also detect new attacks by using heuristic methods. In this paper we propose a hybrid IDS by combining the two approaches in one system. The hybrid IDS is obtained by combining packet header anomaly detection (PHAD) and network traffic anomaly detection (NETAD) which are anomaly-based IDSs with the misuse-based IDS Snort which is an open-source project.The hybrid IDS obtained is evaluated using the MIT Lincoln Laboratories network traffic data (IDEVAL) as a testbed. Evaluation compares the number of attacks detected by misuse-based IDS on its own, with the hybrid IDS obtained combining anomaly-based and misuse-based IDSs and shows that the hybrid IDS is a more powerful system.     

高速网络环境下突发通信量生成算法研究

文章对高速网络模拟中必需的突发(或称自相似)通信量生成方法进行了研究,提出了一种基于分数高斯噪声(FGN,Fractional Gaussian Noise)过程和二进小波变换(DWT,Dyadic Wavelet Transform)的突发通信量生成算法——FGN-DWT算法,并对该算法的精度和计算复杂度进行了数学分析和仿真实验,研究结果表明,利用FGN-DW T算法生成的突发通信量的精度和速度都令人满意,为实际网络模拟的进一步深入研究提供了基础。     

Voice/data integration on Ethernet — backoff and priority considerations

Astudy of the CSMA/CD protocol as given in the Ethernet specifications, for real time (voice in particular) and mixed voice/data environments is described. The objectives of the study were to predict the Ethernet's behaviour under the current protocols, to point out potential improvements within the realm of the current specifications, and to improve understanding of the network behaviour in these environments. Results are given in terms of relative performance of the various system measures given as a function of system and traffic models. It is shown that the support provided to real time voice on Ethernet strongly depends on the assumptions made with regard to the coexisting data user's traffic and that with certain data traffic patterns the observed voice delay is much worse than previously reported. Finally, it is shown that without violating the Ethernet network specification, voice service can be improved by simple adjustments of the Ethernet retransmission ‘backoff’ algorithm.     

Remote detection of bottleneck links using spectral and statistical methods

Persistently saturated links are abnormal conditions that indicate bottlenecks in Internet traffic. Network operators are interested in detecting such links for troubleshooting, to improve capacity planning and traffic estimation, and to detect denial-of-service attacks. Currently bottleneck links can be detected either locally, through SNMP information, or remotely, through active probing or passive flow-based analysis. However, local SNMP information may not be available due to administrative restrictions, and existing remote approaches are not used systematically because of their network or computation overhead. This paper proposes a new approach to remotely detect the presence of bottleneck links using spectral and statistical analysis of traffic. Our approach is passive, operates on aggregate traffic without flow separation, and supports remote detection of bottlenecks, addressing some of the major limitations of existing approaches. Our technique assumes that traffic through the bottleneck is dominated by packets with a common size (typically the maximum transfer unit, for reasons discussed in Section 5.1). With this assumption, we observe that bottlenecks imprint periodicities on packet transmissions based on the packet size and link bandwidth. Such periodicities manifest themselves as strong frequencies in the spectral representation of the aggregate traffic observed at a downstream monitoring point. We propose a detection algorithm based on rigorous statistical methods to detect the presence of bottleneck links by examining strong frequencies in aggregate traffic. We use data from live Internet traces to evaluate the performance of our algorithm under various network conditions. Results show that with proper parameters our algorithm can provide excellent accuracy (up to 95%) even if the traffic through the bottleneck link accounts for less than 10% of the aggregate traffic.     

基于改进CUSUM算法的网络异常流量检测*

首先对网络流量进行异常检测,发现异常后再对数据包进行分析,实施相应措施,有利于降低系统开销。针对在使用CUSUM算法进行流量异常检测时产生的累积和效应问题,提出用自适应算法消除累积和效应的影响,分析了参数的设置对解除报警的延迟时间、误报警等的影响。实验结果表明设计的算法是有效和正确的,可以直接应用于检测SYN洪水攻击等。     

Self similarity analysis via fractional Fourier transform

Self similarity has taken great interest in computer networks since modeling of Ethernet traffic via self similarity. Recent studies have shown that network traffic exhibits long range dependency which could not be modeled with Poisson distribution. Time and frequency domain representations are frequently utilized to better visualize and characterize self similar stochastic processes.Fractional Fourier transform is a generalization of ordinary Fourier transform and find applications in many areas that ordinary Fourier transform has found. In this study, a network traffic analysis via fractional Fourier transform is performed. This study aims to better evaluate self similarity of network traffic via using fractional Fourier transform. Due to their high self similarity degrees, real IPv6 packet traffic is used for the analysis. We also perform analysis with an exact self similar process, fractional Gaussian noise to compare the results.     

Fast Filtered Sampling

Traffic sampled from the network backbone using uniform packet sampling is commonly utilized to detect heavy hitters, estimate flow level statistics, as well as identify anomalies like DDoS attacks and worm scans. Previous work has shown however that this technique introduces flow bias and truncation which yields inaccurate flow statistics and “drowns out” information from small flows, leading to large false positives in anomaly detection.In this paper, we present a new sampling design: Fast Filtered Sampling (FFS), which is comprised of an independent low-complexity filter, concatenated with any sampling scheme at choice. FFS ensures the integrity of small flows for anomaly detection, while still providing acceptable identification of heavy hitters. This is achieved through a filter design which suppresses packets from flows as a function of their size, “boosting” small flows relative to medium and large flows. FFS design requires only one update operation per packet, has two simple control parameters and can work in conjunction with existing sampling mechanisms without any additional changes. Therefore, it accomplishes a lightweight online implementation of the “flow-size dependent” sampling method. Through extensive evaluation on traffic traces, we show the efficacy of FFS for applications such as portscan detection and traffic estimation.     

基于注意力机制的CNN-LSTM的ADS-B异常数据检测

广播式自动相关监视(ADS-B)是民航新一代空中交通管理系统的重要组成部分,由于协议没有数据加密和认证,导致容易受到数据攻击.为了准确检测ADS-B数据攻击,基于ADS-B数据的时序性,提出了一种基于注意力机制的卷积神经网络-长短期记忆网络(convolutional neural networks-long short-term memory, CNN-LSTM)异常数据检测模型.首先,利用CNN提取ADS-B数据的特征,然后以时序形式将特征向量输入到LSTM中,最后使用注意力机制进行网络参数优化,实现对ADS-B数据的预测,通过计算预测误差,来进行异常检测.实验表明,该模型能够很好地检测出模拟的4种类型的异常数据,与其他机器学习方法相比,具有更高的准确率和F1分数.     

Suggestions of efficient self-similar generators

The growth of Grid computing and the Internet has been exponential in recent years. These high-speed communication networks have had a tremendous impact on our civilisation. High-speed communication networks offer a wide range of applications, such as multimedia and data intensive applications, which differ significantly in their traffic characteristics and performance requirements. Many analytical studies have shown that self-similar network traffic can have a detrimental impact on network performance, including amplified queueing delays and packet loss rates in broadband wide area networks. Thus, full understanding of the self-similar nature in teletraffic engineering is an important issue.This paper presents a detailed survey of self-similar generators proposed for generating sequential and fixed-length self-similar pseudo-random sequences for simulation in communication networks. We evaluate and compare the operational properties of the fixed-length and sequential generators of self-similar pseudo-random sequences. The statistical accuracy and time required to produce long sequences are discussed theoretically and studied experimentally. The evaluation of the generators concentrated on two aspects: (i) how accurately self-similar processes can be generated (assuming a given mean, variance and self-similarity parameter H), and (ii) how quickly the generators can generate long self-similar sequences. Overall, our results have revealed that the fastest and most accurate generators of the six sequential and five fixed-length sequence generators considered are the SRP-FGN, FFT and FGN-DW methods.     

Remote dynamic partial reconfiguration: A threat to Internet-of-Things and embedded security applications

The advent of the Internet of Things has motivated the use of Field Programmable Gate Array (FPGA) devices with Dynamic Partial Reconfiguration (DPR) capabilities for dynamic non-invasive modifications to circuits implemented on the FPGA. In particular, the ability to perform DPR over the network is essential in the context of a growing number of Internet of Things (IoT)-based and embedded security applications. However, the use of remote DPR brings with it a number of security threats that could lead to potentially catastrophic consequences in practical scenarios. In this paper, we demonstrate four examples where the remote DPR capability of the FPGA may be exploited by an adversary to launch Hardware Trojan Horse (HTH) attacks on commonly used security applications. We substantiate the threat by demonstrating remotely-launched attacks on Xilinx FPGA-based hardware implementations of a cryptographic algorithm, a true random number generator, and two processor based security applications - namely, a software implementation of a cryptographic algorithm and a cash dispensing scheme. The attacks are launched by on-the-fly transfer of malicious FPGA configuration bitstreams over an Ethernet connection to perform DPR and leak sensitive information. Finally, we comment on plausible countermeasures to prevent such attacks.