基于Chi-square检验的分布式网络入侵检测系统

针对网络攻击的新特点,本文提出了一种基于Chi-square检验的分布式网络入侵检测系统模型CTDIDS。设计并实现了一个基于异常检测的入侵分析引擎。通过对网络数据包的分析,运用Chi-square值比较对系统的行为进行检测。与现有的入侵检测方法相比,本文提出的方法具有更好的环境适应性和数据协同分析能力。实验证明,分布式入侵检测系统CTDIDS具有更高的准确性和扩展性。     

基于免疫智能的网络异常检测算法

网络异常检测模型可以用来检测未知攻击,具有良好的可扩展性,是目前入侵检测系统研究的热点。但目前的异常检测方法存在着误报率较高、检测效率不能满足高速网络实时检测需求等问题。本文通过对免疫智能算法与网络异常研究,提出了一种基于免疫智能的网络异常检测算法AIAIK。理论分析和实验说明改算法具有自然免疫系统的免疫网络、非线性、免疫记忆和克隆选择等良好特性,实验检测效果良好。     

Flow-based anomaly detection in high-speed links using modified GSA-optimized neural network

Ever growing Internet causes the availability of information. However, it also provides a suitable space for malicious activities, so security is crucial in this virtual environment. The network intrusion detection system (NIDS) is a popular tool to counter attacks against computer networks. This valuable tool can be realized using machine learning methods and intrusion datasets. Traditional datasets are usually packet-based in which all network packets are analyzed for intrusion detection in a time-consuming process. On the other hand, the recent spread of 1–10-Gbps-technologies have clearly pointed out that scalability is a growing problem. In this way, flow-based solutions can help to solve the problem by reduction of data and processing time, opening the way to high-speed detection on large infrastructures. Besides, NIDS should be capable of detecting new malicious activities. Artificial neural network-based NIDSs can detect unseen attacks, so a multi-layer perceptron (MLP) neural classifier is used in this study to distinguish benign and malicious traffic in a flow-based NIDS. In this way, a modified gravitational search algorithm (MGSA), as a modern heuristic technique, is employed to optimize the interconnection weights of the neural anomaly detector. The proposed scheme is trained using an enhanced version of the first labeled flow-based dataset for intrusion detection introduced in 2009. In addition, the particle swarm optimization (PSO) algorithm and traditional error back-propagation (EBP) algorithm are employed to train MLP, so performance comparison becomes possible. The experimental results based on the actual network data show that the MGSA-optimized neural anomaly detector is effective for monitoring abnormal traffic flows in the gigabytes traffic environment, and the accuracy is about 97.8 %.     

Joint network-host based malware detection using information-theoretic tools

In this paper, we propose two joint network-host based anomaly detection techniques that detect self-propagating malware in real-time by observing deviations from a behavioral model derived from a benign data profile. The proposed malware detection techniques employ perturbations in the distribution of keystrokes that are used to initiate network sessions. We show that the keystrokes’ entropy increases and the session-keystroke mutual information decreases when an endpoint is compromised by a self-propagating malware. These two types of perturbations are used for real-time malware detection. The proposed malware detection techniques are further compared with three prominent anomaly detectors, namely the maximum entropy detector, the rate limiting detector and the credit-based threshold random walk detector. We show that the proposed detectors provide considerably higher accuracy with almost 100% detection rates and very low false alarm rates.     

融合密度聚类与集成学习的数据库异常检测

目前,针对数据库系统内部攻击与威胁的检测方法较少,且已有的数据库异常检测方案存在代价开销高、检测准确率低等问题.为此,将密度聚类和集成学习融合,提出一种基于密度聚类和集成学习的数据库异常检测方法.利用OPTICS(Ordering Points To Identify the Clustering Structure)密度聚类算法对用户产生的数据库SQL操作日志进行聚类,通过对SQL语句中的各属性进行分析,提取用户的异常行为,形成先验知识;将Bagging、Boosting和Stacking进行组合,形成集成学习模型,以OPTICS聚类形成的先验知识为基础,并利用该集成学习模型对用户行为作进一步分析,并创建用户行为特征库.基于用户形成特征库,对用户行为进行检测.给出了方案的详细构建过程,包括数据预处理、训练、学习模型建立以及异常检测;利用相关实验数据进行测试,结果表明本方案能以较高的效率检测出数据库异常行为,并且在准确率方面优于同类方案.     

基于行为模型的工控异常检测方法研究

目前,工业控制系统(Industrial Control Systems,ICS)网络安全已经成为信息安全领域的重点问题,而检测篡改行为数据及控制程序等攻击是ICS网络安全的难点问题,据此提出了基于行为模型的工控异常检测方法。该方法从工控网络流量中提取行为数据序列,根据ICS的控制和被控过程构建正常行为模型,通过比较分析实时提取的行为数据与模型预测的行为数据,判断是否出现异常。通过实验分析,验证了所提方法能有效实现对篡改行为数据及控制程序等攻击的异常检测。     

An overview of anomaly detection techniques: Existing solutions and latest technological trends

As advances in networking technology help to connect the distant corners of the globe and as the Internet continues to expand its influence as a medium for communications and commerce, the threat from spammers, attackers and criminal enterprises has also grown accordingly. It is the prevalence of such threats that has made intrusion detection systems—the cyberspace’s equivalent to the burglar alarm—join ranks with firewalls as one of the fundamental technologies for network security. However, today’s commercially available intrusion detection systems are predominantly signature-based intrusion detection systems that are designed to detect known attacks by utilizing the signatures of those attacks. Such systems require frequent rule-base updates and signature updates, and are not capable of detecting unknown attacks. In contrast, anomaly detection systems, a subset of intrusion detection systems, model the normal system/network behavior which enables them to be extremely effective in finding and foiling both known as well as unknown or “zero day” attacks. While anomaly detection systems are attractive conceptually, a host of technological problems need to be overcome before they can be widely adopted. These problems include: high false alarm rate, failure to scale to gigabit speeds, etc. In this paper, we provide a comprehensive survey of anomaly detection systems and hybrid intrusion detection systems of the recent past and present. We also discuss recent technological trends in anomaly detection and identify open problems and challenges in this area.     

Training a neural-network based intrusion detector to recognizenovel attacks

While many commercial intrusion detection systems (IDS) are deployed, the protection they afford is modest. State-of-the-art IDS produce voluminous alerts, most false alarms, and function mainly by recognizing the signatures of known attacks so that novel attacks slip past them. Attempts have been made to create systems that recognize the signature of “normal,” in the hope that they will then detect attacks, known or novel. These systems are often confounded by the extreme variability of nominal behavior. The paper describes an experiment with an IDS composed of a hierarchy of neural networks (NN) that functions as a true anomaly detector. This result is achieved by monitoring selected areas of network behavior, such as protocols, that are predictable in advance. While this does not cover the entire attack space, a considerable number of attacks are carried out by violating the expectations of the protocol/operating system designer. Within this focus, the NNs are trained using data that spans the entire normal space. These detectors are able to recognize attacks that were not specifically presented during training. We show that using small detectors in a hierarchy gives a better result than a single large detector. Some techniques can be used not only to detect anomalies, but to distinguish among them     

Multi-stage change-point detection scheme for large-scale simultaneous events

Change-point detection schemes, which represent one type of anomaly detection schemes, are a promising approach for detecting network anomalies, such as attacks and epidemics by unknown viruses and worms. These events are detected as change-points. However, the schemes generally also detect false-positive change-points caused by other events, such as improper parameter setting of detectors. Therefore there is a requirement for a scheme that detects only true-positive change-points caused by attacks and epidemics by unknown viruses and worms. The true-positive change-points tend to occur simultaneously and intensively in very large numbers, while the false-positive change-points tend to occur independently. Therefore, we expect that the multi-stage change-point detection scheme, which performs change-point detection in a distributed manner and takes account of the correlation among multiple change-points, can exclude false-positive change-points by neglecting those that occur independently. In this paper, we propose the multi-stage change-point detection scheme and introduce a weighting function that gives smaller weight to LDs with higher false-positive rate inferred by GD in order to avoid a set of false-positive alerts generated by the low-accuracy detectors from causing high false-positive rate of the scheme. We evaluate the performance of the scheme by a simulation using the parameter values obtained in an experiment using real random scan worms. In the evaluation, we modify AAWP (Analytical Active Worm Propagation) model so that it can derive the number of infected hosts (i.e., attack hosts) more accurately by considering a failure of infection behavior by random scan worms. The simulation results show that our scheme can achieve an optimal performance (detection rate of 1.0 and false-positive rate of 0) while the stand-alone change-point detection scheme, which does not use the correlation among multiple change-points, cannot attain such optimal performance, and our scheme with alert weighting always shows better detection performance than the scheme without alert weighting.     

数据挖掘技术在入侵检测系统中的应用研究

随着以太网的快速发展,基于网络的攻击方式越来越多,传统的入侵检测系统越来越难以应付;将数据挖掘技术引入到入侵检测系统中来,分析网络中各种行为记录中潜在的攻击信息,自动辨别出网络入侵的模式,从而提高系统的检测效率;将K- MEANS算法及DBSCAN算法相综合,应用到入侵检测系统,并针对K- MEANS算法的一些不足进行了改进,提出了通过信息嫡理论的使用解决K- MEANS算法选择初始簇中心问题,然后利用其分类结果完善DBSCAN算法两个关键参数(Eps,Minpts)的设置,通过DB-SCAN算法,进一步地分析可疑的异常聚类,提高聚类的准确度.     

A hybrid intrusion detection system design for computer network security

Intrusions detection systems (IDSs) are systems that try to detect attacks as they occur or after the attacks took place. IDSs collect network traffic information from some point on the network or computer system and then use this information to secure the network. Intrusion detection systems can be misuse-detection or anomaly detection based. Misuse-detection based IDSs can only detect known attacks whereas anomaly detection based IDSs can also detect new attacks by using heuristic methods. In this paper we propose a hybrid IDS by combining the two approaches in one system. The hybrid IDS is obtained by combining packet header anomaly detection (PHAD) and network traffic anomaly detection (NETAD) which are anomaly-based IDSs with the misuse-based IDS Snort which is an open-source project.The hybrid IDS obtained is evaluated using the MIT Lincoln Laboratories network traffic data (IDEVAL) as a testbed. Evaluation compares the number of attacks detected by misuse-based IDS on its own, with the hybrid IDS obtained combining anomaly-based and misuse-based IDSs and shows that the hybrid IDS is a more powerful system.     

Determining malicious executable distinguishing attributes and low-complexity detection

Detection of rapidly evolving malware requires classification techniques that can effectively and efficiently detect zero-day attacks. Such detection is based on a robust model of benign behavior and deviations from that model are used to detect malicious behavior. In this paper we propose a low-complexity host-based technique that uses deviations in static file attributes to detect malicious executables. We first develop simple statistical models of static file attributes derived from the empirical data of thousands of benign executables. Deviations among the attribute models of benign and malware executables are then quantified using information-theoretic (Kullback-Leibler-based) divergence measures. This quantification reveals distinguishing attributes that are considerably divergent between benign and malware executables and therefore can be used for detection. We use the benign models of divergent attributes in cross-correlation and log-likelihood frameworks to classify malicious executables. Our results, using over 4,000 malicious file samples, indicate that the proposed detector provides reasonably high detection accuracy, while having significantly lower complexity than existing detectors.     

“Andromaly”: a behavioral malware detection framework for android devices

This article presents Andromaly—a framework for detecting malware on Android mobile devices. The proposed framework realizes a Host-based Malware Detection System that continuously monitors various features and events obtained from the mobile device and then applies Machine Learning anomaly detectors to classify the collected data as normal (benign) or abnormal (malicious). Since no malicious applications are yet available for Android, we developed four malicious applications, and evaluated Andromaly’s ability to detect new malware based on samples of known malware. We evaluated several combinations of anomaly detection algorithms, feature selection method and the number of top features in order to find the combination that yields the best performance in detecting new malware on Android. Empirical results suggest that the proposed framework is effective in detecting malware on mobile devices in general and on Android in particular.     

基于支持向量数据描述的异常检测方法

提出了一种基于支持向量数据描述算法的异常检测方法。该方法将入侵检测看作是一种单值分类问题,建立正常行为的支持向量描述模型,通过该模型可以检测各种已知和未知的攻击行为。该方法是一种无监督的异常检测方法,能够在包含噪声的数据集进行模型训练,降低了训练集的要求。在KDD CUP99标准入侵检测数据集上进行实验,并与无监督聚类异常检测实验结果相比较,证实该方法能够获得较高检测率和较低误警率。     

云环境下SDN的流量异常检测性能分析

随着复杂的混合云网络逐渐成为云计算发展的瓶颈,软件定义网络(SDN)技术近年来成为学术界和工业界关注的热点。在网络安全领域,对于应用SDN来解决网络攻击的研究尚处于起步阶段,SDN是否能够高效检测来自内部的网络攻击尚无定论。针对该问题,在分析SDN技术框架的基础上,设计基于OpenStack的云环境实验方案。在传统云环境网络和SDN环境下同时测试2种流量异常检测算法,模拟Flood攻击和端口扫描攻击,分析SDN在检测攻击时的精确度和资源使用率。结果表明,在云环境下利用SDN检测内部威胁时比传统网络环境占用更少的物理内存而不影响精确度,但直接在SDN控制器上部署安全应用的方式也存在性能瓶颈。     

分布式网络异常攻击检测模型仿真分析

针对传统的异常攻击检测方法主要以异常攻击行为规则与网络数据隶属度大小进行判别,只能针对已知异常攻击进行检测,对新型异常攻击,检测算法率低,计算数据量大的问题。提出一种新的分布式网络异常攻击检测方式,通过对分布式网络内数据进行迭代聚类将正常和异常数据进行分类,建立矩阵映射模型进行数据矩阵对比,初步对异常攻击数据进行判断。在矩阵中建立粒子密度函数,通过粒子密度变化计算其异常攻击概率,最后对其数据进行加权和波滤确定数据异常攻击特征,建立攻击检测模型。仿真实验表明,优化的分布式网络异常攻击检测模型提高了异常数据攻击检测的自适应性,在网络信号受到攻击信号干扰情况下,仍然能够准确检测出带有攻击特征的小网络异常数据。有效提高了分布式网络的检测正确率,加快了检测速度和稳定性。     

基于大数据技术的网络异常行为检测模型

针对传统的网络异常检测受数据存储、处理能力的限制,存在准确率较低、误报率较高以及无法检测未知攻击的问题。在Spark框架下结合改进的支持向量机和随机森林算法,提出了一种基于大数据技术的网络异常行为检测模型。使用NSL-KDD数据集进行了方法验证,表明该方法在准确率和误报率方面明显优于传统的检测算法,整体检测的准确率和误报率分别为96.61%和2.92%,DOS、Probe、R2L和U2R四种攻击类型的准确率分别达到98.01%、88.29%、94.03%和66.67%,验证了方法的有效性。     

Hybrid unsupervised web-attack detection and classification – A deep learning approach

Web requests made by users of web applications are manipulated by hackers to gain control of web servers. Moreover, detecting web attacks has been increasingly important in the distribution of information over the last few decades. Also, several existing techniques had been performed on detecting vulnerable web attacks using machine learning and deep learning techniques. However, there is a lack in achieving attack detection ratio owing to the utilization of supervised and semi-supervised learning approaches. Thus to overcome the aforementioned issues, this research proposes a hybrid unsupervised detection model a deep learning-based anomaly-based web attack detection. Whereas, the encoded outputs of De-Noising Autoencoder (DAE), as well as Stacked Autoencoder (SAE), are integrated and given to the Generative adversarial network (GAN) as input to improve the feature representation ability to detect the web attacks. Consequently, for classifying the type of attacks, a novel DBM-Bi LSTM-based classification model has been introduced. Which incorporates DBM for binary classification and Bi-LSTM for multi-class classification to classify the various attacks. Finally, the performance of the classifier in terms of recall, precision, F1-Score, and accuracy are evaluated and compared. The proposed method achieved high accuracy of 98%.     

A-GHSOM: An adaptive growing hierarchical self organizing map for network anomaly detection

The growing hierarchical self organizing map (GHSOM) has been shown to be an effective technique to facilitate anomaly detection. However, existing approaches based on GHSOM are not able to adapt online to the ever-changing anomaly detection. This results in low accuracy in identifying intrusions, particularly “unknown” attacks. In this paper, we propose an adaptive GHSOM based approach (A-GHSOM) to network anomaly detection. It consists of four significant enhancements: enhanced threshold-based training, dynamic input normalization, feedback-based quantization error threshold adaptation, and prediction confidence filtering and forwarding. We first evaluate the A-GHSOM approach for intrusion detection using the KDD’99 dataset. Extensive experimental results demonstrate that compared with eight representative intrusion detection approaches, A-GHSOM achieves significant overall accuracy improvement and significant improvement in identifying “unknown” attacks while maintaining low false-positive rates. It achieves an overall accuracy of 99.63%, and 94.04% accuracy in identifying “unknown” attacks while the false positive rate is 1.8%. To avoid drawing research results and conclusions solely based on experiments with the KDD dataset, we have also built a dataset (TD-Sim) that consists of a mixture of live trace data from the Lawrence Berkeley National Laboratory and simulated traffic based on our testbed network, ensuring adequate coverage of a variety of attacks. Performance evaluation with the TD-Sim dataset shows that A-GHSOM adapts to live traffic and achieves an overall accuracy rate of 97.12% while maintaining the false positive rate of 2.6%.     

Network traffic fusion and analysis against DDoS flooding attacks with a novel reversible sketch

Distributed Denial of Service (DDoS) flooding attacks are one of the typical attacks over the Internet. They aim to prevent normal users from accessing specific network resources. How to detect DDoS flooding attacks arises a significant and timely research topic. However, with the continuous increase of network scale, the continuous growth of network traffic brings great challenges to the detection of DDoS flooding attacks. Incomplete network traffic collection or non-real-time processing of big-volume network traffic will seriously affect the accuracy and efficiency of attack detection. Recently, sketch data structures are widely applied in high-speed networks to compress and fuse network traffic. But sketches suffer from a reversibility problem that it is difficult to reconstruct a set of keys that exhibit abnormal behavior due to the irreversibility of hash functions. In order to address the above challenges, in this paper, we first design a novel Chinese Remainder Theorem based Reversible Sketch (CRT-RS). CRT-RS is not only capable of compressing and fusing big-volume network traffic but also has the ability of reversely discovering the anomalous keys (e.g., the sources of malicious or unwanted traffic). Then, based on traffic records generated by CRT-RS, we propose a Modified Multi-chart Cumulative Sum (MM-CUSUM) algorithm that supports self-adaptive and protocol independent detection to detect DDoS flooding attacks. The performance of the proposed detection method is experimentally examined by two open source datasets. The experimental results show that the method can detect DDoS flooding attacks with efficiency, accuracy, adaptability, and protocol independability. Moreover, by comparing with other attack detection methods using sketch techniques, our method has quantifiable lower computation complexity when recovering the anomalous source addresses, which is the most important merit of the developed method.