Intrusion detection in computer networks by a modular ensemble of one-class classifiers

Since the early days of research on intrusion detection, anomaly-based approaches have been proposed to detect intrusion attempts. Attacks are detected as anomalies when compared to a model of normal (legitimate) events. Anomaly-based approaches typically produce a relatively large number of false alarms compared to signature-based IDS. However, anomaly-based IDS are able to detect never-before-seen attacks. As new types of attacks are generated at an increasing pace and the process of signature generation is slow, it turns out that signature-based IDS can be easily evaded by new attacks. The ability of anomaly-based IDS to detect attacks never observed in the wild has stirred up a renewed interest in anomaly detection. In particular, recent work focused on unsupervised or unlabeled anomaly detection, due to the fact that it is very hard and expensive to obtain a labeled dataset containing only pure normal events.The unlabeled approaches proposed so far for network IDS focused on modeling the normal network traffic considered as a whole. As network traffic related to different protocols or services exhibits different characteristics, this paper proposes an unlabeled Network Anomaly IDS based on a modular Multiple Classifier System (MCS). Each module is designed to model a particular group of similar protocols or network services. The use of a modular MCS allows the designer to choose a different model and decision threshold for different (groups of) network services. This also allows the designer to tune the false alarm rate and detection rate produced by each module to optimize the overall performance of the ensemble. Experimental results on the KDD-Cup 1999 dataset show that the proposed anomaly IDS achieves high attack detection rate and low false alarm rate at the same time.     

基于流量异常分析多维优化的入侵检测方法

入侵检测系统在检测和预防各种网络异常行为的过程中,海量和高维的流量数据使其面临着低准确率和高误报率的问题。本文提出一种基于流量异常分析多维优化的入侵检测方法,该方法在入侵检测数据的横向维度和纵向维度两个维度进行优化。在横向维度优化中,对数量较多的类别进行数据抽样,并采用遗传算法得到每个类别的最佳抽样比例参数,完成数据的均衡化。在纵向维度优化中,结合特征与类别的相关分析,采用递归特征添加算法选择特征,并提出平均召回率指标评估特征选择效果,实现训练集的低维高效性。基于优化的入侵检测数据,进一步通过训练数据集得到随机森林分类器,在真实数据集UNSW_NB15评估和验证本文提出的算法。与其他算法相比,本文算法具有高准确率和低误报率,并在攻击类型上取得了有效的召回率。     

Language models for detection of unknown attacks in network traffic

In this paper, we propose a method for network intrusion detection based on language models. Our method proceeds by extracting language features such as n-grams and words from connection payloads and applying unsupervised anomaly detection—without prior learning phase or presence of labeled data. The essential part of this procedure is linear-time computation of similarity measures between language models of connection payloads. Particular patterns in these models decisive for differentiation of attacks and normal data can be traced back to attack semantics and utilized for automatic generation of attack signatures. Results of experiments conducted on two datasets of network traffic demonstrate the importance of high-order n-grams and variable-length language models for detection of unknown network attacks. An implementation of our system achieved detection accuracy of over 80% with no false positives on instances of recent remote-to-local attacks in HTTP, FTP and SMTP traffic.     

Flow-based anomaly detection in high-speed links using modified GSA-optimized neural network

Ever growing Internet causes the availability of information. However, it also provides a suitable space for malicious activities, so security is crucial in this virtual environment. The network intrusion detection system (NIDS) is a popular tool to counter attacks against computer networks. This valuable tool can be realized using machine learning methods and intrusion datasets. Traditional datasets are usually packet-based in which all network packets are analyzed for intrusion detection in a time-consuming process. On the other hand, the recent spread of 1–10-Gbps-technologies have clearly pointed out that scalability is a growing problem. In this way, flow-based solutions can help to solve the problem by reduction of data and processing time, opening the way to high-speed detection on large infrastructures. Besides, NIDS should be capable of detecting new malicious activities. Artificial neural network-based NIDSs can detect unseen attacks, so a multi-layer perceptron (MLP) neural classifier is used in this study to distinguish benign and malicious traffic in a flow-based NIDS. In this way, a modified gravitational search algorithm (MGSA), as a modern heuristic technique, is employed to optimize the interconnection weights of the neural anomaly detector. The proposed scheme is trained using an enhanced version of the first labeled flow-based dataset for intrusion detection introduced in 2009. In addition, the particle swarm optimization (PSO) algorithm and traditional error back-propagation (EBP) algorithm are employed to train MLP, so performance comparison becomes possible. The experimental results based on the actual network data show that the MGSA-optimized neural anomaly detector is effective for monitoring abnormal traffic flows in the gigabytes traffic environment, and the accuracy is about 97.8 %.     

An intelligent algorithm with feature selection and decision rules applied to anomaly intrusion detection

Intrusion detection system (IDS) is to monitor the attacks occurring in the computer or networks. Anomaly intrusion detection plays an important role in IDS to detect new attacks by detecting any deviation from the normal profile. In this paper, an intelligent algorithm with feature selection and decision rules applied to anomaly intrusion detection is proposed. The key idea is to take the advantage of support vector machine (SVM), decision tree (DT), and simulated annealing (SA). In the proposed algorithm, SVM and SA can find the best selected features to elevate the accuracy of anomaly intrusion detection. By analyzing the information from using KDD’99 dataset, DT and SA can obtain decision rules for new attacks and can improve accuracy of classification. In addition, the best parameter settings for the DT and SVM are automatically adjusted by SA. The proposed algorithm outperforms other existing approaches. Simulation results demonstrate that the proposed algorithm is successful in detecting anomaly intrusion detection.     

Intrusion detection in networks using crow search optimization algorithm with adaptive neuro-fuzzy inference system

Intrusion detection system has become the fundamental part for the network security and essential for network security because of the expansion of attacks which causes many issues. This is because of the broad development of internet and access to data systems around the world. For detecting the abnormalities present in the network or system, the intrusion detection system (IDS) is used. Because of the large volume of data, the network gets expanded with false alarm rate of intrusion and detection accuracy decreased. This is one of the significant issues when the network experiences unknown attacks. The principle objective was to expand the accuracy and reduce the false alarm rate (FAR). To address the above difficulties the proposed with Crow Search Optimization algorithm with Adaptive Neuro-Fuzzy Inference System (CSO-ANFIS) is used. The ANFIS is the combination of fuzzy interference system and artificial neural network, and to enhance the performance of the ANFIS model the crow search optimization algorithm is used to optimize the ANFIS. The NSL-KDD data set was used to validate the performance of intrusion detection of the proposed model and the experiment results are compared with other existing techniques for overall performance validation. The results of the intrusion detection based on the NSL-KDD dataset was better and efficient compared with those models because the detection rate was 95.80% and the FAR result was 3.45%.     

基于深度学习的网络流量异常预测方法

针对网络入侵检测系统（NIDS）能够检测当前系统中存在的网络安全事件,但由于自身的高误报率和识别安全事件产生的时延,无法提前对网络安全事件进行准确率较高的预警功能,严重制约了NIDS的实际应用和未来发展的问题,提出了基于深度学习的网络流量异常预测方法。该方法提出了一种结合深度学习算法中长短期记忆网络和卷积神经网络的预测模型,能够训练得到网络流量数据的时空特征,实现预测下一时段网络流量特征变化和网络安全事件分类识别,为NIDS实现网络安全事件的预警功能提供了方法分析。实验通过使用设计好的神经网络框架对入侵检测系统流量数据集CICIDS2017进行了训练和性能测试,在该方法下流量分类的误报率下降到0.26%,总体准确率达到了99.57%,流量特征预测模型R2的最佳效果达到了0.762。     

A hybrid machine learning approach to network anomaly detection

Zero-day cyber attacks such as worms and spy-ware are becoming increasingly widespread and dangerous. The existing signature-based intrusion detection mechanisms are often not sufficient in detecting these types of attacks. As a result, anomaly intrusion detection methods have been developed to cope with such attacks. Among the variety of anomaly detection approaches, the Support Vector Machine (SVM) is known to be one of the best machine learning algorithms to classify abnormal behaviors. The soft-margin SVM is one of the well-known basic SVM methods using supervised learning. However, it is not appropriate to use the soft-margin SVM method for detecting novel attacks in Internet traffic since it requires pre-acquired learning information for supervised learning procedure. Such pre-acquired learning information is divided into normal and attack traffic with labels separately. Furthermore, we apply the one-class SVM approach using unsupervised learning for detecting anomalies. This means one-class SVM does not require the labeled information. However, there is downside to using one-class SVM: it is difficult to use the one-class SVM in the real world, due to its high false positive rate. In this paper, we propose a new SVM approach, named Enhanced SVM, which combines these two methods in order to provide unsupervised learning and low false alarm capability, similar to that of a supervised SVM approach.We use the following additional techniques to improve the performance of the proposed approach (referred to as Anomaly Detector using Enhanced SVM): First, we create a profile of normal packets using Self-Organized Feature Map (SOFM), for SVM learning without pre-existing knowledge. Second, we use a packet filtering scheme based on Passive TCP/IP Fingerprinting (PTF), in order to reject incomplete network traffic that either violates the TCP/IP standard or generation policy inside of well-known platforms. Third, a feature selection technique using a Genetic Algorithm (GA) is used for extracting optimized information from raw internet packets. Fourth, we use the flow of packets based on temporal relationships during data preprocessing, for considering the temporal relationships among the inputs used in SVM learning. Lastly, we demonstrate the effectiveness of the Enhanced SVM approach using the above-mentioned techniques, such as SOFM, PTF, and GA on MIT Lincoln Lab datasets, and a live dataset captured from a real network. The experimental results are verified by m-fold cross validation, and the proposed approach is compared with real world Network Intrusion Detection Systems (NIDS).     

A neuro-genetic based short-term forecasting framework for network intrusion prediction system

Information systems are one of the most rapidly changing and vulnerable systems, where security is a major issue. The number of security-breaking attempts originating inside organizations is increasing steadily. Attacks made in this way, usually done by ＂authorized＂ users of the system, cannot be immediately traced. Because the idea of filtering the traffic at the entrance door, by using firewalls and the like, is not completely successful, the use of intrusion detection systems should be considered to increase the defense capacity of an information system. An intrusion detection system （IDS） is usually working in a dynamically changing environment, which forces continuous tuning of the intrusion detection model, in order to maintain sufficient performance. The manual tuning process required by current IDS depends on the system operators in working out the tuning solution and in integrating it into the detection model. Furthermore, an extensive effort is required to tackle the newly evolving attacks and a deep study is necessary to categorize it into the respective classes. To reduce this dependence, an automatically evolving anomaly IDS using neuro-genetic algorithm is presented. The proposed system automatically tunes the detection model on the fly according to the feedback provided by the system operator when false predictions are encountered. The system has been evaluated using the Knowledge Discovery in Databases Conference （KDD 2009） intrusion detection dataset. Genetic paradigm is employed to choose the predominant features, which reveal the occurrence of intrusions. The neuro-genetic IDS （NGIDS） involves calculation of weightage value for each of the categorical attributes so that data of uniform representation can be processed by the neuro-genetic algorithm. In this system unauthorized invasion of a user are identified and newer types of attacks are sensed and classified respectively by the neuro-genetic algorithm. The experimental results obtained in this work show that the system achieves improvement in terms of misclassification cost when compared with conventional IDS. The results of the experiments show that this system can be deployed based on a real network or database environment for effective prediction of both normal attacks and new attacks.     

面向不平衡样本的物联网入侵检测方法

随着设备的迭代,网络流量呈现指数级别的增长,针对各种应用的攻击行为越来越多,从流量层面识别并对这些攻击流量进行分类具有重要意义。同时,随着物联网设备的激增,针对这些设备的攻击行为也逐渐增多,造成的危害也越来越大。物联网入侵检测方法可以从这些海量的流量中识别出攻击流量,从流量层面保护物联网设备,阻断攻击行为。针对现阶段各类攻击流量检测准确率低以及样本不平衡问题,提出了基于重采样随机森林（RF,random forest）的入侵检测模型——Resample-RF,共包含3种具体算法：最优样本选择算法、基于信息熵的特征归并算法、多分类贪心转化算法。在物联网环境中,针对不平衡样本问题,提出最优样本选择算法,增加小样本所占权重,从而提高模型准确率;针对随机森林特征分裂效率不高的问题,提出基于信息熵的特征归并算法,提高模型运行效率;针对随机森林多分类精度不高的问题,提出多分类贪心转化算法,进一步提高准确率。在两个公开数据集上进行模型的检验,在 IoT-23 数据集上 F1 达到0.99,在Kaggle数据集上F1达到1.0,均具有显著效果。从实验结果中可知,提出的模型具有非常好的效果,能从海量流量中有效识别出攻击流量,较好地防范黑客对应用的攻击,保护物联网设备,从而保护用户。