A Pareto-based multi-objective evolutionary algorithm for automatic rule generation in network intrusion detection systems

Attacks against computer systems are becoming more complex, making it necessary to continually improve the security systems, such as intrusion detection systems which provide security for computer systems by distinguishing between hostile and non-hostile activity. Intrusion detection systems are usually classified into two main categories according to whether they are based on misuse (signature-based) detection or on anomaly detection. With the aim of minimizing the number of wrong decisions, a new Pareto-based multi-objective evolutionary algorithm is used to optimize the automatic rule generation of a signature-based intrusion detection system (IDS). This optimizer, included within a network IDS, has been evaluated using a benchmark dataset and real traffic of a Spanish university. The results obtained in this real application show the advantages of using this multi-objective approach.     

基于Hamming网络的入侵检测技术的研究

分析了异常和误用入侵检测技术存在的一些问题,并结合神经网络的原理,提出了一个新的基于Hamming网络的入侵检测技术。该技术改善了基于特征检测算法中存在的不足,提高了对未知入侵类型的检测能力,并对Hamming网络入侵检测技术进行了分析和测试。     

Intrusion detection in computer networks by a modular ensemble of one-class classifiers

Since the early days of research on intrusion detection, anomaly-based approaches have been proposed to detect intrusion attempts. Attacks are detected as anomalies when compared to a model of normal (legitimate) events. Anomaly-based approaches typically produce a relatively large number of false alarms compared to signature-based IDS. However, anomaly-based IDS are able to detect never-before-seen attacks. As new types of attacks are generated at an increasing pace and the process of signature generation is slow, it turns out that signature-based IDS can be easily evaded by new attacks. The ability of anomaly-based IDS to detect attacks never observed in the wild has stirred up a renewed interest in anomaly detection. In particular, recent work focused on unsupervised or unlabeled anomaly detection, due to the fact that it is very hard and expensive to obtain a labeled dataset containing only pure normal events.The unlabeled approaches proposed so far for network IDS focused on modeling the normal network traffic considered as a whole. As network traffic related to different protocols or services exhibits different characteristics, this paper proposes an unlabeled Network Anomaly IDS based on a modular Multiple Classifier System (MCS). Each module is designed to model a particular group of similar protocols or network services. The use of a modular MCS allows the designer to choose a different model and decision threshold for different (groups of) network services. This also allows the designer to tune the false alarm rate and detection rate produced by each module to optimize the overall performance of the ensemble. Experimental results on the KDD-Cup 1999 dataset show that the proposed anomaly IDS achieves high attack detection rate and low false alarm rate at the same time.     

基于异常和特征的入侵检测系统模型

目前大多数入侵检测系统(Intrusion Detection System，IDS)没有兼备检测已知和未知入侵的能力，甚至不能检测已知入侵的微小变异，效率较低。本文提出了一种结合异常和特征检测技术的IDS。使用单一技术的IDS存在严重的缺点，为提高其效率，唯一的解决方案是两者的结合，即基于异常和特征的入侵检测。异常检测能发现未知入侵，而基于特征的检测能发现已知入侵，结合两者而成的基于异常和特征的入侵检测系统不但能检测已知和未知的入侵，而且能更新基于特征检测的数据库，因而具有很高的效率。     

Intrusion and intrusion detection

Assurance technologies for computer security have failed to have significant impacts in the marketplace, with the result that most of the computers connected to the internet are vulnerable to attack. This paper looks at the problem of malicious users from both a historical and practical standpoint. It traces the history of intrusion and intrusion detection from the early 1970s to the present day, beginning with a historical overview. The paper describes the two primary intrusion detection techniques, anomaly detection and signature-based misuse detection, in some detail and describes a number of contemporary research and commercial intrusion detection systems. It ends with a brief discussion of the problems associated with evaluating intrusion detection systems and a discussion of the difficulties associated with making further progress in the field. With respect to the latter, it notes that, like many fields, intrusion detection has been based on a combination of intuition and brute-force techniques. We suspect that these have carried the field as far as they can and that further significant progress will depend on the development of an underlying theoretical basis for the field. Published online: 27 July 2001     

Towards adaptive character frequency-based exclusive signature matching scheme and its applications in distributed intrusion detection

Network intrusion detection systems (NIDSs), especially signature-based NIDSs, are being widely deployed in a distributed network environment with the purpose of defending against a variety of network attacks. However, signature matching is a key limiting factor to limit and lower the performance of a signature-based NIDS in a large-scale network environment, in which the cost is at least linear to the size of an input string. The overhead network packets can greatly reduce the effectiveness of such detection systems and heavily consume computer resources. To mitigate this issue, a more efficient signature matching algorithm is desirable. In this paper, we therefore develop an adaptive character frequency-based exclusive signature matching scheme (named ACF-EX) that can improve the process of signature matching for a signature-based NIDS. In the experiment, we implemented the ACF-EX scheme in a distributed network environment, evaluated it by comparing with the performance of Snort. In addition, we further apply this scheme to constructing a packet filter that can filter out network packets by conducting exclusive signature matching for a signature-based NIDS, which can avoid implementation issues and improve the flexibility of the scheme. The experimental results demonstrate that, in the distributed network environment, the proposed ACF-EX scheme can positively reduce the time consumption of signature matching and that our scheme is promising in constructing a packet filter to reduce the burden of a signature-based NIDS.     

基于Boosting算法的入侵检测

提出了一种新颖的基于boosting BP 神经网络的入侵检测方法。为了提高BP神经网络的泛化能力,采用改进的Boosting方法,进行网络集成。Boosting方法采用更有效的参数求解方法,即弱分类器的加权参数不但与错误率有关,还与其对正样本的识别能力有关。对“KDD Cup 1999 Data”网络连接数据集进行特征选择和归一化处理之后用于训练神经网络并仿真实验,得到了较高的检测率和较低的误报率,仿真结果表明,提出的入侵检测方法是有效的。     

Wired and wireless intrusion detection system: Classifications, good characteristics and state-of-the-art

In computer and network security, standard approaches to intrusion detection and response attempt to detect and prevent individual attacks. Intrusion Detection System (IDS) and intrusion prevention systems (IPS) are real-time software for risk assessment by monitoring for suspicious activity at the network and system layer. Software scanner allows network administrator to audit the network for vulnerabilities and thus securing potential holes before attackers take advantage of them.

In this paper we try to define the intruder, types of intruders, detection behaviors, detection approaches and detection techniques. This paper presents a structural approach to the IDS by introducing a classification of IDS. It presents important features, advantages and disadvantages of each detection approach and the corresponding detection techniques. Furthermore, this paper introduces the wireless intrusion protection systems.

The goal of this paper is to place some characteristics of good IDS and examine the positioning of intrusion prevention as part of an overall layered security strategy and a review of evaluation criteria for identifying and selecting IDS and IPS. With this, we hope to introduce a good characteristic in order to improve the capabilities for early detection of distributed attacks in the preliminary phases against infrastructure and take a full spectrum of manual and automatic response actions against the source of attacks.     

A hybrid machine learning approach to network anomaly detection

Zero-day cyber attacks such as worms and spy-ware are becoming increasingly widespread and dangerous. The existing signature-based intrusion detection mechanisms are often not sufficient in detecting these types of attacks. As a result, anomaly intrusion detection methods have been developed to cope with such attacks. Among the variety of anomaly detection approaches, the Support Vector Machine (SVM) is known to be one of the best machine learning algorithms to classify abnormal behaviors. The soft-margin SVM is one of the well-known basic SVM methods using supervised learning. However, it is not appropriate to use the soft-margin SVM method for detecting novel attacks in Internet traffic since it requires pre-acquired learning information for supervised learning procedure. Such pre-acquired learning information is divided into normal and attack traffic with labels separately. Furthermore, we apply the one-class SVM approach using unsupervised learning for detecting anomalies. This means one-class SVM does not require the labeled information. However, there is downside to using one-class SVM: it is difficult to use the one-class SVM in the real world, due to its high false positive rate. In this paper, we propose a new SVM approach, named Enhanced SVM, which combines these two methods in order to provide unsupervised learning and low false alarm capability, similar to that of a supervised SVM approach.We use the following additional techniques to improve the performance of the proposed approach (referred to as Anomaly Detector using Enhanced SVM): First, we create a profile of normal packets using Self-Organized Feature Map (SOFM), for SVM learning without pre-existing knowledge. Second, we use a packet filtering scheme based on Passive TCP/IP Fingerprinting (PTF), in order to reject incomplete network traffic that either violates the TCP/IP standard or generation policy inside of well-known platforms. Third, a feature selection technique using a Genetic Algorithm (GA) is used for extracting optimized information from raw internet packets. Fourth, we use the flow of packets based on temporal relationships during data preprocessing, for considering the temporal relationships among the inputs used in SVM learning. Lastly, we demonstrate the effectiveness of the Enhanced SVM approach using the above-mentioned techniques, such as SOFM, PTF, and GA on MIT Lincoln Lab datasets, and a live dataset captured from a real network. The experimental results are verified by m-fold cross validation, and the proposed approach is compared with real world Network Intrusion Detection Systems (NIDS).     

Review of "Enhancing Computer Security with Smart Technology" [Book Review]

This book presents various methods for enhancing the enforcement of computer security. It consists of two parts and nine chapters. Among the topics covered are: basic issues with cyber trust; the need for firewalls; web application security; risk assessment; the relevance of machine learning in computer security; applying machine learning to intrusion detection; scanning and probing techniques; signature-based and anomaly IDs; artificial immune systems; and exploratory multivariate analysis for network security.     

基于NetFlow的网络异常流量检测

NetFlow可以提供网络中IP流的信息。这些流的信息有多种用途，包括网管、网络规划、ISP计费等。在网络安全领域，NetFlow提供的IP流信息可以用来分析网络中的异常流量，这是对现有的基于特征的NIDS的很好的补充。本文介绍了Net—Flow—based Anomaly Traffic Analyzer，一个基于NetFlow的网络异常流量检测系统，并通过一些实验证明了该系统的有效性。     

Human perspective to anomaly detection for cybersecurity

Traditionally signature-based network Intrusion Detection Systems (IDS) rely on inputs from domain experts and can only identify the attacks if they occur as individual event. IDS generate large number of alerts and it becomes very difficult for human users to go through each message. Previous researches have proposed analytics based approaches to analyze IDS alert patterns based on anomaly detection models, multi-steps models or probabilistic approaches. However, due to the complexities of network intrusions, it is impossible to develop all possible attack patterns or to avoid false positives. With the advance in technologies and popularity of networks in our daily life, it is becoming more and more difficult to detect network intrusions. However, no matter how rapid the technologies change, the human behaviors behind the cyber attacks stay relatively constant. This provides us an opportunity to develop an improved system to detect the unusual cyber attacks. In this paper, we developed four network intrusion models based on consideration of human factors. We then tested these models on ITOC Cyber Defense Competition (CDX) 2009 data. Our results are encouraging. These Models are not only able to recognize most network attacks identified by SNORT log alerts, they are also able to distinguish the non-attack network traffic that was potentially missed by SNORT as indicated by ground truth validation of the data.     

基于免疫的入侵检测方法研究

生物的免疫系统和计算机安全系统所面临及需要解决的问题十分类似．采用生物免疫思想的入侵检测技术可以结合异常检测和误用检测的优点．研究了基于免疫的入侵检测方法，对Self集的确定和有效检测器的生戍方法进行了研究和改进，基于反向选择机制提出了一种新的有效检测器生成算法．可以使用较少的有效检测器检测网络中的异常行为，从而提高了有效检测器生成和入侵检测的速度．通过与基于已有的有效检测器生成算法的系统进行比较，使用本文的方法构造的入侵检测系统速度更快．且有较高的准确性．     

Cluster-based Intrusion Detection in Wireless Ad-Hoc Networks

There are inherent vulnerabillties that are not easily preventable in the mobile Ad-Hoc networks.To build a highly secure wireless Ad-Hoc network,intrusion detection and response techniques need to be deployed;The intrusion detection and cluster-based Ad-Hoc networks has been introduced,then,an architecture for better intrusion detection based on cluster using Data Mining in wireless Ad-Hoc networks has been shown.A statistical anomaly detection approach has been used.The anomaly detection and trace analysis have been done locally in each node and possibly through cooperation with clusterhead detection in the network.     

FART在非监督式网络异常检测中的应用

目前的入侵检测系统主要采用的是基于特征的误用方法。另外,近几年出现的基于数据挖掘技术的入侵检测方法则需要依靠带标识的训练数据来保证检测效果,然而在现实环境中,训练数据往往是难以获得的。与之相比,非监督式的异常检测系统则具有独特的优势,它无需大量的带标识的、用于标明各种攻击的训练数据,而只需要寻找和定义正常的分类,因此,它具有在不具备任何先验知识的情况下发现新型攻击的能力。文章提出了一种采用模糊自适应谐振网(fuzzyART)发现网络入侵的新方法,并在最后采用KDDCUP99的测试数据集对该方法进行了评估,证实了该方法在网络异常检测中的有效性。     

入侵检测的可进化模糊规则分类器研究

随着入侵检测技术(IDS)在网络安全领域的作用越来越重要,将多种软计算方法应用到入侵检测技术中是构建智能入侵检测系统的新途径和尝试。本文将模糊数据挖掘技术和遗传算法相结合,提出一种基于遗传算法的模糊规则反复学习的方法,构造具有自适应能力的分类器,并进一步应用到计算机网络的入侵检测中。仿真测试证明了该方法的有效性。     

粒子群优化支持向量机的入侵检测算法

为了提高网络入侵的检测正确率,针对网络入侵检测中特征选择问题,将二值粒子群优化算法(BPSO)用于网络入侵特征选择,结合支持向量机(SVM)提出了一种基于BPSO-SVM的网络入侵检测算法。该算法将网络入侵检测转化为多分类问题,采用wrapper特征选择模型,以SVM为分类器,通过样本训练分类器,根据分类结果,利用BPSO算法在特征空间中进行全局搜索,选择最优特征集进行分类。实验结果表明,BPSO-SVM有效降低了特征维数,显著提高了网络入侵的检测正确率,还大大缩短了检测时间。     

An active learning based TCM-KNN algorithm for supervised network intrusion detection

As network attacks have increased in number and severity over the past few years, intrusion detection is increasingly becoming a critical component of secure information systems and supervised network intrusion detection has been an active and difficult research topic in the field of intrusion detection for many years. However, it hasn't been widely applied in practice due to some inherent issues. The most important reason is the difficulties in obtaining adequate attack data for the supervised classifiers to model the attack patterns, and the data acquisition task is always time-consuming and greatly relies on the domain experts. In this paper, we propose a novel supervised network intrusion detection method based on TCM-KNN (Transductive Confidence Machines for K-Nearest Neighbors) machine learning algorithm and active learning based training data selection method. It can effectively detect anomalies with high detection rate, low false positives under the circumstance of using much fewer selected data as well as selected features for training in comparison with the traditional supervised intrusion detection methods. A series of experimental results on the well-known KDD Cup 1999 data set demonstrate that the proposed method is more robust and effective than the state-of-the-art intrusion detection methods, as well as can be further optimized as discussed in this paper for real applications.     

基于Boosting RBF神经网络的入侵检测

提出一种新颖的基于boosting RBF神经网络的入侵检测方法。将模糊聚类和神经网络技术相结合,提出基于改进的FCM算法和OLS算法相结合的FORBF算法,为了提高RBF神经网络的泛化能力,采用Boosting方法,进行网络集成。以“KDD Cup 1999 Data”网络连接数据集训练神经网络并仿真实验,得到了较高的检测率和较低的误警率。     

Application classification using packet size distribution and port association

Traffic classification is an essential part in common network management applications such as intrusion detection and network monitoring. Identifying traffic by looking at port numbers is only suitable to well-known applications, while signature-based classification is not applicable to encrypted messages. Our preliminary observation shows that each application has distinct packet size distribution (PSD) of the connections. Therefore, it is feasible to classify traffic by analyzing the variances of packet sizes of the connections without analyzing packet payload. In this work, each connection is first transformed into a point in a multi-dimensional space according to its PSD. Then it is compared with the representative points of pre-defined applications and recognized as the application having a minimum distance. Once a connection is identified as a specific application, port association is used to accelerate the classification by combining it with the other connections of the same session because applications usually use consecutive ports during a session. Using the proposed techniques, packet size distribution and port association, a high accuracy rate, 96% on average, and low false positive and false negative rates, 4–5%, are achieved. Our proposed method not only works well for encrypted traffic but also can be easily incorporated with a signature-based method to provide better accuracy.