A hybrid intrusion detection system design for computer network security

Intrusions detection systems (IDSs) are systems that try to detect attacks as they occur or after the attacks took place. IDSs collect network traffic information from some point on the network or computer system and then use this information to secure the network. Intrusion detection systems can be misuse-detection or anomaly detection based. Misuse-detection based IDSs can only detect known attacks whereas anomaly detection based IDSs can also detect new attacks by using heuristic methods. In this paper we propose a hybrid IDS by combining the two approaches in one system. The hybrid IDS is obtained by combining packet header anomaly detection (PHAD) and network traffic anomaly detection (NETAD) which are anomaly-based IDSs with the misuse-based IDS Snort which is an open-source project.The hybrid IDS obtained is evaluated using the MIT Lincoln Laboratories network traffic data (IDEVAL) as a testbed. Evaluation compares the number of attacks detected by misuse-based IDS on its own, with the hybrid IDS obtained combining anomaly-based and misuse-based IDSs and shows that the hybrid IDS is a more powerful system.     

Human perspective to anomaly detection for cybersecurity

Traditionally signature-based network Intrusion Detection Systems (IDS) rely on inputs from domain experts and can only identify the attacks if they occur as individual event. IDS generate large number of alerts and it becomes very difficult for human users to go through each message. Previous researches have proposed analytics based approaches to analyze IDS alert patterns based on anomaly detection models, multi-steps models or probabilistic approaches. However, due to the complexities of network intrusions, it is impossible to develop all possible attack patterns or to avoid false positives. With the advance in technologies and popularity of networks in our daily life, it is becoming more and more difficult to detect network intrusions. However, no matter how rapid the technologies change, the human behaviors behind the cyber attacks stay relatively constant. This provides us an opportunity to develop an improved system to detect the unusual cyber attacks. In this paper, we developed four network intrusion models based on consideration of human factors. We then tested these models on ITOC Cyber Defense Competition (CDX) 2009 data. Our results are encouraging. These Models are not only able to recognize most network attacks identified by SNORT log alerts, they are also able to distinguish the non-attack network traffic that was potentially missed by SNORT as indicated by ground truth validation of the data.     

An overview of anomaly detection techniques: Existing solutions and latest technological trends

As advances in networking technology help to connect the distant corners of the globe and as the Internet continues to expand its influence as a medium for communications and commerce, the threat from spammers, attackers and criminal enterprises has also grown accordingly. It is the prevalence of such threats that has made intrusion detection systems—the cyberspace’s equivalent to the burglar alarm—join ranks with firewalls as one of the fundamental technologies for network security. However, today’s commercially available intrusion detection systems are predominantly signature-based intrusion detection systems that are designed to detect known attacks by utilizing the signatures of those attacks. Such systems require frequent rule-base updates and signature updates, and are not capable of detecting unknown attacks. In contrast, anomaly detection systems, a subset of intrusion detection systems, model the normal system/network behavior which enables them to be extremely effective in finding and foiling both known as well as unknown or “zero day” attacks. While anomaly detection systems are attractive conceptually, a host of technological problems need to be overcome before they can be widely adopted. These problems include: high false alarm rate, failure to scale to gigabit speeds, etc. In this paper, we provide a comprehensive survey of anomaly detection systems and hybrid intrusion detection systems of the recent past and present. We also discuss recent technological trends in anomaly detection and identify open problems and challenges in this area.     

基于异常和特征的入侵检测系统模型

目前大多数入侵检测系统(Intrusion Detection System，IDS)没有兼备检测已知和未知入侵的能力，甚至不能检测已知入侵的微小变异，效率较低。本文提出了一种结合异常和特征检测技术的IDS。使用单一技术的IDS存在严重的缺点，为提高其效率，唯一的解决方案是两者的结合，即基于异常和特征的入侵检测。异常检测能发现未知入侵，而基于特征的检测能发现已知入侵，结合两者而成的基于异常和特征的入侵检测系统不但能检测已知和未知的入侵，而且能更新基于特征检测的数据库，因而具有很高的效率。     

基于虚拟机的运行时入侵检测技术研究

入侵检测技术通常分为误用检测和异常检测两类,误用检测根据攻击模式库检测已知的攻击行为,但却难以防范未知的攻击行为;异常检测技术虽然可以预测偏离正常值阈区间的潜在攻击行为,但却存在较高的误报现象。在虚拟机监视器中对虚拟机操作系统的运行行为进行带外监控,避免了操作系统内监控模块被病毒感染的难题;通过监视虚拟机的运行时行为,对之作组合序列的合法性分析,扩展了误用检测防范长时间段攻击行为的能力,识别通过合法系统调用进行的恶意攻击。测试数据表明,该技术能够较好地检测出复杂组合攻击行为。     

基于相对熵理论的多测度网络异常检测方法

检测率低、误报率高和检测攻击范围不够全面已经成为制约网络异常检测发展的最大障碍,为了提高检测率,降低误报率,扩大检测攻击范围,提出了一种新的网络异常检测方法。首先,对网络流量进行统计分析并引入相对熵理论来表征测度对应的全概率事件;然后,通过加权系数融合多个测度相对熵而得到加权相对熵;最终,以综合的多测度加权相对熵作为网络异常判断的依据。实验数据采用DARPA1999测评数据集,实验结果表明该方法在低误报率的前提下,达到了较高的检测率。     

基于P2DR模型的分布式入侵检测系统设计与实现

文章设计并实现了一个入侵检测系统———SC-IDS,该系统采用滥用和异常相结合的检测方法,分布式的体系结构,符合P2DR模型。克服了传统系统的缺点,如误警率和漏警率高、可扩展性弱、不能适应大规模网络、不能与其它安全产品协同工作等。在实际应用中取得了良好效果。     

Data preprocessing for anomaly based network intrusion detection: A review

Data preprocessing is widely recognized as an important stage in anomaly detection. This paper reviews the data preprocessing techniques used by anomaly-based network intrusion detection systems (NIDS), concentrating on which aspects of the network traffic are analyzed, and what feature construction and selection methods have been used. Motivation for the paper comes from the large impact data preprocessing has on the accuracy and capability of anomaly-based NIDS. The review finds that many NIDS limit their view of network traffic to the TCP/IP packet headers. Time-based statistics can be derived from these headers to detect network scans, network worm behavior, and denial of service attacks. A number of other NIDS perform deeper inspection of request packets to detect attacks against network services and network applications. More recent approaches analyze full service responses to detect attacks targeting clients. The review covers a wide range of NIDS, highlighting which classes of attack are detectable by each of these approaches.Data preprocessing is found to predominantly rely on expert domain knowledge for identifying the most relevant parts of network traffic and for constructing the initial candidate set of traffic features. On the other hand, automated methods have been widely used for feature extraction to reduce data dimensionality, and feature selection to find the most relevant subset of features from this candidate set. The review shows a trend toward deeper packet inspection to construct more relevant features through targeted content parsing. These context sensitive features are required to detect current attacks.     

Hybrid Intrusion Detection with Weighted Signature Generation over Anomalous Internet Episodes

This paper reports the design principles and evaluation results of a new experimental hybrid intrusion detection system (HIDS). This hybrid system combines the advantages of low false-positive rate of signature-based intrusion detection system (IDS) and the ability of anomaly detection system (ADS) to detect novel unknown attacks. By mining anomalous traffic episodes from Internet connections, we build an ADS that detects anomalies beyond the capabilities of signature-based SNORT or Bro systems. A weighted signature generation scheme is developed to integrate ADS with SNORT by extracting signatures from anomalies detected. HIDS extracts signatures from the output of ADS and adds them into the SNORT signature database for fast and accurate intrusion detection. By testing our HIDS scheme over real-life Internet trace data mixed with 10 days of Massachusetts Institute of Technology/Lincoln Laboratory (MIT/LL) attack data set, our experimental results show a 60 percent detection rate of the HIDS, compared with 30 percent and 22 percent in using the SNORT and Bro systems, respectively. This sharp increase in detection rate is obtained with less than 3 percent false alarms. The signatures generated by ADS upgrade the SNORT performance by 33 percent. The HIDS approach proves the vitality of detecting intrusions and anomalies, simultaneously, by automated data mining and signature generation over Internet connection episodes     

基于最小二乘支持向量机的Linux主机入侵检测系统

论文探讨在新的网络软硬件环境、各种新的攻击工具与方法下,建立一个实际的网络入侵异常检测系统的可行性。为此,论文建立一个基于Linux主机的入侵检测实验环境,在同时提供多种正常服务的条件下实施攻击、提取特征并应用最小二乘支持向量机(LS-SVM)检测入侵。结果表明检测系统设计合理,特征提取及检测方法有效。     

Genetic-based Fuzzy IDS for Feature Set Reduction and Worm Hole Attack Detection

The wireless ad-hoc networks are decentralized networks with a dynamic topology that allows for end-to-end communications via multi-hop routing operations with several nodes collaborating themselves, when the destination and source nodes are not in range of coverage. Because of its wireless type, it has lot of security concerns than an infrastructure networks. Wormhole attacks are one of the most serious security vulnerabilities in the network layers. It is simple to launch, even if there is no prior network experience. Signatures are the sole thing that preventive measures rely on. Intrusion detection systems (IDS) and other reactive measures detect all types of threats. The majority of IDS employ features from various network layers. One issue is calculating a huge layered features set from an ad-hoc network. This research implements genetic algorithm (GA)-based feature reduction intrusion detection approaches to minimize the quantity of wireless feature sets required to identify worm hole attacks. For attack detection, the reduced feature set was put to a fuzzy logic system (FLS). The performance of proposed model was compared with principal component analysis (PCA) and statistical parametric mapping (SPM). Network performance analysis like delay, packet dropping ratio, normalized overhead, packet delivery ratio, average energy consumption, throughput, and control overhead are evaluated and the IDS performance parameters like detection ratio, accuracy, and false alarm rate are evaluated for validation of the proposed model. The proposed model achieves 95.5% in detection ratio with 96.8% accuracy and produces very less false alarm rate (FAR) of 14% when compared with existing techniques.     

Training a neural-network based intrusion detector to recognizenovel attacks

While many commercial intrusion detection systems (IDS) are deployed, the protection they afford is modest. State-of-the-art IDS produce voluminous alerts, most false alarms, and function mainly by recognizing the signatures of known attacks so that novel attacks slip past them. Attempts have been made to create systems that recognize the signature of “normal,” in the hope that they will then detect attacks, known or novel. These systems are often confounded by the extreme variability of nominal behavior. The paper describes an experiment with an IDS composed of a hierarchy of neural networks (NN) that functions as a true anomaly detector. This result is achieved by monitoring selected areas of network behavior, such as protocols, that are predictable in advance. While this does not cover the entire attack space, a considerable number of attacks are carried out by violating the expectations of the protocol/operating system designer. Within this focus, the NNs are trained using data that spans the entire normal space. These detectors are able to recognize attacks that were not specifically presented during training. We show that using small detectors in a hierarchy gives a better result than a single large detector. Some techniques can be used not only to detect anomalies, but to distinguish among them     

入侵检测系统中报警验证模块的设计与实现

传统入侵检测系统虽然可以根据特征匹配的方法检测出攻击企图,却无法验证攻击企图是否成功,生成的报警不仅数量巨大而且误警率很高。该文提出一种结合漏洞扫描工具对入侵检测系统生成的报警进行验证的方法,根据被攻击主机是否包含能使攻击成功的漏洞来判定攻击能否成功,对攻击的目标主机不存在对应漏洞的报警降低优先级,从而提高报警质量。说明了报警验证模型各部分的设计和实现方法,系统运行结果显示该方法能有效地压缩报警量,降低误警率,帮助管理员从大量数据中找到最应该关注的真实报警。     

基于NetFlow的网络异常流量检测

NetFlow可以提供网络中IP流的信息。这些流的信息有多种用途，包括网管、网络规划、ISP计费等。在网络安全领域，NetFlow提供的IP流信息可以用来分析网络中的异常流量，这是对现有的基于特征的NIDS的很好的补充。本文介绍了Net—Flow—based Anomaly Traffic Analyzer，一个基于NetFlow的网络异常流量检测系统，并通过一些实验证明了该系统的有效性。     

An intelligent algorithm with feature selection and decision rules applied to anomaly intrusion detection

Intrusion detection system (IDS) is to monitor the attacks occurring in the computer or networks. Anomaly intrusion detection plays an important role in IDS to detect new attacks by detecting any deviation from the normal profile. In this paper, an intelligent algorithm with feature selection and decision rules applied to anomaly intrusion detection is proposed. The key idea is to take the advantage of support vector machine (SVM), decision tree (DT), and simulated annealing (SA). In the proposed algorithm, SVM and SA can find the best selected features to elevate the accuracy of anomaly intrusion detection. By analyzing the information from using KDD’99 dataset, DT and SA can obtain decision rules for new attacks and can improve accuracy of classification. In addition, the best parameter settings for the DT and SVM are automatically adjusted by SA. The proposed algorithm outperforms other existing approaches. Simulation results demonstrate that the proposed algorithm is successful in detecting anomaly intrusion detection.     

Intrusion detection in networks using crow search optimization algorithm with adaptive neuro-fuzzy inference system

Intrusion detection system has become the fundamental part for the network security and essential for network security because of the expansion of attacks which causes many issues. This is because of the broad development of internet and access to data systems around the world. For detecting the abnormalities present in the network or system, the intrusion detection system (IDS) is used. Because of the large volume of data, the network gets expanded with false alarm rate of intrusion and detection accuracy decreased. This is one of the significant issues when the network experiences unknown attacks. The principle objective was to expand the accuracy and reduce the false alarm rate (FAR). To address the above difficulties the proposed with Crow Search Optimization algorithm with Adaptive Neuro-Fuzzy Inference System (CSO-ANFIS) is used. The ANFIS is the combination of fuzzy interference system and artificial neural network, and to enhance the performance of the ANFIS model the crow search optimization algorithm is used to optimize the ANFIS. The NSL-KDD data set was used to validate the performance of intrusion detection of the proposed model and the experiment results are compared with other existing techniques for overall performance validation. The results of the intrusion detection based on the NSL-KDD dataset was better and efficient compared with those models because the detection rate was 95.80% and the FAR result was 3.45%.     

A novel intrusion detection approach learned from the change of antibody concentration in biological immune response

Inspired by the relationship between the antibody concentration and the intrusion network traffic pattern intensity, we present a Novel Intrusion Detection Approach learned from the change of Antibody Concentration in biological immune response (NIDAAC) to reduce false alarm rate without affecting detection rate. In NIDAAC, the concepts and formal definitions of self, nonself, antibody, antigen and detector in the intrusion detection domain are given. Then, in initial IDS, new detectors are generated from the gene library and tested by the negative selection. In every effective IDS node, according to the intrusion network traffic pattern intensity, the change of antibody number is recorded from the process of clone proliferation based on the detector evolution. Finally, building upon the above works, a probabilistic calculation model for intrusion alarm production, which is based on the correlation between the antibody concentration and the intrusion network traffic pattern intensity, is proposed. Compared with Naive Bayes (NB), Multilevel Classifier (AdaBoost) and Hidden Markov Model (HMM), the false alarm rate of NIDAAC is reduced by 8.66%, 4.93% and 6.36%, respectively. Our theoretical analysis and experimental results show that NIDAAC has a better performance than previous approaches.     

A hybrid artificial immune system and Self Organising Map for network intrusion detection

Network intrusion detection is the problem of detecting unauthorised use of, or access to, computer systems over a network. Two broad approaches exist to tackle this problem: anomaly detection and misuse detection. An anomaly detection system is trained only on examples of normal connections, and thus has the potential to detect novel attacks. However, many anomaly detection systems simply report the anomalous activity, rather than analysing it further in order to report higher-level information that is of more use to a security officer. On the other hand, misuse detection systems recognise known attack patterns, thereby allowing them to provide more detailed information about an intrusion. However, such systems cannot detect novel attacks.A hybrid system is presented in this paper with the aim of combining the advantages of both approaches. Specifically, anomalous network connections are initially detected using an artificial immune system. Connections that are flagged as anomalous are then categorised using a Kohonen Self Organising Map, allowing higher-level information, in the form of cluster membership, to be extracted. Experimental results on the KDD 1999 Cup dataset show a low false positive rate and a detection and classification rate for Denial-of-Service and User-to-Root attacks that is higher than those in a sample of other works.     

A Pareto-based multi-objective evolutionary algorithm for automatic rule generation in network intrusion detection systems

Attacks against computer systems are becoming more complex, making it necessary to continually improve the security systems, such as intrusion detection systems which provide security for computer systems by distinguishing between hostile and non-hostile activity. Intrusion detection systems are usually classified into two main categories according to whether they are based on misuse (signature-based) detection or on anomaly detection. With the aim of minimizing the number of wrong decisions, a new Pareto-based multi-objective evolutionary algorithm is used to optimize the automatic rule generation of a signature-based intrusion detection system (IDS). This optimizer, included within a network IDS, has been evaluated using a benchmark dataset and real traffic of a Spanish university. The results obtained in this real application show the advantages of using this multi-objective approach.