Comprehensive approach to intrusion detection alert correlation

Alert correlation is a process that analyzes the alerts produced by one or more intrusion detection systems and provides a more succinct and high-level view of occurring or attempted intrusions. Even though the correlation process is often presented as a single step, the analysis is actually carried out by a number of components, each of which has a specific goal. Unfortunately, most approaches to correlation concentrate on just a few components of the process, providing formalisms and techniques that address only specific correlation issues. This paper presents a general correlation model that includes a comprehensive set of components and a framework based on this model. A tool using the framework has been applied to a number of well-known intrusion detection data sets to identify how each component contributes to the overall goals of correlation. The results of these experiments show that the correlation components are effective in achieving alert reduction and abstraction. They also show that the effectiveness of a component depends heavily on the nature of the data set analyzed.     

Decentralized multi-dimensional alert correlation for collaborative intrusion detection

The growth in coordinated network attacks such as scans, worms and distributed denial-of-service (DDoS) attacks is a profound threat to the security of the Internet. Collaborative intrusion detection systems (CIDSs) have the potential to detect these attacks, by enabling all the participating intrusion detection systems (IDSs) to share suspicious intelligence with each other to form a global view of the current security threats. Current correlation algorithms in CIDSs are either too simple to capture the important characteristics of attacks, or too computationally expensive to detect attacks in a timely manner. We propose a decentralized, multi-dimensional alert correlation algorithm for CIDSs to address these challenges. A multi-dimensional alert clustering algorithm is used to extract the significant intrusion patterns from raw intrusion alerts. A two-stage correlation algorithm is used, which first clusters alerts locally at each IDS, before reporting significant alert patterns to a global correlation stage. We introduce a probabilistic approach to decide when a pattern at the local stage is sufficiently significant to warrant correlation at the global stage. We then implement the proposed two-stage correlation algorithm in a fully distributed CIDS. Our experiments on a large real-world intrusion data set show that our approach can achieve a significant reduction in the number of alert messages generated by the local correlation stage with negligible false negatives compared to a centralized scheme. The proposed probabilistic threshold approach gains a significant improvement in detection accuracy in a stealthy attack scenario, compared to a naive scheme that uses the same threshold at the local and global stages. A large scale experiment on PlanetLab shows that our decentralized architecture is significantly more efficient than a centralized approach in terms of the time required to correlate alerts.     

Advanced probabilistic approach for network intrusion forecasting and detection

Recently, as damage caused by Internet threats has increased significantly, one of the major challenges is to accurately predict the period and severity of threats. In this study, a novel probabilistic approach is proposed effectively to forecast and detect network intrusions. It uses a Markov chain for probabilistic modeling of abnormal events in network systems. First, to define the network states, we perform K-means clustering, and then we introduce the concept of an outlier factor. Based on the defined states, the degree of abnormality of the incoming data is stochastically measured in real-time. The performance of the proposed approach is evaluated through experiments using the well-known DARPA 2000 data set and further analyzes. The proposed approach achieves high detection performance while representing the level of attacks in stages. In particular, our approach is shown to be very robust to training data sets and the number of states in the Markov model.     

An anomaly detection approach for multiple monitoring data series based on latent correlation probabilistic model

Condition monitoring systems are widely used to monitor the working condition of equipment, generating a vast amount and variety of monitoring data in the process. The main task of surveillance focuses on detecting anomalies in these routinely collected monitoring data, intended to help detect possible faults in the equipment. However, with the rapid increase in the volume of monitoring data, it is a nontrivial task to scan all the monitoring data to detect anomalies. In this paper, we propose an approach called latent correlation-based anomaly detection (LCAD) that efficiently and effectively detects potential anomalies from a large number of correlative isomerous monitoring data series. Instead of focusing on one or more isomorphic monitoring data series, LCAD identifies anomalies by modeling the latent correlation among multiple correlative isomerous monitoring data series, using a probabilistic distribution model called the latent correlation probabilistic model, which helps to detect anomalies according to their relations with the model. Experimental results on real-world data sets show that when dealing with a large number of correlative isomerous monitoring data series, LCAD yields better performances than existing anomaly detection approaches.     

A logic-based model to support alert correlation in intrusion detection

Managing and supervising security in large networks has become a challenging task, as new threats and flaws are being discovered on a daily basis. This requires an in depth and up-to-date knowledge of the context in which security-related events occur. Several tools have been proposed to support security operators in this task, each of which focuses on some specific aspects of the monitoring. Many alarm fusion and correlation approaches have also been investigated. However, most of these approaches suffer from two major drawbacks. First, they only take advantage of the information found in alerts, which is not sufficient to achieve the goals of alert correlation, that is to say to reduce the overall amount of alerts, while enhancing their semantics. Second, these techniques have been designed on an ad hoc basis and lack a shared data model that would allow them to reason about events in a cooperative way. In this paper, we propose a federative data model for security systems to query and assert knowledge about security incidents and the context in which they occur. This model constitutes a consistent and formal ground to represent information that is required to reason about complementary evidences, in order to confirm or invalidate alerts raised by intrusion detection systems.     

Enhancing network intrusion detection classifiers using supervised adversarial training

The performance of classifiers has a direct impact on the effectiveness of intrusion detection system. Thus, most researchers aim to improve the detection performance of classifiers. However, classifiers can only get limited useful information from the limited number of labeled training samples, which usually affects the generalization of classifiers. In order to enhance the network intrusion detection classifiers, we resort to adversarial training, and a novel supervised learning framework using generative adversarial network for improving the performance of the classifier is proposed in this paper. The generative model in our framework is utilized to continuously generate other complementary labeled samples for adversarial training and assist the classifier for classification, while the classifier in our framework is used to identify different categories. Meanwhile, the loss function is deduced again, and several empirical training strategies are proposed to improve the stabilization of the supervised learning framework. Experimental results prove that the classifier via adversarial training improves the performance indicators of intrusion detection. The proposed framework provides a feasible method to enhance the performance and generalization of the classifier.

     

基于分类器相关性的Adaboost人脸检测算法

为了提高传统Adaboost算法的集成性能,提出一种基于分类器相关性的Adaboost算法.该方法在弱分类器的训练过程中加入分类器的相关性判定,使每一个弱分类器的生成不仅与当前分类器有关,而且与前面若干个分类器相关,并将由此生成的弱分类器组合成新的强分类器.在CMU正面人脸检测集上的仿真结果表明,较传统的Adaboost算法,基于分类器相关性的Adaboost人脸检测算法具有更好的检测效率,同时降低了误检率.     

An algebraic approach to revising propositional rule-based knowledge bases

One of the important topics in knowledge base revision is to introduce an efficient implementation algorithm. Algebraic approaches have good characteristics and implementation method; they may be a choice to solve the problem. An algebraic approach is presented to revise propositional rule-based knowledge bases in this paper. A way is firstly introduced to transform a propositional rule-based knowledge base into a Petri net. A knowledge base is represented by a Petri net, and facts are represented by the initial marking. Thus, the consistency check of a knowledge base is equivalent to the reachability problem of Petri nets. The reachability of Petri nets can be decided by whether the state equation has a solution; hence the consistency check can also be implemented by algebraic approach. Furthermore, algorithms are introduced to revise a propositional rule-based knowledge base, as well as extended logic programming. Compared with related works, the algorithms presented in the paper are efficient, and the time complexities of these algorithms are polynomial.     

一种基于免疫的入侵检测关联报警模型

在入侵检测系统Snort的基础上,结合网络实时危险评估技术,提出了一种基于免疫的网络入侵检测报警模型SAIM。给出了网络环境下记忆细胞的表示方法,以及记忆细胞实时危险计算过程,建立了主机分类及总体实时危险计算方程,在此基础上给出了网络入侵检测报警模型。理论分析和试验结果均表明,SAIM模型能有效进行关联报警,提高报警质量。     

基于概率推理的知识图谱中实体间关联度计算

针对知识图谱中实体间的关联关系存在不确定性、实体间关联度计算复杂度高等问题,提出一种基于贝叶斯网的实体间关联度的计算方法.针对知识图谱做预处理,利用剪枝后获取的核心子图构建贝叶斯网,提出基于知识图谱的贝叶斯网构建方法;利用贝叶斯网作为知识图谱中实体之间关联关系的量化和推理框架,基于贝叶斯网的概率推理,提出知识图谱中实体...     

Decision tree based light weight intrusion detection using a wrapper approach

The objective of this paper is to construct a lightweight Intrusion Detection System (IDS) aimed at detecting anomalies in networks. The crucial part of building lightweight IDS depends on preprocessing of network data, identifying important features and in the design of efficient learning algorithm that classify normal and anomalous patterns. Therefore in this work, the design of IDS is investigated from these three perspectives. The goals of this paper are (i) removing redundant instances that causes the learning algorithm to be unbiased (ii) identifying suitable subset of features by employing a wrapper based feature selection algorithm (iii) realizing proposed IDS with neurotree to achieve better detection accuracy. The lightweight IDS has been developed by using a wrapper based feature selection algorithm that maximizes the specificity and sensitivity of the IDS as well as by employing a neural ensemble decision tree iterative procedure to evolve optimal features. An extensive experimental evaluation of the proposed approach with a family of six decision tree classifiers namely Decision Stump, C4.5, Naive Baye’s Tree, Random Forest, Random Tree and Representative Tree model to perform the detection of anomalous network pattern has been introduced.     

基于Spark框架的分布式入侵检测方法

为以较低的误报率和较高的检测率对攻击和恶意行为进行实时检测,基于Spark框架和位置敏感哈希算法,提出一种分布式数据流聚类方法DSCLS ,能够处理实时数据流,可根据数据流速进行横向分布式扩展。基于DSCLS分布式聚类算法,建立网络入侵检测系统,能够高速实时分析数据流,聚类相关模式,实时检测已知攻击和入侵,能够对未知的新型攻击进行检测。理论分析和实验结果表明,与主流的数据流聚类算法D‐Stream相比, DSCLS方法能够有效提高检测率并降低误报率,在时间性能和可扩展性方面更有优势。     

An approach of classifiers fusion based on hierarchical modifications

Applied Intelligence - Classifiers fusion is considered as an effective way to promote the accuracy of pattern recognition. In practice, its performance is mainly limited by potentials and...     

用遗传算法改进基于专家系统的入侵检测系统

基于专家系统的入侵检测系统的检测性能很大程度依赖于专家系统的规则集.为了提高基于专家系统的入侵检测系统的检测能力,使用遗传算法来对专家系统的规则集进行动态更新.但是基本遗传算法并不能有效对规则集进行动态更新,所以从编码、适应度函数、交叉等几个方面对遗传算法进行了改进.对如何使用改进的遗传算法对专家系统的规则集进行动态更新提出了一种实现方案.     

Land-cover classification from multiple classifiers using decision fusion based on the probabilistic graphical model

A novel method of using different classification algorithms in an integrated manner by adaptively weighted decision level fusion was proposed. The proposed fusion scheme involves two steps. First, we processed the data using each classifier separately and provided probability estimations for each pixel of the considered classes. Then, the results are aggregated on the basis of the decision rule of probabilistic graphical model according to the capabilities of classifiers and ancillary information. The method was tested and validated through the Landsat 8 operational land imager data using two different classifiers, namely, maximum likelihood classifier and support vector machine. The proposed method provided higher accuracy improvement than the separate use of different classifiers and that complex landscapes, such as mountainous regions, have higher accuracy improvement than the relatively homogenous ones. Moreover, the method can handle more than two types of classifiers and effectively introduce additional ancillary information for adaptive weight selection. These findings can help promote our proposed method as an emerging approach for land-cover classification through remote sensing technology.     

Integrating intrusion alert information to aid forensic explanation: An analytical intrusion detection framework for distributive IDS

The objective of this research is to show an analytical intrusion detection framework (AIDF) comprised of (i) a probability model discovery approach, and (ii) a probabilistic inference mechanism for generating the most probable forensic explanation based on not only just the observed intrusion detection alerts, but also the unreported signature rules that are revealed in the probability model. The significance of the proposed probabilistic inference is its ability to integrate alert information available from IDS sensors distributed across subnets. We choose the open source Snort to illustrate its feasibility, and demonstrate the inference process applied to the intrusion detection alerts produced by Snort. Through a preliminary experimental study, we illustrate the applicability of AIDF for information integration and the realization of (i) a distributive IDS environment comprised of multiple sensors, and (ii) a mechanism for selecting and integrating the probabilistic inference results from multiple models for composing the most probable forensic explanation.     

基于系统调用序列分析的入侵检测方法

提出了一种改进的基于系统调用序列分析的入侵检测方法,该方法对审计数据首先进行MLSI现象的检测,在发现MLSI之后,再与正常库进行匹配,以检测是否有入侵行为.理论分析和实验表明,MLSI能够有效地标识入侵,通过查找MLSI,再进行异常检测的方法可以大大地降低系统的开销,这些都说明该方法是有效和可行的.     

A triangle area based nearest neighbors approach to intrusion detection

Intrusion detection is a necessary step to identify unusual access or attacks to secure internal networks. In general, intrusion detection can be approached by machine learning techniques. In literature, advanced techniques by hybrid learning or ensemble methods have been considered, and related work has shown that they are superior to the models using single machine learning techniques. This paper proposes a hybrid learning model based on the triangle area based nearest neighbors (TANN) in order to detect attacks more effectively. In TANN, the k-means clustering is firstly used to obtain cluster centers corresponding to the attack classes, respectively. Then, the triangle area by two cluster centers with one data from the given dataset is calculated and formed a new feature signature of the data. Finally, the k-NN classifier is used to classify similar attacks based on the new feature represented by triangle areas. By using KDD-Cup ’99 as the simulation dataset, the experimental results show that TANN can effectively detect intrusion attacks and provide higher accuracy and detection rates, and the lower false alarm rate than three baseline models based on support vector machines, k-NN, and the hybrid centroid-based classification model by combining k-means and k-NN.     

基于卡尔曼滤波和相关系数的异常检测方法

由于安全问题是无线传感器网络应用面临的重要挑战之一,提出了一种基于卡尔曼滤波和相关系数相结合的异常入侵检测方法,该方法首先对正常情况下无线传感器网络节点的流量应用卡尔曼滤波进行预测,然后根据传感器节点的流量预测序列和实际流量序列的相关系数变化来进行异常检测.该算法具有能量有效、轻量级、可用性的特点.实验结果表明了该方法的有效性.     

一种基于半监督学习的物联网入侵检测系统

针对目前大多数物联网入侵检测系统误报率高、响应不及时、无法自主调查攻击行为的不足,提出了用半监督学习作为入侵检测的检测方法。同时,在Fuzzy C-means(FCM)算法的基础上提出了Random Fuzzy C-means (RFCM)算法的框架与实现。首先通过随机森林得到初始化的模型;然后,通过指定两个置信度参数,每轮得到分类结果置信度高的无标记样本;再将这些样本加入到原始有标记的样本集合里进行模型的二次训练,通过多轮迭代得到最终模型。实验表明,在NSL-KDD的入侵检测数据集里,模型具备良好的泛化性能。