基于优化Apriori算法的入侵检测系统模型设计

复合攻击是网络入侵的主要形式之一。如何检测复合攻击是当前入侵检测研究的一个重要方向,经过对复合攻击模式的大量研究,提出了一种基于自动调节的警报关联模型。为了提高入侵检测系统的效率,针对入侵检测系统的特点,将数据挖掘技术引入模型中。阐述了使用为关联规则提取所优化的Apriori算法,对日志文件进行特征分析与知识发掘的入侵检测系统模型的设计。     

一种基于多因素的告警关联方法

入侵检测系统作为保护网络安全的重要工具已被广泛使用,其通常产生大量冗余度高、误报率高的告警。告警关联分析通过对底层告警进行综合分析与处理,揭示出其中包含的多步攻击行为。许多告警关联方法通过在历史告警中挖掘频繁模式来构建攻击场景,方法容易受冗余告警、误报影响,挖掘出的多步攻击链在某些情况下不能反映出真实的多步攻击行为。为此,提出一种基于多因素的多步攻击关联方法。通过聚合原始告警以得到超级告警,降低冗余告警带来的影响;将超级告警构造成超级告警时间关系图,同时结合超级告警间的多因素关联度评价函数从时间关系图中挖掘出多步攻击场景。实验结果表明,该方法能克服冗余告警及大部分误报带来的负面影响、有效地挖掘出多步攻击链。     

Improving the quality of alerts and predicting intruder’s next goal with Hidden Colored Petri-Net

Intrusion detection systems (IDS) often provide poor quality alerts, which are insufficient to support rapid identification of ongoing attacks or predict an intruder’s next likely goal. In this paper, we propose a novel approach to alert postprocessing and correlation, the Hidden Colored Petri-Net (HCPN). Different from most other alert correlation methods, our approach treats the alert correlation problem as an inference problem rather than a filter problem. Our approach assumes that the intruder’s actions are unknown to the IDS and can be inferred only from the alerts generated by the IDS sensors. HCPN can describe the relationship between different steps carried out by intruders, model observations (alerts) and transitions (actions) separately, and associate each token element (system state) with a probability (or confidence). The model is an extension to Colored Petri-Net (CPN). It is so called “hidden” because the transitions (actions) are not directly observable but can be inferred by looking through the observations (alerts). These features make HCPN especially suitable for discovering intruders’ actions from their partial observations (alerts) and predicting intruders’ next goal. Our experiments on DARPA evaluation datasets and the attack scenarios from the Grand Challenge Problem (GCP) show that HCPN has promise as a way to reducing false positives and negatives, predicting intruder’s next possible action, uncovering intruders’ intrusion strategies after the attack scenario has happened, and providing confidence scores.     

一种基于信息熵的IDS告警预处理方法

针对目前入侵检测系统存在的海量重复告警、误报率偏高、告警质量低下等问题，提出一种基于信息熵的IDS告警预处理方法，用于减少误告警，聚合相似告警，生成代表单步攻击意图的超告警。首先，对IDS告警进行特征提取，用告警密度、告警周期值、源IP对应的目的IP数与攻击源威胁度这4个特征的信息熵融合结果表示一条告警所具有的特征信息量。通过与误告警的特征向量进行互雷尼信息熵的计算，从而识别出误告，并且去除误告。然后对误告去除后的告警按照IP对应关系，划分为2类：一种源IP对应一种目的IP的告警以及一种源IP对应多种目的IP的告警。分别对2类告警进行特征统计，构造5维特征信息熵向量，采用DBSCAN算法将信息量相同或者相似的告警进行聚类。最后对各个类别的告警进行动态时间窗口划分，并构建出代表单步攻击意图的超告警。实验结果表明，基于信息熵的告警预处理方法误告去除率为87.43%，告警聚合率达到98.63%，具有较好的误告去除效果以及较高的告警聚合率。     

入侵响应中基于事件相关性的攻击预测算法

目前的入侵检测系统(IDS)中的响应单元只对检测出的当前安全事件做出响应,而忽略了攻击事件间隐藏的关系及攻击的最终目的。本文针对上述问题在IDS的响应单元中提出了一个利用攻击事件间的相关性对攻击的最终目的进行预测的算法。实验证明该算法提高了网络的预警能力,减少了对误报的响应,并能发现分析引擎的漏报情况。     

Processing intrusion detection alert aggregates with time series modeling

The main use of intrusion detection systems (IDS) is to detect attacks against information systems and networks. Normal use of the network and its functioning can also be monitored with an IDS. It can be used to control, for example, the use of management and signaling protocols, or the network traffic related to some less critical aspects of system policies. These complementary usages can generate large numbers of alerts, but still, in operational environment, the collection of such data may be mandated by the security policy. Processing this type of alerts presents a different problem than correlating alerts directly related to attacks or filtering incorrectly issued alerts.We aggregate individual alerts to alert flows, and then process the flows instead of individual alerts for two reasons. First, this is necessary to cope with the large quantity of alerts – a common problem among all alert correlation approaches. Second, individual alert’s relevancy is often indeterminable, but irrelevant alerts and interesting phenomena can be identified at the flow level. This is the particularity of the alerts created by the complementary uses of IDSes.Flows consisting of alerts related to normal system behavior can contain strong regularities. We propose to model these regularities using non-stationary autoregressive models. Once modeled, the regularities can be filtered out to relieve the security operator from manual analysis of true, but low impact alerts. We present experimental results using these models to process voluminous alert flows from an operational network.     

入侵检测报警信息融合系统的构建与实现

针对目前入侵检测系统（IDS）存在的误报、漏报等问题,首先分析了存在误警的原因,设计并实现了一个入侵检测报警信息融合系统的模型。该模型提出一种相似隶属函数对报警事件进行关联,最后对系统进行了实验验证。结果表明该系统能有效地减少报警数量,降低误报、漏报率,从而提高了报警的有效性。同时通过事件关联完成攻击场景的重构,为加深对攻击者攻击意图的了解带来了方便,达到预警的目的,具有较强的实用价值。     

入侵检测报警信息融合系统的构建与实现

针对目前入侵检测系统(IDS)存在的误报、漏报等问题,首先分析了存在误警的原因,设计并实现了一个入侵检测报警信息融合系统的模型。该模型提出一种相似隶属函数对报警事件进行关联,最后对系统进行了实验验证。结果表明该系统能有效地减少报警数量,降低误报、漏报率,从而提高了报警的有效性。同时通过事件关联完成攻击场景的重构,为加深对攻击者攻击意图的了解带来了方便,达到预警的目的,具有较强的实用价值。     

TCP有限状态机和协议解析在伪警报去除中的应用

面对入侵检测系统(IDS)产生的海量警报,提出了一种基于协议解析和传输控制协议(TCP)有限状态机的伪警报去除方法。对于无连接的请求/应答协议,同时分析请求数据包的攻击特征和应答数据包的返回状态码来去除伪警报;对于TCP,在协议分析的基础上建立TCP数据包的有限状态机的模型,通过判断系列数据包是否为同一TCP连接、是否包含攻击序列来去除伪警报。在DARPA2000的数据集上的实验结果表明,此方法的误警率平均降低了59.47%,对TCP和请求/应答协议的警报的识别率达到76.67%。该方法简单又有效,依赖IDS的攻击特征库,可以插件的形式在线实现。     

入侵检测系统中告警相关部件的设计与实现

随着网络的迅猛发展,管理入侵检测系统产生的大量告警变得越来越重要。本文基于因果相关的思想,设计并实现了一个入侵检测系统中的告警相关部件。实验表明,该部件能有效减少告警数量,其告警减少率达到83．26％。     

Decentralized multi-dimensional alert correlation for collaborative intrusion detection

The growth in coordinated network attacks such as scans, worms and distributed denial-of-service (DDoS) attacks is a profound threat to the security of the Internet. Collaborative intrusion detection systems (CIDSs) have the potential to detect these attacks, by enabling all the participating intrusion detection systems (IDSs) to share suspicious intelligence with each other to form a global view of the current security threats. Current correlation algorithms in CIDSs are either too simple to capture the important characteristics of attacks, or too computationally expensive to detect attacks in a timely manner. We propose a decentralized, multi-dimensional alert correlation algorithm for CIDSs to address these challenges. A multi-dimensional alert clustering algorithm is used to extract the significant intrusion patterns from raw intrusion alerts. A two-stage correlation algorithm is used, which first clusters alerts locally at each IDS, before reporting significant alert patterns to a global correlation stage. We introduce a probabilistic approach to decide when a pattern at the local stage is sufficiently significant to warrant correlation at the global stage. We then implement the proposed two-stage correlation algorithm in a fully distributed CIDS. Our experiments on a large real-world intrusion data set show that our approach can achieve a significant reduction in the number of alert messages generated by the local correlation stage with negligible false negatives compared to a centralized scheme. The proposed probabilistic threshold approach gains a significant improvement in detection accuracy in a stealthy attack scenario, compared to a naive scheme that uses the same threshold at the local and global stages. A large scale experiment on PlanetLab shows that our decentralized architecture is significantly more efficient than a centralized approach in terms of the time required to correlate alerts.     

多步攻击告警关联模型构建与实现

为精简入侵检测系统产生的大量报警信息和分析攻击者的目的和动机,提出了新的报警信息关联模型。该模型通过事件关联把具有相似关系的报警信息关联后存储为元报警,然后根据报警类型知识库转换为超报警,最后根据超报警之间的因果关系进行攻击关联,构建出攻击关联图。实验表明,该模型提高了报警处理效率,对识别攻击意图和提高报警准确性有较好的效果。     

分布式入侵告警关联分析

为了精简分布式入侵检测系统中重复性的、不完善的或不完整的告警数据，降低误告警率，解决具有因果关系和非因果关系共存的告警关联问题，提出了一种分级关联算法．利用告警数据的检测时间属性的接近度将关联分析分为两类：概率关联和因果关联．给出了自调节增量贝叶斯分类器和实时因果关联算法，从而实现了多种特征混合的告警关联，提高了告警关联率．使用MIT Lincoln Lab提供的2000 DARPA入侵检测攻击场景数据集LLDOS1．0对该算法进行了性能测试，实验结果验证了算法的有效性．     

基于IDMEF和分类的报警聚合

如何聚合来自不同IDS的报警以减少重复报警是分布式IDS研究的重要问题。设计了一种入侵检测消息交换格式(IDMEF)报警消息格式规范,通过对IDMEF数据模型中的Alert类扩展而成,可统一描述不同IDS上报的报警信息。为有效聚合IDMEF报警,对基于分类和相似度的原始报警聚合算法进行了改进,并在Snort平台上实现了报警聚合模块原型和IDMEF报警生成模块原型。在DARPA99数据集上的实验结果证明了该方案的有效性。     

Risk balance defense approach against intrusions for network server

The paper presents a new defense approach based on risk balance to protect network servers from intrusion activities. We construct and implement a risk balance system, which consists of three modules, including a comprehensive alert processing module, an online risk assessment module, and a risk balance response decision-making module. The alert processing module improves the information quality of intrusion detection system (IDS) raw alerts by reducing false alerts rate, forming alert threads, and computing general parameters from the alert threads. The risk assessment module provides accurate evaluation of risks accordingly to alert threads. Based on the risk assessment, the response decision-making module is able to make right response decisions and perform very well in terms of noise immunization. Having advantages over conventional intrusion response systems, the risk balancer protects network servers not by directly blocking intrusion activities but by redirecting related network traffics and changing service platform. In this way, the system configurations that favor attackers are changed, and attacks are stopped with little impact on services to users. Therefore, the proposed risk balance approach is a good solution to not only the trade-off between the effectiveness and the negative effects of responses but also the false response problems caused by both IDS false-positive alerts and duplicated alerts.     

Human perspective to anomaly detection for cybersecurity

Traditionally signature-based network Intrusion Detection Systems (IDS) rely on inputs from domain experts and can only identify the attacks if they occur as individual event. IDS generate large number of alerts and it becomes very difficult for human users to go through each message. Previous researches have proposed analytics based approaches to analyze IDS alert patterns based on anomaly detection models, multi-steps models or probabilistic approaches. However, due to the complexities of network intrusions, it is impossible to develop all possible attack patterns or to avoid false positives. With the advance in technologies and popularity of networks in our daily life, it is becoming more and more difficult to detect network intrusions. However, no matter how rapid the technologies change, the human behaviors behind the cyber attacks stay relatively constant. This provides us an opportunity to develop an improved system to detect the unusual cyber attacks. In this paper, we developed four network intrusion models based on consideration of human factors. We then tested these models on ITOC Cyber Defense Competition (CDX) 2009 data. Our results are encouraging. These Models are not only able to recognize most network attacks identified by SNORT log alerts, they are also able to distinguish the non-attack network traffic that was potentially missed by SNORT as indicated by ground truth validation of the data.     

基于攻击图的复合入侵关联及预测方法

当前的大多数漏洞扫描器和入侵检测系统只能检测汇报孤立的漏洞和攻击。但网络中真正的威胁来自那些技术精湛的黑客,他们综合利用各种攻击手段绕开安全策略,逐步获得权限。这种复合攻击能渗透进看似防御严密的网络。攻击图是一种重要的网络安全漏洞分析工具,能用来关联警报,假设漏报,预测下一步的警报,对系统管理员理解威胁的本质从而采取适当对策是关键的。本文提出一种基于攻击图来关联并预测复合网络入侵的方法,该方法在实际网络环境中有良好的表现。     

分布式IDS的报警关联定义

网络规模越来越大。传统的IDS往往存在漏报误报率高、报警太低级的问题,因而不能及时准确反映整个系统的安全态势。网络管理人员不得不面对海量的原始报警信息,如大海捞针般地寻找可能的安全威胁和攻击来源。本文首先讨论了现有IDS的不足;之后给出了报警关联的定义。本文的研究成果已经在“网络安全监控与预警系统”（十五863项目）中得到应用,对分布式入侵检测系统的报警关联设计有重要的参考价值。     

基于可拓学的网络安全报警分析技术研究*

用户的网络管理需要建立一种新型的综合网络安全管理解决方案,即统一网络安全管理。特别关注于其中的一个关键功能——报警分析。其思路是以IDS报警为中心,将报警分析过程分解为包含报警评估与报警相关的两级关联分析模式。为了有效克服现今IDS报警分析技术中存在的问题和局限,顺应网络安全管理的统一化趋势,引入在解决矛盾问题方面极具优势的可拓学,以保证网络安全报警分析各种功能在技术上的实现。     

入侵检测系统中报警验证模块的设计与实现

传统入侵检测系统虽然可以根据特征匹配的方法检测出攻击企图,却无法验证攻击企图是否成功,生成的报警不仅数量巨大而且误警率很高。该文提出一种结合漏洞扫描工具对入侵检测系统生成的报警进行验证的方法,根据被攻击主机是否包含能使攻击成功的漏洞来判定攻击能否成功,对攻击的目标主机不存在对应漏洞的报警降低优先级,从而提高报警质量。说明了报警验证模型各部分的设计和实现方法,系统运行结果显示该方法能有效地压缩报警量,降低误警率,帮助管理员从大量数据中找到最应该关注的真实报警。