用神经网络驱动的模糊推理入侵检测方法

提出了神经网络驱动模糊推理的入侵检测方法,利用神经网络的学习能力,对不清楚规则的复杂系统的输入输出特性进行适当的非线性划分,自动形成舰则集和相应的隶属关系,克服了在多维空间上经验性的确定隶属函数的困难。对于神经网络的训练数据,采用人工数据,克服了神经网络监督学习和获取已知输出的训练数据的困难。试验证明,这种技术具有很好的灵敏度和鲁棒性,而且,能够检测出未知的入侵行为。     

网络管理告警数据库中时序规则挖掘的一种新方法

提出一种基于“最小发生的双时间窗口约束”时序规则挖掘新方法。该方法依据“双时间窗口”约束和“最小发生”判据,可判别在一个时间窗内的哪些告警事件导致了另一个时间窗内告警集合事件的产生,快速寻找出不同网络设备告警与其它网络设备告警之间的关联知识。通过对采集某省级IP网的告警数据库进行规则挖掘实验,表明该方法可以准确、快速地挖掘出海量网络告警数据库中大量有意义的时序规则,这些规则可作为选验知识来指导网络智能化故障定位、诊断及预测。     

Automated adaptive intrusion containment in systems of interacting services

Large scale distributed systems typically have interactions among different services that create an avenue for propagation of a failure from one service to another. The failures being considered may be the result of natural failures or malicious activity, collectively called disruptions. To make these systems tolerant to failures it is necessary to contain the spread of the occurrence automatically once it is detected. The objective is to allow certain parts of the system to continue to provide partial functionality in the system in the face of failures. Real world situations impose several constraints on the design of such a disruption tolerant system of which we consider the following – the alarms may have type I or type II errors; it may not be possible to change the service itself even though the interaction may be changed; attacks may use steps that are not anticipated a priori; and there may be bursts of concurrent alarms. We present the design and implementation of a system named Adepts as the realization of such a disruption tolerant system. Adepts uses a directed graph representation to model the spread of the failure through the system, presents algorithms for determining appropriate responses and monitoring their effectiveness, and quantifies the effect of disruptions through a high level survivability metric. Adepts is demonstrated on a real e-commerce testbed with actual attack patterns injected into it.     

An incremental neural learning framework and its application to vehicle diagnostics

This paper presents a framework for incremental neural learning (INL) that allows a base neural learning system to incrementally learn new knowledge from only new data without forgetting the existing knowledge. Upon subsequent encounters of new data examples, INL utilizes prior knowledge to direct its incremental learning. A number of critical issues are addressed including when to make the system learn new knowledge, how to learn new knowledge without forgetting existing knowledge, how to perform inference using both the existing and the newly learnt knowledge, and how to detect and deal with aged learnt systems. To validate the proposed INL framework, we use backpropagation (BP) as a base learner and a multi-layer neural network as a base intelligent system. INL has several advantages over existing incremental algorithms: it can be applied to a broad range of neural network systems beyond the BP trained neural networks; it retains the existing neural network structures and weights even during incremental learning; the neural network committees generated by INL do not interact with one another and each sees the same inputs and error signals at the same time; this limited communication makes the INL architecture attractive for parallel implementation. We have applied INL to two vehicle fault diagnostics problems: end-of-line test in auto assembly plants and onboard vehicle misfire detection. These experimental results demonstrate that the INL framework has the capability to successfully perform incremental learning from unbalanced and noisy data. In order to show the general capabilities of INL, we also applied INL to three general machine learning benchmark data sets. The INL systems showed good generalization capabilities in comparison with other well known machine learning algorithms.     

Highly reliable message-passing mechanism for cluster file system

With the increase in personal computer clusters in popularity and quantity, message passing between nodes has been an important issue for high failure rate in the network. File access in a cluster file system often contains several sub-operations; each includes one or more network transmissions. Any network failures cause the file system service unavailable. In this paper, we describe a highly reliable message-passing mechanism (HR-NET), which tolerates both software and hardware network failures. HR-NET provides fine-grained, connection-level failover across redundant communication paths. With it, the file system can keep passing messages because HR-NET handles failures automatically by either recovery from network failures or failed over to a backup; therefore, it screens network failures from requests or data transmission of cluster file system. Load balance for messages is also achieved to relieve network traffic. For transmission timeout, HR-NET proposes a priority-based message scheduling which dynamically manages messages in an appropriate order to tolerate request–response failures between clients and servers. HR-NET is implemented upon standard network protocol stack. Performance results show that HR-NET can provide almost full underlying network bandwidth with average 6.17% throughput loss and provide a fast recovery. Experiments with cluster file system show that the overall performance degradation is below 8% due to failover of HR-NET while the reliability is highly enhanced.     

环境重现的非稳态流量异常检测系统设计

网络入侵检测中的流量异常检测方法存在着虚警率较高的问题，为此提出了结合环境重现的非稳态流量异常检测系统。该系统采取了基于发生新事件的先验概率和趋势来评估异常的思想，建立非稳态正常模型，采用环境重现进行进一步数据挖掘，最后给出了综合系统的原型设计。     

基于数据挖掘的自适应入侵检测框架设计

数据挖掘、人工神经网络和机器学习等技术在入侵检测中的广泛应用,大幅度地提高了检测引擎的精度,但误用检测中的漏报率和异常检测中的误报率仍然是入侵检测中的难题。论文结合误用检测和异常检测的特点,利用机器学习思想,设计实现了一种新型的具有自适应能力的复合式入侵检测系统。     

基于攻击图与报警相似性的混合报警关联模型

为了揭示入侵检测系统所生成的报警数据之间的关联关系和重构入侵攻击场景,提出了一种基于攻击图与报警数据相似性分析的混合报警关联模型。该模型结合攻击图和报警数据分析的优点,首先根据入侵攻击的先验知识定义初始攻击图,描述报警数据间的因果关联关系,再利用报警数据的相似性分析修正初始攻击图的部分缺陷,进而实现报警关联。实验结果表明,混合关联模型能够较好地恢复攻击场景,并能够完全修复攻击图中单个攻击步骤的缺失。     

Using Network Fault Predictions to Enable IP Traffic Management

IP traffic management is important for the continued growth of the Internet. Several traffic management algorithms exist today. However, to enable these algorithms it is necessary to provide reliable alarms relating to network performance bottlenecks and failures. In this work we propose an algorithm to obtain reliable predictive alarms for network fault conditions. The algorithm is based on modeling network fault behavior. The algorithm has been successfully tested on two production networks. Predictive alarms were obtained for four different types of failures: file server failures, network access problems, protocol implementation errors, and runaway processes. The potential of using this model to do fault classification is also discussed. In addition, it is shown that the proposed algorithm performs better than the majority-vote scheme.     

Using Bayesian networks for root cause analysis in statistical process control

Despite their fame and capability in detecting out-of-control conditions, control charts are not effective tools for fault diagnosis. There are other techniques in the literature mainly based on process information and control charts patterns to help control charts for root cause analysis. However these methods are limited in practice due to their dependency on the expertise of practitioners. In this study, we develop a network for capturing the cause and effect relationship among chart patterns, process information and possible root causes/assignable causes. This network is then trained under the framework of Bayesian networks and a suggested data structure using process information and chart patterns. The proposed method provides a real time identification of single and multiple assignable causes of failures as well as false alarms while improving itself performance by learning from mistakes. It also has an acceptable performance on missing data. This is demonstrated by comparing the performance of the proposed method with methods like neural nets and K-Nearest Neighbor under extensive simulation studies.     

A distributed parallel alarm management strategy for alarm reduction in chemical plants

A distributed parallel alarm management strategy based on massive historical alarms and distributed clustering algorithm is proposed to reduce the number of alarms presented to operators in modern chemical plants. Due to the large and growing scale of historical alarms as the basis of analysis, it is difficult for traditional alarm management strategy to store and analyze all alarms efficiently. In this paper, by designing the row key and storage structure in a distributed extensible NoSQL database, the strategy spreads alarm data in a group of commercial machines, which ensures the capacity and scalability of the whole system. Meanwhile, Distributed Parallel Query Model (DPQM) proposed as a unified query model provides efficient query and better integration of distributed platform. Based on the characteristics of alarms and time-delay correlation of alarm occurrence, alarm similarity criteria are proposed to effectively identify repetitive and homologous alarms. In order to group massive alarm data, a new distributed clustering algorithm is designed to work concurrently in MapReduce frameworks. The test results using alarm data from real chemical plants show that the strategy is better than traditional method based on MySQL at system performance, and provides excellent redundant alarm suppression in both normal situation and alarm flooding situation.     

New data mining technique to enhance IDS alarms quality

The intrusion detection systems (IDSs) generate large number of alarms most of which are false positives. Fortunately, there are reasons for triggering alarms where most of these reasons are not attacks. In this paper, a new data mining technique has been developed to group alarms and to produce clusters. Hereafter, each cluster abstracted as a generalized alarm. The generalized alarms related to root causes are converted to filters to reduce future alarms load. The proposed algorithm makes use of nearest neighboring and generalization concepts to cluster alarms. As a clustering algorithm, the proposed algorithm uses a new measure to compute distances between alarms features values. This measure depends on background knowledge of the monitored network, making it robust and meaningful. The new data mining technique was verified with many datasets, and the averaged reduction ratio was about 82% of the total alarms. Application of the new technique to alarms log greatly helps the security analyst in identifying the root causes; and then reduces the alarm load in the future.     

基于对抗学习和知识蒸馏的神经网络压缩算法

针对基于深度学习的人脸识别模型难以在嵌入式设备进行部署和实时性能差的问题,深入研究了现有的模型压缩和加速算法,提出了一种基于知识蒸馏和对抗学习的神经网络压缩算法。算法框架由三部分组成,预训练的大规模教师网络、轻量级的学生网络和辅助对抗学习的判别器。改进传统的知识蒸馏损失,增加指示函数,使学生网络只学习教师网络正确识别的分类概率;鉴于中间层特征图具有丰富的高维特征,引入对抗学习策略中的判别器,鉴别学生网络与教师网络在特征图层面的差异;为了进一步提高学生网络的泛化能力,使其能够应用于不同的机器视觉任务,在训练的后半部分教师网络和学生网络相互学习,交替更新,使学生网络能够探索自己的最优解空间。分别在CASIA WEBFACE和CelebA两个数据集上进行验证,实验结果表明知识蒸馏得到的小尺寸学生网络相较全监督训练的教师网络,识别准确率仅下降了1.5%左右。同时将本研究所提方法与面向特征图知识蒸馏算法和基于对抗学习训练的模型压缩算法进行对比,所提方法具有较高的人脸识别准确率。     

基于聚类的入侵检测研究综述

入侵检测通过收集各种网络数据,从中分析和发现可能的入侵攻击行为。为了增强入侵检测从海量数据中发现攻击行为的能力和提高入侵检测的智能性,数据挖掘被引入到入侵检测领域,以实现智能化的知识发现和入侵检测模型的建立。聚类分析是数据挖掘中的一种重要的技术,能够通过无监督的学习过程发现隐藏的模式,具有独立地发现知识的能力。现有大量关于其在入侵检测领域的应用研究,各种聚类分析方法及改进措施被用于从不同的训练数据集建立入侵检测模型,成为对整个检测系统的一个有力补充。对现有文献中典型的基于聚类的入侵检测模型作了全面的介绍和适当的比较分析,提出了进一步的研究建议。     

多维频繁情节挖掘在电信告警信息分析中的应用

电信网络每天都要产生大量的告警信息,这些信息中隐藏着网络结构相关的有用知识。基于对电信网络告警信息的特点的分析和针对现有挖掘方式的不足,论文提出一种从电信网络告警信息中挖掘频发的模式知识的思想方法———多维频繁情节挖掘。挖掘的多维频繁情节可以帮助网络管理人员分析告警信息和诊断故障。     

Preprocessing expert system for mining association rules in telecommunication networks

Recently, the application of association rules mining becomes an important research area in alarm correlation analysis. However, the original alarms in the telecommunication networks cannot be used to mine association rules directly. This paper proposes a novel preprocessing expert system model to deal with the original alarms. This model uses two important techniques, of which the time window technique is used for converting original alarms into transactions, and the neural network technique can classify the alarms with different levels according to the characteristics of telecommunication networks in order to mine the weighted association rules. Simulation results and the real-world applications demonstrate the effectiveness and practicality of this preprocessing expert system.     

基于知识点与错误率关联的个性化智能组卷模型

大数据环境下的个性化学习模型研究是大规模网络学习环境下的研究热点,本文针对传统的智能组卷策略存在数据训练不足、个性化特点不突出、题库试题知识点分布不均匀等问题,将大数据运用于组卷之中,提出了基于知识点权重与错误率关联的个性化训练模型,优化了抽题的法则并使得个性化特点更精确,在一定程度上有利于学生对薄弱点和盲点的深入理解与消化.本文采用将每章节题目的知识点转化为树形进行管理的方法,并在知识点树中加入知识点错误率元素,来优化基于知识点的抽题结果,研究出适合个人学习情况的个性化模拟练习策略.最后将此新研究模型应用于教学教育系统进行实验研究,研究表明对此关键点的改进更有利于普遍提升学生的整体成绩.