Evolving fuzzy neural networks for supervised/unsupervised onlineknowledge-based learning

This paper introduces evolving fuzzy neural networks (EFuNNs) as a means for the implementation of the evolving connectionist systems (ECOS) paradigm that is aimed at building online, adaptive intelligent systems that have both their structure and functionality evolving in time. EFuNNs evolve their structure and parameter values through incremental, hybrid supervised/unsupervised, online learning. They can accommodate new input data, including new features, new classes, etc., through local element tuning. New connections and new neurons are created during the operation of the system. EFuNNs can learn spatial-temporal sequences in an adaptive way through one pass learning and automatically adapt their parameter values as they operate. Fuzzy or crisp rules can be inserted and extracted at any time of the EFuNN operation. The characteristics of EFuNNs are illustrated on several case study data sets for time series prediction and spoken word classification. Their performance is compared with traditional connectionist methods and systems. The applicability of EFuNNs as general purpose online learning machines, what concerns systems that learn from large databases, life-long learning systems, and online adaptive systems in different areas of engineering are discussed.     

Self-adaptive and dynamic clustering for online anomaly detection

As recent Internet threats are evolving more rapidly than ever before, one of the major challenges in designing an intrusion detection system is to provide early and accurate detection of emerging threats. In this study, a novel framework is developed for fully unsupervised training and online anomaly detection. The framework is designed so that an initial model is constructed and then it gradually evolves according to the current state of online data without any human intervention. In the framework, a self-organizing map (SOM) that is seamlessly combined with K-means clustering is transformed into an adaptive and dynamic algorithm suitable for real-time processing. The performance of the proposed approach is evaluated through experiments using the well-known KDD Cup 1999 data set and further experiments using the honeypot data recently collected from Kyoto University. It is shown that the proposed approach can significantly increase the detection rate while the false alarm rate remains low. In particular, it is capable of detecting new types of attacks at the earliest possible time.     

在线自适应网络异常检测系统模型与算法

随着因特网等计算机网络应用的增加,安全问题越来越突出,对具有主动防御特征的入侵检测系统的需求日趋紧迫.提出一个轻量级的在线自适应网络异常检测系统模型,给出了相关算法.系统能够对实时网络数据流进行在线学习和检测,在少量指导下逐渐构建网络的正常模式库和入侵模式库,并根据网络使用特点动态进行更新.在检测阶段,系统能够对异常数据进行报警,并识别未曾见过的新入侵.系统结构简单,计算的时间复杂度和空间复杂度都很低,满足在线处理网络数据的要求.在DARPAKDD99入侵检测数据集上进行测试,10%训练集数据和测试集数据以数据流方式顺序一次输入系统,在40s之内系统完成所有学习和检测任务,并达到检测率91.32%和误报率0.43%的结果.实验结果表明系统实用性强,检测效果令人满意,而且在识别新入侵上有良好的表现.     

基于TSK模糊控制系统的网络异常检测

在网络异常检测中,为了提高对异常状态的检测率,降低对正常状态的误判率,该文提出利用TSK模糊控制系统进行网络异常检测的新方法。在对TSK模糊控制系统的训练中采取梯度下降算法,充分发挥梯度下降局部细致搜索优势。实验数据采用KDDCUP99数据集,实验结果表明,基于梯度下降的模糊控制系统提高了异常检测的准确性。     

DENFIS: dynamic evolving neural-fuzzy inference system and itsapplication for time-series prediction

This paper introduces a new type of fuzzy inference systems, denoted as dynamic evolving neural-fuzzy inference system (DENFIS), for adaptive online and offline learning, and their application for dynamic time series prediction. DENFIS evolve through incremental, hybrid (supervised/unsupervised), learning, and accommodate new input data, including new features, new classes, etc., through local element tuning. New fuzzy rules are created and updated during the operation of the system. At each time moment, the output of DENFIS is calculated through a fuzzy inference system based on m-most activated fuzzy rules which are dynamically chosen from a fuzzy rule set. Two approaches are proposed: (1) dynamic creation of a first-order Takagi-Sugeno-type fuzzy rule set for a DENFIS online model; and (2) creation of a first-order Takagi-Sugeno-type fuzzy rule set, or an expanded high-order one, for a DENFIS offline model. A set of fuzzy rules can be inserted into DENFIS before or during its learning process. Fuzzy rules can also be extracted during or after the learning process. An evolving clustering method (ECM), which is employed in both online and offline DENFIS models, is also introduced. It is demonstrated that DENFIS can effectively learn complex temporal sequences in an adaptive way and outperform some well-known, existing models     

A-GHSOM: An adaptive growing hierarchical self organizing map for network anomaly detection

The growing hierarchical self organizing map (GHSOM) has been shown to be an effective technique to facilitate anomaly detection. However, existing approaches based on GHSOM are not able to adapt online to the ever-changing anomaly detection. This results in low accuracy in identifying intrusions, particularly “unknown” attacks. In this paper, we propose an adaptive GHSOM based approach (A-GHSOM) to network anomaly detection. It consists of four significant enhancements: enhanced threshold-based training, dynamic input normalization, feedback-based quantization error threshold adaptation, and prediction confidence filtering and forwarding. We first evaluate the A-GHSOM approach for intrusion detection using the KDD’99 dataset. Extensive experimental results demonstrate that compared with eight representative intrusion detection approaches, A-GHSOM achieves significant overall accuracy improvement and significant improvement in identifying “unknown” attacks while maintaining low false-positive rates. It achieves an overall accuracy of 99.63%, and 94.04% accuracy in identifying “unknown” attacks while the false positive rate is 1.8%. To avoid drawing research results and conclusions solely based on experiments with the KDD dataset, we have also built a dataset (TD-Sim) that consists of a mixture of live trace data from the Lawrence Berkeley National Laboratory and simulated traffic based on our testbed network, ensuring adequate coverage of a variety of attacks. Performance evaluation with the TD-Sim dataset shows that A-GHSOM adapts to live traffic and achieves an overall accuracy rate of 97.12% while maintaining the false positive rate of 2.6%.     

SoHyFIS-Yager: A self-organizing Yager based Hybrid neural Fuzzy Inference System

The Hybrid neural Fuzzy Inference System (HyFIS) is a multilayer adaptive neural fuzzy system for building and optimizing fuzzy models using neural networks. In this paper, the fuzzy Yager inference scheme, which is able to emulate the human deductive reasoning logic, is integrated into the HyFIS model to provide it with a firm and intuitive logical reasoning and decision-making framework. In addition, a self-organizing gaussian Discrete Incremental Clustering (gDIC) technique is implemented in the network to automatically form fuzzy sets in the fuzzification phase. This clustering technique is no longer limited by the need to have prior knowledge about the number of clusters present in each input and output dimensions. The proposed self-organizing Yager based Hybrid neural Fuzzy Inference System (SoHyFIS-Yager) introduces the learning power of neural networks to fuzzy logic systems, while providing linguistic explanations of the fuzzy logic systems to the connectionist networks. Extensive simulations were conducted using the proposed model and its performance demonstrates its superiority as an effective neuro-fuzzy modeling technique.     

Fast-flux hunter: a system for filtering online fast-flux botnet

Fast-flux networks is a domain name system (DNS) technique used by botnets, which is hiding some attack like phishing and malware delivery sites behind associate dynamical network of compromised hosts acting as proxies, that sometimes hosts malicious content. Detection of fast-flux networks continues to be a difficult issue attributable to the similar behavior between these networks and alternative legitimate infrastructures, like server farms and content distribution networks. This study seeks to improve the detection and prediction of the unknown “zero-day” online fast-flux botnet. This improvement will be achieved using a new system called the fast-flux hunter (FFH), which supports a new adaptive evolving fuzzy neural network algorithm. The FFH system is a hybrid between the supervised and unsupervised online knowledge-based learning systems. The core mechanism of the FFH is based on the inherent feature of the fast-flux networks. It uses a collection of DNS traffic information. The FFH is able to scan over 7615 domain records and extract 14 distinct features for each domain. The FFH decreases the classification method’s error rate. The FFH has a detection accuracy rate of approximately 98 % and is compatible with life-long learning systems, footprint-consuming memory, and high-speed systems.

     

基于张量分解的数据流异常检测

本文对基于分布式的演化数据流的连续异常检测问题进行了形式化描述,提出一种在滑动窗口中基于张量分解的异常检测算法--WSTA.该算法将各分布结点上的数据流作为全局数据流的子张量,通过分布结点与中心节点的通信,在分布结点的滑动窗口中自适应抽样生成概要数据结构矩阵.对该数据矩阵进行张量分解得到特征向量,然后采用基于距离的异常检测方法发现异常点.基于大量真实数据集的实验表明,此算法具有良好的适用性和可扩展性.     

基于PSO算法的模糊神经网络的网络异常检测

在网络异常检测中,为了提高对异常状态的检测率,降低对正常状态的误判率,提出一种基于粒子群优化算法训练模糊神经网络进行网络异常检测的新方法。在对模糊神经网络训练中采取PSO算法和梯度下降算法相结合的方法,充分发挥PSO全局寻优的能力和梯度下降局部细致搜索优势。实验数据采用KDD CUP99数据集,实验结果表明,该学习算法与传统的梯度下降法（GD）相比,收敛速度快,具有更好的全局收敛性,提高了异常检测的准确性,同时该方法对于新的异常也有较高检测率。     

An adaptive algorithm for anomaly and novelty detection in evolving data streams

In the era of big data, considerable research focus is being put on designing efficient algorithms capable of learning and extracting high-level knowledge from ubiquitous data streams in an online fashion. While, most existing algorithms assume that data samples are drawn from a stationary distribution, several complex environments deal with data streams that are subject to change over time. Taking this aspect into consideration is an important step towards building truly aware and intelligent systems. In this paper, we propose GNG-A, an adaptive method for incremental unsupervised learning from evolving data streams experiencing various types of change. The proposed method maintains a continuously updated network (graph) of neurons by extending the Growing Neural Gas algorithm with three complementary mechanisms, allowing it to closely track both gradual and sudden changes in the data distribution. First, an adaptation mechanism handles local changes where the distribution is only non-stationary in some regions of the feature space. Second, an adaptive forgetting mechanism identifies and removes neurons that become irrelevant due to the evolving nature of the stream. Finally, a probabilistic evolution mechanism creates new neurons when there is a need to represent data in new regions of the feature space. The proposed method is demonstrated for anomaly and novelty detection in non-stationary environments. Results show that the method handles different data distributions and efficiently reacts to various types of change.     

Integrative connectionist learning systems inspired by nature: current models, future trends and challenges

The so far developed and widely utilized connectionist systems (artificial neural networks) are mainly based on a single brain-like connectionist principle of information processing, where learning and information exchange occur in the connections. This paper extends this paradigm of connectionist systems to a new trend—integrative connectionist learning systems (ICOS) that integrate in their structure and learning algorithms principles from different hierarchical levels of information processing in the brain, including neuronal-, genetic-, quantum. Spiking neural networks (SNN) are used as a basic connectionist learning model which is further extended with other information learning principles to create different ICOS. For example, evolving SNN for multitask learning are presented and illustrated on a case study of person authentification based on multimodal auditory and visual information. Integrative gene-SNN are presented, where gene interactions are included in the functioning of a spiking neuron. They are applied on a case study of computational neurogenetic modeling. Integrative quantum-SNN are introduced with a quantum Hebbian learning, where input features as well as information spikes are represented by quantum bits that result in exponentially faster feature selection and model learning. ICOS can be used to solve more efficiently challenging biological and engineering problems when fast adaptive learning systems are needed to incrementally learn in a large dimensional space. They can also help to better understand complex information processes in the brain especially how information processes at different information levels interact. Open questions, challenges and directions for further research are presented.     

A new systematic design for Habitually Linear Evolving TS Fuzzy Model

In this paper, a systematic design of habitually evolving Takagi-Sugeno (TS) fuzzy systems, suggested for online prediction of processes with uncertainty, is introduced. A Habitually Linear Evolving Fuzzy System (HLEFS) starts off with an adaptive linear model and evolves into a TS fuzzy model whenever the linear model is unable to mitigate the output error. The number of rules in the HLEFS is controlled by an adaptive threshold on the error. The structure of the HLEFS tends to return to the adaptive linear model as soon as possible, and that is why we have dubbed the proposed model ‘Habitually’ Linear. Three theorems are stated and proved in a sequence in support of the HLEFS ability to keep the output error in a relatively small bound. It is shown that the adaptive linear model may not be good enough when the process changes abruptly and nonlinearly—what we call a Transient Significant Disturbance. In this case, it is proved that evolving into a TS fuzzy system with the proposed algorithm can mitigate the error.The performance of HLEFS in forecasting of daily electrical power consumption is studied and compared with that of four famous existing evolving fuzzy systems. Obtained results demonstrate the applicability and effectiveness of the proposed method in keeping the prediction error low with less number of fuzzy rules.     

一种新颖的异常检测自适应模型

针对目前异常入侵检测系统误报率过高、自适应能力不强等问题,提出知识库的完备度、自相似度等概念,构造一种新颖的异常入侵检测自适应模型.使入侵检测系统能够根据自身的学习情况自动调节异常和正常的判断准则,从而增强系统的自适应能力,有效降低系统的误报率,提高入侵检测的准确度.     

基于免疫和模糊综合评判的入侵检测模型研究

应用人体免疫系统的特异性免疫的分类，设计了一个入侵检测模型，将入侵检测模块分为固有检测模块和适应性检测模块。固有检测模块考虑继承目前已有的知识；适应性检测模块针对目前异常检测算法难以确定评判正常和异常的阈值以及检测特征数量多难以综合评判的问题，提出了一种具体的异常检测算法——FLADA。该算法借鉴了模糊数学的理论，采用模糊综合评判和层次分析法相结合。实验证明，该方法不仅能准确地检测出已知攻击，还能较好地检测出未知攻击。     

Deep Domain-Adversarial Anomaly Detection With One-Class Transfer Learning

Despite the big success of transfer learning techniques in anomaly detection, it is still challenging to achieve good transition of detection rules merely based on the preferred data in the anomaly detection with one-class classification, especially for the data with a large distribution difference. To address this challenge,a novel deep one-class transfer learning algorithm with domain-adversarial training is proposed in this paper. First, by integrating a hypersphere adaptation constraint into...     

Human behavior clustering for anomaly detection

This paper aims to address the problem of modeling human behavior patterns captured in surveillance videos for the application of online normal behavior recognition and anomaly detection. A novel framework is developed for automatic behavior modeling and online anomaly detection without the need for manual labeling of the training data set. The framework consists of the following key components. 1) A compact and effective behavior representation method is developed based on spatial-temporal interest point detection. 2) The natural grouping of behavior patterns is determined through a novel clustering algorithm, topic hidden Markov model (THMM) built upon the existing hidden Markov model (HMM) and latent Dirichlet allocation (LDA), which overcomes the current limitations in accuracy, robustness, and computational efficiency. The new model is a four-level hierarchical Bayesian model, in which each video is modeled as a Markov chain of behavior patterns where each behavior pattern is a distribution over some segments of the video. Each of these segments in the video can be modeled as a mixture of actions where each action is a distribution over spatial-temporal words. 3) An online anomaly measure is introduced to detect abnormal behavior, whereas normal behavior is recognized by runtime accumulative visual evidence using the likelihood ratio test (LRT) method. Experimental results demonstrate the effectiveness and robustness of our approach using noisy and sparse data sets collected from a real surveillance scenario.     

Extensions of vector quantization for incremental clustering

In this paper, we extend the conventional vector quantization by incorporating a vigilance parameter, which steers the tradeoff between plasticity and stability during incremental online learning. This is motivated in the adaptive resonance theory (ART) network approach and is exploited in our paper for forming a one-pass incremental and evolving variant of vector quantization. This variant can be applied for online clustering, classification and approximation tasks with an unknown number of clusters. Additionally, two novel extensions are described: one concerns the incorporation of the sphere of influence of clusters in the vector quantization learning process by selecting the ‘winning cluster’ based on the distances of a data point to the surface of all clusters. Another one introduces a deletion of cluster satellites and an online split-and-merge strategy: clusters are dynamically split and merged after each incremental learning step. Both strategies prevent the algorithm to generate a wrong cluster partition due to a bad a priori setting of the most essential parameter(s). The extensions will be applied to clustering of two- and high-dimensional data, within an image classification framework and for model-based fault detection based on data-driven evolving fuzzy models.     

An approach to online identification of Takagi-Sugeno fuzzy models.

An approach to the online learning of Takagi-Sugeno (TS) type models is proposed in the paper. It is based on a novel learning algorithm that recursively updates TS model structure and parameters by combining supervised and unsupervised learning. The rule-base and parameters of the TS model continually evolve by adding new rules with more summarization power and by modifying existing rules and parameters. In this way, the rule-base structure is inherited and up-dated when new data become available. By applying this learning concept to the TS model we arrive at a new type adaptive model called the Evolving Takagi-Sugeno model (ETS). The adaptive nature of these evolving TS models in combination with the highly transparent and compact form of fuzzy rules makes them a promising candidate for online modeling and control of complex processes, competitive to neural networks. The approach has been tested on data from an air-conditioning installation serving a real building. The results illustrate the viability and efficiency of the approach. The proposed concept, however, has significantly wider implications in a number of fields, including adaptive nonlinear control, fault detection and diagnostics, performance analysis, forecasting, knowledge extraction, robotics, behavior modeling.     

基于主题隐马尔科夫模型的人体异常行为识别

针对基于监控视频的人体异常行为识别问题,提出了基于主题隐马尔科夫模型的人体异常行为识别方法,即通过无任何人工标注的视频训练集自动学习人体行为模型,并能够应用学到的人体行为模型实时检测异常行为和识别正常行为。这一方法主要围绕"低层视频表示-中层语义行为建模-高层语义分类"3个方面进行:1)基于时-空间兴趣点构建了一种紧凑的和有效的视频表示方法。2)提出一种新颖的语义主题模型(Topic Model,TM)——主题隐马尔科夫模型(Topic Hidden Markov Model,THMM),它能够自然地分组视频中检测到的人体行为。主题隐马尔科夫模型基于已有的马尔科夫模型和主题模型构造,不但聚类运动词汇成简单动作,而且聚类简单动作成全局行为,同时建模了行为时间上的相关性。THMM是一个4层贝叶斯主题模型,它将视频序列建模为行为的马尔科夫链,同时行为是视频序列中某些视频剪辑(Clip)的概率分布;将视频剪辑建模为动作的随机组合,同时动作是视频剪辑中运动词汇的概率分布。克服了传统隐马尔科夫模型和主题模型在人体复杂行为建模过程中精度、鲁棒性和计算效率上的不足。3)提出运行时累积的异常性测度及其在线异常行为检测方法和基于在线似然比检验(Likelihood Ratio Test,LRT)的实时正常行为分类方法,从而克服了实时行为识别过程中由于缺乏充分的视觉证据而引发的行为类型歧义,能完较好地完成监控场景中实时异常行为检测和在线正常行为识别的任务。取自实际监控场景的实验数据集上的实验结果证明了本方法的有效性。