Two-tier network anomaly detection model: a machine learning approach

Network anomaly detection is one of the most challenging fields in cyber security. Most of the proposed techniques have high computation complexity or based on heuristic approaches. This paper proposes a novel two-tier classification models based on machine learning approaches Naïve Bayes, certainty factor voting version of KNN classifiers and also Linear Discriminant Analysis for dimension reduction. Experimental results show a desirable and promising gain in detection rate and false alarm compared with other existing models. The model also trained by two generated balance training sets using SMOTE method to evaluate the chosen similarity measure for dealing with imbalanced network anomaly data sets. The two-tier model provides low computation time due to optimal dimension reduction and feature selection, as well as good detection rate against rare and complex attack types which are so dangerous because of their close similarity to normal behaviors like User to Root and Remote to Local. All evaluation processes experimented by NSL-KDD data set.     

融合帧间时序关系的标准胎儿四腔心超声切面自动获取

目的 超声医师手动探查与采集胎儿心脏切面图像时,常因频繁的手动暂停与截图操作而错失心脏切面最佳获取时机。而单纯采用深层视觉目标检测或分类网络自动获取切面时,因无法确保网络重点关注切面图像中相对较小的心脏区域的细粒度特征,导致高误检率;另外,不同的心脏解剖部件的最佳成像时刻也常常不同步。针对上述问题,提出一种目标检测与分类网络相结合,同时融合关键帧间时序关系的标准四腔心（four-chamber,4CH）切面图像自动获取算法。方法 首先,利用自行构建的胎儿心脏超声切面数据集训练目标检测网络,实现四腔心区域和降主动脉区域的快速准确定位。接着,当检测到在一定时间窗内的视频帧存在降主动脉区域时,将包含四腔心目标的候选区域提取后送入利用自建的标准四腔心区域图像集训练好的分类网络,进一步分类出标准四腔心区域。最后,通过时序关系确定出可靠的降主动脉区域,将可靠降主动脉的检测置信度及同一时间窗内各个切面图像中四腔心区域在分类模型中的输出,加权计算得到标准四腔心切面图像的得分。结果 采用本文构建的数据集训练的YOLOv5x(you only look once version 5 extra large...     

基于NMEA-BP的WSN数据流异常检测算法

针对常规BPNN(Back Propagation Neural Network)容易陷入局部最优解、收敛速度慢等问题,提出了一种基于小生境思维进化NMEA(Niche Mind Evolutionary Algorithm)及BPNN的传感器数据流异常检测算法(NMEA-BP).该算法利用NMEA的全局搜索性优化BPNN的参数,获得BPNN的最优权阈值,从而提高异常检测的准确性.为了评估算法的性能,使用因特尔伯克利实验室数据集IBRL(Intel Berkeley Research Lab)及带标记的传感网络数据集LWSNDR(Labeled Wireless Sensor Network Data Repository)完成了仿真实验,并与基于常规BPNN、支持向量机(Support Vector Machine)和极限学习机(Extreme Learning Machine)等3种异常检测算法作对比.仿真实验结果表明,与上述3种算法相比,NMEA-BP算法对各个数据集都具有较高的检测率和较低的误报率,检测率平均达到99.45％,误报率平均仅为1.45％.此外,NMEA-BP异常检测算法的模型训练时间比传统的BPNN异常检测算法平均减少30％以上.     

基于直推式方法的网络异常检测方法

网络异常检测技术是入侵检测领域研究的热点和难点内容,目前仍然存在着误报率较高、对建立检测模型的数据要求过高、在复杂的网络环境中由于"噪音"的影响而导致检测率不高等问题.基于改进的TCM-KNN(transductive confidence machines for K-nearest neighbors)置信度机器学习算法,提出了一种网络异常检测的新方法,能够在高置信度的情况下,使用训练的正常样本有效地对异常进行检测.通过大量基于著名的KDD Cup 1999数据集的实验,表明其相对于传统的异常检测方法在保证较高检测率的前提下,有效地降低了误报率.另外,在训练集有少量"噪音"数据干扰的情况下,其仍能保证较高的检测性能;并且在采用"小样本"训练集以及为了避免"维灾难"而进行特征选取等优化处理后,其性能没有明显的削减.     

基于加权样本选择与主动学习的视频异常行为检测算法*

随着日益突出的公共安全问题,视频监控异常行为检测成为计算机视觉的一个研究热点.文中结合视频异常检测数据集,提出基于加权样本选择与主动学习的视频异常行为检测算法.根据视频监控数据集的分布特性,选择合适的权重值消除不平衡数据集对分类器的影响,通过主动学习的方式选取少量异常不确定样本,不断迭代更新检测模型,适应复杂多变的异常事件.实验表明,在UCSD异常行为检测数据集上,相比传统方法,文中方法具有更好的检测性能     

基于混合式聚类算法的离群点挖掘在异常检测中的应用研究

为了提高异常检测系统的检测率,降低误警率,解决现有异常检测所存在的问题,将离群点挖掘技术应用到异常检测中,提出了一种基于混合式聚类算法的异常检测方法(NADHC)。该方法将基于距离的聚类算法与基于密度的聚类算法相结合从而形成新的混合聚类算法,通过k-中心点算法找出簇中心,进而去除隐蔽性较高的少量攻击行为样本,再将重复增加样本的方法结合基于密度的聚类算法计算出异常度,从而判断出异常行为。最后在KDD CUP 99数据集上进行实验仿真,验证了所提算法的可行性和有效性。     

一种基于局部时空特征的视频异常检测方案

针对目前大多数视频异常检测方案在局部异常检测上的不足,提出一种基于局部时空特征的视频异常检测方案。该方案先提取运动描述符,再量化拆分,对每个特征描述符使用不同标度的时间空间滤波器,获得各时间空间区域的平滑估计,为训练和测试视频计算出各区域的局部K最邻近(KNN)距离,根据上述局部KNN距离,得出测试和训练视频的总体分值。对总体分值排名,确定异常。将该方案在公共数据集(UCSD数据集、人群异常UMN数据集、U型转弯数据集)上进行测试,结果表明,该方案的误差率、曲线下面积等性能指标优于现有的视频异常检测算法。     

基于K-MEANS聚类的分支定界算法在网络异常检测中的应用

网络异常检测技术是入侵检测领域研究的热点之一。在异常检测中,针对其存在的对训练集中关键数据的 选取不准确、选取过程耗时较长、检测的误报率过高等问题,结合经典的K-MEANS算法和分支定界算法,建立起一 种网络异常检测模型,以有效地提高在大量训练集中选取关键数据的准确率,同时降低数据选取的时耗。通过大量基 于著名的KDD Cup 1999数据集的实验,表明此模型能够达到较高的检则准确性,并能有效地控制检测错误报警的发 生。     

基于特征压缩与分支剪裁的网络异常检测算法

提出一种改进的直推式网络异常检测算法,利用K-L变换降低计算欧氏距离特征向量的维数,采用分支限界树剪裁减少欧氏距离的计算次数。基于KDD CUP99数据集的实验验证了改进算法能提高网络异常检测的实时性,通过与基于单类支持向量机的异常检测算法的性能对比结果表明,改进算法在保证一定误报率的情况下具有较高的检测率。     

一种基于深度迁移学习的滚动轴承早期故障在线检测方法

近年来, 深度学习技术已在滚动轴承故障检测和诊断领域取得了成功应用, 但面对不停机情况下的早期故障在线检测问题, 仍存在着早期故障特征表示不充分、误报警率高等不足. 为解决上述问题, 本文从时序异常检测的角度出发, 提出了一种基于深度迁移学习的早期故障在线检测方法. 首先, 提出一种面向多域迁移的深度自编码网络, 通过构建具有改进的最大均值差异正则项和Laplace正则项的损失函数, 在自适应提取不同域数据的公共特征表示同时, 提高正常状态和早期故障状态之间特征的差异性; 基于该特征表示, 提出一种基于时序异常模式的在线检测模型, 利用离线轴承正常状态的排列熵值构建报警阈值, 实现在线数据中异常序列的快速匹配, 同时提高在线检测结果的可靠性. 在XJTU-SY数据集上的实验结果表明, 与现有代表性早期故障检测方法相比, 本文方法具有更好的检测实时性和更低的误报警数.     

改进的字节频度负载异常入侵检测方法

数据集内容的特性对基于负载的网络异常入侵检测系统准确度有很大影响。本文分析了训练集数据包之间的内容特性差异对基于字节频度分布的模型的影响,较大的差异可能会导致分组计算频度均值的模型产生较高的误报率。本文据此提出了一种改进的模型—单包频度分布模型,以单个数据包的频度分布特征构成正常行为集,并以聚类方法控制其规模。在模拟数据集和DARPA99数据集上的实验表明,训练集数据包内容特性的差异确实导致基于均值的字节频度模型产生更多的误报,单包频度分布模型则不受影响,它有更高的检测准确度,在同等检测率下误报率更低。在数据包相互完全不同的情况下,基于均值的模型甚至失效。可认为单包频度分布模型对具有丰富动态内容的网络服务将有良好的适应能力。     

网络流量异常检测中分类器的提取与训练方法研究

随着网络安全领域研究的不断深入,研究者提出了各种类型的流量异常检测方法,基于分类的方法是其中很重要的一类.但是因为网络环境的多样性和动态变化性,在训练数据集上具有很高精度的检测系统实际部署时可能出现大量的误报.文中针对训练模型难于获取以及部署环境的动态变化性问题,对分类器的选择、使用和训练方法进行了研究.首先把网络流量数据投影到不同维度的Hash直方图上构建检测向量,在检测向量的基础上对比了各类分类器,选用能够处理高维数据、泛化能力强的SVDD进行异常检测;采用增减式在线训练算法对分类器进行不断训练,提高异常检测系统的精度并减少训练成本;最后采用多步关联检测算法优化检测精度,并在新增样本中剔除明显的异常样本,减少训练成本提高分类精度.通过大量的真实网络流量数据验证了上述方法具有较高的检准率和较低的误报率,并能够有效减少训练成本.     

Intrusion detection in computer networks by a modular ensemble of one-class classifiers

Since the early days of research on intrusion detection, anomaly-based approaches have been proposed to detect intrusion attempts. Attacks are detected as anomalies when compared to a model of normal (legitimate) events. Anomaly-based approaches typically produce a relatively large number of false alarms compared to signature-based IDS. However, anomaly-based IDS are able to detect never-before-seen attacks. As new types of attacks are generated at an increasing pace and the process of signature generation is slow, it turns out that signature-based IDS can be easily evaded by new attacks. The ability of anomaly-based IDS to detect attacks never observed in the wild has stirred up a renewed interest in anomaly detection. In particular, recent work focused on unsupervised or unlabeled anomaly detection, due to the fact that it is very hard and expensive to obtain a labeled dataset containing only pure normal events.The unlabeled approaches proposed so far for network IDS focused on modeling the normal network traffic considered as a whole. As network traffic related to different protocols or services exhibits different characteristics, this paper proposes an unlabeled Network Anomaly IDS based on a modular Multiple Classifier System (MCS). Each module is designed to model a particular group of similar protocols or network services. The use of a modular MCS allows the designer to choose a different model and decision threshold for different (groups of) network services. This also allows the designer to tune the false alarm rate and detection rate produced by each module to optimize the overall performance of the ensemble. Experimental results on the KDD-Cup 1999 dataset show that the proposed anomaly IDS achieves high attack detection rate and low false alarm rate at the same time.     

图结构多尺度变换的视频异常检测

目的 在监控场景的视频异常检测中,存在数据量大和检测速度慢的问题,为此提出图结构多尺度变换下的视频异常检测方法。方法 针对视频中光流特征的空间结构存在关联性,提出构建光流特征网络图结构,并在相关约束下利用光流特征图结构的迭代尺度化变换,有效降低视频异常检测中的光流特征数量,从而完成特征优化。光流特征图结构的尺度化变换首先利用光流特征图结构的图拉普拉斯矩阵所对应的最大特征向量的极性来筛选顶点,完成图的下采样操作;接着利用Kron规约构建顶点间的内在连接,重新构建光流特征图结构。结果 该方法能够提高视频异常检测算法的检测速度,但这是在略微降低检测精度的前提下实现的。在UMN数据集中,当尺度化图结构仅一次时的检测精度下降了3.2%,但检测速度提升了19.1%。这对整个视频集的检测速度的提升有明显效果。当尺度化次数为两次时的检测精度下降了7.3%,但这时检测效果达不到实际要求。此时,当尺度化图结构仅一次时异常检测的效果能达到预期。在Web数据集中,当尺度化图结构仅一次时,检测精度下降了1.9%,但检测速度提升了32%;尺度化两次时,检测精度降低了4.8%,检测速度提升了51%。因此,需要根据检测精度与检测速度的综合考虑后,选择尺度化次数是一次还是两次。但是随着尺度化次数的提高,这时检测效果就不能符合要求。结论 本文利用不规则的网络图结构来更好地表述特征之间的空间关系,并且多尺度变换后图结构也能表述特征间仍然保留有较强的空间关系。在不同的视频监控场景下,根据对检测精度与检测速度的综合考虑后选择合适的尺度化次数,从而实现快速异常检测。     

边缘计算应用：传感数据异常实时检测算法

随着物联网技术的不断发展,已逐步进入“万物互联”的新时代.针对物联网中实时采集的传感数据总体质量低下的问题,提出基于边缘计算的传感数据异常实时检测算法.该算法首先对相应的传感数据以“时间序列”的形式进行表示,并建立基于边缘计算的分布式传感数据异常检测模型;其次利用单源时间序列自身的连续性以及多源时间序列之间的相关性,分别对实时传感数据中出现的数据异常进行有效检测,并分别形成相应的异常检测结果集;最后将上述2个异常检测结果集进行有效地融合处理,从而得到更加准确的异常数据检测结果.通过实验验证该算法的检测准确性和有效性,结果显示：该算法检测时间短并且异常检出率高.     

一种SVM入侵检测的融合新策略

入侵检测是计算机网络安全中不可或缺的组成部分,其中异常检测更是该领域研究的热点内容。现有的检测方法中,SVM 能够在小样本条件下保持良好的检测状态。但是单一的SVM检测仍存在检测率不高、误报率过高等局限性。结合D-S证据理论,提出一种基于多SVM融合的异常检测方法,有效地弥补单个SVM检测的局限性。通过KDD99评测数据的评测实验表明,该方法有效地提高了入侵检测率的同时降低了误报率,大幅度地提高了入侵检测系统的检测性能。     

A-GHSOM: An adaptive growing hierarchical self organizing map for network anomaly detection

The growing hierarchical self organizing map (GHSOM) has been shown to be an effective technique to facilitate anomaly detection. However, existing approaches based on GHSOM are not able to adapt online to the ever-changing anomaly detection. This results in low accuracy in identifying intrusions, particularly “unknown” attacks. In this paper, we propose an adaptive GHSOM based approach (A-GHSOM) to network anomaly detection. It consists of four significant enhancements: enhanced threshold-based training, dynamic input normalization, feedback-based quantization error threshold adaptation, and prediction confidence filtering and forwarding. We first evaluate the A-GHSOM approach for intrusion detection using the KDD’99 dataset. Extensive experimental results demonstrate that compared with eight representative intrusion detection approaches, A-GHSOM achieves significant overall accuracy improvement and significant improvement in identifying “unknown” attacks while maintaining low false-positive rates. It achieves an overall accuracy of 99.63%, and 94.04% accuracy in identifying “unknown” attacks while the false positive rate is 1.8%. To avoid drawing research results and conclusions solely based on experiments with the KDD dataset, we have also built a dataset (TD-Sim) that consists of a mixture of live trace data from the Lawrence Berkeley National Laboratory and simulated traffic based on our testbed network, ensuring adequate coverage of a variety of attacks. Performance evaluation with the TD-Sim dataset shows that A-GHSOM adapts to live traffic and achieves an overall accuracy rate of 97.12% while maintaining the false positive rate of 2.6%.     

An efficient subsequence search for video anomaly detection and localization

This paper presents a novel framework for anomaly event detection and localization in crowded scenes. For anomaly detection, one-class support vector machine with Bayesian derivation is applied to detect unusual events. We also propose a novel event representation, called subsequence, which refers to a time series of spatial windows in proximity. Unlike recent works encoded an event with a 3D bounding box which may contain irrelevant information, e.g. background, a subsequence can concisely capture the unstructured property of an event. To efficiently locate anomalous subsequences in a video space, we propose the maximum subsequence search. The proposed search algorithm integrates local anomaly scores into a global consistent detection so that the start and end of an abnormal event can be determined under false and missing detections. Experimental results on two public datasets show that our method is robust to the illumination change and achieve at least 80% localization rate which approximately doubles the accuracy of recent works. This study concludes that anomaly localization is crucial in finding abnormal events.     

Video behavior profiling for anomaly detection

This paper aims to address the problem of modelling video behaviour captured in surveillancevideos for the applications of online normal behaviour recognition and anomaly detection. A novelframework is developed for automatic behaviour profiling and online anomaly sampling/detectionwithout any manual labelling of the training dataset. The framework consists of the followingkey components: (1) A compact and effective behaviour representation method is developed basedon discrete scene event detection. The similarity between behaviour patterns are measured basedon modelling each pattern using a Dynamic Bayesian Network (DBN). (2) Natural grouping ofbehaviour patterns is discovered through a novel spectral clustering algorithm with unsupervisedmodel selection and feature selection on the eigenvectors of a normalised affinity matrix. (3) Acomposite generative behaviour model is constructed which is capable of generalising from asmall training set to accommodate variations in unseen normal behaviour patterns. (4) A run-timeaccumulative anomaly measure is introduced to detect abnormal behaviour while normal behaviourpatterns are recognised when sufficient visual evidence has become available based on an onlineLikelihood Ratio Test (LRT) method. This ensures robust and reliable anomaly detection and normalbehaviour recognition at the shortest possible time. The effectiveness and robustness of our approachis demonstrated through experiments using noisy and sparse datasets collected from both indoorand outdoor surveillance scenarios. In particular, it is shown that a behaviour model trained usingan unlabelled dataset is superior to those trained using the same but labelled dataset in detectinganomaly from an unseen video. The experiments also suggest that our online LRT based behaviourrecognition approach is advantageous over the commonly used Maximum Likelihood (ML) methodin differentiating ambiguities among different behaviour classes observed online.     

基于流量异常分析多维优化的入侵检测方法

入侵检测系统在检测和预防各种网络异常行为的过程中,海量和高维的流量数据使其面临着低准确率和高误报率的问题。本文提出一种基于流量异常分析多维优化的入侵检测方法,该方法在入侵检测数据的横向维度和纵向维度两个维度进行优化。在横向维度优化中,对数量较多的类别进行数据抽样,并采用遗传算法得到每个类别的最佳抽样比例参数,完成数据的均衡化。在纵向维度优化中,结合特征与类别的相关分析,采用递归特征添加算法选择特征,并提出平均召回率指标评估特征选择效果,实现训练集的低维高效性。基于优化的入侵检测数据,进一步通过训练数据集得到随机森林分类器,在真实数据集UNSW_NB15评估和验证本文提出的算法。与其他算法相比,本文算法具有高准确率和低误报率,并在攻击类型上取得了有效的召回率。