Network Intrusion Detection in Internet of Blended Environment Using Ensemble of Heterogeneous Autoencoders (E-HAE)

Contemporary attackers, mainly motivated by financial gain, consistently devise sophisticated penetration techniques to access important information or data. The growing use of Internet of Things (IoT) technology in the contemporary convergence environment to connect to corporate networks and cloud-based applications only worsens this situation, as it facilitates multiple new attack vectors to emerge effortlessly. As such, existing intrusion detection systems suffer from performance degradation mainly because of insufficient considerations and poorly modeled detection systems. To address this problem, we designed a blended threat detection approach, considering the possible impact and dimensionality of new attack surfaces due to the aforementioned convergence. We collectively refer to the convergence of different technology sectors as the internet of blended environment. The proposed approach encompasses an ensemble of heterogeneous probabilistic autoencoders that leverage the corresponding advantages of a convolutional variational autoencoder and long short-term memory variational autoencoder. An extensive experimental analysis conducted on the TON_IoT dataset demonstrated 96.02% detection accuracy. Furthermore, performance of the proposed approach was compared with various single model (autoencoder)-based network intrusion detection approaches: autoencoder, variational autoencoder, convolutional variational autoencoder, and long short-term memory variational autoencoder. The proposed model outperformed all compared models, demonstrating F1-score improvements of 4.99%, 2.25%, 1.92%, and 3.69%, respectively.     

Feature Selection for Detecting ICMPv6-Based DDoS Attacks Using Binary Flower Pollination Algorithm

Internet Protocol version 6 (IPv6) is the latest version of IP that goal to host 3.4 × 10³⁸ unique IP addresses of devices in the network. IPv6 has introduced new features like Neighbour Discovery Protocol (NDP) and Address Auto-configuration Scheme. IPv6 needed several protocols like the Address Auto-configuration Scheme and Internet Control Message Protocol (ICMPv6). IPv6 is vulnerable to numerous attacks like Denial of Service (DoS) and Distributed Denial of Service (DDoS) which is one of the most dangerous attacks executed through ICMPv6 messages that impose security and financial implications. Therefore, an Intrusion Detection System (IDS) is a monitoring system of the security of a network that detects suspicious activities and deals with a massive amount of data comprised of repetitive and inappropriate features which affect the detection rate. A feature selection (FS) technique helps to reduce the computation time and complexity by selecting the optimum subset of features. This paper proposes a method for detecting DDoS flooding attacks (FA) based on ICMPv6 messages using a Binary Flower Pollination Algorithm (BFPA-FA). The proposed method (BFPA-FA) employs FS technology with a support vector machine (SVM) to identify the most relevant, influential features. Moreover, The ICMPv6-DDoS dataset was used to demonstrate the effectiveness of the proposed method through different attack scenarios. The results show that the proposed method BFPA-FA achieved the best accuracy rate (97.96%) for the ICMPv6 DDoS detection with a reduced number of features (9) to half the total (19) features. The proven proposed method BFPA-FA is effective in the ICMPv6 DDoS attacks via IDS.     

基于滑动窗口的直升机序列异常检测算法

无标签的序列在异常检测算法中往往存在着对数据的信息掌握不全面、不能合理使用的情况,而采用深度学习的技术实现检测时往往对其计算的解释性欠佳;对于攻克这些难题,以直升机飞行数据为例对时间序列的反常检测问题展开了深入研究,并利用Iforest技术和PCA算法,给出了一个采用滑动窗口的时间序列异常检测方法,利用从滑动窗口采集信息的时间变化状态等数据信息,将序列异常检测问题转换为点异常检测问题;同时以auc评分为衡量标准,从带有时刻特殊标志的多个信息集上检验了检测效率的提高;在无标签的直升机飞行数据集上进行实验,验证了算法的有效性,并通过对比检测过程中不同特征变量的变化情况,从算法层面和现实层面上阐述了算法的可解释性。     

基于图自编码器的无监督多变量时间序列异常检测

针对多变量时间序列复杂的时间相关性和高维度使得异常检测性能较差的问题,以对抗训练框架为基础提出基于图自编码的无监督多变量时间序列异常检测模型.首先,将特征转换为嵌入向量来表示;其次,将划分好的时间序列结合嵌入向量转换为图结构数据;然后,用两个图自编码器模拟对抗训练重构数据样本;最后,根据测试数据在模型训练下的重构误差进行异常判定.将提出的方法与5种基线异常检测方法进行比较.实验结果表明,提出的模型在测试数据集获得了最高的F1分数,总体性能分F1分数比最新的异常检测模型USAD提高了28.4%.可见提出的模型有效提高异常检测性能.     

基于DAE和GRU组合的流量异常检测方法

流量异常检测能够有效识别网络流量数据中的攻击行为,是一种重要的网络安全防护手段。近年来,深度学习在流量异常检测领域得到了广泛应用,现有的深度学习模型进行流量异常检测存在两个问题:一是数据受噪声影响导致检测鲁棒性差、准确率低;二是数据特征维度高以及模型参数多导致训练和检测速度慢。为了在降低流量数据噪声影响的基础上提高检测速度和准确性,本文提出了一种基于去噪自编码器(Denoising Auto Encoder,DAE)和门控循环单元(Gated Recurrent Unit,GRU)组合的流量异常检测方法。首先设计了基于DAE的流量特征提取算法,采用小批量梯度下降算法对DAE进行训练,通过最小化含噪声数据的重构向量与原始输入向量间的差异,有效提取具有较强鲁棒性的流量特征,降低特征维度。然后设计了基于GRU的异常检测算法,利用提取的低维流量特征数据训练GRU,从而构建异常流量分类器,实现对攻击流量的准确检测。最后在NSL-KDD、UNSW-NB15、CICIDS2017数据集上的实验结果表明:与其他的机器学习、深度学习方法相比,本文所提方法的检测准确率最大提升了18.71%。同时,本文方法可以实现较高的精确率、召回率和检测效率,同时具有较低的误报率。在面对数据受到噪声破坏时,具有较强的检测鲁棒性。     

基于钉板分布稀疏变分自编码器的异常检测算法研究

异常检测由于其广泛的应用一直是数据挖掘中一个重要的研究分支,它有助于研究人员获得重要的信息进而对数据做出更好的决策。提出了一种基于钉板分布稀疏变分自编码器的异常检测模型。首先,使用离散-连续混合模型钉板分布作为变分自编码器的先验,模拟隐变量所在空间的稀疏性,得到数据特征的稀疏表示;其次,以所提出的自编码器构建深度支持向量网络,对特征空间进行压缩,并采用最优超球体区分正常数据和异常数据;再次,以数据特征和超球体中心之间的欧氏距离完成异常检测;最后,在基准数据集MNIST （modifiednational institute of standards and technology database）和Fashion-MNIST上的实验评估表明,与现存的异常检测算法相比,本文所提出的算法具有更好的检测效果。     

E级高性能计算机的维护故障诊断系统研究

E级计算机系统规模巨大,使得故障异常总量随之增多,导致诊断发现的难度增加,因此,迫切需要一套更加准确高效的实时维护故障诊断系统,对硬件系统进行全面的异常及故障信息实时检测、故障诊断及故障预测。传统故障诊断系统在面对数万节点规模的诊断时存在执行效率低、异常检测误报率高的问题,异常检测及故障诊断的覆盖率不足。对异常及故障检测、故障诊断与故障预测相关技术进行研究,分析技术原理及适用性,并结合E级高性能计算机实际工程需求,设计一套满足数E级高性能计算机需求的维护故障诊断系统。基于维护系统的结构组成设计可扩展的边缘诊断架构,将高性能计算机系统知识、专家知识与数理统计、机器学习相融合给出故障检测、诊断及预测算法,并针对专用场景建立预测模型。实验结果表明,该系统具有较好的可扩展性,能在10 s内完成对十万个节点规模系统的故障诊断,与传统故障诊断系统相比,异常检测某特定指标误报率从3.3%降低到几乎为0,硬件故障检测覆盖率从90.2%提升至96%以上,硬件故障诊断覆盖率从71%提升至约94%,能较准确地预测多个重要应用场景下的故障。     

一种基于降维密度聚类的船舶异常轨迹识别方法

目的 有效分析和探索海洋船舶时空轨迹行为模式,提高船舶轨迹聚类的效率与质量,更好地检测真实船舶的异常行为。方法 针对当前船舶轨迹数据研究中存在的对多维特征信息利用不足、检测效率不高、检测精度较差等问题,提出一种精确度高、能自主识别分析多维特征的船舶异常轨迹识别方法。首先利用随机森林分类器评估多维特征重要性,构建轨迹特征的最优组合;然后提出一种降维密度聚类方法,将T–分布随机邻域嵌入(T–SNE)和自适应密度聚类(DBSCAN)模型结合,通过构建特征选择层和无监督聚类层实现对数据元素非线性关系的高效提取以及对聚类参数的智能选择;最后根据聚类结果构建类簇特征向量,计算距离阈值判别轨迹相似度,实现轨迹异常检测模型的构建。结果 以UCI数据集为例,降维密度聚类方法对4、13、30、64维特征数据集的F1分数能达到0.9 048、0.9 534、0.8 218、0.6 627,多个聚类指标均优于DBSCAN、K–Means等常见聚类算法的。结论 研究结果表明,降维密度聚类方法能有效提取数据多维特征结构,实现聚类参数自适应,弥补密度聚类中参数难以确定的问题,有效实现对多种类型船舶轨迹异常的识别。     

Fine-Grained Multivariate Time Series Anomaly Detection in IoT

Sensors produce a large amount of multivariate time series data to record the states of Internet of Things (IoT) systems. Multivariate time series timestamp anomaly detection (TSAD) can identify timestamps of attacks and malfunctions. However, it is necessary to determine which sensor or indicator is abnormal to facilitate a more detailed diagnosis, a process referred to as fine-grained anomaly detection (FGAD). Although further FGAD can be extended based on TSAD methods, existing works do not provide a quantitative evaluation, and the performance is unknown. Therefore, to tackle the FGAD problem, this paper first verifies that the TSAD methods achieve low performance when applied to the FGAD task directly because of the excessive fusion of features and the ignoring of the relationship’s dynamic changes between indicators. Accordingly, this paper proposes a multivariate time series fine-grained anomaly detection (MFGAD) framework. To avoid excessive fusion of features, MFGAD constructs two sub-models to independently identify the abnormal timestamp and abnormal indicator instead of a single model and then combines the two kinds of abnormal results to detect the fine-grained anomaly. Based on this framework, an algorithm based on Graph Attention Neural Network (GAT) and Attention Convolutional Long-Short Term Memory (A-ConvLSTM) is proposed, in which GAT learns temporal features of multiple indicators to detect abnormal timestamps and A-ConvLSTM captures the dynamic relationship between indicators to identify abnormal indicators. Extensive simulations on a real-world dataset demonstrate that the proposed algorithm can achieve a higher F1 score and hit rate than the extension of existing TSAD methods with the benefit of two independent sub-models for timestamp and indicator detection.     

Prioritizing Information for the Discovery of Phenomena

We consider the problem of prioritizing a collection of discrete pieces of information, or transactions. The goal is to rank the transactions in such a way that the user can best pursue a subset of the transactions in hopes of discovering those which were generated by an interesting source. The problem is shown to differ from traditional classification in several fundamental ways. Ranking algorithms are divided into classes, depending on the amount of information they may utilize. We demonstrate that while ranking by the least constrained algorithm class is consistent with classification, such is not the case for a more constrained class of algorithms. We demonstrate also that while optimal ranking by the former class is easy, optimal ranking by the latter class is NP-hard. Finally, we present detectors which solve optimally restricted versions of the ranking problem, including symmetric anomaly detection.