Network intrusion detection based on deep learning model optimized with rule-based hybrid feature selection

ABSTRACT

Network Intrusion Detection System (NIDS) is often used to classify network traffic in an attempt to protect computer systems from various network attacks. A major component for building an efficient intrusion detection system is the preprocessing of network traffic and identification of essential features which is essential for building robust classifier. In this study, a NIDS based on deep learning model optimized with rule-based hybrid feature selection is proposed. The architecture is divided into three phases namely: hybrid feature selection, rule evaluation and detection. Several search methods and attribute evaluators were combined for features selection to enhance experimentation and comparison. The results obtained showed that the number of selected features will not affect the detection accuracy of the feature selection algorithms, but directly proportional to the performance of the base classifier. Results from the performance comparison proved that the proposed method outperforms other related methods with reduction of false alarm rate, high accuracy rate, reduced training and testing time of 1.2%, 98.8%, 7.17s and 3.11s, respectively. Finally, the simulation experiments on standard evaluation metrics showed that the proposed method is suitable for attack classification in NIDS.     

Effective network intrusion detection using stacking-based ensemble approach

The increasing demand for communication between networked devices connected either through an intranet or the internet increases the need for a reliable and accurate network defense mechanism. Network intrusion detection systems (NIDSs), which are used to detect malicious or anomalous network traffic, are an integral part of network defense. This research aims to address some of the issues faced by anomaly-based network intrusion detection systems. In this research, we first identify some limitations of the legacy NIDS datasets, including a recent CICIDS2017 dataset, which lead us to develop our novel dataset, CIPMAIDS2023-1. Then, we propose a stacking-based ensemble approach that outperforms the overall state of the art for NIDS. Various attack scenarios were implemented along with benign user traffic on the network topology created using graphical network simulator-3 (GNS-3). Key flow features are extracted using cicflowmeter for each attack and are evaluated to analyze their behavior. Several different machine learning approaches are applied to the features extracted from the traffic data, and their performance is compared. The results show that the stacking-based ensemble approach is the most promising and achieves the highest weighted F1-score of 98.24%.

     

Language models for detection of unknown attacks in network traffic

In this paper, we propose a method for network intrusion detection based on language models. Our method proceeds by extracting language features such as n-grams and words from connection payloads and applying unsupervised anomaly detection—without prior learning phase or presence of labeled data. The essential part of this procedure is linear-time computation of similarity measures between language models of connection payloads. Particular patterns in these models decisive for differentiation of attacks and normal data can be traced back to attack semantics and utilized for automatic generation of attack signatures. Results of experiments conducted on two datasets of network traffic demonstrate the importance of high-order n-grams and variable-length language models for detection of unknown network attacks. An implementation of our system achieved detection accuracy of over 80% with no false positives on instances of recent remote-to-local attacks in HTTP, FTP and SMTP traffic.     

OS-Guard: on-site signature based framework for multimedia surveillance data management

This paper presents OS-Guard(On-Site Guard), a novel on-site signature based framework for multimedia surveillance data management. One of the major concerns in widespread deployment of multimedia surveillance systems is the enormous amount of data collected from multiple media streams that need to be communicated, observed and stored for crime alerts and forensic analysis. This necessitates investigating efficient data management techniques to solve this problem. This work aims to tackle this problem, motivated by the following observation, more data does not mean more information. OS-Guard is a novel framework that attempts to collect informative data and filter out non-informative data on-site, thus taking a step towards solving the data management problem. In the framework, both audio and video cues are utilized by extracting features from the incoming data stream and the resultant real valued feature data is binarized for efficient storage and processing. A feature selection process based on association rule mining selects discriminant features. A short representative sample of the whole database is generated using a novel reservoir sampling algorithm that is stored onsite and used with an support vector machine to classify an important event. Initial experiments for a Bank ATM monitoring scenario demonstrates promising results.     

Virtual indexing based methods for estimating node connection degrees

It is difficult to accurately measure node connection degrees for a high speed network, since there is a massive amount of traffic to be processed. In this paper, we present a new virtual indexing method for estimating node connection degrees for high speed links. It is based on the virtual connection degree sketch (VCDS) where a compact sketch of network traffic is built by generating multiple virtual bitmaps for each network node. Each virtual bitmap consists of a fixed number of bits selected randomly from a shared bit array by a new method for recording the traffic flows of the corresponding node. The shared bit array is efficiently utilized by all nodes since every bit is shared by the virtual bitmaps of multiple nodes. To reduce the “noise” contaminated in a node’s virtual bitmaps due to sharing, we propose a new method to generate the “filtered” bitmap used to estimate node connection degree. Furthermore, we apply VCDS to detect super nodes often associated with traffic anomalies. Since VCDS need a large amount of extra memory to store node addresses, we also propose a new data structure, the reversible virtual connection degree sketch, which identifies super node addresses analytically without the need of extra memory space but at a small increase in estimation error. Furthermore we combine the VCDS and RVCDS based methods with a uniform flow sampling technique to reduce memory complexities. Experiments are performed based on the actual network traffic and testing results show that the new methods are more memory efficient and more accurate than existing methods.     

基于RBF神经网络的可信加密流量分类方法

网络流量分类广泛应用于网络资源分配、流量调度、入侵检测系统等研究领域。随着加密协议的普及和网络流量快速发展,基于深度学习的流量分类器由于其自动提取特征的特性和较高的分类准确性,逐渐受到科研人员的重视,但是面向网络流量分类的可信程度方面却不曾有研究。本文提出一种基于RBF神经网络对加密网络流量进行可信分类的方法。所提算法建立在RBF网络的思想上并采用一种新的损失函数和质心更新方案来进行训练,通过使用梯度惩罚强制检测输入的变化,能够有效地检测分布外的数据。在2个公共的ISCX VPN-nonVPN和USTC-TFC2016流量数据集上,与同类算法相比,所提算法取得了最好的分布外检测结果,在AUROC指标上达到98.55%。实验结果表明所提算法在具有较高分类性能的同时,能够有效地检测出分布外的流量数据,从而提高流量分类的可信性。     

Network anomaly detection with incomplete audit data

With the ever increasing deployment and usage of gigabit networks, traditional network anomaly detection based Intrusion Detection Systems (IDS) have not scaled accordingly. Most, if not all IDS assume the availability of complete and clean audit data. We contend that this assumption is not valid. Factors like noise, mobility of the nodes and the large amount of network traffic make it difficult to build a traffic profile of the network that is complete and immaculate for the purpose of anomaly detection. In this paper, we attempt to address these issues by presenting an anomaly detection scheme, called SCAN (Stochastic Clustering Algorithm for Network Anomaly Detection), that has the capability to detect intrusions with high accuracy even with incomplete audit data. To address the threats posed by network-based denial-of-service attacks in high speed networks, SCAN consists of two modules: an anomaly detection module that is at the core of the design and an adaptive packet sampling scheme that intelligently samples packets to aid the anomaly detection module. The noteworthy features of SCAN include: (a) it intelligently samples the incoming network traffic to decrease the amount of audit data being sampled while retaining the intrinsic characteristics of the network traffic itself; (b) it computes the missing elements of the sampled audit data by utilizing an improved expectation–maximization (EM) algorithm-based clustering algorithm; and (c) it improves the speed of convergence of the clustering process by employing Bloom filters and data summaries.     

Traffic conduction analysis model with time series rule mining

The traffic density situation in a traffic network, especially traffic congestion, exhibits characteristics similar to thermodynamic heat conduction, e.g., the traffic congestion in one section can be conducted to other adjacent sections of the traffic network sequentially. Analyzing this conduction facilitates the forecasting of future traffic situation; therefore, a navigation system can reduce traffic congestion and improve transportation mobility. This study describes a methodology for traffic conduction analysis modeling based on extracting important time-related conduction rules using a type of evolutionary algorithm named Genetic Network Programming (GNP). The extracted rules construct a useful model for forecasting future traffic situations and analyzing traffic conduction. The proposed methodology was implemented and experimentally evaluated using a large scale real-time traffic simulator, SOUND/4U.     

多特征融合的行为识别模型

目的 视频行为识别和理解是智能监控、人机交互和虚拟现实等诸多应用中的一项基础技术,由于视频时空结构的复杂性,以及视频内容的多样性,当前行为识别仍面临如何高效提取视频的时域表示、如何高效提取视频特征并在时间轴上建模的难点问题。针对这些难点,提出了一种多特征融合的行为识别模型。方法 首先,提取视频中高频信息和低频信息,采用本文提出的两帧融合算法和三帧融合算法压缩原始数据,保留原始视频绝大多数信息,增强原始数据集,更好地表达原始行为信息。其次,设计双路特征提取网络,一路将融合数据正向输入网络提取细节特征,另一路将融合数据逆向输入网络提取整体特征,接着将两路特征加权融合,每一路特征提取网络均使用通用视频描述符——3D ConvNets （3D convolutional neural networks）结构。然后,采用BiConvLSTM （bidirectional convolutional long short-term memory network）网络对融合特征进一步提取局部信息并在时间轴上建模,解决视频序列中某些行为间隔相对较长的问题。最后,利用Softmax最大化似然函数分类行为动作。结果 为了验证本文算法的有效性,在公开的行为识别数据集UCF101和HMDB51上,采用5折交叉验证的方式进行整体测试与分析,然后针对每类行为动作进行比较统计。结果表明,本文算法在两个验证集上的平均准确率分别为96.47%和80.03%。结论 通过与目前主流行为识别模型比较,本文提出的多特征模型获得了最高的识别精度,具有通用、紧凑、简单和高效的特点。     

基于IXP2400千兆防火墙包分类算法的设计与实现*

针对千兆网下包过滤防火墙,提出了HSBIPG(Hash Search Based on IP Group)包分类算法,并分析了算法的优缺点,基于该算法用IXP2400实现了线速千兆包过滤防火墙,通过实验证明了此算法是可行和高效的。     

使用FPGA进行网络入侵监测

FPGA技术已经被广泛用于实时网络入侵监测.一个称为BV-TCAM的数据包分类体系,它是用来实现以FPGA为基础的网络入侵监测系统(NIDS).这个分类器每秒钟报出网络连接中的多个以比特单位的匹配,它结合了三重内容可设定地址的存储(TCAM)和比特向量算法(BV).     

残差密集空间金字塔网络的城市遥感图像分割

目的 遥感图像语义分割是根据土地覆盖类型对图像中每个像素进行分类,是遥感图像处理领域的一个重要研究方向。由于遥感图像包含的地物尺度差别大、地物边界复杂等原因,准确提取遥感图像特征具有一定难度,使得精确分割遥感图像比较困难。卷积神经网络因其自主分层提取图像特征的特点逐步成为图像处理领域的主流算法,本文将基于残差密集空间金字塔的卷积神经网络应用于城市地区遥感图像分割,以提升高分辨率城市地区遥感影像语义分割的精度。方法 模型将带孔卷积引入残差网络,代替网络中的下采样操作,在扩大特征图感受野的同时能够保持特征图尺寸不变;模型基于密集连接机制级联空间金字塔结构各分支,每个分支的输出都有更加密集的感受野信息;模型利用跳线连接跨层融合网络特征,结合网络中的高层语义特征和低层纹理特征恢复空间信息。结果 基于ISPRS （International Society for Photogrammetry and Remote Sensing） Vaihingen地区遥感数据集展开充分的实验研究,实验结果表明,本文模型在6种不同的地物分类上的平均交并比和平均F₁值分别达到69.88%和81.39%,性能在数学指标和视觉效果上均优于SegNet、pix2pix、Res-shuffling-Net以及SDFCN （symmetrical dense-shortcut fully convolutional network）算法。结论 将密集连接改进空间金字塔池化网络应用于高分辨率遥感图像语义分割,该模型利用了遥感图像不同尺度下的特征、高层语义信息和低层纹理信息,有效提升了城市地区遥感图像分割精度。     

结合上下文编码与特征融合的SAR图像分割

目的 图像分割的中心任务是寻找更强大的特征表示,而合成孔径雷达(synthetic aperture radar, SAR)图像中斑点噪声阻碍特征提取。为加强对SAR图像特征的提取以及对特征充分利用,提出一种改进的全卷积分割网络。方法 该网络遵循编码器—解码器结构,主要包括上下文编码模块和特征融合模块两部分。上下文编码模块(contextual encoder module, CEM)通过捕获局部上下文和通道上下文信息增强对图像的特征提取;特征融合模块(feature fusion module, FFM)提取高层特征中的全局上下文信息,将其嵌入低层特征,然后将增强的低层特征并入解码网络,提升特征图分辨率恢复的准确性。结果 在两幅真实SAR图像上,采用5种基于全卷积神经网络的分割算法作为对比,并对CEM与CEM-FFM分别进行实验。结果显示,该网络分割结果的总体精度(overall accuracy, OA)、平均精度(average accuracy, AA)与Kappa系数比5种先进算法均有显著提升。其中,网络在OA上表现最好,CEM在两幅SAR图像上OA分别为91.082%和90...     

A dyadic multi-resolution deep convolutional neural wavelet network for image classification

For almost the past four decades, image classification has gained a lot of attention in the field of pattern recognition due to its application in various fields. Given its importance, several approaches have been proposed up to now. In this paper, we will present a dyadic multi-resolution deep convolutional neural wavelets’ network approach for image classification. This approach consists of performing the classification of one class versus all the other classes of the dataset by the reconstruction of a Deep Convolutional Neural Wavelet Network (DCNWN). This network is based on the Neural Network (NN) architecture, the Fast Wavelet Transform (FWT) and the Adaboost algorithm. It consists, first, of extracting features using the FWT based on the Multi-Resolution Analysis (MRA). These features are used to calculate the inputs of the hidden layer. Second, those inputs are filtered by using the Adaboost algorithm to select the best ones corresponding to each image. Third, we create an AutoEncoder (AE) using wavelet networks of all images. Finally, we apply a pooling for each hidden layer of the wavelet network to obtain a DCNWN that permits the classification of one class and rejects all other classes of the dataset. Classification rates given by our approach show a clear improvement compared to those cited in this article.

     

基于Markov的无线传感器网络入侵检测机制

本文采用Markov线性预测模型,为无线传感器网络设计了一种基于流量预测的拒绝服务攻击检测方案——MPDD。在该方案中,每个节点基于流量预测判断和检测异常网络流量,无需特殊的硬件支持和节点之间的合作;提出了一种报警评估机制,有效提高方案的检测准确度,减少了预测误差或信道误码所带来的误报。仿真实验结果表明,Markov模型具有较高的预测精度,能够实时地预测传感器网络流量;MPDD方案能够快速、有效地检测拒绝服务攻击且消耗资源较少。     

A Duty Cycle-Based Gateway Selection Algorithm for LoRaWAN Downlink Communication

Long Range Wide Area Network (LoRaWAN) has been developed to meet the requirements for the enormous device-to-device communication of Internet of Things (IoT) networks, which consist of a large number of participating devices spread over large coverage areas with low data rates and low power consumption. It supports communications in both directions, uplink, and downlink directions. However, the downlink communication in the current LoRaWAN raises the bottleneck issue at gateways due to the used gateway selection algorithm. This paper proposes a novel gateway selection algorithm based on the duty cycle time-off values for the existing gateways, Duty Cycle Gateway Selection (DCGS), to direct acknowledgment packets as downlink traffic towards the most suitable gateway. Thus, the proposed system avoids subsequent retransmission of previously sent traffic that leads to excessive traffic overloading the network. The proposed system avoids exhausting a gateway duty cycle with downlink traffic by distributing the downlink traffic among available gateways based on the duty cycle time off. DCGS is evaluated using FloRa and INET frameworks in the well-known network simulator OMNeT++. The result shows the superior performance of the proposed approach over the existing Signal-to-Noise ratio (SNR) based selection mechanism. It clearly indicates that the DCGS maintains a better confirmed packet delivery rate while reducing number of retransmissions, collisions, and power consumption.     

Making the gigabit IPsec VPN architecture secure

By placing the security processors directly in the data path to secure traffic without the aid of additional outside devices or software, the flow-through security device creates a virtual private network that maximizes network processing unit host offload with minimal system integration effort. A virtual private network uses the Internet protocol security (IPsec) framework to provide confidentiality, data integrity, and end point authentication. These features protect corporate data from being viewed or tampered with while in transit over the Internet. Additionally, the VPN supports data compression, which increases Internet performance between sites. Metropolitan area networks and storage area networks lead the trend toward gigabit Ethernet installations that seek to provide higher speed and better security. The decreasing cost of gigabit devices and their increasing availability in PCs are driving the use of gigabit MAN, while increasing data rates drive the use of gigabit SANs as the bit rate to hard-disk media approaches 1 gigabit per second.     

SDN环境下基于DBN的DDoS攻击检测

软件定义网络(SDN)作为新型网络架构模式,其安全威胁主要来自DDoS攻击,建立高效的DDoS攻击检测系统是网络安全管理的重要内容.在SDN环境下,针对DDoS的入侵检测算法具有支持协议少、实用性差等缺陷,为此,提出一种基于深度信念网络(DBN)的DDoS攻击检测算法.分析SDN环境下DDoS攻击的机制,通过Mininet模拟SDN的网络拓扑结构,并使用Wireshark完成DDoS流量数据包的收集和检测.实验结果表明,与XGBoost、随机森林、支持向量机算法相比,该算法具有攻击检测准确性高、误报率低、检测速率快和易于扩展等优势,综合性能较好.     

Automatic Traffic Balance Algorithm Based on Traffic Engineering

Routing protocols can decide for data packets which route is reachable and co-optimal, and may cause data packets to swarm into certain links, thus causing congestion on those links. General traffic engineering (GTE) technology provides ER-LSP/CR-LSP in MPLS networks to avoid this kind of congestion. However, GTE takes only the current data flow into account and establishes an ER-LSP/CR-LSP for this current data flow in order to guarantee QoS. Although this could resolve the issue raised by routing protocols, it may also waste some resource. In this article we focus on optimization of traffic engineering and propose an automatic traffic balance algorithm based on GTE technology. Dengyin Zhang received the BS, MS, and PhD degrees from Nanjing University of Posts & Telecommunications, China, in 1986, 1989, and 2004, respectively. He is presently an associate professor at Nanjing University of Posts & Telecommunications. His research interests include computer networks, communication systems, signal and information processing. Zhiyun Tang received the MS degree in computer science and technology from Nanjing University of Posts & Telecommunications, China, in 2005. His research interests include MPLS technology, QoS control and resource management in wired and wireless networks. Ruchuan Wang born in 1943, he is a professor in College of Computer at Nanjing University of Posts and Telecommunications. He advises doctorial graduate students majoring in Computer Software, Computer Network, E-Commence and Network Security and Mobile Agents.