Bayesian Neural Networks for Internet Traffic Classification

Internet traffic identification is an important tool for network management. It allows operators to better predict future traffic matrices and demands, security personnel to detect anomalous behavior, and researchers to develop more realistic traffic models. We present here a traffic classifier that can achieve a high accuracy across a range of application types without any source or destination host-address or port information. We use supervised machine learning based on a Bayesian trained neural network. Though our technique uses training data with categories derived from packet content, training and testing were done using features derived from packet streams consisting of one or more packet headers. By providing classification without access to the contents of packets, our technique offers wider application than methods that require full packet/payloads for classification. This is a powerful advantage, using samples of classified traffic to permit the categorization of traffic based only upon commonly available information     

基于SVM和聚类的Internet流识别方法

利用支持向量机与聚类结合的方法,通过训练具有不同的特点几类应用对网络流量进行分类,这些应用是从校园网的骨干网中抓获的.鉴别选择算法的设计是用来获取用以进行分类的最佳组合特征.基于无偏向性的训练和测试样本,优化方法拥有比较高的准确性.所有的特征参数从实时捕获的数据包包头中计算出来的,显示出实时网络流量的高精度分类是可以实现的.实验结果表明该方法是有效的.     

混合模式的网络流量分类方法

为了更好地满足用户对各类Internet业务服务质量越来越精细的要求,流量分类是网络管理的重要环节之一。通过分析、对比基于端口号匹配、特征字段分析和流统计特征的机器学习分类方法的应用现状及其优缺点,针对单一分类方法存在的分类准确度不高、分类时间长等问题,提出一种混合模式的网络流量分类方案。此方案结合端口号匹配和机器学习分类方法,采用输出结果可视化的自组织映射网络算法实现网络流量在应用层的分类。实验表明,该方案能有效地实现对网络流量应用类型的分类,分类结果可视化效果好。     

Network traffic classification based on ensemble learning and co-training

Classification of network traffic is the essential step for many network researches. However,with the rapid evolution of Internet applications the effectiveness of the port-based or payload-based identifi-cation approaches has been greatly diminished in recent years. And many researchers begin to turn their attentions to an alternative machine learning based method. This paper presents a novel machine learning-based classification model,which combines ensemble learning paradigm with co-training tech-niques. Compared to previous approaches,most of which only employed single classifier,multiple clas-sifiers and semi-supervised learning are applied in our method and it mainly helps to overcome three shortcomings:limited flow accuracy rate,weak adaptability and huge demand of labeled training set. In this paper,statistical characteristics of IP flows are extracted from the packet level traces to establish the feature set,then the classification model is created and tested and the empirical results prove its feasibility and effectiveness.     

基于卷积神经网络的加密流量分类方法

针对传统加密网络流量分类方法准确率较低、泛用性不强、易侵犯隐私等问题,提出了一种基于卷积神经网络的加密流量分类方法,避免依赖原始流量数据,防止过度拟合特定应用程序的字节结构。针对网络流量的数据包大小和到达时间信息,设计了一种将原始流量转换为二维图片的方法,直方图中每个单元格代表到达相应时间间隔的具有相应大小数据包的数量,不依赖数据包有效载荷,避免了侵犯隐私;针对LeNet-5卷积神经网络模型进行了优化以提高分类精度,嵌入Inception模块进行多维特征提取并进行特征融合,使用1*1卷积来控制输出的特征维度;使用平均池化层和卷积层替代全连接层,提高计算速度且避免过拟合;使用对象检测任务中的滑动窗口方法,将每个网络单向流划分为大小相等的块,确保单个会话中训练集中的块和测试集中的块没有重叠,扩充了数据集样本。在ISCX数据集上的分类实验结果显示,针对应用流量分类任务,准确率达到了 95%以上。对比实验结果表明,训练集和测试集类型不同时,传统分类方法出现了显著的精度下降乃至失效,而所提方法的准确率依然达到了89.2%,证明了所提方法普适于加密流量与非加密流量。进行的所有实验均基于不平衡数据集,如果对数据集进行平衡化处理,准确率可能会进一步提高。     

Data preprocessing for anomaly based network intrusion detection: A review

Data preprocessing is widely recognized as an important stage in anomaly detection. This paper reviews the data preprocessing techniques used by anomaly-based network intrusion detection systems (NIDS), concentrating on which aspects of the network traffic are analyzed, and what feature construction and selection methods have been used. Motivation for the paper comes from the large impact data preprocessing has on the accuracy and capability of anomaly-based NIDS. The review finds that many NIDS limit their view of network traffic to the TCP/IP packet headers. Time-based statistics can be derived from these headers to detect network scans, network worm behavior, and denial of service attacks. A number of other NIDS perform deeper inspection of request packets to detect attacks against network services and network applications. More recent approaches analyze full service responses to detect attacks targeting clients. The review covers a wide range of NIDS, highlighting which classes of attack are detectable by each of these approaches.Data preprocessing is found to predominantly rely on expert domain knowledge for identifying the most relevant parts of network traffic and for constructing the initial candidate set of traffic features. On the other hand, automated methods have been widely used for feature extraction to reduce data dimensionality, and feature selection to find the most relevant subset of features from this candidate set. The review shows a trend toward deeper packet inspection to construct more relevant features through targeted content parsing. These context sensitive features are required to detect current attacks.     

基于SSOM的网络流量分类方法

针对目前基于端口号匹配和特征码识别的流量分类方法准确率低、应用范围受限等问题,提出一种基于有监督的自组织映射(SSOM)的网络流量分类方法。该方法使用已标注类别的网络流量训练集,通过改变自组织映射(SOM)训练过程中的权值调整规则,使输出层中获胜神经元的选择更容易,各类别之间划分更清晰,从而提高分类性能。实验结果表明,SSOM的分辨率及拓扑连续性均优于SOM,对网络流量分类具有更高的准确率。     

Internet traffic clustering with side information

Internet traffic classification is a critical and essential functionality for network management and security systems. Due to the limitations of traditional port-based and payload-based classification approaches, the past several years have seen extensive research on utilizing machine learning techniques to classify Internet traffic based on packet and flow level characteristics. For the purpose of learning from unlabeled traffic data, some classic clustering methods have been applied in previous studies but the reported accuracy results are unsatisfactory. In this paper, we propose a semi-supervised approach for accurate Internet traffic clustering, which is motivated by the observation of widely existing partial equivalence relationships among Internet traffic flows. In particular, we formulate the problem using a Gaussian Mixture Model (GMM) with set-based equivalence constraint and propose a constrained Expectation Maximization (EM) algorithm for clustering. Experiments with real-world packet traces show that the proposed approach can significantly improve the quality of resultant traffic clusters.     

基于复杂网络社团划分的网络流量分类

随着网络的高速发展以及各种应用的不断涌现,采用端口号映射或有效负载分析的方法进行流量分类与应用识别已难以满足应用的需求。以流为网络节点、流之间统计特征的相似度为边,构建流相关网络模型,利用Newman快速社团划分算法(NFCD)对流相关网络模型进行社团划分,得到了流的聚类结果,实现了网络流量的分类,并与先前的两种无监督的流量分类算法(K-Means,DBSCAN)进行了对比。实验结果显示,利用NFCD算法具有更高的准确率,并能产生更好的聚类效果,且不受输入参数影响。     

Application classification using packet size distribution and port association

Traffic classification is an essential part in common network management applications such as intrusion detection and network monitoring. Identifying traffic by looking at port numbers is only suitable to well-known applications, while signature-based classification is not applicable to encrypted messages. Our preliminary observation shows that each application has distinct packet size distribution (PSD) of the connections. Therefore, it is feasible to classify traffic by analyzing the variances of packet sizes of the connections without analyzing packet payload. In this work, each connection is first transformed into a point in a multi-dimensional space according to its PSD. Then it is compared with the representative points of pre-defined applications and recognized as the application having a minimum distance. Once a connection is identified as a specific application, port association is used to accelerate the classification by combining it with the other connections of the same session because applications usually use consecutive ports during a session. Using the proposed techniques, packet size distribution and port association, a high accuracy rate, 96% on average, and low false positive and false negative rates, 4–5%, are achieved. Our proposed method not only works well for encrypted traffic but also can be easily incorporated with a signature-based method to provide better accuracy.     

Online NetFPGA decision tree statistical traffic classifier

Classifying online network traffic is becoming critical in network management and security. Recently, new classification methods based on analysis of statistical features of transport layer traffic have been proposed. While these new methods address the limitations of the port based and payload based traffic classification, the current software-based solutions are not fast enough to deal with the traffic of today’s high-speed networks. In this paper, we propose an online statistical traffic classifier using the C4.5 machine learning algorithm running on the NetFPGA platform. Our NetFPGA classifier is constructed by adding three main modules to the NetFPGA reference switch design; a Netflow module, a feature extractor module, and a C4.5 search tree classifier. The proposed classifier is able to classify the input traffics at the maximum line speed of the NetFPGA platform, i.e. 8 Gbps without any packet loss. Our method is based on the statistical features of the first few packets of a flow. The flow is classified just a few micro seconds after receiving the desired number of packets.     

一种基于随机森林算法的MQTT异常流量检测方法

工业物联网系统所面临的网络安全威胁随着物联网技术的广泛应用日益增加,信息安全问题已成为其发展过程中的一大挑战。MQTT（Message Queuing Telemetry Transport）协议是物联网通信的主流协议,基于该协议的物联网通信安全研究是当前研究的热点话题。传统的流量识别技术如深度包检测无法有效地识别符合包格式的异常流量,而基于机器学习理论的异常流量识别技术则表现出很好的效果。对此提出一种基于随机森林算法的MQTT异常流量检测方法,实现整体高于90%的MQTT异常流量识别准确度,与其他常用分类模型相比拥有更好的识别效果。     

Extraction of shift invariant wavelet features for classification of images with different sizes

An effective shift invariant wavelet feature extraction method for classification of images with different sizes is proposed. The feature extraction process involves a normalization followed by an adaptive shift invariant wavelet packet transform. An energy signature is computed for each subband of these invariant wavelet coefficients. A reduced subset of energy signatures is selected as the feature vector for classification of images with different sizes. Experimental results show that the proposed method can achieve high classification accuracy of 98.5 percent and outperforms the other two image classification methods.     

基于侧信道特征的安全代理流量分类方法

安全代理被越来越多的互联网用户用于规避网络审查和访问受限资源,因此安全代理流量的分类对于网络安全和网络管理具有重要意义。为弥补深度包检测技术在过滤和识别不良信息上的不足,提高防火墙流量探测能力,提出一种安全代理流量分类方法。提取用于安全代理流量分类的侧信道特征,包括有效载荷长度序列、信号序列等,使用机器学习和深度学习算法对Shadowsocks、V2Ray、Freegate、Ultrasurf 4种被广泛使用的安全代理流量进行识别。实验结果表明,通过提取与有效载荷内容无关的侧信道特征进行分类,与MLP、LSMP等算法相比,该方法在准确率、F1值等性能方面均有提升。     

基于迁移学习的加密恶意流量检测方法

现有加密恶意流量检测方法需要利用大量准确标记的样本进行训练,以达到较好的检测效果。但在实际网络环境中,加密流量数据由于其内容不可见而难以进行正确标记。针对上述问题,提出了一种基于迁移学习的加密恶意流量检测方法,首次将基于ImageNet数据集预训练的模型Efficientnet-B0,迁移到加密流量数据集上,保留其卷积层结构和参数,对全连接层进行替换和再训练,利用迁移学习的思想实现小样本条件下的高性能检测。该方法利用端到端的框架设计,能够直接从原始流量数据中提取特征并进行检测和细粒度分类,避免了繁杂的手动特征提取过程。实验结果表明,该方法对正常、恶意流量的二分类准确率能够达到99.87%,加密恶意流量细粒度分类准确率可达到98.88%,并且在训练集中各类流量样本数量减少到100条时,也能够达到96.35%的细粒度分类准确率。     

网络背景流量的分类与识别研究综述

互联网流量分类是识别网络应用和分类相应流量的过程,这被认为是现代网络管理和安全系统中最基本的功能。与应用相关的流量分类是网络安全的基础技术。传统的流量分类方法包括基于端口的预测方法和基于有效载荷的深度检测方法。在目前的网络环境下,传统的方法存在一些实际问题,如动态端口和加密应用,因此采用基于流量统计特征的机器学习（ML）技术来进行流量分类识别。机器学习可以利用提供的流量数据进行集中自动搜索,并描述有用的结构模式,这有助于智能地进行流量分类。起初使用朴素贝叶斯方法进行网络流量分类的识别和分类,对特定流量进行实验时,表现较好,准确度可达90%以上,但对点对点传输网络流量（P2P）等流量识别准确度仅能达到50%左右。然后有使用支持向量机（SVM）和神经网络（NN）等方法,神经网络方法使整体网络流量的分类准确度能达到80%以上。多项研究结果表明,对于多种机器学习方法的使用和后续的改进,很好地提高了流量分类的准确性。     

Semisupervised Encrypted Traffic Identification Based on Auxiliary Classification Generative Adversarial Network

The rapidly increasing popularity of mobile devices has changed the methods with which people access various network services and increased network traffic markedly. Over the past few decades, network traffic identification has been a research hotspot in the field of network management and security monitoring. However, as more network services use encryption technology, network traffic identification faces many challenges. Although classic machine learning methods can solve many problems that cannot be solved by port- and payload-based methods, manually extract features that are frequently updated is time-consuming and labor-intensive. Deep learning has good automatic feature learning capabilities and is an ideal method for network traffic identification, particularly encrypted traffic identification; Existing recognition methods based on deep learning primarily use supervised learning methods and rely on many labeled samples. However, in real scenarios, labeled samples are often difficult to obtain. This paper adjusts the structure of the auxiliary classification generation adversarial network (ACGAN) so that it can use unlabeled samples for training, and use the wasserstein distance instead of the original cross entropy as the loss function to achieve semisupervised learning. Experimental results show that the identification accuracy of ISCX and USTC data sets using the proposed method yields markedly better performance when the number of labeled samples is small compared to that of convolutional neural network (CNN) based classifier.     

支持向量机的半监督网络流量分类方法

针对传统网络流量分类方法准确率低、开销大、应用范围受限等问题,提出一种支持向量机（SVM）的半监督网络流量分类方法。该方法在SVM训练中,使用增量学习技术在初始和新增样本集中动态地确定支持向量,避免不必要的重复训练,改善因出现新样本而造成原分类器分类精度降低、分类时间长的情况;改进半监督Tri-training方法对分类器进行协同训练,同时使用大量未标记和少量已标记样本对分类器进行反复修正, 减少辅助分类器的噪声数据,克服传统协同验证对分类算法及样本类型要求苛刻的不足。实验结果表明,该方法可明显提高网络流量分类的准确率和效率。     

使用机器学习算法分类P2P流量的方法*

P2P应用的快速增长,带来网络拥塞等诸多问题,而传统的基于端口与有效载荷的P2P流量分类方法存在着很多缺陷。以抽取独立于端口、协议和有效载荷的P2P流的信息作为特征,用提出的基于ReliefF-CFS的方法选择流的特征子集,研究使用机器学习算法对P2P流量进行分类的方法,也研究了利用流的前向N个报文的统计信息作为特征,分类P2P流量的方法。实验结果显示提出的方法取得了较好的分类准确率。     

结合多特征识别的恶意加密流量检测方法

随着加密流量的广泛使用,越来越多恶意软件也利用加密流量来传输恶意信息,由于其传输内容不可见,传统的基于深度包分析的检测方法带来精度下降和实时性不足等问题。本文通过分析恶意加密流量和正常流量的会话和协议,提出了一种结合多特征的恶意加密流量检测方法,该方法提取了加密流量会话的包长与时间马尔科夫链、包长与时间分布及包长与时间统计等方面的统计特征,结合握手阶段的TLS加密套件使用、证书及域名等协议特征,构建了863维的特征向量,利用机器学习方法对加密流量进行检测,从而发现恶意加密流量。测试结果表明,结合多特征的恶意加密流量检测方法能达到98%以上的分类准确性及99.8%以上召回率,且在保持相当的分类准确性基础上,具有更好的鲁棒性,适用性更广。