Session level flow classification by packet size distribution and session grouping

Classifying traffic into specific network applications is essential for application-aware network management and it becomes more challenging because modern applications complicate their network behaviors. While port number-based classifiers work only for some well-known applications and signature-based classifiers are not applicable to encrypted packet payloads, researchers tend to classify network traffic based on behaviors observed in network applications. In this paper, a session level flow classification (SLFC) approach is proposed to classify network flows as a session, which comprises of flows in the same conversation. SLFC first classifies flows into the corresponding applications by packet size distribution (PSD) and then groups flows as sessions by port locality. With PSD, each flow is transformed into a set of points in a two-dimension space and the distances between each flow and the representatives of pre-selected applications are computed. The flow is recognized as the application having a minimum distance. Meanwhile, port locality is used to group flows as sessions because an application often uses consecutive port numbers within a session. If flows of a session are classified into different applications, an arbitration algorithm is invoked to make the correction. The evaluation shows that SLFC achieves high accuracy rates on both flow and session classifications, say 99.9% and 99.98%, respectively. When SLFC is applied to online classification, it is able to make decisions quickly by checking at most 300 packets for long-lasting flows. Based on our test data, an average of 72% of packets in long-lasting flows can be skipped without reducing the classification accuracy rates.     

基于信息熵的加密会话检测方法

传统协议分析方法在检测网络加密会话时大都通过端口识别,在加密应用使用非常规端口或者在周知明文端口出现加密流量时无法进行有效的检测.为此,提出基于信息熵的加密会话检测方法.该方法先对数据流按端口进行会话重组,再计算会话数据包字符熵,进而统计出整个会话字符熵,判断熵值是否属于训练模型正态分布置信区间,通过信息分布均匀度来检测加密会话.实验表明,该方法无需特征指纹库,且检测准确率高,并能实现实时检测和处理.     

Hybrid P2P traffic classification with heuristic rules and machine learning

Peer-to-peer (P2P) applications have become more and more popular in recent years. Although they make our lives easier, increasing P2P traffic leads to many problems in management and security. Classifying P2P traffic accurately is becoming more critical for network management and P2P malware detection. Many methods have been proposed for P2P traffic classification, such as port-based, signature-based, pattern-based, and statistics-based methods. However, with the development of anti-identification techniques from port disguise to payload encryption or even packet size controlling, a single method is not enough to classify P2P traffic accurately. In this paper, an improved two-step hybrid P2P traffic classifier is proposed. The first step is a signature-based classifier at the packet-level combined with connection heuristics. The second step consists of a statistics-based classifier and pattern heuristics, and classifies the remaining unknown traffic at the flow level. Based on the analysis of various machine learning algorithms, the statistics-based classifier is implemented with REPTree, a decision tree algorithm. Through verification with real datasets, it is shown that our hybrid scheme provides high accuracy and low overhead compared to other hybrid schemes.     

An output queueing analysis of multipath ATM switches

The performance of output buffers in multipath ATM switches is closely related to the output traffic distribution, which characterizes the packet arrival rate at each output link connected to the output buffer of a given output port. Many multipath switches produce nonuniform output traffic distributions even if the input traffic patterns are uniform. Focusing on the nonuniform output traffic distribution, we analyze the output buffer performances of several multipath switches under both uniform and nonuniform input traffic patterns. It is shown that the output traffic distributions are different for the various multipath switches and the output buffer performance measured in terms of packet loss probability and mean waiting time improves as the nonuniformity of the output traffic distribution becomes higher.     

An SVM-based machine learning method for accurate internet traffic classification

Accurate and timely traffic classification is critical in network security monitoring and traffic engineering. Traditional methods based on port numbers and protocols have proven to be ineffective in terms of dynamic port allocation and packet encapsulation. The signature matching methods, on the other hand, require a known signature set and processing of packet payload, can only handle the signatures of a limited number of IP packets in real-time. A machine learning method based on SVM (supporting vector machine) is proposed in this paper for accurate Internet traffic classification. The method classifies the Internet traffic into broad application categories according to the network flow parameters obtained from the packet headers. An optimized feature set is obtained via multiple classifier selection methods. Experimental results using traffic from campus backbone show that an accuracy of 99.42% is achieved with the regular biased training and testing samples. An accuracy of 97.17% is achieved when un-biased training and testing samples are used with the same feature set. Furthermore, as all the feature parameters are computable from the packet headers, the proposed method is also applicable to encrypted network traffic.     

The effects of feature selection on the classification of encrypted botnet

Many applications today are using an encrypted channel to secure their communication and transactions. Though, their security is often challenged by adversaries such as Botnet. Botnet leverages the encrypted channel to launch attacks and amplify the impact of attacks. The numbers of Botnet attacks over an encrypted channel are increasing and continue to cause a great loss of money. This study proposes an encrypted Botnet detection technique based on packet header analysis. This technique does not require deep packet inspection and intense traffic analysis. However, the proposed technique requires the analysis of the features taken from the packet header, which are essential for detection. The study endeavors to show that features selected can significantly affect the classification of encrypted Botnet. Therefore, in this paper, the researchers focus on the effects of feature selection on the classification of encrypted Botnet. The researchers use different classification mode (full training and 10-fold cross-validation) mainly by using seven features (7-features) and three features (3-features). Seven features are the number of features extracted from the packet header, and after the feature selection, only three features out of the seven features have weight (value). Therefore, the three features are the most significant features from the seven features that have been extracted. Generally, the result shows that classification with three most significant features provides higher true positive compared to the 7-features classification. Different machine learning algorithms have been used for the classification. Relatively, the results show that the True Positives are higher for 3-features classification than 7-features classification.

     

基于卷积神经网络的加密流量分类方法

针对传统加密网络流量分类方法准确率较低、泛用性不强、易侵犯隐私等问题,提出了一种基于卷积神经网络的加密流量分类方法,避免依赖原始流量数据,防止过度拟合特定应用程序的字节结构。针对网络流量的数据包大小和到达时间信息,设计了一种将原始流量转换为二维图片的方法,直方图中每个单元格代表到达相应时间间隔的具有相应大小数据包的数量,不依赖数据包有效载荷,避免了侵犯隐私;针对LeNet-5卷积神经网络模型进行了优化以提高分类精度,嵌入Inception模块进行多维特征提取并进行特征融合,使用1*1卷积来控制输出的特征维度;使用平均池化层和卷积层替代全连接层,提高计算速度且避免过拟合;使用对象检测任务中的滑动窗口方法,将每个网络单向流划分为大小相等的块,确保单个会话中训练集中的块和测试集中的块没有重叠,扩充了数据集样本。在ISCX数据集上的分类实验结果显示,针对应用流量分类任务,准确率达到了95%以上。对比实验结果表明,训练集和测试集类型不同时,传统分类方法出现了显著的精度下降乃至失效,而所提方法的准确率依然达到了89.2%,证明了所提方法普适于加密流量与非加密流量。进行的所有实验均基于不平衡数据集,...     

GBDT与LR融合模型在加密流量识别中的应用

随着网络应用服务类型的多样化以及网络流量加密技术的不断发展,加密流量识别已经成为网络安全领域的一个重大挑战。传统的流量识别技术如深度包检测无法有效地识别加密流量,而基于机器学习理论的加密流量识别技术则表现出很好的效果。因此,本文提出一种融合梯度提升决策树算法(GBDT)与逻辑回归(LR)算法的加密流量分类模型,使用贝叶斯优化(BO)算法进行超参数调整,利用与时间相关的流特征对普通加密流量与VPN加密流量进行识别,实现了整体高于90%的流量识别准确度,与其他常用分类模型相比拥有更好的识别效果。     

结合多特征识别的恶意加密流量检测方法

随着加密流量的广泛使用,越来越多恶意软件也利用加密流量来传输恶意信息,由于其传输内容不可见,传统的基于深度包分析的检测方法带来精度下降和实时性不足等问题.本文通过分析恶意加密流量和正常流量的会话和协议,提出了一种结合多特征的恶意加密流量检测方法,该方法提取了加密流量会话的包长与时间马尔科夫链、包长与时间分布及包长与时间...     

Clustering botnet communication traffic based on n-gram feature selection

Recognized as one the most serious security threats on current Internet infrastructure, botnets can not only be implemented by existing well known applications, e.g. IRC, HTTP, or Peer-to-Peer, but also can be constructed by unknown or creative applications, which makes the botnet detection a challenging problem. Previous attempts for detecting botnets are mostly to examine traffic content for bot command on selected network links or by setting up honeypots. Traffic content, however, can be encrypted with the evolution of botnet, and as a result leading to a fail of content based detection approaches. In this paper, we address this issue and propose a new approach for detecting and clustering botnet traffic on large-scale network application communities, in which we first classify the network traffic into different applications by using traffic payload signatures, and then a novel decision tree model is used to classify those traffic to be unknown by the payload content (e.g. encrypted traffic) into known application communities where network traffic is clustered based on n-gram features selected and extracted from the content of network flows in order to differentiate the malicious botnet traffic created by bots from normal traffic generated by human beings on each specific application. We evaluate our approach with seven different traffic trace collected on three different network links and results show the proposed approach successfully detects two IRC botnet traffic traces with a high detection rate and an acceptable low false alarm rate.     

Improving the performance of passive network monitoring applications with memory locality enhancements

Passive network monitoring is the basis for a multitude of systems that support the robust, efficient, and secure operation of modern computer networks. Emerging network monitoring applications are more demanding in terms of memory and CPU resources due to the increasingly complex analysis operations that are performed on the inspected traffic. At the same time, as the traffic throughput in modern network links increases, the CPU time that can be devoted for processing each network packet decreases. This leads to a growing demand for more efficient passive network monitoring systems in which runtime performance becomes a critical issue.In this paper we present locality buffering, a novel approach for improving the runtime performance of a large class of CPU and memory intensive passive monitoring applications, such as intrusion detection systems, traffic characterization applications, and NetFlow export probes. Using locality buffering, captured packets are being reordered by clustering packets with the same port number before they are delivered to the monitoring application. This results in improved code and data locality, and consequently, in an overall increase in the packet processing throughput and decrease in the packet loss rate. We have implemented locality buffering within the widely used libpcap packet capturing library, which allows existing monitoring applications to transparently benefit from the reordered packet stream without modifications. Our experimental evaluation shows that locality buffering improves significantly the performance of popular applications, such as the Snort IDS, which exhibits a 21% increase in the packet processing throughput and is able to handle 67% higher traffic rates without dropping any packets.     

Catching Remote Administration Trojans (RATs)

A Remote Administration Trojan (RAT ) allows an attacker to remotely control a computing system and typically consists of a server invisibly running and listening to specific TCP /UDP ports on a victim machine as well as a client acting as the interface between the server and the attacker. The accuracy of host and/or network‐based methods often employed to identify RATs highly depends on the quality of Trojan signatures derived from static patterns appearing in RAT programs and/or their communications. Attackers may also obfuscate such patterns by having RATs use dynamic ports, encrypted messages, and even changing Trojan banners. In this paper, we propose a comprehensive framework termed RAT Catcher, which reliably detects and ultimately blocks RAT malicious activities even when Trojans use multiple evasion techniques. Employing network‐based methods and functioning in inline mode to inspect passing packets in real time, our RAT Catcher collects and maintains status information for every connection and conducts session correlation to greatly improve detection accuracy. The RAT Catcher re‐assembles packets in each data stream and dissects the resulting aggregation according to known Trojan communication protocols, further enhancing its traffic classification. By scanning not only protocol headers but also payloads, RAT Catcher is a truly application‐layer inspector that performs a range of corrective actions on identified traffic including alerting, packet dropping, and connection termination. We show the effectiveness and efficiency of RAT Catcher with experimentation in both laboratory and real‐world settings. Copyright © 2007 John Wiley & Sons, Ltd.     

基于特征融合的恶意加密流量识别

随着加密技术的全面应用, 越来越多的恶意软件同样采用加密的方式隐藏自身的网络活动, 导致基于规则和特征的传统方法无法满足准确性和普适性的要求. 针对上述问题, 提出一种层次特征融合和注意力的恶意加密流量识别方法. 算法具备层次结构, 依次提取数据包的特征和会话流的特征, 前一阶段设计全局混合池化方法进行特征融合; 后一阶段使用注意力机制提高BiLSTM网络分析序列关系的能力. 最终, 实验采用CIC-AndMal 2017数据集进行验证, 结果表明: 模型设计合理, 相比TextCNN模型和HST-MHSA模型, 漏报率分别降低5.8%和2.6%, 加权F1值分别提高4.7%和3.5%, 在恶意加密流量识别和分类方面体现良好的优化效果.     

Classification of packet contents for malware detection

Many existing schemes for malware detection are signature-based. Although they can effectively detect known malwares, they cannot detect variants of known malwares or new ones. Most network servers do not expect executable code in their in-bound network traffic, such as on-line shopping malls, Picasa, Youtube, Blogger, etc. Therefore, such network applications can be protected from malware infection by monitoring their ports to see if incoming packets contain any executable contents. This paper proposes a content-classification scheme that identifies executable content in incoming packets. The proposed scheme analyzes the packet payload in two steps. It first analyzes the packet payload to see if it contains multimedia-type data (such as avi, wmv, jpg){{\tt avi, wmv, jpg})} . If not, then it classifies the payload either as text-type (such as txt, jsp, asp){{\tt txt, jsp, asp})} or executable. Although in our experiments the proposed scheme shows a low rate of false negatives and positives (4.69% and 2.53%, respectively), the presence of inaccuracies still requires further inspection to efficiently detect the occurrence of malware. In this paper, we also propose simple statistical and combinatorial analysis to deal with false positives and negatives.     

Bayesian Neural Networks for Internet Traffic Classification

Internet traffic identification is an important tool for network management. It allows operators to better predict future traffic matrices and demands, security personnel to detect anomalous behavior, and researchers to develop more realistic traffic models. We present here a traffic classifier that can achieve a high accuracy across a range of application types without any source or destination host-address or port information. We use supervised machine learning based on a Bayesian trained neural network. Though our technique uses training data with categories derived from packet content, training and testing were done using features derived from packet streams consisting of one or more packet headers. By providing classification without access to the contents of packets, our technique offers wider application than methods that require full packet/payloads for classification. This is a powerful advantage, using samples of classified traffic to permit the categorization of traffic based only upon commonly available information     

Towards adaptive character frequency-based exclusive signature matching scheme and its applications in distributed intrusion detection

Network intrusion detection systems (NIDSs), especially signature-based NIDSs, are being widely deployed in a distributed network environment with the purpose of defending against a variety of network attacks. However, signature matching is a key limiting factor to limit and lower the performance of a signature-based NIDS in a large-scale network environment, in which the cost is at least linear to the size of an input string. The overhead network packets can greatly reduce the effectiveness of such detection systems and heavily consume computer resources. To mitigate this issue, a more efficient signature matching algorithm is desirable. In this paper, we therefore develop an adaptive character frequency-based exclusive signature matching scheme (named ACF-EX) that can improve the process of signature matching for a signature-based NIDS. In the experiment, we implemented the ACF-EX scheme in a distributed network environment, evaluated it by comparing with the performance of Snort. In addition, we further apply this scheme to constructing a packet filter that can filter out network packets by conducting exclusive signature matching for a signature-based NIDS, which can avoid implementation issues and improve the flexibility of the scheme. The experimental results demonstrate that, in the distributed network environment, the proposed ACF-EX scheme can positively reduce the time consumption of signature matching and that our scheme is promising in constructing a packet filter to reduce the burden of a signature-based NIDS.     

基于MSE协议特征的BT加密流量识别方法

P2P（Peer-to-Peer）系统在文件共享、协同计算、流媒体等领域获得了广泛应用。随着P2P技术的发展,越来越多的P2P应用对数据进行加密传输,加大了对其流量的识别难度。通过对MSE（Message Stream Encryption）协议特征的分析,提出了还原MSE协议消息流,实现BT（BitTorrent）加密流量识别的方法。修改了开源BT客户端Vuze,利用其收集的真实BT流量信息来检验本方法,结果表明该方法与现有的DPI（deep packet inspection）技术结合,对网络中BT流量进行识别,具有较高的召回率和准确率,同时保持了较低的误报率。     

Intrusion detection in computer networks by a modular ensemble of one-class classifiers

Since the early days of research on intrusion detection, anomaly-based approaches have been proposed to detect intrusion attempts. Attacks are detected as anomalies when compared to a model of normal (legitimate) events. Anomaly-based approaches typically produce a relatively large number of false alarms compared to signature-based IDS. However, anomaly-based IDS are able to detect never-before-seen attacks. As new types of attacks are generated at an increasing pace and the process of signature generation is slow, it turns out that signature-based IDS can be easily evaded by new attacks. The ability of anomaly-based IDS to detect attacks never observed in the wild has stirred up a renewed interest in anomaly detection. In particular, recent work focused on unsupervised or unlabeled anomaly detection, due to the fact that it is very hard and expensive to obtain a labeled dataset containing only pure normal events.The unlabeled approaches proposed so far for network IDS focused on modeling the normal network traffic considered as a whole. As network traffic related to different protocols or services exhibits different characteristics, this paper proposes an unlabeled Network Anomaly IDS based on a modular Multiple Classifier System (MCS). Each module is designed to model a particular group of similar protocols or network services. The use of a modular MCS allows the designer to choose a different model and decision threshold for different (groups of) network services. This also allows the designer to tune the false alarm rate and detection rate produced by each module to optimize the overall performance of the ensemble. Experimental results on the KDD-Cup 1999 dataset show that the proposed anomaly IDS achieves high attack detection rate and low false alarm rate at the same time.     

Interval-based flow watermarking for tracing interactive traffic

Tracing interactive attack traffic that traverses stepping stones (i.e., intermediate hosts) is challenging, as the packet headers, lengths, and contents can all be changed by the stepping stones. The traffic timing (delays between packets) has therefore been studied as a means of tracing traffic. One such technique uses traffic timing as a side channel into which a watermark, or identifying tag, can be embedded to aid with tracing. The effectiveness of such techniques is greatly reduced when the packet count of the traffic is changed at the stepping stone. Such transformations may occur as a result of either active countermeasures (e.g. chaff packets, flow splitting) by an adversary attempting to defeat tracing, or by incidental repacketization of the traffic by network interfaces.This paper presents a new method of embedding a watermark in traffic timing, for purposes of tracing the traffic in the presence of flow splitting, chaff packets, timing perturbation, and repacketization. This method uses an invariant characteristic of two connection flows which are part of the same stepping stone chain, namely, the elapsed time of the flows. The duration of each flow is sliced into short fixed-length intervals. Packet timing is adjusted to manipulate the packet count in specific intervals (without adding or deleting any packets), for purposes of embedding the watermark. The method is self-synchronizing and does not require clock synchronization between the watermark encoder and decoder.A statistical analysis of the method, with no assumptions or limitations concerning the distribution of packet times, proves the effectiveness of the method given a sufficient number of packets, despite natural and/or deliberate repacketization and countermeasures by an adversary. The method has been implemented and tested on a large number of SSH traffic flows. The results demonstrate that 100% detection rates and very low false positive rates are achieved under conditions of multiple countermeasures, and using only a few hundred packets.