基于深度报文检测和深度流检测的骨干网流量特征分析

Based on the massive data collected with a passive network monitoring equipment placed in China's backbone, we present a deep insight into the network backbone traffic and evaluate various ways for improving traffic classifying efficiency in this paper. In particular, the study has scrutinized the network traffic in terms of protocol types and signatures, flow length, and port distribution, from which meaningful and interesting insights on the current Internet of China from the perspective of both the packet and flow levels are derived. We show that the classification efficiency can be greatly improved by using the information of preferred ports of the network applications. Quantitatively, we find two traffic duration thresholds, with which 40% of TCP flows and 70% of UDP flows can be excluded from classification processing while the impact on classification accuracy is trivial, i.e., the classification accuracy can still reach a high level by saving 85% of the resources.     

Refined identification of hybrid traffic in DNS tunnels based on regression analysis

DNS (Domain Name System) tunnels almost obscure the true network activities of users, which makes it challenging for the gateway or censorship equipment to identify malicious or unpermitted network behaviors. An efficient way to address this problem is to conduct a temporal-spatial analysis on the tunnel traffic. Nevertheless, current studies on this topic limit the DNS tunnel to those with a single protocol, whereas more than one protocol may be used simultaneously. In this paper, we concentrate on the refined identification of two protocols mixed in a DNS tunnel. A feature set is first derived from DNS query and response flows, which is incorporated with deep neural networks to construct a regression model. We benchmark the proposed method with captured DNS tunnel traffic, the experimental results show that the proposed scheme can achieve identification accuracy of more than 90%. To the best of our knowledge, the proposed scheme is the first to estimate the ratios of two mixed protocols in DNS tunnels.     

A guaranteed minimum throughput service for TCP flows using measurement‐based admission control

We propose a new scheme for a network service that guarantees a minimum throughput to flows accepted by admission control (AC). The whole scheme only uses a small set of packet classes in a core‐stateless network. At the ingress of the network each flow packet is marked into one of the sets of classes, and within the network, each class is assigned a different discarding priority. The AC method is based on edge‐to‐edge per‐flow throughput measurements using the first packets of the flow, and it requires flows to send with a minimum rate. We evaluate the scheme through simulations in a simple bottleneck topology with different traffic loads consisting of TCP flows that carry files of varying sizes. We use a modified TCP source with a new algorithm that forces the source to send with a minimum rate. We compare our scheme with the best‐effort service and we study the influence of the measurement duration on the scheme's performance. The results prove that the scheme guarantees the requested throughput to accepted flows and achieves a high utilization of network resources. Copyright © 2006 John Wiley & Sons, Ltd.     

基于IPFIX的DNS异常行为检测方法

提出了一种基于IPFIX（IP数据流信息输出）网络流量数据准确检测可疑和异常DNS、识别DNS流量放大攻击行为的算法。该算法已在清华大学校园网实际部署运行,能够有效检测到校园网内部DNS的异常行为并发送告警信息,从而及时控制攻击行为,实现异常流量的及时监测和预警。     

High‐speed flow‐based classification on FPGA

Analyzing the composition of Internet traffic has many applications nowadays, like tracking bandwidth‐consuming applications, QoS‐based traffic engineering and lawful interception of illegal traffic. Even though many flow‐based classification methods, such as support vector machines (SVM), have demonstrated their accuracy, few practical implementations of lightweight classifiers exist. We consider in this paper the design of a real‐time SVM traffic classifier at hundreds of Gb/s to allow online detection of categories of applications. We also implement a high‐speed flow reconstruction algorithm able to handle one million concurrent flows. The solution is based on the massive parallelism and low‐level network interface access of FPGA boards. We find maximum supported bit rates up to 408 Gb/s for classification and up to 20 GB/s for flow reconstruction for the most challenging trace. Results are confirmed using a commercial Combov2 board with a Virtex 5 FPGA. Copyright © 2014 John Wiley & Sons, Ltd.     

Delay analysis in practical wireless network coding

Network coding provides a powerful mechanism for improving performance of wireless networks. In this paper, we present an analytical approach for end‐to‐end delay analysis in wireless networks that employs inter‐session network coding. Prior work on performance analysis in wireless network coding mainly focuses on the throughput of the overall network. Our approach aims to analyze the delay of each flow in the network. The theoretical basis of our approach is network calculus. In order to use network calculus to analyze the performance of traffic flows in the network, we have to address three specific problems: identifying traffic flows, characterizing broadcast links, and measuring coding opportunities. We propose solutions for these problems and discuss the practical issues when applying the approach in practice. We make three main contributions. First, we obtain theoretical formulations for computing the queueing delay bounds of traffic flows in wireless networks with network coding. Second, with the formulations, we figure out the factors that affect the queueing delay of a flow and find that first‐in first‐out scheduling cannot fully exploit the benefit of network coding. Third, in order to exploit our findings, we introduce a new scheduling scheme that can improve the performance of current practical wireless network coding. Copyright © 2012 John Wiley & Sons, Ltd.     

Sparsely‐deployed relay node assisted routing algorithm for vehicular ad hoc networks

In this paper, we study the issue of routing in a vehicular ad hoc network with the assistance of sparsely deployed auxiliary relay nodes at some road intersections in a city. In such a network, vehicles keep moving, and relay nodes are static. The purpose of introducing auxiliary relay nodes is to reduce the end‐to‐end packet delivery delay. We propose a sparsely deployed relay node assisted routing (SRR) algorithm, which differs from existing routing protocols on how routing decisions are made at road intersections where static relay nodes are available such that relay nodes can temporarily buffer a data packet if the packet is expected to meet a vehicle leading to a better route with high probability in certain time than the current vehicles. We further calculate the joint probability for such a case to happen on the basis of the local vehicle traffic distribution and also the turning probability at an intersection. The detailed procedure of the protocol is presented. The SRR protocol is easy to implement and requires little extra routing information. Simulation results show that SRR can achieve high performance in terms of end‐to‐end packet delivery latency and delivery ratio when compared with existing protocols. Copyright © 2013 John Wiley & Sons, Ltd.     

一种面向流量异常检测的概率流抽样方法

针对基于概率抽样的网络流量异常检测数据集构造过程中无法同时兼顾大、小流抽样需求及未区分flash crowd与流量攻击等问题,该文提出一种面向流量异常检测的概率流抽样方法。在对数据流按目的、源IP地址进行分类的基础上,将每类数据流抽样率定义为其目的、源IP地址抽样率的最大值,并在抽样过程中对数据流抽样数目向上取整,保证每类数据流至少被抽样一次,使抽样得到的数据集可有效反映原始流量在大、小流和源、目的IP地址方面的分布性。采用源IP地址熵刻画异常流源IP地址分散度,并基于源IP地址熵阈值设计攻击流抽样算法,降低由flash crowd引起的非攻击异常流抽样概率。仿真结果表明,该方法能同时满足大、小流抽样需求,具有较强的异常流抽样能力,可抽样到所有与异常流相关的可疑源、目的IP地址,并能在抽样过程中过滤非攻击异常流。     

Proactive multipath routing with a predictive mechanism in software‐defined networks

With the growth of network traffic volume, link congestion cannot be avoided efficiently with conventional routing protocols. By utilizing the single shortest‐path routing algorithm from link state advertisement information, standard routing protocols lack of global awareness and are difficult to be modified in a traditional network environment. Recently, software‐defined network (SDN) provided innovative architecture for researchers to program their own network protocols. With SDN, we can divert heavy traffic to multiple paths in order to resolve link congestion. Furthermore, certain network traffics come in periodic fashion such as peak hours at working days so that we can leverage forecasting for resource management to improve its performance. In this paper, we propose a proactive multipath routing with a predictive mechanism (PMRP) to achieve high‐performance congestion resolution. PMRP has two main concepts: (a) a proactive mechanism where PMRP deploys M/M/1 queue and traffic statistics to simulate weighted delay for possible combinations of multipaths placement of all subnet pairs, and leverage genetic algorithm for accelerating selection of optimized solution, and (b) a predictive mechanism whereby PMRP uses exponential smoothing for demand traffic volumes and variance predictions. Experimental results show a 49% reduction in average delay as compared with single shortest routing, and a 16% reduction in average delay compared with utilization & topology‐aware multipath routing (UTAMP). With the predictive mechanism, PMRP can decrease an additional 20% average delay. Furthermore, PMRP reduces 93% of flow table usage on average as compared with UTAMP.     

A parameterizable methodology for Internet traffic flow profiling

We present a parameterizable methodology for profiling Internet traffic flows at a variety of granularities. Our methodology differs from many previous studies that have concentrated on end-point definitions of flows in terms of state derived from observing the explicit opening and closing of TCP connections. Instead, our model defines flows based on traffic satisfying various temporal and spatial locality conditions, as observed at internal points of the network. This approach to flow characterization helps address some central problems in networking based on the Internet model. Among them are route caching, resource reservation at multiple service levels, usage based accounting, and the integration of IP traffic over an ATM fabric. We first define the parameter space and then concentrate on metrics characterizing both individual flows as well as the aggregate flow profile. We consider various granularities of the definition of a flow, such as by destination network, host-pair, or host and port quadruple. We include some measurements based on case studies we undertook, which yield significant insights into some aspects of Internet traffic, including demonstrating (i) the brevity of a significant fraction of IP flows at a variety of traffic aggregation granularities, (ii) that the number of host-pair IP flows is not significantly larger than the number of destination network flows, and (iii) that schemes for caching traffic information could significantly benefit from using application information     

Dynamic link sleeping reconfigurations for green traffic engineering

The high volume of energy consumption has become a great concern to the Internet community because of high energy waste on redundant network devices. One promising scheme for energy savings is to reconfigure network elements to sleep mode when traffic demand is low. However, due to the nature of today's traditional IP routing protocols, network reconfiguration is generally deemed to be harmful because of routing table reconvergence. To make these sleeping network elements, such as links, robust to traffic disruption, we propose a novel online scheme called designate to sleep algorithm that aims to remove network links without causing traffic disruption during energy‐saving periods. Considering the nature of diurnal traffic, there could be traffic surge in the network because of reduced network capacity. We therefore propose a complementary scheme called dynamic wake‐up algorithm that intelligently wakes up minimum number of sleeping links needed to control such dynamicity. This is contrary to the normal paradigm of either reverting to full topology and sacrificing energy savings or employing on‐the‐fly link weight manipulation. Using the real topologies of GEANT and Abilene networks respectively, we show that the proposed schemes can save a substantial amount of energy without affecting network performance.     

Network level perspective in web sessions troubleshooting

In the last years, the quantity of data and the number of applications carried over web traffic have been continuously increasing and nowadays web browsing accounts for most of the Internet traffic. In such a scenario, a poor browsing experience can result very annoying to the end user, and the effective identification of the root cause of such bad performance is of primary interest to both the users and the network operators. In this paper, we present a unified framework, based on a novel lightweight open‐source publicly available probe and on an original statistical diagnosis algorithm, to correctly and effectively point out the segment of a web connection (eg, local client, backbone network, and DNS server) responsible for a poor web browsing experience. The extensive experimental evaluation carried out in the paper demonstrates the effectiveness of the proposed approach to diagnose poor quality of experience at a large scale.     

Outsourcing automated QoS control of home routers for a better online game experience

Consumer network access links can become bottlenecks when faced with heterogeneous network traffic where real-time traffic from network games finds itself competing with nongame traffic for access to bandwidth. We would like to prioritize network game traffic over these bandwidth restricted links. However, the limited resources of consumer access devices make this problematic. We propose a solution whereby the classification of flows is outsourced to an ISPbased system. The access device is then notified of flow classifications and can apply a simple flow prioritization rule. We have developed a prototype of this system and found it viable in terms of functionality, timeliness of classification, and scalability.     

Prediction‐based MAC‐layer sensing in cognitive radio networks

We consider a cognitive radio network which coexists with multiple primary users (PUs) and secondary users (SUs) transmit over time‐varying channels. In this scenario, one problem of the existing work is the poor performances of throughput and fairness due to variances of SUs' channel conditions and PUs' traffic patterns. To solve this problem, we propose a novel prediction‐based MAC‐layer sensing algorithm. In the proposed algorithm, the SUs' channel quality information and the probability of the licensed channel being idle are predicted. Through the earlier predicted information, we schedule the SUs to sense and transmit on different licensed channels. Specifically, multiple significant factors, including network throughput and fairness, are jointly considered in the proposed algorithm. Then, we formulate the prediction‐based sensing scheduling problem as an optimization problem and solve it with the Hungarian algorithm in polynomial time. Simulation results show that the proposed prediction‐based sensing scheduling algorithm could achieve a good tradeoff between network throughput and fairness among SUs. Copyright © 2014 John Wiley & Sons, Ltd.     

Estimating Dynamic Traffic Matrices by Using Viable Routing Changes

In this paper we propose a new approach for dealing with the ill-posed nature of traffic matrix estimation. We present three solution enhancers: an algorithm for deliberately changing link weights to obtain additional information that can make the underlying linear system full rank; a cyclo-stationary model to capture both long-term and short-term traffic variability, and a method for estimating the variance of origin-destination (OD) flows. We show how these three elements can be combined into a comprehensive traffic matrix estimation procedure that dramatically reduces the errors compared to existing methods. We demonstrate that our variance estimates can be used to identify the elephant OD flows, and we thus propose a variant of our algorithm that addresses the problem of estimating only the heavy flows in a traffic matrix. One of our key findings is that by focusing only on heavy flows, we can simplify the measurement and estimation procedure so as to render it more practical. Although there is a tradeoff between practicality and accuracy, we find that increasing the rank is so helpful that we can nevertheless keep the average errors consistently below the 10% carrier target error rate. We validate the effectiveness of our methodology and the intuition behind it using commercial traffic matrix data from Sprint's Tier-1 backbone.     

Genetic algorithm‐based routing method for enhanced video delivery over software defined networks

Video streaming has emerged as a killer application in today's Internet, delivering a tremendous amount of media contents to millions of users at any given time. Such a heavy traffic load demands an effective routing method. In this paper, an effective routing method, named GA‐SDN, is developed based on software defined network (SDN) technique. To facilitate the researchers in this field to evaluate the video delivery quality over SDN, an evaluation framework and its associated source codes are provided. The framework integrates the H.264 Scalable Video coding streaming Evaluation Framework (SVEF) with the Mininet emulator. Through this framework, video processing researchers can evaluate their proposed coding algorithms in an SDN‐enabled network emulator, while network operators or executives can evaluate the impact of real video streams on the developing network architectures or protocols. Experiment results demonstrate the usefulness of myEvalSVC_SDN and prove that GA‐SDN outperforms traditional Bellman‐Ford routing algorithm in terms of packet drop rate, throughput, and average peak signal‐to‐noise ratio.     

Exploiting packet‐sampling measurements for traffic characterization and classification

The use of packet sampling for traffic measurement has become mandatory for network operators to cope with the huge amount of data transmitted in today's networks, powered by increasingly faster transmission technologies. Therefore, many networking tasks must already deal with such reduced data, more available but less rich in information. In this work we assess the impact of packet sampling on various network monitoring‐activities, with a particular focus on traffic characterization and classification. We process an extremely heterogeneous dataset composed of four packet‐level traces (representative of different access technologies and operational environments) with a traffic monitor able to apply different sampling policies and rates to the traffic and extract several features both in aggregated and per‐flow fashion, providing empirical evidences of the impact of packet sampling on both traffic measurement and traffic classification. First, we analyze feature distortion, quantified by means of two statistical metrics: most features appear already deteriorated under low sampling step, no matter the sampling policy, while only a few remain consistent under harsh sampling conditions, which may even cause some artifacts, undermining the correctness of measurements. Second, we evaluate the performance of traffic classification under sampling. The information content of features, even though deteriorated, still allows a good classification accuracy, provided that the classifier is trained with data obtained at the same sampling rate of the target data. The accuracy is also due to a thoughtful choice of a smart sampling policy which biases the sampling towards packets carrying the most useful information. Copyright © 2012 John Wiley & Sons, Ltd.     

Quality-Driven Capacity-Aware Resources Optimization Design for Ubiquitous Wireless Networks

In this paper, we propose a probability-statistical capacity-prediction scheme to provide probabilistic quality-of-service (QoS) guarantees under the high traffic load of IEEE 802.11 wireless multimedia Mesh networks. The proposed scheme perceives the state of wireless link based on the MAC retransmission statistics and calculates the statistical channel capacity especially under the saturated traffic load. Via a cross-layer design approach, the scheme allocates network resource and forwards data packets by taking the interference among flows and the channel capacity into consideration. Extensive experiments have been carried out on the basis of IEEE 802.11 protocols in order to demonstrate the superiority of the proposed scheme over the existing location-based QoS optimization delivery algorithm in terms of retransmission count, successful delivery rate, and end-to-end delay on the condition of time-varying multi-hop wireless links.     

A deep learning method to detect network intrusion through flow‐based features

In this paper, we present a deep neural network model to enhance the intrusion detection performance. A deep learning architecture combining convolution neural network and long short‐term memory learns spatial‐temporal features of network flows automatically. Flow features are extracted from raw network traffic captures, flows are grouped, and the consecutive N flow records are transformed into a two‐dimensional array like an image. These constructed two‐dimensional feature vectors are normalized and forwarded to the deep learning model. Transformation of flow information assures deep learning in a computationally efficient manner. Overall, convolution neural network learns spatial features, and long short‐term memory learns temporal features from a sequence of network raw data packets. To maximize the detection performance of the deep neural network and to reach at the highest statistical metric values, we apply the tree‐structured Parzen estimator seeking the optimum parameters in the parameter hyper‐plane. Furthermore, we investigate the impact of flow status interval, flow window size, convolution filter size, and long short‐term memory units to the detection performance in terms of level in statistical metric values. The presented flow‐based intrusion method outperforms other publicly available methods, and it detects abnormal traffic with 99.09% accuracy and 0.0227 false alarm rate.     

A Method for Classifying Packets into Network Flows Based on GHSOM

Recently, various applications and services are used in the Internet. Load balancing the increasing network traffic in real time can improve the network quality. The flow control technologies become much more important than before. Our research proposes an intelligent network flow identifying method, which is based on the neural network algorithm, GHSOM. In this paper, we suggest to utilize the structural classification of GHSOM for training the properties of packets, such as timestamp, source and destination. Based on our proposed normalization, IP network flows can be formed autonomously during the learning process. The combination use of the new normalization with the GHSOM can divide a flow to several sub-IP flows. This paper indicates that a flow shall consist of several sub-IP flows, and sub-IP flow shall consist of several IP packets. The experiments show that IP packets can be divided to flow and sub-IP flow classes properly. Furthermore, those repeated jumbo sub-IP flows can be used to discover communicating errors or abnormal attacks.