Long-term forecasting of Internet backbone traffic

We introduce a methodology to predict when and where link additions/upgrades have to take place in an Internet protocol (IP) backbone network. Using simple network management protocol (SNMP) statistics, collected continuously since 1999, we compute aggregate demand between any two adjacent points of presence (PoPs) and look at its evolution at time scales larger than 1 h. We show that IP backbone traffic exhibits visible long term trends, strong periodicities, and variability at multiple time scales. Our methodology relies on the wavelet multiresolution analysis (MRA) and linear time series models. Using wavelet MRA, we smooth the collected measurements until we identify the overall long-term trend. The fluctuations around the obtained trend are further analyzed at multiple time scales. We show that the largest amount of variability in the original signal is due to its fluctuations at the 12-h time scale. We model inter-PoP aggregate demand as a multiple linear regression model, consisting of the two identified components. We show that this model accounts for 98% of the total energy in the original signal, while explaining 90% of its variance. Weekly approximations of those components can be accurately modeled with low-order autoregressive integrated moving average (ARIMA) models. We show that forecasting the long term trend and the fluctuations of the traffic at the 12-h time scale yields accurate estimates for at least 6 months in the future.     

Real-time traffic transmission over the Internet

Multimedia applications require the transmission of real-time streams over a network. These streams often exhibit variable bandwidth requirements, and require high bandwidths and guarantees from the network. This creates problems when such streams are delivered over the Internet. To solve these problems, recently, a small set of differentiated services has been introduced. Among these, Premium Service is suitable for transmitting real-time stored stream (full knowledge of the stream characteristics). It uses a bandwidth allocation mechanism (BAM) based on the stream peak rate. Due to the variable bandwidth requirement, the peak rate BAM can waste large amount of bandwidth. In this paper we propose a new BAM that uses less bandwidth than the peak rate BAM, while providing the same service. Our BAM does not affect the real-time stream quality of service (QoS) and does not require any modification to the Premium Service Architecture. We also introduce several frame dropping mechanisms that further reduce bandwidth consumption subject to a QoS constraint when coupled with the above BAM. The proposed BAM and the dropping mechanisms are evaluated using Motion JPEG and MPEG videos and are shown to be effective in reducing bandwidth requirements. Further, since VCR operations are very useful in video streaming, we propose a mechanism that introduces these operations in our BAM. Through simulations we show the effectiveness of this mechanism     

Internet traffic clustering with side information

Internet traffic classification is a critical and essential functionality for network management and security systems. Due to the limitations of traditional port-based and payload-based classification approaches, the past several years have seen extensive research on utilizing machine learning techniques to classify Internet traffic based on packet and flow level characteristics. For the purpose of learning from unlabeled traffic data, some classic clustering methods have been applied in previous studies but the reported accuracy results are unsatisfactory. In this paper, we propose a semi-supervised approach for accurate Internet traffic clustering, which is motivated by the observation of widely existing partial equivalence relationships among Internet traffic flows. In particular, we formulate the problem using a Gaussian Mixture Model (GMM) with set-based equivalence constraint and propose a constrained Expectation Maximization (EM) algorithm for clustering. Experiments with real-world packet traces show that the proposed approach can significantly improve the quality of resultant traffic clusters.     

基于机器学习的Internet流量分类

提出了基于机器学习的Internet流量分类框架,研究了支持向量机(SVM)在权威流量数据集全部子集上的分类效果和特征选择对分类效果的影响.实验结果表明,SVM对流量分类问题具有较高的分类精度和稳定性,通过特征选择,可以有效降低数据的维数,并一定程度上提高分类的精度.     

An orchestration approach for unwanted Internet traffic identification

A simple examination of Internet traffic shows a wide mix of relevant and unwanted traffic. The latter is becoming increasingly harmful to network performance and service availability, while often consuming precious network and processing resources. Coordinated attacks, such as distributed denial-of-services (DDoS), large-scale scans, and worm outbreaks, occur in multiple networks simultaneously and become extremely difficult to detect using an individual detection engine. This paper presents the specification of a new orchestration-based approach to detect, and, as far as possible, to limit the actions of these coordinated attacks. Core to the proposal is a framework that coordinates the receiving of a multitude of alerts and events from detectors, evaluates this input to detect or prove the existence of anomalies, and consequently chooses the best action course. This framework is named Orchestration-oriented Anomaly Detection System (OADS). We also describe an OADS prototype implementation of the proposed infrastructure and analyze initial results obtained through experimentation with this prototype.     

Long-range dependence ten years of Internet traffic modeling

Self-similarity and scaling phenomena have dominated Internet traffic analysis for the past decade. With the identification of long-range dependence (LRD) in network traffic, the research community has undergone a mental shift from Poisson and memory-less processes to LRD and bursty processes. Despite its widespread use, though, LRD analysis is hindered by the difficulty of actually identifying dependence and estimating its parameters unambiguously. The authors outline LRD findings in network traffic and explore the current lack of accuracy and robustness in LRD estimation. In addition, they present recent evidence that packet arrivals appear to be in agreement with the Poisson assumption in the Internet core.     

Internet traffic modeling by means of Hidden Markov Models

In this work, we propose a Hidden Markov Model for Internet traffic sources at packet level, jointly analyzing Inter Packet Time and Packet Size. We give an analytical basis and the mathematical details regarding the model, and we test the flexibility of the proposed modeling approach with real traffic traces related to common Internet services with strong differences in terms of both applications/users and protocol behavior: SMTP, HTTP, a network game, and an instant messaging platform. The presented experimental analysis shows that, even maintaining a simple structure, the model is able to achieve good results in terms of estimation of statistical parameters and synthetic series generation, taking into account marginal distributions, mutual, and temporal dependencies. Moreover we show how, by exploiting such temporal dependencies, the model is able to perform short-term prediction by observing traffic from real sources.     

基于Internet的楼宇自动化系统中的数据通信

介绍了基于Internet的楼宇自动化系统的概念及其子系统的功能和设计目标,在此基础上分析了数据通信的基本原理,提出了系统中的数据通信整体设计方案和实现方法,以便做到局域和远程信息的实时监控、数据资源的综合共享以及全局事件的快速处理和一体化的科学管理,最后给出了用VC 6．0实现的例子。     

避免数据包重复采集的分布式流量测量算法

针对D PC算法存在的问题,提出一种“标记”数据包头的分布式流量测量算法。为避免同一数据包被不同测量点重复采集,在网络入口测量点处,对到达的数据包进行抽样,修改被抽取数据包头中标志位的预留位为1;在网络的中间节点,通过查看数据包的预留位判断数据包是否已被抽取,对于已被抽取的数据包,不再重复记录,对于尚未被抽取的数据包,根据抽样规则决定是否抽取,避免多次采集相同数据包导致计算资源和存储资源的重复消耗。通过理论推导和实例验证了该算法的有效性,为分布式网络流量测量的扩展应用提供了解决途径。     

一种新型的高速IP网络的流量测量体系

网络流量测量是网络管理的基础之一,流量测量体系结构的研究是网络流量测量及其应用的基础和重要环节。对传统的网络流量测量方法的特点进行了分析,对高速网络流量测量中的关键问题进行了研究,采用数据流归并技术,以提高实用性为目的,Linux平台和开源软件的基础上提出了一种新型的基于分布式探针的高速IP网络流量测量体系,并对其构成及特点进行了分析。该测量体系提供了实用的高速测量平台和数据分析接口,具有广阔的应用前景。     

Towards low-complexity Internet traffic engineering: The Adaptive Multi-Path algorithm

In this paper, we describe a novel algorithm for dynamic traffic engineering in the Internet called Adaptive Multi-Path routing (AMP). The main objective of AMP is to distribute load within a network domain in a continuous manner by offloading congested links in real-time. The algorithm’s recursive signaling structure represents one of our central contributions, as it restricts information exchange to neighboring nodes while still maintaining global propagation of congestion information. This paper starts with a comprehensive survey of related literature, before we introduce AMP in depth and present our exhaustive performance evaluation which is carried out using full implementations of the algorithm both in packet-level and flow-level simulation environments. The investigated state-of-the-art topologies and traffic models provide a detailed insight into the behavior of AMP in realistic operators’ networks of substantial size and complexity, whereas our results demonstrate both the traffic engineering efficiency and the stability of AMP. Finally, we present important areas of application for AMP in emerging networking architectures and conclude the paper with an outlook on future research.     

基于机器自学习的互联网加密业务流早期识别

为精确高效地识别加密类业务流,给出了一种基于机器自学习的互联网加密业务流早期识别方法.该方法利用加密前后变化不明显的流量统计特征结合机器自学习方法进行识别.首先基于特征与业务类型的互熵来遴选出最优特征用于分类;然后利用所选特征给出了加密业务流总体识别模型,并对模型中的自学习阶段及识别阶段进行了创新,仅选取最能反映协议特点的每条业务流的前几个数据包进行早期识别,达到了对加密业务流高效识别的效果;最后对识别方法进行了性能分析和实验,实验结果表明,基于所选取的最优特征,仅利用每务流前5个数据包即可得到90%以上的流识别精确度.     

Address switching: Reforming the architecture and traffic of Internet

The success of the Internet is largely ascribable to the packet-switching scheme, which, however, also presents major challenges. Having identified three missing links in the current Internet architecture based on our long-term experiences of designing and operating large-scale backbones, we put forward a new, but incrementally deployable, network scheme—address switching. The address switching has both the advantages of packet switching and circuit switching; it supplies the missing links in the current Internet architecture and can reform the Internet traffic. Our analysis, protocol design and experiments indicate that the address switching can greatly improve the quality of service (QoS), security and routing scalability of today’s Internet. So it can provide flexible, high-performance and “per-service” networking for the scientific research communities. Moreover, it can provide a fairer and more sustainable business model for the commodity Internet. Supported by the China Next Generation Internet Project (Grant No. CNGI-04-13-2T), and the National Basic Research Program of China (Grant No. 041710001)     

Bandwidth optimization for Internet traffic in generalized processor sharing servers

Bandwidth optimization is considered when several classes of Internet traffic are served in generalized processor sharing (GPS) servers. Internet traffic shows self-similar patterns that make it difficult to obtain analytical performance in GPS. Thus, for performance estimation of different classes of traffic, we use fluid simulation techniques that can reduce the simulation complexity, compared to packet-level simulation. Using the relationship between the guaranteed bandwidth vector and the corresponding performance, we propose a bandwidth optimization problem to minimize the total bandwidth such that performance requirements are satisfied. We use an exterior penalty function method to solve the optimization problem. However, a penalized objective function may have local minimum which is not a global minimum. Thus, we propose a new methodology to circumvent the limitation of the exterior penalty function method.     

物联网异常流量检测算法研究

主要研究利用无监督机器学习算法检测物联网中的网络异常流量,采用开源数据集Intrusion Detection Evaluation Dataset,介绍了异常流量的特征选取,深入研究了三种无监督机器学习算法,分别是One-class Support Vector Machine (One-class SVM)、K-means和Isolation Forest。分析了这三种无监督机器学习算法的原理以及不足的地方,介绍了目前研究的状况。采用wireshark对pcap流量包进行解析、预处理操作,然后使用One-class SVM、K-means和Isolation Forest这三种无监督机器学习算法对训练集进行机器学习,对测试集进行正确率、误报率、漏报率、效率等方面进行评估。实验结果显示K-means算法多方面要优于其他两种算法,主要体现在正确率、误报率、漏报率和效率上。     

Fairness of traffic controls for inelastic flows in the Internet

In best-effort networks, fairness has been used as a criterion to guide the design of traffic controls. The notion of fairness has evolved over time, from simple equality to a form of equality modulated by the user’s need (e.g. max–min and proportional fairness). However, fairness has always been defined on a per-user basis for a deterministic workload. In this paper, we argue that we must redefine the notion of fairness when we study traffic controls for the co-existence of elastic and inelastic traffics. Our results indicate that subjecting inelastic flows to fairness congestion control on a per-flow basis does not necessarily maximize the network’s utility. Instead, inelastic flows may follow their own form of traffic control, such as admission control (without congestion control). At the aggregate level, our results indicate that it still makes sense to maintain a balance between elastic and inelastic traffic. In order to support our arguments, we develop a methodology for comparing different traffic controls for given utility functions and different workloads, both deterministic and stochastic.     

Image-based retro-reflectivity measurement of traffic signs in day time

The visibility of a traffic sign at night depends on its retro-reflectivity, a property that needs to be frequently monitored to ensure transportation safety. In the U.S., Federal Highway Administration (FHWA) maintains regulations to ensure minimum retro-reflectivity levels. Current measurement techniques either (a) use vehicle mounted device during the night, or (b) use manual handheld devices during the day. The former is expensive due to nighttime labor cost. The latter is time-consuming and unsafe. To address these limitations, this paper presents a computer vision-based technique to measure retro-reflectivity during daytime using a vehicle mounted device. The presented algorithms simulate nighttime visibility of traffic signs from images taken during daytime and measure their retro-reflectivity. The technique is faster, cheaper, and safer as it neither requires nighttime operation nor requires manual sign inspection. It also satisfies FHWA measurement guidelines both in terms of granularity and accuracy. The performance of the presented technique is evaluated under various testing conditions. The results are promising and demonstrate a strong potential in lowering inspection cost and improving safety in practical applications on retro-reflectivity measurement.     

互联网拓扑测量技术与工程实践

互联网拓扑测量是网络空间测绘的重要手段之一,相关技术与平台已经日趋成熟。首先介绍当前互联网拓扑测量技术与平台;然后介绍一个基于虚拟专用服务器的拓扑测量系统VTopo;最后,给出一次地区互联网拓扑测量试验,结果表明VTopo独立发现大量拓扑信息,与CAIDA Ark数据集具有很大的互补性。     

Routing Internet traffic in heterogeneous mesh networks: Analysis and algorithms

In practical wireless mesh networks (WMNs), gateways are subject to hard capacity limits on the aggregate number of flows (in terms of bit rate) that they can support. Thus, if traffic is routed in the mesh network without considering those constraints, as well as the traffic distribution, some gateways or intermediate mesh routers may rapidly get overloaded, and the network resources can be unevenly utilized. To address this problem, in this paper we firstly develop a multi-class queuing network model to analyze feasible throughput allocations, as well as average end-to-end delay, in heterogeneous WMNs. Guided by our analysis, we design a Capacity-Aware Route Selection algorithm (CARS), which allocates network paths to downstream and upstream Internet flows so as to ensure a more balanced utilization of wireless network resources and gateways’ fixed connections. Through simulations in a number of different network scenarios we show that the CARS scheme significantly outperforms conventional shortest path routing, as well as an alternative routing method that distributes the traffic load on the gateway nodes to minimize its variance.