Combining OpenFlow and sFlow for an effective and scalable anomaly detection and mitigation mechanism on SDN environments

Software Defined Networks (SDNs) based on the OpenFlow (OF) protocol export control-plane programmability of switched substrates. As a result, rich functionality in traffic management, load balancing, routing, firewall configuration, etc. that may pertain to specific flows they control, may be easily developed. In this paper we extend these functionalities with an efficient and scalable mechanism for performing anomaly detection and mitigation in SDN architectures. Flow statistics may reveal anomalies triggered by large scale malicious events (typically massive Distributed Denial of Service attacks) and subsequently assist networked resource owners/operators to raise mitigation policies against these threats. First, we demonstrate that OF statistics collection and processing overloads the centralized control plane, introducing scalability issues. Second, we propose a modular architecture for the separation of the data collection process from the SDN control plane with the employment of sFlow monitoring data. We then report experimental results that compare its performance against native OF approaches that use standard flow table statistics. Both alternatives are evaluated using an entropy-based method on high volume real network traffic data collected from a university campus network. The packet traces were fed to hardware and software OF devices in order to assess flow-based data-gathering and related anomaly detection options. We subsequently present experimental results that demonstrate the effectiveness of the proposed sFlow-based mechanism compared to the native OF approach, in terms of overhead imposed on usage of system resources. Finally, we conclude by demonstrating that once a network anomaly is detected and identified, the OF protocol can effectively mitigate it via flow table modifications.     

Flow-based anomaly detection in high-speed links using modified GSA-optimized neural network

Ever growing Internet causes the availability of information. However, it also provides a suitable space for malicious activities, so security is crucial in this virtual environment. The network intrusion detection system (NIDS) is a popular tool to counter attacks against computer networks. This valuable tool can be realized using machine learning methods and intrusion datasets. Traditional datasets are usually packet-based in which all network packets are analyzed for intrusion detection in a time-consuming process. On the other hand, the recent spread of 1–10-Gbps-technologies have clearly pointed out that scalability is a growing problem. In this way, flow-based solutions can help to solve the problem by reduction of data and processing time, opening the way to high-speed detection on large infrastructures. Besides, NIDS should be capable of detecting new malicious activities. Artificial neural network-based NIDSs can detect unseen attacks, so a multi-layer perceptron (MLP) neural classifier is used in this study to distinguish benign and malicious traffic in a flow-based NIDS. In this way, a modified gravitational search algorithm (MGSA), as a modern heuristic technique, is employed to optimize the interconnection weights of the neural anomaly detector. The proposed scheme is trained using an enhanced version of the first labeled flow-based dataset for intrusion detection introduced in 2009. In addition, the particle swarm optimization (PSO) algorithm and traditional error back-propagation (EBP) algorithm are employed to train MLP, so performance comparison becomes possible. The experimental results based on the actual network data show that the MGSA-optimized neural anomaly detector is effective for monitoring abnormal traffic flows in the gigabytes traffic environment, and the accuracy is about 97.8 %.     

Fast Filtered Sampling

Traffic sampled from the network backbone using uniform packet sampling is commonly utilized to detect heavy hitters, estimate flow level statistics, as well as identify anomalies like DDoS attacks and worm scans. Previous work has shown however that this technique introduces flow bias and truncation which yields inaccurate flow statistics and “drowns out” information from small flows, leading to large false positives in anomaly detection.In this paper, we present a new sampling design: Fast Filtered Sampling (FFS), which is comprised of an independent low-complexity filter, concatenated with any sampling scheme at choice. FFS ensures the integrity of small flows for anomaly detection, while still providing acceptable identification of heavy hitters. This is achieved through a filter design which suppresses packets from flows as a function of their size, “boosting” small flows relative to medium and large flows. FFS design requires only one update operation per packet, has two simple control parameters and can work in conjunction with existing sampling mechanisms without any additional changes. Therefore, it accomplishes a lightweight online implementation of the “flow-size dependent” sampling method. Through extensive evaluation on traffic traces, we show the efficacy of FFS for applications such as portscan detection and traffic estimation.     

基于异构网络处理平台的可配置并行路由查表算法研究

基于通用多核的网络转发性能难以满足高速网络流量线速处理的需求.软硬件结合的异构网络处理平台以其较高的性能和灵活性在网络处理领域得到广泛应用,但是如何基于异构平台实现高效的路由查表算法仍需进行深入研究,多核资源利用率低、共享冲突严重和访存次数多的问题是制约传统路由查表算法在异构网络处理平台实现性能提升的主要问题.为此,基于异构网络处理平台（network processing platform,简称NPP)提出一种可配置并行路由查表机制（configurable parallel lookup,简称CPL).CPL中的多线程并行查找和路由表的多副本存储技术在提高多核资源利用率的同时,实现了零冲突访问路由表项.此外,考虑到不同场景下路由前缀分布的差异,CPL支持通过配置对多级路由表的组织结构进行调整,从而有效地减少了路由表访问次数.最后在NPP上,对CPL和传统的查表算法进行性能测试和对比,验证了CPL的可用性和高效性.     

基于分数Langevin方程的通信量模型及其应用

建立了一个考虑QoS和多流复用的通信量模型。模型中的通信量由大量有相同源和目的地址的,独立同分布的数据流汇聚而成,其中每个数据流又由许多独立的符合Pareto分布的ON/OFF支流构成。文章首先分析了支流构成的数据流的概率分布,然后分析得到了数据流复用后的流速的概率分布,接着结合分数Langevin运动方程,得到了一个流量计算公式,该公式属于扩展的分数布朗运动。通过引入分数Langevin运动方程可以将外部影响因子,内部影响因子和Hurst参数有机地联系起来,给通信量的单分形和多重分形一个清晰合理的物理解释。最后使用该模型分析了VoIP中多流复用后的QoS参数,得到了几个重要结论。     

Evaluating TCP-friendliness in light of Concurrent Multipath Transfer

In prior work, a CMT protocol using SCTP multihoming (termed SCTP-based CMT) was proposed and investigated for improving application throughput. SCTP-based CMT was studied in (bottleneck-independent) wired networking scenarios with ns-2 simulations. This paper studies the TCP-friendliness of CMT in the Internet. In this paper, we surveyed historical developments of the TCP-friendliness concept and argued that the original TCP-friendliness doctrine should be extended to incorporate multihoming and SCTP-based CMT.Since CMT is based on (single-homed) SCTP, we first investigated TCP-friendliness of single-homed SCTP. We discovered that although SCTP’s congestion control mechanisms were intended to be “similar” to TCP’s, being a newer protocol, SCTP specification has some of the proposed TCP enhancements already incorporated which results in SCTP performing better than TCP. Therefore, SCTP obtains larger share of the bandwidth when competing with a TCP flavor that does not have similar enhancements. We concluded that SCTP is TCP-friendly, but achieves higher throughput than TCP, due to SCTP’s better loss recovery mechanisms just as TCP-SACK and TCP-Reno perform better than TCP-Tahoe.We then investigated the TCP-friendliness of CMT. Via QualNet simulations, we found out that one two-homed CMT association has similar or worse performance (for smaller number of competing TCP flows) than the aggregated performance of two independent, single-homed SCTP associations while sharing the link with other TCP connections, for the reason that a CMT flow creates a burstier data traffic than independent SCTP flows. When compared to the aggregated performance of two-independent TCP connections, one two-homed CMT obtains a higher share of the tight link bandwidth because of better loss recovery mechanisms in CMT. In addition, sharing of ACK information makes CMT more resilient to losses. Although CMT obtains higher throughput than two independent TCP flows, CMT’s AIMD-based congestion control mechanism allows other TCP flows to co-exist in the network. Therefore, we concluded that CMT is TCP-friendly, similar to two TCP-Reno flows are TCP-friendly when compared to two TCP-Tahoe flows.     

Enhanced Internet security by a distributed traffic control service based on traffic ownership

The frequency and intensity of Internet attacks are rising at an alarming pace. Several technologies and concepts were proposed for fighting distributed denial of service (DDoS) attacks: traceback, pushback, i3, SOS and Mayday. This paper shows that in the case of DDoS reflector attacks they are either ineffective or even counterproductive. We then propose the novel concept of traffic ownership and describe a system that extends control over network traffic by network users to the Internet using adaptive traffic processing devices. We safely delegate partial network management capabilities from network operators to network users. All network packets with a source or destination address “owned” by a network user can now also be controlled within the Internet instead of only at the network user's Internet uplink. By limiting the traffic control features and by restricting the realm of control to the “owner” of the traffic, we can rule out misuse of this system. Applications of our system are manifold: prevention of source address spoofing, DDoS attack mitigation, distributed firewall-like filtering, new ways of collecting traffic statistics, service-level agreement validation, traceback, distributed network debugging, support for forensic analyses and many more. A use case illustrates how our system enables network users to prevent and react to DDoS attacks.     

基于多核处理器的L7-Filter规则匹配改进算法

针对多核处理器的体系结构和网络数据流在时间上的局部性特点,提出了一种基于多核处理器的分链动态适应算法。该算法通过对网络数据流进行类型分类并根据网络数据流的时间局部性对规则链进行动态优化,从而有效减少了多核处理器下L7-Filter对网络数据流的匹配次数,显著提升了规则匹配效率。仿真实验结果表明:在网络数据包个数相同条件下,所提算法在性能上约有7%的提高。随着网络数据包个数的增加,性能优越性更加明显。     

A generalizable dynamic flow pairing method for traffic classification

The goal of network traffic classification is to identify the protocols or types of protocols in the network traffic. In particular, the identification of network traffic with high resource consumption, such as peer-to-peer (P2P) traffic, represents a great concern for Internet Service Providers (ISP) and network managers. Most current flow-based classification approaches report high accuracy without paying attention to the generalization ability of the classifier. However, without this ability, a classifier may not be suitable for on-line classification. In this paper, a number of experiments on real traffic help to elucidate the reason for this lack of generalization. It is also shown that one way to attain the generalization ability is by using dynamic classifiers. From these results, a dynamic classification approach based on the pairing of flows according to a similarity criterion is proposed. The pairing method is not a classifier by itself. Rather, its goal is to determine in a fast way that two given flows are similar enough to conclude they correspond to the same protocol. Combining this method with a classifier, most of the flows do not need to be explicitly evaluated by the later, so that the computational overhead is reduced without a significant reduction in accuracy. In this paper, as a case study, we explore complementing the pairing method with payload inspection. In the experiments performed, the pairing approach generalizes well to traffic obtained in different conditions and scenarios than that used for calibration. Moreover, a high portion of the traffic unclassified by payload inspection is categorized with the pairing method.     

软件定义网络中延迟满足的路由选择与实时调度更新

由于数据流的动态性和流量负载转移,软件定义网络（software defined networking,简称SDN）需要频繁更新数据平面以优化网络性能.大多数已有路由更新策略首先根据网络当前流量状态确定目标路由配置,然后更新数据流的路由.然而,由于交换机基于TCAM（ternary content addressable memory）进行流表更新的速度较慢,导致路由更新的延迟通常较大.当网络规模大或网络拓扑结构经常变化时,路由更新的延迟可能更大.研究发现,大多数数据流的持续时间很短且整个网络的流量强度在一段时间后会发生变化.如果路由更新延迟过长,更新后的路由配置可能不再有效.为此,研究了SDN的实时路由更新问题,提出了延迟满足的路由选择和调度更新策略（delay satisfied route selection and updating scheme,简称DSRSU）.与大多数现有研究不同,DSRSU同时从控制平面路径选择和数据平面的更新调度两方面来联合优化,降低路由更新的延迟.路径选择阶段只选择部分数据流进行路由更新;更新调度阶段通过建立更新关系图挖掘数据流的更新先后顺序,进一步加快路由更新速度.仿真分析结果表明,与现有几种路由更新策略相比,DSRSU能够在大幅度降低路由更新延迟的同时,达到与现有策略相似的网络性能.     

基于通用多核平台的入侵检测系统研究

为应对网络流量快速增长问题,提出一种基于通用多核平台的入侵检测系统结构。在系统设计基础上,分析、验证了硬件平台、资源分配模式和流量特征等关键因素对系统处理性能的影响。实验表明,网络流量的流数、单位时间内报文包数等指标对系统性能的影响更大;在启用多核处理器超线程技术并将检测引擎与CPU绑定时,系统性能可以得到有效提高;系统易于实现,性价比高。     

An Architecture for Network Congestion Control and Charging of Non-cooperative Traffic

This paper proposes an architecture capable of reducing network congestion caused by the intense use of non-cooperative traffic. A charging scheme is imposed to all traffic that is carried over the UDP protocol (non-cooperative) given its intrinsic priority over TCP. Prices are calculated according to the degree of starvation undergone by cooperative TCP flows. When TCP flows experience a low performance, charges are high for non-cooperative flows and so the architecture tends to block new incoming UDP traffic. Knowledge about flows status is obtained through the use of flow protocol technology. Resources are reserved using firewall rules and custom-queueing mechanisms. An implementation of the architecture is made and tests show the effectiveness of our proposal in a real network scenario.     

An online framework for catching top spreaders and scanners

Flow level information is important for many applications in network measurement and analysis. In this work, we tackle the “Top Spreaders” and “Top Scanners” problems, where hosts that are spreading the largest numbers of flows, especially small flows, must be efficiently and accurately identified. The identification of these top users can be very helpful in network management, traffic engineering, application behavior analysis, and anomaly detection.We propose novel streaming algorithms and a “Filter-Tracker-Digester” framework to catch the top spreaders and scanners online. Our framework combines sampling and streaming algorithms, as well as deterministic and randomized algorithms, in such a way that they can effectively help each other to improve accuracy while reducing memory usage and processing time. To our knowledge, we are the first to tackle the “Top Scanners” problem in a streaming way. We address several challenges, namely: traffic scale, skewness, speed, memory usage, and result accuracy. The performance bounds of our algorithms are derived analytically, and are also evaluated by both real and synthetic traces, where we show our algorithm can achieve accuracy and speed of at least an order of magnitude higher than existing approaches.     

基于流的网络流量特征分析

网络流量特征分析是提高网络性能的基础．其自相似特征是一个普遍存在的现象．通过对主干链路上的流量进行基于流的流量特征的分析，结果表明流间隔时间序列在小时间尺度上的自相似程度较弱，而大时间尺度上的自相似程度较强．进一步的分析表明，流的大小以及ICMP流对流的自相似特征有显著的影响，特别是流大小为1个包的流对其影响更大．     

Multi-core application performance optimization using a constrained tandem queueing model

Recently, performance optimization on multi-core platforms has received more and more research interest especially for networking applications. While a number of case studies have been made with success, there is however a lack of analytical treatment on the modeling of parallel processing in applications. In this paper, we tackle the performance optimization problem using queueing analysis. Modeling macroscopic pipeline processes with a constrained tandem model, an optimization principle is derived. We then employ the derived principle to analyze two network applications: intrusion detection and image retrieval. Both theoretical performance analysis and simulation results confirm the effectiveness of our optimization principle.     

基于流谱理论的SSL/TLS协议攻击检测方法

网络攻击检测在网络安全中扮演着重要角色.网络攻击检测的对象主要为僵尸网络、SQL注入等攻击行为.随着安全套接层/安全传输层(SSL/TLS)加密协议的广泛使用,针对SSL/TLS协议本身发起的SSL/TLS攻击日益增多,因此通过搭建网络流采集环境,构建了包含4种SSL/TLS攻击网络流与正常网络流的网络流数据集.针对当...     

A calculus for stochastic QoS analysis

The issue of Quality of Service (QoS) performance analysis in packet-switched networks has drawn a lot of attention in the networking community. There is a lot of work including an elegant theory under the name of network calculus, which focuses on analysis of deterministic worst case QoS performance bounds. In the meantime, researchers have studied stochastic QoS performance for specific schedulers. However, most previous works on deterministic QoS analysis or stochastic QoS analysis have only considered a server that provides deterministic service, i.e. deterministically bounded rate service. Few have considered the behavior of a stochastic server that provides input flows with variable rate service, for example wireless links. In this paper, we propose a stochastic network calculus to analyze the end-to-end stochastic QoS performance of a system with stochastically bounded input traffic over a series of deterministic and stochastic servers. We also prove that a server serving an aggregate of flows can be regarded as a stochastic server for individual flows within the aggregate. Based on this, the proposed framework is further applied to analyze per-flow stochastic QoS performance under aggregate scheduling.     

多核平台入侵检测系统负载均衡算法设计与实现*

负载均衡是基于多核平台实现高速入侵检测系统的关键技术之一。基于真实流量统计分析发现的流阈值与流数目、流字节数之间变化的规律,提出只调整较大流的动态分流算法HCLF,并实现了原型系统。实验测试表明,与静态哈希算法和新流调整算法相比,HCLF算法在负载均衡度、系统丢包率方面具有显著的优越性,改善了多核平台高速入侵检测系统对突发流量和应用环境的适应性。     

基于NDN的区块链数据同步方法

近年来，区块链技术在各个领域受到了广泛的关注并取得了阶段性的成功，然而区块链技术的自身缺陷，限制了其在重要领域的发展。这是因为区块链技术通过P2P网络基于TCP架构的通信方式，使得其对相同数据内容的请求是独立的，从而在数据同步时造成大量的冗余流量，进而占用较高的带宽而导致了较大的数据传输时延，并在最终拉低了区块链的整体性能。由此，提出“NDN+区块链”结构，以期借助命名数据网络（named data networking，NDN）自有的优势，有效解决区块链数据传输效率低下的问题，从而扩大区块链的技术优势。通过仿真模拟实验的分析，得出“NDN＋区块链”结构相较于传统的TCP区块链，在带宽占用及数据同步时延方面确有很好的表现效果。     

空间高效的数据包公平抽样算法

数据包公平抽样通过牺牲长流的包抽样率以换取更高的短流包抽样率,因而比均匀随机包抽样更能保证数据流之间的公平性.现有的公平抽样算法SGS(sketch guided sampling)存在空间效率低、短流估计误差大的问题.提出了一种空间高效的数据包公平抽样算法SEFS(space-efficient fair sampling).SEFS算法的新颖之处在于采用多解析度抽样统计器对数据流流量作近似估计,各个统计器由d-left哈希表实现.采用在OC-48和OC-192骨干网采集的真实流量数据,在数据流流量测量以及长流检测的应用背景下,对SEFS算法和SGS算法的性能进行了比较.实验结果表明,与SGS算法相比,SEFS算法在空间复杂度降低65%的前提下,仍具有更高的估计精度.特别是对于占网络数据流绝大多数的短流而言,SEFS算法估计精度高的优势更为明显.