Improving the resilience of content distribution networks to large scale distributed denial of service attacks

Distributed Denial of Service (DDoS) attacks remain a daunting challenge for Internet service providers. Previous work on countering these attacks has focused primarily on attacks at a single server location and the associated network infrastructure. Increasingly, however, high-volume sites are served via content distribution networks (CDNs). In this paper, we propose two mechanisms to withstand and deter DDoS attacks on CDN-hosted Web sites and the CDN infrastructure. First, we present a novel CDN request routing algorithm which allows CDN proxies to effectively distinguish attacks from the requests from actual users. The proposed scheme, based on the keyed hash function, can significantly improve the resilience of CDNs to DDoS attacks. In particular, the resilience of a CDN, consisting of n proxies, becomes O(n²) with the proposed approach, when compared to a site hosted by a single server. We present performance numbers from a controlled test environment to show that the proposed approach is effective. Second, we introduce novel site allocation algorithms based on the well-established theory on binary codes. The proposed allocation algorithm guarantees an upper bound on the level of service outage of a CDN-hosted site even when a DoS attack on another site on the same CDN has been successful. Together, our schemes significantly improve the resilience of the Web sites hosted by CDNs, and complement other work on countering DoS.     

Edge Video CDN: A Wi-Fi Content Hotspot Solution

The emergence of smart edge-network content item hotspots, which are equipped with huge storage space (e.g., several GBs), opens up the opportunity to study the possibility of delivering videos at the edge network. Different from both the conventional content item delivery network (CDN) and the peer-to-peer (P2P) scheme, this new delivery paradigm, namely edge video CDN, requires up to millions of edge hotspots located at users’ homes/offices to be coordinately managed to serve mobile video content item. Specifically, two challenges are involved in building edge video CDN, including how edge content item hotspots should be organized to serve users, and how content items should be replicated to them at different locations to serve users. To address these challenges, we propose our data-driven design as follows. First, we formulate an edge region partition problem to jointly maximize the quality experienced by users and minimize the replication cost, which is NP-hard in nature, and we design a Voronoi-like partition algorithm to generate optimal service cells. Second, to replicate content items to edge-network content item hotspots, we propose an edge request prediction based replication strategy, which carries out the replication in a server peak offloading manner. We implement our design and use trace-driven experiments to verify its effectiveness. Compared with conventional centralized CDN and popularity-based replication, our design can significantly improve users’ quality of experience, in terms of users’ perceived bandwidth and latency, up to 40%.     

基于流媒体服务的IP网络内容分发系统的设计

IP网络中数据量巨大的流媒体内容使服务器和骨干网络承受的压力越来越大。内容分发系统对保证流媒体QoS起到至关重要的作用。该文通过分析内容分发网络(CDN)和P2P两种新型内容分发技术，提出了多源网状散列式P2P模型和一种改进的内容缓存算法，设计了基于CDN和P2P技术的内容分发系统。系统可同时满足请求多和请求少的情况，提高了流媒体QoS。     

Techniques for increasing the stream capacity of a high-performancemultimedia server

High-performance servers and high-speed networks will form the backbone of the infrastructure required for distributed multimedia information systems. A server for an interactive distributed multimedia system may require thousands of gigabytes of storage space and a high I/O bandwidth. In order to maximize the system utilization, and thus minimize the cost, it is essential that the load be balanced among each of the server's components, viz. the disks, the interconnection network and the scheduler. Many algorithms for maximizing retrieval capacity from the storage system have been proposed in the literature. This paper presents techniques for improving the server capacity by assigning media requests to the nodes of a server so as to balance the load on the interconnection network and the scheduling nodes. Five policies for request assignment-round-robin (RR), minimum link allocation (MLA), minimum contention allocation (MCA), weighted minimum link allocation (WMLA) and weighted minimum contention allocation (WMCA)-are developed. The performance of these policies on a server model developed by the authors (1995) is presented. We also consider the issue of file replication, and develop two schemes for storing the replicas: the parent group-based round-robin placement (PGBRRP) scheme, and the group-wide round-robin placement (GWRRP) scheme. The performance of the request assignment policies in the presence of file replication is presented     

双机热备系统中的数据同步技术研究

双机热备系统中数据的及时同步在很大程度上影响系统的性能。针对传统双机热备系统中数据复制的连续性以及检索更新文件时间长的不足,提出了基于链路延迟的文件复制和基于单向哈希函数的更新文件检索。通过对这两种方法的分析,说明该改善在很大程度上提高了双机对等节点间数据同步的效率。     

A paracasting model for concurrent access to replicated Internet content

In this paper, we develop a model to study how to effectively download a document from a set of replicated servers. We propose a generalized application-layer anycasting protocol, known as paracasting, to advocate concurrent access of a subset of replicated servers to cooperatively satisfy a client's request. Each participating server satisfies the request in part by transmitting a subset of the requested file to the client. The client can recover the complete file when different parts of the file sent from the participating servers are received. This model allows us to estimate the average time to download a file from the set of homogeneous replicated servers, and the request blocking probability when each server can accept and serve a finite number of concurrent requests. Our results show that the file download time drops when a request is served concurrently by a larger number of homogeneous replicated servers, although the performance improvement quickly saturates when the number of servers increases. If the total number of requests that a server can handle simultaneously is finite, the request blocking probability increases with the number of replicated servers used to serve a request concurrently. Therefore, paracasting is effective when a small number of servers, say, up to four, are used to serve a request concurrently.     

Modelling and performance analysis of content sharing and distribution in community networks with infrastructure support

We consider the problem of sharing a viral file between users in a local community network (e.g., college and office campuses). Community computing is computer networking among and between users in a geographically bounded setting for local purposes and activities. Due to the community-oriented nature of such networks, it is likely that users of a community network would like to share content. Peer-to-Peer (P2P) networks have turned out to be one of the most innovative paradigms for sharing content on the Internet. In this paper, we analyze the performance of P2P content sharing in community networks and investigate the role that infrastructure nodes (helpers) can play to enhance the performance of content sharing and distribution. We model the evolution of content demand in a community network. The use of this demand prediction model allows us to design a delicate P2P-with-helpers content distribution system. Our insights that we obtain using fluid-flow model increase our understanding of how helper provisioning affects the performance of content sharing and distribution. The derived results show that significant reduction in both the cost of distributing content and the average content download time can be realized when only few infrastructure nodes in the community network play the role helpers and cache P2P objects.     

Building a peer-to-peer content distribution network with high performance,scalability and robustness

Content distribution networks (CDN) are fundamental, yet expensive technologies for distributing the content of web-servers to large audiences. The P2P model is a perfect match to build a low-cost and scalable CDN infrastructure for popular websites by exploiting the underutilized resources of their user communities. However, building a P2P-based CDN is not a straightforward endeavor. In contrast to traditional CDNs, peers are autonomous and volunteer participants with their own heterogeneous interests that should be taken into account in the design of the P2P system. Moreover, churn rate is much higher than in dedicated CDN infrastructures, which can easily destabilize the system and severely degrade the performance. Finally and foremostly, while many P2P systems abstract any topological information about the underlying network, a top priority of a CDN is to incorporate locality-awareness in query routing in order to locate close-by content. This paper aims at building a P2P CDN with high performance, scalability and robustness. Our proposed protocols combine DHT efficiency with gossip robustness and take into account the interests and localities of peers. In short, Flower-CDN provides a hybrid and locality-aware routing infrastructure for user queries. PetalUp-CDN is a highly scalable version of Flower-CDN that dynamically adapts to variable rates of participation and prevent overload situations. In addition, we ensure the robustness of our P2P CDN via low-cost maintenance protocols that can detect and recover from churn and dynamicity. Our extensive performance evaluation shows that our protocols yield high performance gains under both static and highly dynamic environments. Furthermore, they incur acceptable and tunable overhead. Finally we provide main guidelines to deploy Flower-CDN for the public use.     

Analysis of Replica Placement under Expiration-Based Consistency Management

Expiration-based consistency management is widely used to keep replicated contents up-to-date in the Internet. The effectiveness of replication can be characterized by the communication costs of client accesses and consistency management. Both costs depend on the locations of the replicas. This paper investigates the problem of placing replicas in a network where replica consistency is managed by the expiration-based scheme. Our objective is to minimize the total cost of client accesses and consistency management. By analyzing the communication cost of recursive validations for cascaded replicas, we prove that in the optimal placement scheme, the nodes not assigned replicas induce a connected subgraph that includes the origin server. Our results are generic in that they apply to any request arrival patterns. Based on the analysis, an O(D)-time algorithm is proposed to compute the optimal placement of the replicas, where D is the sum of the number of descendants over all nodes in the routing tree     

基于CDN和P2P的分布式网络存储系统

把用户的文件分片后均衡存储在不同的分布式存储节点上,并利用虚拟目录服务器和基于P2P—DHT的目录服务器把文件元数据与文件数据片高效地对应起来,以提供高效目录服务,分布式存储节点以P2P方式工作以快速完成用户对文件数据的请求任务。分布式网络存储系统DNSS充分利用了CDN和P2P的技术优势,有较高的可用性、可靠性和可扩展性。DNSS已经在中国科学技术大学应用。     

城域网上CDN技术的应用

CDN是一个建立并覆盖在互联网,并由分布在不同区域的节点服务器群组成的虚拟网络。CDN采用缓存、复制、负载均衡和客户请求重定向等技术,将信息资源推向网络边缘,使得客户可以从"最近最好"的服务器快速访问到所需的内容,从而提高了终端用户的访问速度。文中简要介绍CDN的概念和技术,包括CDN基本工作原理,内容路由的设计原则和机制比较,节点内容引擎比较。通过CDN的技术方案,为宽带的发展提供一些技术参考。     

PROSE: Proactive, Selective CDN Participation for P2P Streaming

Many production peer-to-peer (P2P) streaming systems use content delivery networks (CDN) to protect the user’s quality of experiences. Thus, how to efficiently utilize the capacity of CDN (e.g., which peers receive services from the CDN nodes) is a problem of practical significance. Existing solutions adopt a passive, on-demand approach, which is inefficient in utilizing CDN resources. In this paper, we propose PROSE, a simple, novel scheme to achieve proactive, selective CDN participation for P2P streaming. PROSE introduces novel concepts such as choke point expansion nodes/super nodes and leads to efficient, light-weighted, and distributed algorithms to identify and serve these nodes using CDN. Our experimental results show that PROSE achieves at least 10%～25% performance improvement and 2～4 times overhead reduction compared with existing general CDN-P2P-hybrid schemes.     

内容分发网络性能测量方法研究与实现

内容分发网络 (CDN)是新型的网络加速技术 .提出了一种新的 CDN性能测量方法 :将 CDN性能测量融入到网络测量基础设施之中 ,CDN提供者、内容提供者或终端用户可以根据需要随时、方便的测量、评价 CDN性能 ,克服了以往测量方法测量周期长、测量规模扩展性差以及数据资源利用率低等缺点 .基于此方法 ,设计并实现了一个集成在 CNMI下的 CDN性能测量系统 :CDNPMS,系统使用灵活简单 ,可以从 CDN外部测量用户感知的 (Client- perceived) CDN性能 ,并且用此系统对目前最大的 Akamai CDN进行了成功测量 .     

Freenet中的密钥和索引机制综述

Freenet,又叫自由网,是一种分布式信息存储和搜索系统,设计定位于保障信息的私有性和有效性。系统的运行类似于一个具有位置无关性特征的分布式文件系统,这个文件系统由许多独立的计算机组成,这些计算机允许用户匿名进行文件的插入、存储和请求。在自由网中,发布的匿名性,文件的标识、发布、存储、请求都与自由网的密钥和索引机制有着相当密切的联系,该文对自由网中的密钥机制做了一定程度的分析、探索,以求能对自由网的运行机制的进一步研究和了解有所帮助。     

Performance evaluations of data-centric information retrieval schemes for DTNs

Mobile nodes in some challenging network scenarios, e.g. battlefield and disaster recovery scenarios, suffer from intermittent connectivity and frequent partitions. Disruption Tolerant Network (DTN) technologies are designed to enable communications in such environments. Several DTN routing schemes have been proposed. However, not much work has been done on designing schemes that provide efficient information access in such challenging network scenarios. In this paper, we explore how a content-based information retrieval system can be designed for DTNs. There are three important design issues, namely (a) how data should be replicated and stored at multiple nodes, (b) how a query is disseminated in sparsely connected networks, and (c) how a query response is routed back to the issuing node. We first describe how to select nodes for storing the replicated copies of data items. We consider the random and the intelligent caching schemes. In the random caching scheme, nodes that are encountered first by a data-generating node are selected to cache the extra copies while in the intelligent caching scheme, nodes that can potentially meet more nodes, e.g. faster nodes, are selected to cache the extra data copies. The number of replicated data copies K can be the same for all data items or varied depending on the access frequencies of the data items. In this work, we consider fixed, proportional and square-root replication schemes. Then, we describe two query dissemination schemes: (a) W-copy Selective Query Spraying (WSS) scheme and (b) L-hop Neighborhood Spraying (LNS) scheme. In the WSS scheme, nodes that can move faster are selected to cache the queries while in the LNS scheme, nodes that are within L-hops of a querying node will cache the queries. For message routing, we use an enhanced Prophet scheme where a next-hop node is selected only if its predicted delivery probability to the destination is higher than a certain threshold. We conduct extensive simulation studies to evaluate different combinations of the replication and query dissemination algorithms. Our results reveal that the scheme that performs the best is the one that uses the WSS scheme combined with binary spread of replicated data copies. The WSS scheme can achieve a higher query success ratio when compared to a scheme that does not use any data and query replication. Furthermore, the square-root and proportional replication schemes provide higher query success ratio than the fixed copy approach with varying node density. In addition, the intelligent caching approach can further improve the query success ratio by 5.3–15.8% with varying node density. Our results using different mobility models reveal that the query success ratio degrades at most 7.3% when the Community-Based model is used compared to the Random Waypoint (RWP) model [J. Broch et al., A Performance Comparison of Multihop wireless Ad hoc Network Routing Protocols, ACM Mobicom, 1998, pp. 85–97]. Compared to the RWP and the Community-Based mobility models, the UmassBusNet model from the DieselNet project [X. Zhang et al., Modeling of a Bus-based Disruption Tolerant Network Trace, Proceedings of ACM Mobihoc, 2007.] achieves much lower query success ratio because of the longer inter-node encounter time.     

基于DHT的拓扑感知节点聚集算法

在文件共享、流媒体和协作计算等P2P应用模型中,节点间采用单播通信并构建出对应的覆盖网络.由于覆盖网络通常建立在已有的底层网络之上,节点随机加入系统将导致上下层网络拓扑不匹配,不仅增加了节点间通信延时而且给底层网络带来较大的带宽压力.当前的拓扑匹配算法尚存在可扩展性低、节点聚集时延长等问题.在网络坐标算法和DHT算法基础之上,提出一种分布式的拓扑感知节点聚集算法TANRA,利用等距同心圆簇对节点二维网络坐标平面进行等面积划分,并根据节点所处区域进行多层命名空间中区间的一一映射.由于保留了节点之间的邻近关系,从而可使用DHT基本的"发布"和"搜索"原语进行相邻节点聚集.仿真结果表明,TANRA算法在大规模节点数时能有效保证网络拓扑匹配,并且具有较低的加入延时.     

DiCAS: An Efficient Distributed Caching Mechanism for P2P Systems

Peer-to-peer networks are widely criticized for their inefficient flooding search mechanism. Distributed Hash Table (DHT) algorithms have been proposed to improve the search efficiency by mapping the index of a file to a unique peer based on predefined hash functions. However, the tight coupling between indices and hosting peers incurs high maintenance cost in a highly dynamic network. To properly balance the tradeoff between the costs of indexing and searching, we propose the distributed caching and adaptive search (DiCAS) algorithm, where indices are passively cached in a group of peers based on a predefined hash function. Guided by the same function, adaptive search selectively forwards queries to "matched” peers with a high probability of caching the desired indices. The search cost is reduced due to shrunk searching space. Different from the DHT solutions, distributed caching loosely maps the index of a file to a group of peers in a passive fashion, which saves the cost of updating indices. Our simulation study shows that the DiCAS protocol can significantly reduce the network search traffic with the help of small cache space contributed by each individual peer.     

Designing content distribution networks for optimal cost and performance

One strategy for alleviating excess latency (delay) in the Internet is the caching of web content at multiple locations. This reduces the number of hops necessary to reach the desired content. This strategy is used for web content such as html pages, images, streaming video, and Internet radio. The network of servers which store this content, and the collections of objects stored on each server, is called a content distribution network (CDN). In order to optimally design a CDN, given a network topology with available server storage capacity at various points in the network, one must decide which object collections to place on each server in order to achieve performance or cost objectives. The placements must be within the storage limits of the servers and must reflect the request patterns for each collection of objects to be cached. Researchers have suggested formulations for the CDN problem which address performance by minimizing latency (the average number of hops is a commonly accepted measure of latency) from client to content, or formulations that focus on minimizing cost of storage and/or bandwidth. In this research, we develop a model which allows for the simultaneous treatment of performance and cost, present examples to illustrate the application of the model and perform a detailed designed experiment to gain insights into cost/hops tradeoff for a variety of network parameters.     

Cooperative content dissemination in multi-channel WLAN hotspots

It is common for hotspots in airports, cafes, and malls to deploy info-stations on their access points to provide local information such as flight schedule, multimedia access, sales and discount deals. The dissemination service, however, is limited by the wireless throughput in hotspots and cannot scale to a large number of users. This paper designs a new architecture that significantly improves the throughput of info-stations. We observe that usually the number of collocated info-stations is smaller than the number of orthogonal channels in IEEE 802.11, leaving some orthogonal channels unused, and thus wasting their capacity. We also observe that many users of an info-station download the same objects. Thus, we advocate an architecture, called hybrid-WLAN (H-WLAN), where mobile nodes can retrieve objects (multimedia content or information) from the info-station in the infrastructure mode, or exploit the idle 802.11 channels to cooperatively share their content with neighboring nodes in an ad-hoc manner. Our H-WLAN contains two components. The first component directs each object request to a nearby ad-hoc node that has the requested object, while avoids overloading any of the nodes. The second component balances the load across channels taking into account the popularity of the disseminated objects. We evaluate our design via simulations. The results show that our H-WLAN improves channel utilization and provides significantly higher throughput.     

DHT网络基于债务关系的公平文件交换

P2P(peer to peer)网络中,节点的自私行为极大地降低了系统的可用性.基于债务关系的文件交换网络,构建了一种促进合作的激励机制.同时,该机制保证了文件交换的公平性.激励机制的关键在于DHT(distributed hash table)网络邻居有限的固有特征,因而节点间的交互易于形成重复博弈.DFFE(debt relationship based fair file exchange in DHT network)协议只需维护很少的本地节点交互信息,协议开销小、网络扩展性好.网络路由采用基于一跳信息的贪婪算法.理性节点间的博弈存在纳什均衡,其策略选择的近似算法具有渐进收敛性.仿真实验表明了激励机制的有效性和在动态网络中性能的稳定性.