A scalable P2P platform for the knowledge grid

The knowledge grid needs to operate with a scalable platform to provide large-scale intelligent services. A key function of such a platform is to efficiently support various complex queries in a dynamic large-scale network environment. This paper proposes a platform to support index-based path queries by incorporating a semantic overlay with an underlying structured P2P network that provides object location and management services. Various distributed indexing structures can be dynamically formed by publishing, semantic objects as indexing nodes. Queries are forwarded along the chains of semantic object pointers to search for objects. We investigate the deployment of a scalable distributed trie index for broadcast queries on key strings, propose a decentralized load balancing method for solving the problem of uneven load distribution incurred by heterogeneity of loads and node capacities and by the distributed trie index, and give an approach for improving the availability of the semantic overlay and its trie index. Experiments demonstrate the scalability of the proposed platform.     

基于Hilbert曲线的双层覆盖网模型

资源定位和负栽均衡是P2P网络系统中急需解决的核心问题。本文给出的模型将连续相邻的结点组成一个无结构的结点组，结点组之间利用Chord协议连接成为一个全局结构化的环形拓扑结构，并在该拓扑结构中引入HBLS资源查找算法和动态负载均衡双层环状模型调度策略。整个方案不仅具有很好的负载平衡特性，而且为查找提供了充分的灵活性性。仿真实验表明，该模型在资源定位和负载平衡两个方面都优于原Chord。     

Virtual Tree: A robust architecture for interval valid queries in dynamic distributed systems

This paper studies the problem of answering aggregation queries, satisfying the interval validity semantics, in a distributed system prone to continuous arrival and departure of participants. The interval validity semantics states that the query answer must be calculated considering contributions of at least all processes that remained in the distributed system for the whole query duration. Satisfying this semantics in systems experiencing unbounded churn is impossible due to the lack of connectivity and path stability between processes. This paper presents a novel architecture, namely Virtual Tree, for building and maintaining a structured overlay network with guaranteed connectivity and path stability in settings characterized by bounded churn rate. The architecture includes a simple query answering algorithm that provides interval valid answers. The overlay network generated by the Virtual Tree architecture is a tree-shaped topology with virtual nodes constituted by clusters of processes and virtual links constituted by multiple communication links connecting processes located in adjacent virtual nodes. We formally prove a bound on the churn rate for interval valid queries in a distributed system where communication latencies are bounded by a constant unknown by processes. Finally, we carry out an extensive experimental evaluation that shows the degree of robustness of the overlay network generated by the virtual tree architecture under different churn rates.     

Laurel:一种混合式数据分发覆盖网

覆盖网是各种数据分发应用的基础架构.在节点波动的网络环境中实现快速而准确的数据分发,对覆盖网提出了两个要求:高效的数据路由;较强的系统鲁棒性.已有的覆盖网构建方法多侧重于某个方面的优化,因而未能充分权衡数据路由效率与系统鲁棒性.提出了一种混合式数据分发覆盖网--Laurel.Laurel通过簇间多重结构化拓扑与簇内非结...     

Replica-aided load balancing in overlay networks

In recent years, there have been rapid advances in network infrastructure and technologies for end-user communication. However, because of network dynamics and resource limitation, providing scalable end-user communication services is challenging when the applications are utilized on a large-scale. To address this challenge, a replica-aided load balancing scheme (RALB) is proposed for enabling the nodes in an overlay networks to support the communication applications for a large number of users. This paper makes three unique contributions. First, we study the existing load balancing schemes and identify their weakness in handling time-varying workloads with frequent load fluctuations. Second, we introduce a sophisticated cost model for load balancing cost estimation, which captures the dependencies between the factors (e.g., the load, message number, and link latency). Third, we propose a performance tuning technique to minimize the load balancing cost. The extensive experiments show that RALB effectively reduces the load imbalance and eliminates the load balancing cost when compared to the existing load balancing schemes.     

Locality-Aware and Churn-Resilient Load-Balancing Algorithms in Structured Peer-to-Peer Networks

Structured peer-to-peer overlay networks, like distributed hash tables (DHTs), map data items to the network based on a consistent hashing function. Such mapping for data distribution has an inherent load balance problem. Data redistribution algorithms based on randomized matching of heavily loaded nodes with light ones can deal with the dynamics of DHTs. However, they are unable to consider the proximity of the nodes simultaneously. There are other methods that rely on auxiliary networks to facilitate locality-aware load redistribution. Due to the cost of network construction and maintenance, the locality-aware algorithms can hardly work for DHTs with churn. This paper presents a locality-aware randomized load-balancing algorithm to deal with both the proximity and network churn at the same time. We introduce a factor of randomness in the probing of lightly loaded nodes in a range of proximity. We further improve the efficiency by allowing the probing of multiple candidates (d-way) at a time. Simulation results show the superiority of the locality-aware two-way randomized algorithm in comparison with other random or locality-aware algorithms. In DHTs with churn, it performs no worse than the best chum-resilient algorithm. It takes advantage of node capacity heterogeneity and achieves good load balance effectively even in a skewed distribution of items     

Large Scaling Unstructured Peer-to-Peer Networks with Heterogeneity-Aware Topology and Routing

Peer-to-peer (P2P) file sharing systems such as Gnutella have been widely acknowledged as the fastest-growing Internet applications ever. The P2P model has many potential advantages, including high flexibility and serverless management. However, these systems suffer from the well-known performance mismatch between the randomly constructed overlay network topology and the underlying IP-layer topology. This paper proposes to structure the P2P overlay topology using a heterogeneity-aware multitier topology to better balance the load at peers with heterogeneous capacities and to prevent low-capability nodes from throttling the performance of the system. An analytical model is developed to enable the construction and maintenance of heterogeneity-aware overlay topologies with good node connectivity and better load balance. We also develop an efficient routing scheme, called probabilistic selective routing, that further utilizes heterogeneity-awareness to enhance the routing performance. We evaluate our design through simulations. The results show that our multitier topologies alone can provide eight to 10 times improvement in the messaging cost, two to three orders of magnitude improvement in terms of load balancing, and seven to eight times lower topology construction and maintenance costs when compared to Gnutella's random power-law topology. Moreover, our heterogeneity-aware routing scheme provides further improvements on all evaluation metrics, when used with our heterogeneity-aware overlay topologies     

A dynamic load balancing scheme with incentive mechanism in heterogeneous structured P2P networks

Dynamic load imbalance is a basic and inherent problem in structured P2P networks. Most existing research suffers from the problems of inefficiency in globally managing the nodes’ load information and consumption of network bandwidth. This paper describes the mechanisms for collecting and globally managing the dynamic load of each node, and based on which to present a load balancing strategy which transfers the load from overloaded to under loaded nodes so as to improve load balancing efficiency. In order to encourage the rational and selfish nodes to actively participate in the load balancing process, we also propose an incentive mechanism in dynamic load balancing, by which the differentiated services could be provided for the nodes according to their load balancing abilities. The simulation results indicate that our approach could tackle the load imbalance problem in structured P2P networks effectively and efficiently in terms of the load distribution and the transferred load volume.     

Avoidance of churn rate through temporal centralization in Chord

In this paper we have introduced a concept called temporal centralization to handle high churn rate without disturbing the original decentralized architecture of Peer-to-Peer overlay system. The frequent joining/leaving of nodes in a P2P system costs high. We know that the well-known structured system like Chord handles query in O(logn), but node joining/leaving is O(logn)², where n is the number of nodes available in the system. Therefore, with high churn rate, it is hard to maintain the cost of routing table. In our approach, updation of all routing tables are not done immediately after a node joining the system. We introduce the concept of temporal centralization to Chord protocol that reduces the churn rate retaining the same number of steps for query processing. The simulation results show the improvement of performance of P2P network reducing transient node population.     

基于HeteroPastry的P2P网络代理系统

针对集中式网络代理服务器经常产生瓶颈的问题,提出以P2P方式提供网络代理服务的解决方案。参与系统的节点组成一个基于HeteroPastry协议的P2P覆盖网,系统在覆盖网中挑选一些合适的节点组成代理链,以响应用户的不同请求。实验结果证明,该系统可以在提供高质量服务的同时有效地平衡网络负载。     

Shortest-path routing in randomized DHT-based Peer-to-Peer systems

Randomized DHT-based Peer-to-Peer (P2P) systems grant nodes certain flexibility in selecting their overlay neighbors, leading to irregular overlay structures but to better overall performance in terms of path latency, static resilience and local convergence. However, routing in the presence of overlay irregularity is challenging. In this paper, we propose a novel routing protocol, RASTER, that approximates shortest overlay routes between nodes in randomized DHTs. Unlike previously proposed routing protocols, RASTER encodes and aggregates routing information. Its simple bitmap-encoding scheme together with the proposed RASTER routing algorithm enable a performance edge over current overlay routing protocols. RASTER provides a forwarding overhead of merely a small constant number of bitwise operations, a routing performance close to optimal, and a better resilience to churn. RASTER also provides nodes with the flexibility to adjust the size of the maintained routing information based on their storage/processing capabilities. The cost of storing and exchanging encoded routing information is manageable and grows logarithmically with the number of nodes in the system.     

基于覆盖网络的海上信息栅格无线传输系统研究

文章在通信资源的形式化描述基础上,设计了一种信息栅格的传输体系,为一体化信息系统提供透明的传输服务。针对多重网络传输服务与网络性能的多样性,提出了一种基于作战任务的层次覆盖网络模型,以及实现网络负载均衡的QoS路由算法。     

基于P2P的应用层组播结构研究

按照单播树、多播树和网状3种重叠网络构建方式分析现有的P2P应用层组播结构，提出构建P2P应用层组播时深度与宽度的平衡、如何利用叶子结点、网络波动的影响等几个普遍问题。给出了一种新的自适应出度的P2P应用层组播算法，试验证明利用结点带宽的异质性可以有效平衡深度、结点度，提高叶子结点的利用率。     

A scalable multi-attribute hybrid overlay for range queries on the cloud

Cloud computing has become a promising paradigm as next generation computing model, by providing computation, software, data access, and storage services that do not need to know the location of physical resources interconnected across the globe providing such services. In such an environment, important issues as information sharing and resource/service discovery arise. In order to overcome critical limitations in centralized approaches for information sharing and resource/service discovery, this paper proposes a framework of a scalable multi-attribute hybrid overlay featured with decentralized information sharing, flexible resource/service discovery, fault tolerance and load balancing. Additionally, the proposed hybrid overlay integrates a structured P2P system with an unstructured one to support complex queries. Mechanisms such as load balancing and fault tolerance implemented in our proposed system to improve the overall system performance are also discussed. Experimental results show that the performance of the proposed approach is feasible and stable, as the proposed hybrid overlay improves system performance by reducing the number of routing hops and balancing the load by migrating requests.     

基于DHT的P2P系统的负载均衡算法

在基于DHT的结构化P2P系统中，DHT的使用以及节点处理能力的不同导致系统中节点的负载不均衡．现有的负载均衡算法存在两个不足：①负载的转移没有考虑节点之间的链路延迟；②算法依赖于系统中固定位置的某些节点．提出了分布式负载均衡算法：每个节点周期性的收集系统局部负载信息，然后选择链路延迟较小的节点进行负载转移．算法依赖于系统中的所有节点，解决了单点失败问题．同时，负载的转移是在链路延迟较小的节点之间进行的．仿真实验表明，①对于各种系统利用率，该算法都可以获得理想的负载均衡效果；②算法可以使负载转移开销减少45％以上．     

Elastic Routing Table with Provable Performance for Congestion Control in DHT Networks

Consistent hashing-based DHT networks have an inherent load balancing problem. The problem becomes more severe in heterogeneous networks with nonuniform and time-varying popular files. Existing DHT load balancing algorithms are mainly focused on the issues caused by node heterogeneity. To deal with skewed lookups, this paper presents an elastic routing table (ERT) mechanism for query load balancing, based on the observation that high-degree nodes tend to receive more traffic load. The mechanism allows each node to have a routing table of variable size corresponding to node capacities. The indegree and outdegree of the routing table can also be adjusted dynamically in response to the change of file popularity and network churn. Theoretical analysis proves that the routing table degree is bounded. The ERT mechanism facilitates locality-aware randomized query forwarding to further improve lookup efficiency. By relating query forwarding to a supermarket customer service model, we prove that a two-way randomized query forwarding policy should lead to an exponential improvement in query processing time over random walking. Simulation results demonstrate the effectiveness of the ERT mechanism and its related query forwarding policy for congestion and query load balancing. In comparison with existing "virtual-server”-based load balancing algorithms and other routing table control approaches, the ERT-based congestion control protocol yields significant improvement in query lookup efficiency.     

负载敏感的P2P覆盖网

P2P网络较好地实现了大范围分布式环境下的节点自组织,但面向实际应用时,由于节点能力的差异带来了负载均衡问题.按照混合层次网络架构,基于Treap树设计了一种P2P覆盖网,根据负载率的优先级构造最小堆,并动态维护,实现稳定化操作.节点通过Treap树的信息汇聚机制获取后代节点的负载率,以此为基础实现负载均衡策略.仿真结...     

Cooperative Media Data Streaming with Scalable Video Coding

Live peer-to-peer (P2P) streaming has become a promising approach for broadcasting non-interactive media content from a server to a large number of interested clients. However, it still faces many challenges such as high churn rate of peer clients, uplink bandwidth constraints of participating peers, and heterogeneity of client throuput capacities. This paper presents a new P2P network called LSONet, a collaborative peer-to-peer streaming framework for scalable layer-encoded bit streams. The contributions are the combination of the advantages of both layered conding and mesh-based packet exchange. With layered coding, it overcomes overlay bandwidth limitatioins and heterogeneity of client capacities. With mesh based overlay streaming, it can better handle peer churns, as compared to tree-based solutions. For achieving these targets, this paper employs a gossip-based data-driven scheme for partnership formation, and proposes two algorithms, optimized transmission policy (OTP) and graceful degradation scheme (GDS), for multi-layers allocation. The proposed system is completely self-organizing, and in a fully distributed fashion. Extensive simulations show that LSONet achieves higher quality of service by peer-assisted streaming and layered video coding. Also, through comparison, results show that the system outperforms some previous schemes in resource utilization and is more robust and resilient for nodes departure, which demonstrate that it is well-suited for quality adaptive live streaming applications.     

一种新的结构化对等网负载均衡算法

研究结构化对等网(P2P)中的负载均衡问题,P2P网络的节点、延迟和处理能力差异性很大,当前负载均衡算法忽略节点差异性,造成网络负载极不平衡,容易出现"热点"问题,使负载变化大。为了更好的均衡P2P负载,提出一种新的网络负载均衡算法。算法充分考虑节点之间的差异性,对物理节点的地址空间进行动态分配,对于热点资源下载采用局部搜索算法找到邻居节点,并自动把负载转移到轻载节点上,保证节点间负载均衡。仿真结果表明,新网络负载均衡算法加快了负载均衡速度,使P2P网络负载均衡更加均衡,能够很好保持系统稳定性。     

基于SkipGraph的P2P覆盖网信任证链发现

提出一种基于skip图P2P覆盖网模型的分布式信任证存储和发现机制。为保证索引和发现效率,采用RT1^r语言作为信任证描述语言,其语义属性作为skip图中的索引关键字,构造了基于关键字前缀相似和支持范围查询的P2P覆盖网。通过试验评测和分析,该机制具有较高的查询效率和负载均衡机制。