语义对等覆盖网中社区结构的发现和评价

最近,通过建立语义覆盖网络来提高大规模分布式网络环境中信息检索服务的性能成为对等计算领域的研究热点.目前,研究者们在语义覆盖协议和搜索算法方面已经做了大量研究,证明了语义覆盖在基于对等网络模型的内容定位应用方面极为有效.然而,分析和评价语义覆盖网络特征的研究工作确非常有限.文中通过建立数学模型和设计启发式回溯-贪婪混合算法、确认了语义覆盖网络的一种主要内在特性——社区结构特性.利用评价模型比较了SemreX语义覆盖网络和Gnutella网络的性能,实验结果显示SemreX覆盖网具有显著的社区结构特征,而Gnutella网络却没有这样的特征.另外,通过分别在两种覆盖网中仿真洪泛协议发现具有显著社区结构特征的覆盖网在内容定位方面效率更高.     

SemreX:一种基于语义相似度的P2P覆盖网络

对等(peer-to-peer)网络的非集中结构、良好的自治性及容错性等特征,使其可能成为Internet上有效的信息共享模型.然而,内容定位问题仍然是大规模P2P网络中信息共享所面临的挑战.SemreX系统是一种P2P网络环境下的文献检索系统.针对SemreX系统,提出一种基于语义相似度的P2P拓扑管理和查询路由算法.仿真实验结果表明,语义拓扑能够有效地提高系统的搜索效率.     

Distributed Suffix Tree Overlay for Peer-to-Peer Search

Establishing an appropriate semantic overlay on peer-to-peer (P2P) networks to obtain both semantic ability and scalability is a challenge. Current DHT-based P2P networks are limited in their ability to support a semantic search. This paper proposes the distributed suffix tree (DST) overlay as the intermediate layer between the DHT overlay and the semantic overlay to support the search of a keyword sequence. Its time cost is sublinear with the length of the keyword sequence. Analysis and experiments show that the DST-based search is fast, load-balanced, and useful in realizing an accurate content search on P2P networks.     

A dynamic coalition formation game for search efficient adaptive overlay construction in unstructured peer-to-peer networks

A great number of recent works deal with improving search in peer-to-peer systems, specifically by clustering peers into semantic groups. When the process of clustering is predetermined and static, it suffers from lack of adaptation to highly dynamic peer-to-peer environments. We model the problem as a non-superadditive coalition game with non-transferable utility characteristic function, and propose a distributed dynamic coalition formation algorithm through myopic best-reply with experiment rule to solve the coalition formation problem. Coalitions are formed by peers with similar interests considering geographical proximity. The overlay network is dynamically reconfigured over time based on the changes in the interests or locations of the individual peers. The convergence of the proposed algorithm using “core solution” concept is studied. The simulation results show that the proposed algorithm can efficiently reduce the search time, although the overhead of the overlay adaptation is slightly higher.     

Enabling Dynamic Querying over Distributed Hash Tables

Dynamic querying (DQ) is a search technique used in unstructured peer-to-peer (P2P) networks to minimize the number of nodes that is necessary to visit to reach the desired number of results. In this paper, we introduce the use of the DQ technique in structured P2P networks. In particular, we present a P2P search algorithm, named DQ-DHT (Dynamic Querying over a Distributed Hash Table), to perform DQ-like searches over DHT-based overlays. The aim of DQ-DHT is twofold: allowing arbitrary queries to be performed in structured P2P networks and providing dynamic adaptation of the search according to the popularity of the resources to be located. DQ-DHT has been particularly designed for use in those distributed environments, like computational grids, where it is necessary to support arbitrary queries for searching resources on the basis of complex criteria or semantic features. This paper describes the DQ-DHT algorithm using Chord as basic overlay and analyzes its performance in comparison with DQ in unstructured networks.     

Exploiting semantic proximities for content search over p2p networks

In this paper, we address the issue of content search over peer-to peer networks. We use the concept of semantic proximity that exploits the commonalities of interests exhibited among peer users so as to decompose the network into semantic clusters. We initially define search entropy, as a metric indicating the average number of packets required to locate the requested content. Then, spectral clustering is used to organize the peer nodes into semantic clusters so that (a) the probability that a node locates content within its own cluster is maximized, while simultaneously; (b) the respective probability of finding this content outside this cluster is minimized. The proposed semantic partitioning algorithm is then extended into a hierarchical two-tier scheme, in which practical issues arising for the deployment of a peer-to-peer (p2p) application can be more easily addressed. After the system has been initialized, a dynamic algorithm places new users that join the p2p network into appropriately selected clusters and also handles peer departures without the need for matrix eigen decomposition process which is necessary for the assessment of the initial static partitioning. Our experimental results validate that (a) our static partitioning outperforms traditional and novel search techniques and (b) our dynamic algorithm is able to efficiently track the system’s progression maintaining the search entropy close to the initially assessed levels.     

Adding structure to unstructured peer-to-peer networks: the use of small-world graphs

The “small-world” graph structure is pervasive and is observed to arise “without-design” or “naturally” in many practical systems such as the World Wide Web. In contrast to natural systems, overlay networks provide an opportunity to design structure. We seek the advantages of designing overlay topologies with small-world properties to support file sharing in peer-to-peer networks. We focus on two metrics of performance: (a) search protocol performance, a local gain perceived directly by peer-to-peer network users and (b) network utilization, a global property that is of interest to network service providers. We propose a class of overlay topologies and show, by simulation, that a particular topology instance of this class where every node has many close neighbors and few random neighbors (i.e., a small-world graph) exhibits very good properties. In this overlay topology, the chances of locating files are high, and the nodes where these files are found are, on average, close to the query source. This improvement in search protocol performance is achieved while decreasing the traffic load on the links in the underlying network. We propose a simple greedy algorithm to construct such overlay topologies where each node operates independently and in a decentralized manner to select its neighbors.     

Semantic partitioning of peer-to-peer search space

Peer-to-peer (P2P) networks are beginning to form the infrastructure of future applications. Computers are organized in P2P overlay networks to facilitate search queries with reasonable cost. So, scalability is a major aim in design of P2P networks. In this paper, to obtain a high factor of scalability, we partition network search space using a consistent static shared upper ontology. We name our approach semantic partition tree (SPT). All resources and queries are annotated using the upper ontology and queries are semantically routed in the overlay network. Also, each node indexes addresses of other nodes that possess contents expressible by the concept it maintains. So, our approach can be conceived as an ontology-based distributed hash table (DHT). Also, we introduce a lookup service for the network which is very scalable and independent of the network size and just depends on depth of the ontology tree. Further, we introduce a broadcast algorithm on the network. We present worst case analysis of both lookup and broadcast algorithms and measure their performance using simulation. The results show that our scheme is highly scalable and can be used in real P2P applications.     

Robust random number generation for peer-to-peer systems

We consider the problem of designing an efficient and robust distributed random number generator for peer-to-peer systems that is easy to implement and works even if all communication channels are public. A robust random number generator is crucial for avoiding adversarial join–leave attacks on peer-to-peer overlay networks. We show that our new generator together with a light-weight rule recently proposed in [B. Awerbuch, C. Scheideler, Towards a scalable and robust DHT, in: Proc. of the 18th ACM Symp. on Parallel Algorithms and Architectures, SPAA, 2006. See also http://www14.in.tum.de/personen/scheideler] for keeping peers well distributed can keep various structured overlay networks in a robust state even under a constant fraction of adversarial peers.     

T-Man: Gossip-based fast overlay topology construction

Large-scale overlay networks have become crucial ingredients of fully-decentralized applications and peer-to-peer systems. Depending on the task at hand, overlay networks are organized into different topologies, such as rings, trees, semantic and geographic proximity networks. We argue that the central role overlay networks play in decentralized application development requires a more systematic study and effort towards understanding the possibilities and limits of overlay network construction in its generality. Our contribution in this paper is a gossip protocol called T-Man that can build a wide range of overlay networks from scratch, relying only on minimal assumptions. The protocol is fast, robust, and very simple. It is also highly configurable as the desired topology itself is a parameter in the form of a ranking method that orders nodes according to preference for a base node to select them as neighbors. The paper presents extensive empirical analysis of the protocol along with theoretical analysis of certain aspects of its behavior. We also describe a practical application of T-Man for building Chord distributed hash table overlays efficiently from scratch.     

Dealing with network partitions in structured overlay networks

Structured overlay networks form a major class of peer-to-peer systems, which are touted for their abilities to scale, tolerate failures, and self-manage. Any long-lived Internet-scale distributed system is destined to face network partitions. Although the problem of network partitions and mergers is highly related to fault-tolerance and self-management in large-scale systems, it has hardly been studied in the context of structured peer-to-peer systems. These systems have mainly been studied under churn (frequent joins/failures), which as a side effect solves the problem of network partitions, as it is similar to massive node failures. Yet, the crucial aspect of network mergers has been ignored. In fact, it has been claimed that ring-based structured overlay networks, which constitute the majority of the structured overlays, are intrinsically ill-suited for merging rings. In this paper, we present an algorithm for merging multiple similar ring-based overlays when the underlying network merges. We examine the solution in dynamic conditions, showing how our solution is resilient to churn during the merger, something widely believed to be difficult or impossible. We evaluate the algorithm for various scenarios and show that even when falsely detecting a merger, the algorithm quickly terminates and does not clutter the network with many messages. The algorithm is flexible as the tradeoff between message complexity and time complexity can be adjusted by a parameter.     

Exploiting Geographical and Temporal Locality to Boost Search Efficiency in Peer-to-Peer Systems

As a hot research topic, many search algorithms have been presented and studied for unstructured peer-to-peer (P2P) systems during the past few years. Unfortunately, current approaches either cannot yield good lookup performance, or incur high search cost and system maintenance overhead. The poor search efficiency of these approaches may seriously limit the scalability of current unstructured P2P systems. In this paper, we propose to exploit two-dimensional locality to improve P2P system search efficiency. We present a locality-aware P2P system architecture called Foreseer, which explicitly exploits geographical locality and temporal locality by constructing a neighbor overlay and a friend overlay, respectively. Each peer in Foreseer maintains a small number of neighbors and friends along with their content filters used as distributed indices. By combining the advantages of distributed indices and the utilization of two-dimensional locality, our scheme significantly boosts P2P search efficiency while introducing only modest overhead. In addition, several alternative forwarding policies of Foreseer search algorithm are studied in depth on how to fully exploit the two-dimensional locality.     

Enhancing Search Performance on Gnutella-Like P2P Systems

The big challenges facing the search techniques on Gnutella-like peer-to-peer networks are search efficiency and quality of search results. In this paper, leveraging information retrieval (IR) algorithms such as Vector Space Model (VSM) and relevance ranking algorithms, we present GES (Gnutella with Efficient Search) to improve search performance. The key idea is that GES uses a distributed topology adaptation algorithm to organize semantically relevant nodes into same semantic groups by using the notion of node vector. Given a query, GES employs an efficient search protocol to direct the query to the most relevant semantic groups for answers, thereby achieving high recall with probing only a small fraction of nodes. To the best of our knowledge, GES is the first to identify node vector size as an important role in impacting search performance and to show that the node vector size offers a good trade-off between search performance and bandwidth cost. Moreover, GES adopts automatic query expansion and local data clustering to improve search performance. We show that GES is efficient and even outperforms the centralized node clustering system SETS. For example, in the scenario where node capacity is heterogeneous, GES can achieve 73 percent recall when probing only 20 percent nodes, outperforming SETS by about 18 percent.     

PGSW-OS: a novel approach for resource management in a semantic web operating system based on a P2P grid architecture

A web operating system is an operating system that users can access from any hardware at any location. A peer-to-peer (P2P) grid uses P2P communication for resource management and communication between nodes in a grid and manages resources locally in each cluster, and this provides a proper architecture for a web operating system. Use of semantic technology in web operating systems is an emerging field that improves the management and discovery of resources and services. In this paper, we propose PGSW-OS (P2P grid semantic Web OS), a model based on a P2P grid architecture and semantic technology to improve resource management in a web operating system through resource discovery with the aid of semantic features. Our approach integrates distributed hash tables (DHTs) and semantic overlay networks to enable semantic-based resource management by advertising resources in the DHT based upon their annotations to enable semantic-based resource matchmaking. Our model includes ontologies and virtual organizations. Our technique decreases the computational complexity of searching in a web operating system environment. We perform a simulation study using the Gridsim simulator, and our experiments show that our model provides enhanced utilization of resources, better search expressiveness, scalability, and precision.     

一种基于分组的语义对等网络

以语义网络理论为基础,结合GCNET拓扑结构,提出一种基于分组的语义对等网络——Semantic GCNET,充分利用GCNET网络具有小世界特征的优点,确保其搜索限制在与查询主题相关的局部节点子集中,解决其他一些语义对等网络对主题群搜索低效的问题,克服一些语义对等网络仅支持精确匹配查找的缺点。实验结果表明,Semantic GCNET具有高效的语义查询性能和查全率。     

A hierarchical overlay with cluster-based reputation tree for dynamic peer-to-peer systems

Traditional peer-to-peer technologies and systems assume that people operate with desktop computers in fixed broadband networks. When people with modern mobile devices now access Internet and Web services much in the manner they used to on desktop computers, the classical peer-to-peer overlay models can be vulnerable in wireless and mobile networks. This paper proposes a hierarchical overlay architecture based on partially central and semi-structured overlay models for the deployment of peer-to-peer systems in dynamic network environments. To keep up system scalability and efficacy, this architecture design exploits peer locality and network proximity, and contends with several problems of peer churn, peer mobility, search redundancy and traffic overhead that become much stickier in dynamic network environments. This design also integrates the reputation notion to mitigate the free-riding problem in peer-to-peer systems. According to a special cluster-based reputation tree, the hierarchical overlay is adjustable to moderate unfair or imbalanced resource utilization over the system. Furthermore, the cluster hierarchy is resilient to any points of failure at peer clusters in the overlay topology. Therefore, the effort of this study achieves an efficient and robust overlay architecture in dynamic network environments. Simulation results show that the proposed architecture is not only scalable to peer population, but also sustainable to peer- and network-initiated dynamics and influences in peer-to-peer systems.     

A novel robust on-line protocol for load-balancing in structured peer-to-peer systems

In this paper, we revisit the problem of load-balancing structured peer-to-peer systems with on-line protocols. Load-balancing is of major significance for large-scale decentralized networks in terms of enhanced scalability and performance. The main incentives behind balancing schemes are under-utilization of bandwidth and computer resources. Therefore, our methods focus mainly on task-skew. Specifically, we address the problem with on-line protocols on the basis of migration and enhanced availability. In particular, the cornerstones of our methods are the notions of virtual nodes, replication and Multiple realities, combined altogether with allocation techniques based on balls-in-bins games. The rationale of our dynamic protocol to depend exclusively on peer load distribution preserves intact the structural properties and search efficiency of the overlay used as an indexing infrastructure, while preserving the semantic information of the data (e.g., range partitioned network). We also propose an effective load-aware mechanism to facilitate robust operations that counteract against contingent churn failures. Finally, our work is complemented with extensive experiments using both real and synthetic data sets.     

A Semantic Searching Scheme in Heterogeneous Unstructured P2P Networks

Semantic-based searching in peer-to-peer (P2P) networks has drawn significant attention recently. A number of semantic searching schemes, such as GES proposed by Zhu Y et al., employ search models in Information Retrieval (IR). All these IR-based schemes use one vector to summarize semantic contents of all documents on a single node. For example, GES derives a node vector based on the IR model: VSM (Vector Space Model). A topology adaptation algorithm and a search protocol are then designed according to the similarity between node vectors of different nodes. Although the single semantic vector is suitable when the distribution of documents in each node is uniform, it may not be efficient when the distribution is diverse. When there are many categories of documents at each node, the node vector representation may be inaccurate. We extend the idea of GES and present a new class-based semantic searching scheme (CSS) specifically designed for unstructured P2P networks with heterogeneous single-node document collection. It makes use of a state-of-the-art data clustering algorithm, online spherical k-means clustering (OSKM), to cluster all documents on a node into several classes. Each class can be viewed as a virtual node. Virtual nodes are connected through virtual links. As a result, the class vector replaces the node vector and plays an important role in the class-based topology adaptation and search process. This makes CSS very efficient. Our simulation using the IR benchmark TREC collection demonstrates that CSS outperforms GES in terms of higher recall, higher precision, and lower search cost.     

Dynamic Multicast in Overlay Networks with Linear Capacity Constraints

In a peer-to-peer overlay network, the phenomenon of multiple overlay links sharing bottleneck physical links leads to correlation of overlay link capacities. We are able to more accurately model the overlay by incorporating these linear capacity constraints (LCCs). We formulate the problem of maximizing bandwidth in overlay multicast using our LCC model. We show that finding a maximum bandwidth multicast tree in an overlay network with LCC is NP-complete. Therefore, an efficient heuristics algorithm is designed to solve the problem. Extensive simulations show that our algorithm is able to construct multicast trees that are optimal or extremely close to optimal, with significantly higher bandwidth than trees formed in overlays with no LCC. Furthermore, we develop a fully distributed algorithm for obtaining near-optimal multicast trees, by means of gossip-based algorithms and a restricted but inherently distributed class of LCC (node-based LCC). We demonstrate that the distributed algorithm converges quickly to the centralized optimal and is highly scalable.