P2P-based resource discovery in dynamic grids allowing multi-attribute and range queries

A key point for the efficient use of large grid systems is the discovery of resources, and this task becomes more complicated as the size of the system grows up. In this case, large amounts of information on the available resources must be stored and kept up-to-date along the system so that it can be queried by users to find resources meeting specific requirements (e.g. a given operating system or available memory). Thus, three tasks must be performed, (1) information on resources must be gathered and processed, (2) such processed information has to be disseminated over the system, and (3) upon users’ requests, the system must be able to discover resources meeting some requirements using the processed information. This paper presents a new technique for the discovery of resources in grids which can be used in the case of multi-attribute (e.g. {OS = Linux & memory = 4 GB}) and range queries (e.g. {50 GB < disk-space < 100 GB}). This technique relies on the use of content summarisation techniques to perform the first task mentioned before and strives at the main drawback found in proposals from literature using summarization. This drawback is related to scalability, and is tackled by means of using Peer-to-Peer (P2P) techniques, namely Routing Indices (RIs), to perform the second and third tasks.Another contribution of this work is a performance evaluation conducted by means of simulations of the EU DataGRID Testbed which shows the usefulness of this approach compared to other proposals from literature. More specifically, the technique presented in this paper improves on the scalability and produces good performance. Besides, the parameters involved in the summary creation have been tuned and the most suitable values for the presented test case have been found.     

Squid: Enabling search in DHT-based systems

A fundamental problem in large scale, decentralized distributed systems is the efficient discovery of information. This paper presents Squid, a peer-to-peer information discovery system that supports flexible searches and provides search guarantees. The fundamental concept underlying the approach is the definition of multi-dimensional information spaces and the maintenance of locality in these spaces. The key innovation is a dimensionality reducing indexing scheme that effectively maps the multi-dimensional information space to physical peers while preserving lexical locality. Squid supports complex queries containing partial keywords, wildcards and ranges. Analytical and simulation results show that Squid is scalable and efficient.     

一种非结构化P2P的随机有向图拓扑模型

在非结构化P2P系统中,P2P拓扑直接影响了网络的性能,如何构建P2P拓扑是一件重要的问题.采用一个简单的映射将非结构化P2P抽象成一个G(V,E)图,在G(V,E)及其非结构化P2P拓扑的特点基础上,提出了一种非结构化P2P的随机有向图拓扑模型,并讨论了该拓扑模型的度分布、连通性和直径,为研究非结构化P2P系统的性能和算法协议提供了仿真的理论基础.     

Winnowing: Protecting P2P systems against pollution through cooperative index filtering

Pollution (i.e., sharing of corrupted files, or contaminating index information with bogus index records) is a de facto problem in many file sharing peer-to-peer (P2P) systems in use today. Pollution squanders network resources and frustrates users with unprofitable downloads (due to corrupted files) and unproductive download trials (due to bogus index records). In this paper, we propose a novel distributed hash table (DHT)-based anti-pollution scheme called winnowing. Winnowing aims to reduce or eliminate decoy index records (pointing to nonexisting or corrupted files) held by DHT (i.e., index) nodes in the system, so that download attempts based on the remaining (clean) index records are more likely to yield satisfactory results. To achieve this goal, two techniques are used: (1) publish verification is performed by index nodes to counteract index pollution and (2) privacy-preserving object reputation is integrated into the DHT to reduce the impact of content and metadata pollution. By integrating these techniques, winnowing converges quickly to a near-optimal solution. Winnowing has the added benefit that it does not reveal a peer's download history to other downloading peers.The publish verification of winnowing has been implemented on top of the latest eMule client, and extensive data has been collected from the Kad network using this modified client. The measurement results are summarized, and the findings from the measurement study are incorporated into an analytical model. The model demonstrates the robustness of the privacy-preserving object reputation of winnowing to a variety of pollution attacks, and to attacks on winnowing itself. The results of analysis are confirmed by means of event-driven simulations.     

A proactive low-overhead file replication scheme for structured P2P content delivery networks

File replication is a widely used technique for high performance in peer-to-peer content delivery networks. A file replication technique should be efficient and at the same time facilitates efficient file consistency maintenance. However, most traditional methods do not consider nodes’ available capacity and physical location in file replication, leading to high overhead for both file replication and consistency maintenance. This paper presents a proactive low-overhead file replication scheme, namely Plover. By making file replicas among physically close nodes based on nodes’ available capacities, Plover not only achieves high efficiency in file replication but also supports low-cost and timely consistency maintenance. It also includes an efficient file query redirection algorithm for load balancing between replica nodes. Theoretical analysis and simulation results demonstrate the effectiveness of Plover in comparison with other file replication schemes. It dramatically reduces the overhead of both file replication and consistency maintenance compared to other schemes. In addition, it yields significant improvements in reduction of overloaded nodes.     

Probabilistic resource allocation in heterogeneous distributed systems with random failures

The problem of finding efficient workload distribution techniques is becoming increasingly important today for heterogeneous distributed systems where the availability of compute nodes may change spontaneously over time. Resource-allocation policies designed for such systems should maximize the performance and, at the same time, be robust against failure and recovery of compute nodes. Such a policy, based on the concepts of the Derman–Lieberman–Ross theorem, is proposed in this work, and is applied to a simulated model of a dedicated system composed of a set of heterogeneous image processing servers. Assuming that each image results in a “reward” if its processing is completed before a certain deadline, the goal for the resource allocation policy is to maximize the expected cumulative reward. An extensive analysis was done to study the performance of the proposed policy and compare it with the performance of some existing policies adapted to this environment. Our experiments conducted for various types of task-machine heterogeneity illustrate the potential of our method for solving resource allocation problems in a broad spectrum of distributed systems that experience high failure rates.     

Virtual Tree: A robust architecture for interval valid queries in dynamic distributed systems

This paper studies the problem of answering aggregation queries, satisfying the interval validity semantics, in a distributed system prone to continuous arrival and departure of participants. The interval validity semantics states that the query answer must be calculated considering contributions of at least all processes that remained in the distributed system for the whole query duration. Satisfying this semantics in systems experiencing unbounded churn is impossible due to the lack of connectivity and path stability between processes. This paper presents a novel architecture, namely Virtual Tree, for building and maintaining a structured overlay network with guaranteed connectivity and path stability in settings characterized by bounded churn rate. The architecture includes a simple query answering algorithm that provides interval valid answers. The overlay network generated by the Virtual Tree architecture is a tree-shaped topology with virtual nodes constituted by clusters of processes and virtual links constituted by multiple communication links connecting processes located in adjacent virtual nodes. We formally prove a bound on the churn rate for interval valid queries in a distributed system where communication latencies are bounded by a constant unknown by processes. Finally, we carry out an extensive experimental evaluation that shows the degree of robustness of the overlay network generated by the virtual tree architecture under different churn rates.     

Internet-based manufacturing: A review and a new infrastructure for distributed intelligent manufacturing

Global competition and rapidly changing customer requirements are forcing major changes in the production styles and configuration of manufacturing enterprises. Traditional centralized manufacturing systems are not able to meet these requirements. In recent years, the internet has become the worldwide information platform for the sharing of information and data. Information processing is an important challenge in an internet-based manufacturing environment, and must facilitate distribution, heterogeneity, autonomy and cooperation. This paper reviews recent trends and major developments in internet-based manufacturing. Broad categories of distributed information management are identified, based upon the development of computer integrated manufacturing, and the possible direction of future research is indicated.     

基于历史信息的对等网络负载均衡算法

针对当前对等网络中存在的负载不均衡问题,提出了基于历史信息查询的负载均衡算法(HILBA),以提高系统的整体心?实验结果表明,此算法在节点同构和异构环境下都提高了系统的负载均衡能力.     

Enhanced DHT-based P2P Architecture for Effective Resource Discovery and Management

Recently, Peer-to-Peer (P2P) has become a popular paradigm for building distributed systems, aiming to provide resource localization and sharing in large-scale networks. However, advanced searching for resources remains an open issue. The flooding technique used by some P2P systems is expensive in bandwidth usage, and shows a serious lack in scalability. Also, more efficient systems based on distributed hash tables (DHT) lack in query expressiveness and flexibility. This paper addresses this issue by discussing existing solutions, and proposing a novel approach to support advanced multi-keyword queries in the context of P2P systems. It extends the existing, and widely established DHT-based localization frameworks. This new approach provides an effective resource localization framework; it can substantially reduce bandwidth consumption and improve load balancing over the network. Moreover, various kinds of applications can be deployed on top of this generic framework. As a relevant use case, this paper describes a novel service discovery and management application.

Sporadic decentralized resource maintenance for P2P distributed storage networks

In this paper, we propose a novel decentralized resource maintenance strategy for peer-to-peer (P2P) distributed storage networks. Our strategy relies on the Wuala overlay network architecture, (The WUALA Project). While the latter is based, for the resource distribution among peers, on the use of erasure codes, e.g., Reed–Solomon codes, here we investigate the system behavior when a simple randomized network coding strategy is applied. We propose to replace the Wuala regular and centralized strategy for resource maintenance with a decentralized strategy, where users regenerate new fragments sporadically, namely every time a resource is retrieved. Both strategies are analyzed, analytically and through simulations, in the presence of either erasure and network coding. It will be shown that the novel sporadic maintenance strategy, when used with randomized network coding, leads to a fully decentralized solution with management complexity much lower than common centralized solutions.     

CycloidGrid: A proximity-aware P2P-based resource discovery architecture in volunteer computing systems

Volunteer computing which benefits from idle cycles of volunteer resources over the Internet can integrate the power of hundreds to thousands of resources to achieve high computing power. In such an environment the resources are heterogeneous in terms of CPU speed, RAM, disk capacity, and network bandwidth. So finding a suitable resource to run a particular job becomes difficult. Resource discovery architecture is a key factor for overall performance of peer-to-peer based volunteer computing systems. The main contribution of this paper is to develop a proximity-aware resource discovery architecture for peer-to-peer based volunteer computing systems. The proposed resource discovery algorithm consists of two stages. In the first stage, it selects resources based on the requested quality of service and current load of peers. In the second stage, a resource with higher priority to communication delay is selected among the discovered resources. Communication delay between two peers is computed by a network model based on queuing theory, taking into account the background traffic of the Internet. Simulation results show that the proposed resource discovery algorithm improves the response time of user’s requests by a factor of 4.04 under a moderate load.     

基于P2P的工作流系统的实现

介绍了基于中心索引目录拓扑和分布式非结构化拓扑的P2P的工作流系统的实现方法,并力图保持和当今各种流行工作流系统接口的最大兼容。从中可以看出两种P2P工作流系统的优缺点,为工作流系统的P2P化提供另一种可行的方案。     

A UML-based quantitative framework for early prediction of resource usage and load in distributed real-time systems

This paper presents a quantitative framework for early prediction of resource usage and load in distributed real-time systems (DRTS). The prediction is based on an analysis of UML 2.0 sequence diagrams, augmented with timing information, to extract timed-control flow information. It is aimed at improving the early predictability of a DRTS by offering a systematic approach to predict, at the design phase, system behavior in each time instant during its execution. Since behavioral models such as sequence diagrams are available in early design phases of the software life cycle, the framework enables resource analysis at a stage when design decisions are still easy to change. Though we provide a general framework, we use network traffic as an example resource type to illustrate how the approach is applied. We also indicate how usage and load analysis of other types of resources (e.g., CPU and memory) can be performed in a similar fashion. A case study illustrates the feasibility of the approach.

Solution for the broadcasting in the Kademlia peer-to-peer overlay

Structured peer-to-peer networks are capable of fast and efficient lookup operations as a distributed hash table. The topology of these networks makes it possible to send broadcast messages among nodes, either for the purpose of providing a complex query service for participants, or to disseminate information valuable for all nodes. In this article a broadcast algorithm for the Kademlia XOR topology is presented. The algorithm, which was developed specifically for Kademlia, uses replication mechanisms similar to that of the storage and retrieval service of this overlay topology. This allows for increased reliability and speed of the broadcast, and also efficient operation, as the routing table for lookups are already available and can be used. An analytical model is presented, which can be used to calculate the required level of replication for any desired reliability at runtime, and is validated with simulation as well.     

P2P-based multidimensional indexing methods: A survey

P2P-based multidimensional index (MI) is a hotspot which absorbs many researchers to dedicate them into. However, no summarization or review on this technology has been made at present. To the best of our knowledge, this is the first work on reviewing P2P-based MI. This paper innovatively adopts visualization technique to show the research groups and then analyzes investigating style of research groups. Based on evolution of P2P-based MI inheriting from centralized MI and P2P, we divide P2P-based MI methods into 4 categories: extending centralized MI, extending P2P, combining centralized MI and P2P, and miscellaneous. For each category, the paper selects classical techniques and describes them in detail. This is the first time of doing the classification job over massive related works. Finally, load balancing and update strategies are described and discussed for they are important factors related to performance. We believe many researchers will get benefits from our work for further studies.     

Control of complex distributed systems with distributed intelligent agents

Control of spatially distributed systems is a challenging problem because of their complex nature, nonlinearity, and generally high order. The lack of accurate and computationally efficient model-based techniques for large, spatially distributed systems leads to challenges in controlling the system. Agent-based control structures provide a powerful tool to manage distributed systems by utilizing (organizing) local and global information obtained from the system. A hierarchical, agent-based system with local and global controller agents is developed to control networks of interconnected chemical reactors (CSTRs). The global controller agent dynamically updates local controller agent’s objectives as the reactor network conditions change. One challenge posed is control of the spatial distribution of autocatalytic species in a network of reactors hosting multiple species. The multi-agent control system is able to intelligently manipulate the network flow rates such that the desired spatial distribution of species is achieved. Furthermore, the robustness and flexibility of the agent-based control system is illustrated through examples of disturbance rejection and scalability with respect to the size of the network.     

A hybrid approach for estimating document frequencies in unstructured P2P networks

Scalable search and retrieval over numerous web document collections distributed across different sites can be achieved by adopting a peer-to-peer (P2P) communication model. Terms and their document frequencies are the main components of text information retrieval and as such need to be computed, aggregated, and distributed throughout the system. This is a challenging problem in the context of unstructured P2P networks, since the local document collections may not reflect the global collection in an accurate way. This might happen due to skews in the distribution of documents to peers. Moreover, central assembly of the total information is not a scalable solution due to the excessive cost of storage and maintenance, and because of issues related to digital rights management. In this paper, we present an efficient hybrid approach for aggregation of document frequencies using a hierarchical overlay network for a carefully selected set of the most important terms, together with gossip-based aggregation for the remaining terms in the collections. Furthermore, we present a cost analysis to compute the communication cost of hybrid aggregation. We conduct experiments on three document collections, in order to evaluate the quality of the proposed hybrid aggregation.     

Context-based caching and routing for P2P web service discovery

In modern heterogeneous environments, such as mobile, pervasive and ad-hoc networks, architectures based on web services offer an attractive solution for effective communication and inter-operation. In such dynamic and rapidly evolving environments, efficient web service discovery is an important task. Usually this task is based on the input/output parameters or other functional attributes, however this does not guarantee the validity or successful utilization of retrieved web services. Instead, non-functional attributes, such as device power features, computational resources and connectivity status, that characterize the context of both service providers and consumers play an important role to the quality and usability of discovery results. In this paper we introduce context-awareness in web service discovery, enabling the provision of the most appropriate services at the right location and time. We focus on context-based caching and routing for improving web service discovery in a mobile peer-to-peer environment. We conducted a thorough experimental study, using our prototype implementation based on the JXTA framework, while simulations are employed for testing the scalability of the approach. We illustrate the advantages that this approach offers, both by evaluating the context-based cache performance and by comparing the efficiency of location-based routing to broadcast-based approaches. Recommended by: Zakaria Maamar     

Verification of distributed control systems in intelligent manufacturing

This paper presents an application of formal methods for validation of flexible manufacturing systems controlled by distributed controllers. A software tool verification environment for distributed applications (VEDA) is developed for modeling and verification of distributed control systems. The tool provides an integrated environment for formal, model-based verification of the execution control of function blocks following the new international standard IEC61499. The modeling is performed in a closed-loop way using manually developed models of plants and automatically generated models of controllers.