EMMP :a highly efficient membership management protocol

Gossip (or epidemic) algorithms have recently become popular solutions to multicast message dissemination in peer-to-peer systems. Nevertheless, it is not straightforward to apply gossip to on-demand streaming because it often fails to achieve a timely delivery. To solve this problem and taking into account the characteristic of peers randomly joining and leaving in peer-to-peer systems, an Efficient Membership Management Protocol (EMMP) has been presented. Every node only needs to keep contact with O (log(N)) nodes, and EMMP can support the reliable dissemination of messages. Considering the “distance” between peers, it causes the major data to be transmitted in a local area and reduces the backbone’s traffic, and speeds up the dissemination of messages between peers. This paper has adopted the “goodfriend” mechanism to reduce the influence on the system when a peer fails or leaves. Simulation results show that EMMP is highly efficient, and both the redundancy and the delay of the system are well solved.     

Practical load balancing for content requests in peer-to-peer networks

This paper studies the problem of balancing the demand for content in a peer-to-peer network across heterogeneous peer nodes that hold replicas of the content. Previous decentralized load balancing techniques in distributed systems base their decisions on periodic updates containing information about load or available capacity observed at the serving entities. We show that these techniques do not work well in the peer-to-peer context; either they do not address peer node heterogeneity, or they suffer from significant load oscillations which result in unutilized capacity. We propose a new decentralized algorithm, Max-Cap, based on the maximum inherent capacities of the replica nodes. We show that unlike previous algorithms, it is not tied to the timeliness or frequency of updates, and consequently requires significantly less update overhead. Yet, Max-Cap can handle the heterogeneity of a peer-to-peer environment without suffering from load oscillations. Mema Roussopoulos is an Assistant Professor of Computer Science on the Gordon McKay Endowment at Harvard University. Before joining Harvard, she was a Postdoctoral Fellow in the Computer Science Department at Stanford University. She received her PhD and Master’s degrees in Computer Science from Stanford, and her Bachelor’s degree in Computer Science from the University of Maryland at College Park. Her interests are in the areas of distributed systems, networking, and mobile and wireless computing. Mary Baker is a Senior Research Scientist at HP Labs. Her research interests include distributed systems, networks, mobile systems, security, and digital preservation. Before joining HP Labs she was on the faculty of the computer science department at Stanford University where she ran the MosquitoNet project. She received her PhD from the University of California at Berkeley.     

Routing Queries through a Peer-to-Peer InfoBeacons Network Using Information Retrieval Techniques

In the InfoBeacons system, a peer-to-peer network of beacons cooperates to route queries to the best information sources. Many internet sources are unwilling to provide more cooperation than simple searching to aid in the query routing.We adapt techniques from information retrieval to deal with this lack of cooperation. In particular, beacons determine how to route queries based on information cached from sources’ responses to queries. In this paper, we examine alternative architectures for routing queries between beacons and to data sources. We also examine how to improve the routing by probing sources in an informed way to learn about their content. Results of experiments using a beacon network to search 2,500 information sources demonstrates the effectiveness of our system; for example, our techniques require contacting up to 71 percent fewer sources than existing peer-to-peer random walk techniques.     

Applying Peer-to-Peer Techniques to Grid Replica Location Services

Peer-to-peer systems offer attractive system management properties, including the ability of components that join the network to self-organize; scalability up to tens of thousands of members; the ability of the network to automatically repair its topology after node failures; and techniques for maintaining redundant information to improve reliability and load balancing. We investigate applying peer-to-peer techniques to Grid services that are oriented toward resource discovery. In particular, we apply the Chord structured peer-to-peer overlay network to the Globus Replica Location Service, which allows registration and discovery of data replicas. We describe the design and performance of a Peer-to-Peer Replica Location Service (P-RLS) that uses the Chord algorithm to self-organize P-RLS servers and exploits the Chord overlay network to replicate P-RLS mappings adaptively. We present performance measurements and simulation results for the P-RLS system. We also discuss outstanding issues for applying peer-to-peer techniques to Grid resource discovery services.     

Japanese activities of micromachine and its future prospects

 Every corner of advanced R&D seems to head toward micronization. Micronization research in mechanics started recently but an intensive and comprehensive approach has been taken since 1992 when both the government R&D project and Micromachine Center were initiated. Micromachine is a new technological paradigm and has the same potential benefit as today’s macromachine. Micromachine will take considerable time span to be completed for full-fledge application. Until then, a wide range of approaches, i.e. R&D, dissemination, international exchange are indispensable. Received: 22 October 1996 / Accepted: 28 October 1996     

Stepwise fair-share buffering for gossip-based peer-to-peer data dissemination

We consider buffer management in support of large-scale gossip-based peer-to-peer data dissemination protocols. Coupled with an efficient buffering mechanism, system-wide buffer usage can be optimized while providing reliability and scalability in such protocols. We propose a novel approach, stepwise fair-share buffering, that provides uniform load distribution and reduces the overall buffer usage where every peer has a partial view of the system. We report and discuss the comparative performance results with existing buffering approaches as well as random buffering which serves as a benchmark. We present separate evaluations of bufferer selection and gossip-based data dissemination. Reliability, content dissemination time, message delay, buffering delay, and minimum buffer requirements are considered as the key metrics investigated through simulations. The performance of our approach in the case of multiple senders, link failures with multiple bufferers, and scalability to larger networks are investigated. Several power-law and hierarchical overlay topologies are considered. Analytical bounds for reliability of dissemination are also provided.     

Resilient and coherence preserving dissemination of dynamic data using cooperating peers

The focus of our work is to design and build a dynamic data distribution system that is coherence-preserving, i.e. the delivered data must preserve associated coherence requirements (the user-specified bound on tolerable imprecision) and resilient to failures. To this end, we consider a system in which a set of repositories cooperate with each other and the sources, forming a peer-to-peer network. In this system, necessary changes are pushed to the users so that they are automatically informed, about changes of interest. We present techniques 1) to determine when to push an update from one repository to another for coherence maintenance, 2) to construct an efficient dissemination tree for propagating changes from sources to cooperating repositories, and 3) to make the system resilient to failures. An experimental evaluation using real world traces of dynamically changing data demonstrates that 1) careful dissemination of updates through a network of cooperating repositories can substantially lower the cost of coherence maintenance, 2) unless designed carefully, even push-based systems experience considerable loss in fidelity due to message delays and processing costs, 3) the computational and communication cost of achieving resiliency is made to be low, and 4) surprisingly, adding resiliency actually improve fidelity even in the absence of failures.     

基于聚类的对等网络负载优化算法研究

减少系统的资源消耗是对等网络设计的目标之一。该文对基于聚类的对等网络建立了负载的抽象模型,根据实际系统中采集的数据分析了节点的动态特性,并分别在节点能力相同、节点的能力的异构和系统处于动态的情况下,对聚类的数目进行了优化分析,为设计和构造对等网络提供了依据。     

Skyframe: a framework for skyline query processing in peer-to-peer systems

This paper looks at the processing of skyline queries on peer-to-peer (P2P) networks. We propose Skyframe, a framework for efficient skyline query processing in P2P systems, which addresses the challenges of quick response time, low network communication cost and query load balancing among peers. Skyframe consists of two querying methods: one is optimized for network communication while the other focuses on query response time. These methods are different in the way in which the query search space is defined. In particular, the first method uses a high dominating point that has a large dominating region to prune the search space to achieve a low cost in network communication. On the other hand, the second method relaxes the search space in order to allow parallel query processing to speed up query response. Skyframe achieves query load balancing by both query load conscious data space splitting/merging during the join/departure of nodes and dynamic load migration. We further show how to apply Skyframe to both the P2P systems supporting multi-dimensional indexing and the P2P systems supporting single-dimensional indexing. Finally, we have conducted extensive experiments on both real and synthetic data sets over two existing P2P systems: CAN (Ratnasamy in A scalable content-addressable network. In: Proceedings of SIGCOMM Conference, pp. 161–172, 2001) and BATON (Jagadish et al. in A balanced tree structure for peer-to-peer networks. In: Proceedings of VLDB Conference, pp. 661–672, 2005) to evaluate the effectiveness and scalability of Skyframe.     

Metamodels for Computer-based Engineering Design: Survey and recommendations

The use of statistical techniques to build approximations of expensive computer analysis codes pervades much of today’s engineering design. These statistical approximations, or metamodels, are used to replace the actual expensive computer analyses, facilitating multidisciplinary, multiobjective optimization and concept exploration. In this paper, we review several of these techniques, including design of experiments, response surface methodology, Taguchi methods, neural networks, inductive learning and kriging. We survey their existing application in engineering design, and then address the dangers of applying traditional statistical techniques to approximate deterministic computer analysis codes. We conclude with recommendations for the appropriate use of statistical approximation techniques in given situations, and how common pitfalls can be avoided.     

Runtime and Programming Support for Memory Adaptation in Scientific Applications via Local Disk and Remote Memory

The ever increasing memory demands of many scientific applications and the complexity of today’s shared computational resources still require the occasional use of virtual memory, network memory, or even out-of-core implementations, with well known drawbacks in performance and usability. In Mills et al. (Adapting to memory pressure from within scientific applications on multiprogrammed COWS. In: International Parallel and Distributed Processing Symposium, IPDPS, Santa Fe, NM, 2004), we introduced a basic framework for a runtime, user-level library, MMlib, in which DRAM is treated as a dynamic size cache for large memory objects residing on local disk. Application developers can specify and access these objects through MMlib, enabling their application to execute optimally under variable memory availability, using as much DRAM as fluctuating memory levels will allow. In this paper, we first extend our earlier MMlib prototype from a proof of concept to a usable, robust, and flexible library. We present a general framework that enables fully customizable memory malleability in a wide variety of scientific applications. We provide several necessary enhancements to the environment sensing capabilities of MMlib, and introduce a remote memory capability, based on MPI communication of cached memory blocks between ‘compute nodes’ and designated memory servers. The increasing speed of interconnection networks makes a remote memory approach attractive, especially at the large granularity present in large scientific applications. We show experimental results from three important scientific applications that require the general MMlib framework. The memory-adaptive versions perform nearly optimally under constant memory pressure and execute harmoniously with other applications competing for memory, without thrashing the memory system. Under constant memory pressure, we observe execution time improvements of factors between three and five over relying solely on the virtual memory system. With remote memory employed, these factors are even larger and significantly better than other, system-level remote memory implementations.     

A multi-agent based user context Bayesian neural network analysis system

The increasing user mobility demands placed upon IT services necessitates an environment that enables users to access optimal services at any time and in any place. This study presents research conducted to develop a system that is capable of analyzing user IT service patterns and tendencies and provides the necessary service resources by sharing each user’s context information. First, each user’s context information is gathered to provide the multi-agent software training data necessary to describe user operations in a hybrid peer-to-peer (P2P) structured communication environment. Next, the data collected about each user’s mobile device is analyzed through a Bayesian based neural network system to identify the user’s tendency and extract essential service information. This information provides a communication configuration allowing the user access to the best communication service between the user’s mobile device and the local server at any time and in any place, thereby enhancing the ubiquitous computing environment.     

Real-time data dissemination in mobile peer-to-peer networks

Mobile peer-to-peer networks have found many uses such as streaming of audio and video data. There are circumstances, such as emergency situations and disaster recovery, when real-time delivery is a fundamental requirement. The problem is challenging due to the limited network capacity, the variable transmission rates and the unpredictability with respect to the network conditions in the mobile peer-to-peer network.In this paper we address the problem of real-time data dissemination of multimedia streams in mobile peer-to-peer networks. Four routing algorithms are proposed based on a packet's deadline, priority or a combination of these metrics. They are simulated under different setups in a mobile peer-to-peer network with Bluetooth connectivity and nodes broadcasting audio and video streams using different priorities. We compare the performance of the algorithms using a number of metrics. Detailed experimental results are presented. Based on these results, propositions on the usage of the algorithms and the design of network requirements are presented.     

Meta-searches in peer-to-peer networks

We propose a method for carrying out enhanced collaborative searches, called meta-searches, in peer-to-peer networks. In addition to performing regular searches, our method supports searches based on other network users’ previous searches on the same or similar topic. In essence, when a user performs a search, s/he will receive not only the usual result set, but also information on other users’ previous results, as well as relevancy information (such as how many times a resource that appeared in the result set was successfully downloaded). The core components of meta-search are query relevancy calculation, query matching algorithms, and relevancy file format. In this paper we discuss the underlying concepts and principles, and describe the component design in detail. Meta-search provides a way of benefiting from other users’ successful searches without any additional effort, thus potentially improving the efficiency and experience of a search.     

Human centered design in the air traffic control system

A review of the current air traffic control system is undertaken from the perspective of human centered design, focusing on the development of today’s system, the problems in today’s system, and the challenges going forward. Today’s system evolved around the operators in the system (mainly air traffic controllers and pilots), rather than being designed based on specific engineering analyses. This human centered focus has helped make air transportation remarkably safe, but has also made the air traffic control system somewhat inscrutable. This opaqueness of how the system operates poses significant problems for current attempts to transform the system into its “next generation” with significantly improved capacity. Research advances in human centered computing research required in order for this transformation work to proceed are discussed, specifically advances in computing the safety of complex human-integrated systems, understanding and measuring situation awareness, and visualizations of complex data.     

Large-scale network intrusion detection based on distributed learning algorithm

As network traffic bandwidth is increasing at an exponential rate, it’s impossible to keep up with the speed of networks by just increasing the speed of processors. Besides, increasingly complex intrusion detection methods only add further to the pressure on network intrusion detection (NIDS) platforms, so the continuous increasing speed and throughput of network poses new challenges to NIDS. To make NIDS usable in Gigabit Ethernet, the ideal policy is using a load balancer to split the traffic data and forward those to different detection sensors, which can analyze the splitting data in parallel. In order to make each slice contains all the evidence necessary to detect a specific attack, the load balancer design must be complicated and it becomes a new bottleneck of NIDS. To simplify the load balancer this paper put forward a distributed neural network learning algorithm (DNNL). Using DNNL a large data set can be split randomly and each slice of data is presented to an independent neural network; these networks can be trained in distribution and each one in parallel. Completeness analysis shows that DNNL’s learning algorithm is equivalent to training by one neural network which uses the technique of regularization. The experiments to check the completeness and efficiency of DNNL are performed on the KDD’99 Data Set which is a standard intrusion detection benchmark. Compared with other approaches on the same benchmark, DNNL achieves a high detection rate and low false alarm rate.     

Efficient Multimedia Distribution in Source Constraint Networks

In recent years, the number of peer-to-peer (P2P) applications has increased significantly. One important problem in many P2P applications is how to efficiently disseminate data from a single source to multiple receivers on the Internet. A successful model used for analyzing this problem is a graph consisting of nodes and edges, with a capacity assigned to each edge. In some situations however, it is inconvenient to use this model. To that end, we propose to study the problem of efficient data dissemination in a source constraint network. A source constraint network is modeled as a graph in which, the capacity is associated with a node, rather than an edge. The contributions of this paper include (a) a quantitative data dissemination in any source constraint network, (b) a set of topologies suitable for data dissemination in P2P networks, and (c) an architecture and implementation of a P2P system based on the proposed optimal topologies. We will present the experimental results of our P2P system deployed on PlanetLab nodes demonstrating that our approach achieves near optimal throughput while providing scalability, low delay and bandwidth fairness among peers.     

Badumna: A decentralised network engine for virtual environments

Many of today’s virtual environment applications, such as massively multiplayer online games, involve a large and rapidly changing set of users. The server-based architecture of these applications makes its hard for them to support the dynamic user base and provide a scalable solution. Badumna, a scalable network engine for large-scale virtual environments, was developed to address these issues. The approach is based on a decentralised architecture using a structured peer-to-peer network. In order to ensure a highly scalable and efficient design, Badumna uses techniques that include neighbour introductions and the formation of dynamic bounded regions to reduce the load on the peer-to-peer network. The performance of Badumna is evaluated using extensive simulations. Results indicate that Badumna is able to scale to a large number of users without placing any additional overheads on the network that degrade performance.     

Runtime support for scalable programming in Java

The paper research is concerned with enabling parallel, high-performance computation—in particular development of scientific software in the network-aware programming language, Java. Traditionally, this kind of computing was done in Fortran. Arguably, Fortran is becoming a marginalized language, with limited economic incentive for vendors to produce modern development environments, optimizing compilers for new hardware, or other kinds of associated software expected of by today’s programmers. Hence, Java looks like a very promising alternative for the future. The paper will discuss in detail a particular environment called HPJava. HPJava is the environment for parallel programming—especially data-parallel scientific programming—in Java. Our HPJava is based around a small set of language extensions designed to support parallel computation with distributed arrays, plus a set of communication libraries. A high-level communication API, Adlib, is developed as an application level communication library suitable for our HPJava. This communication library supports collective operations on distributed arrays. We include Java Object as one of the Adlib communication data types. So we fully support communication of intrinsic Java types, including primitive types, and Java object types.     

Managing Data Placement in Memory Systems with Multiple Memory Controllers

Modern processors such as Tilera’s Tile64, Intel’s Nehalem, and AMD’s Opteron are migrating memory controllers (MCs) on-chip, while maintaining a large, flat memory address space. This trend to utilize multiple MCs will likely continue and a core or socket will consequently need to route memory requests to the appropriate MC via an inter- or intra-socket interconnect fabric similar to AMD’s HyperTransport^TM, or Intel’s Quick-Path Interconnect^TM. Such systems are therefore subject to non-uniform memory access (NUMA) latencies because of the time spent traveling to remote MCs. Each MC will act as the gateway to a particular region of the physical memory. Data placement will therefore become increasingly critical in minimizing memory access latencies. Increased competition for memory resources will also increase the memory access latency variation in future systems. Proper allocation of workload data to the appropriate MC will be important in decreasing the variation and average latency when servicing memory requests. The allocation strategy will need to be aware of queuing delays, on-chip latencies, and row-buffer hit-rates for each MC. In this paper, we propose dynamic mechanisms that take these factors into account when placing data in appropriate slices of physical memory. We introduce adaptive first-touch page placement, and dynamic page-migration mechanisms to reduce DRAM access delays for multi-MC systems. We also introduce policies that can handle data placement in memory systems that have regions with heterogeneous properties. The proposed policies yield average performance improvements of 6.5% for adaptive first-touch page-placement, and 8.9% for a dynamic page-migration policy for a system with homogeneous DRAM DIMMs. We also show improvements in systems that contain DIMMs with different performance characteristics.