Metacomputing in practice: a distributed compute server for pharmaceutical industry

We describe a distributed high-performance compute server that has been implemented for running compute-intensive applications on a mixture of HPC systems interconnected by Inter-and Intranet. With a practical industrial background, our work focusses on high availability, efficient job load balancing, security, and the easy integration of HPC computing into the daily work-flow at pharmaceutical companies.

The work was done in the course of the ESPRIT project P A Distributed Pharmaceutical Application Server The client software is implemented in Java. All results are displayed in a web browser and can be forwarded to the next stage of applications used in the drug design cycle. The server software handles the job load balancing between the participating HPC nodes and is capable of managing multi-site applications.

Our environment currently supports four key applications that are used in rational drug design and drug target identification. They range from the automatic functional annotation of protein sequences to three-dimensional protein structure prediction tools and protein comparison applications.     

Enhanced iterative-deepening search

Iterative-deepening searches mimic a breadth-first node expansion with a series of depth-first searches that operate with successively extended search horizons. They have been proposed as a simple way to reduce the space complexity of best-first searches like A* from exponential to linear in the search depth. But there is more to iterative-deepening than just a reduction of storage space. As the authors show, the search efficiency can be greatly improved by exploiting previously gained node information. The information management techniques considered here owe much to their counterparts from the domain of two-player games, namely the use of fast-execution memory functions to guide the search. The authors' methods not only save node expansions, but are also faster and easier to implement than previous proposals     

Analyzing the performance of SMP memory allocators with iterative MapReduce applications

The standard memory allocators of shared memory systems (SMPs) often provide poor performance, because they do not sufficiently reflect the access latencies of deep NUMA architectures with their on-chip, off-chip, and off-blade communication. We analyze memory allocation strategies for data-intensive MapReduce applications on SMPs with up to 512 cores and 2 TB memory. We compare the efficiency of the MapReduce frameworks MR-Search and Phoenix++ and provide performance results on two benchmark applications, k-means and shortest-path search.     

The Grid Application Toolkit: Toward Generic and Easy Application Programming Interfaces for the Grid

Core Grid technologies are rapidly maturing, but there remains a shortage of real Grid applications. One important reason is the lack of a simple and high-level application programming toolkit, bridging the gap between existing Grid middleware and application-level needs. The Grid Application Toolkit (GAT), as currently developed by the EC-funded project GridLab, provides this missing functionality. As seen from the application, the GAT provides a unified simple programming interface to the Grid infrastructure, tailored to the needs of Grid application programmers and users. A uniform programming interface will be needed for application developers to create a new generation of "Grid-aware" applications. The GAT implementation handles both the complexity and the variety of existing Grid middleware services via so-called adaptors. Complementing existing Grid middleware, GridLab also provides high-level services to implement the GAT functionality. We present the GridLab software architecture, consisting of the GAT, environment-specific adaptors, and GridLab services. We elaborate the concepts underlying the GAT and outline the corresponding application programming interface. We present the functionality of GridLab's high-level services and demonstrate how a dynamic Grid application can easily benefit from the GAT. All GridLab software is open source and can be downloaded from the project Web site.     

Modeling Replica Availability in Large Data Grids

Large Grid systems not only provide massive aggregated computing power but also an unprecedented amount of distributed storage space. Unfortunately, the dynamic behavior of the Grid, caused by varying resource availability, unpredictable data updates, and the impact of local site policies makes it difficult to exploit the full capabilities of Data Grids.We present an analytical model for determining the optimal number of replica servers, catalog servers, and catalog sizes to guarantee a given overall reliability in the face of unreliable components. Our model captures the characteristics of peer-to-peer-like environments as well as that of Grid systems. Empirical simulations confirm the accuracy of our analytical model.     

Book reviews

Ohne Zusammenfassung

---

Book reviews

     

Heterogeneous Message Passing and a Link to Resource Management

Parallel process communication and system resource management have been seen as two separate entities in parallel and distributed systems. This causes difficulties in the dynamic mapping of newly spawned processes, because the application has little or no information on the availability, the connectivity and the current work-load of the target system. As a consequence, process mappings are often sub-optimal, overloading resources on one system while other processors are idling. We present a software system named "PLUS" that provides interprocess communication between different message passing models such as MPI, PVM and PARIX, and access to resource management systems for optimal process mapping and task migration.PLUS is a light-weight, extensible and efficient communication interface. With only four commands, PLUS is almost transparent ot the application code. Our current implementation supports inter-process communication between PVM, MPI and PARIX, but it can be easily extended to other vendor-specific message passing libraries. As PLUS has been designed for wide area networks, much effort has been spent on portability and on optimizing the communication speed across internet and also intranet links.     

Fast algebraic methods for interval constraint problems

We describe an effective generic method for solving constraint problems, based on Tarski’s relation algebra, using path-consistency as a pruning technique. We investigate the performance of this method on interval constraint problems. Time performance is affected strongly by the path-consistency calculations, which involve the calculation of compositions of relations. We investigate various methods of tuning composition calculations, and also path-consistency computations. Space performance is affected by the branching factor during search. Reducing this branching factor depends on the existence of ‘nice’ subclasses of the constraint domain. Finally, we survey the statistics of consistency properties of interval constraint problems. Problems of up to 500 variables may be solved in expected cubic time. Evidence is presented that the ‘phase transition’ occurs in the range 6 ≤ n.c ≤15, where n is the number of variables, and c is the ratio of non-trivial constraints to possible constraints. This revised version was published online in June 2006 with corrections to the Cover Date.