Computational proteomics on the grid

The large number of protein sequences, provided by genomic projects at an increasing pace, constitutes a challenge for large scale computational studies of protein structure and thermodynamics. Grid technology is very suitable to face this challenge, since it provides a way to access the resources needed in compute and data intensive applications. In this work, we concentrate on the grid-aware implementation of a protein structure prediction algorithm.     

Fault-aware grid scheduling using performance prediction by workload modeling

Computational grids hold great promise in utilizing geographically separated heterogeneous resources to solve large-scale complex problems. However, they suffer from a number of major technical hurdles, including distributed resource management and effective job scheduling. The main focus of this work is devoted on online scheduling of real time applications in distributed environments such as grids. Specifically, we are interested in applications with several independent tasks, each task with a prespecified lifecycle called deadline. Here, our goal is to schedule applications within an optimum overall time considering the specified deadlines. To achieve this, the resource performance prediction based on workload modeling and with the help of queuing techniques is employed. Afterward, a mathematical neural model is used to schedule the subtasks of the application. The main contributions of this work is to incorporate the impatiency factor as well as resource fault in performance modeling of nondedicated distributed systems, and also presenting an efficient and fast parallel scheduling algorithm under time constraint and heterogeneous resources. The proposed model is appropriate for implementation on parallel machines and in O(1) time. The new model was implemented on GridSim toolkit and under various conditions and with different parameters to evaluate the performance of scheduling algorithm. Simulation outcomes have shown that approximately in 87.8% of cases, our model schedules the tasks in such a way that all constraints are satisfied.

Bacteria foraging optimization for protein sequence analysis on the grid

Scientific applications such as protein sequence analysis require a coordination of resources. This is due to hundreds and hundreds of protein sequences being deposited into data banks by the research community which results in an extensive database search when one wants to find a similar protein sequence. This search becomes easier and the time taken is reduced when it is conducted in a grid environment implemented using the Globus tool kit. This paper proposes the use of Bacteria Foraging Optimization (BFO) for finding similar protein sequences in the existing databases. Usage of BFO further reduces the time taken by a resource to execute the user’s requests. Also, the resources utilized in the proposed method are better balanced compared to the existing scheduling algorithms. Also, it is found that the number of tasks executed is more compared to the existing algorithms even though there is a fall in the execution of tasks as the number of resources increases which might be due to network failure etc. The proposed BFO has been compared with the existing First Come First Serve (FCFS) and Minimum Execution Time (MET) scheduling algorithms and it has been found that the proposed BFO performs well compared to the existing algorithms in terms of makespan, resource utilization and minimization in the case of non-execution of client requests.     

A hybrid policy for fault tolerant load balancing in grid computing environments

Due to the emergence of grid computing over the Internet, there is a need for a hybrid load balancing algorithm which takes into account the various characteristics of the grid computing environment. Hence, this research proposes a fault tolerant hybrid load balancing strategy namely AlgHybrid_LB, which takes into account grid architecture, computer heterogeneity, communication delay, network bandwidth, resource availability, resource unpredictability and job characteristics. AlgHybrid_LB juxtaposes the strong points of neighbor-based and cluster based load balancing algorithms. Our main objective is to arrive at job assignments that could achieve minimum response time and optimal computing node utilization. Major achievements include low complexity of proposed approach and drastic reduction of number of additional communications induced due to load balancing. A simulation of the proposed approach using Grid Simulation Toolkit (GridSim) is conducted. Experimental results show that the proposed algorithm performs very well in a large grid environment.     

A parallel micro evolutionary algorithm for heterogeneous computing and grid scheduling

This work presents a novel parallel micro evolutionary algorithm for scheduling tasks in distributed heterogeneous computing and grid environments. The scheduling problem in heterogeneous environments is NP-hard, so a significant effort has been made in order to develop an efficient method to provide good schedules in reduced execution times. The parallel micro evolutionary algorithm is implemented using MALLBA, a general-purpose library for combinatorial optimization. Efficient numerical results are reported in the experimental analysis performed on both well-known problem instances and large instances that model medium-sized grid environments. The comparative study of traditional methods and evolutionary algorithms shows that the parallel micro evolutionary algorithm achieves a high problem solving efficacy, outperforming previous results already reported in the related literature, and also showing a good scalability behavior when facing high dimension problem instances.     

An adaptive multi-policy grid service for biological sequence comparison

In the last decade, we have observed an unprecedented development in molecular biology. An extremely high number of organisms have been sequenced in genome projects and included in genomic databases, for further analysis. These databases present an exponential growth rate and they are intensively accessed daily, all over the world. Once a sequence is obtained, its function and/or structure must be determined. Direct experimentation is considered to be the most reliable method to do that. However, the experiments that must be conducted are very complex and time consuming. For this reason, it is far more productive to use computational methods to infer biological information from a sequence. This is usually done by comparing the new sequence with sequences that already had their characteristics determined. BLAST is the most widely used heuristic tool for sequence comparison. Thousands of BLAST searches are made daily, all over the world. In order to further reduce the BLAST execution time, cluster and grid environments can be effectively used. This paper proposes and evaluates an adaptive task allocation framework to perform BLAST searches in a grid environment. The framework, called PackageBLAST, provides an infrastructure that executes distributed BLAST genomic database comparisons. In addition, it is flexible since the user can choose or incorporate new task allocation strategies. Furthermore, we propose a mechanism to compute grid nodes’ execution weight, adapting the chosen allocation policy to the observed computational power and local load of the nodes. Our results present very good speedups. For instance, in a 16-machine heterogeneous grid testbed, a speedup of 14.59 was achieved, reducing the BLAST execution time from 30.88 min to 2.11 min. Also, we show that the adaptive task allocation strategy was able to handle successfully the complexity of a grid environment.     

A probabilistic and adaptive scheduling algorithm using system-generated predictions for inter-grid resource sharing

Rapid advancement and more readily availability of Grid technologies have encouraged many businesses and researchers to establish Virtual Organizations (VO) and make use of their available desktop resources to solve computing intensive problems. These VOs, however, work as disjointed and independent communities with no resource sharing between them. We, in previous work, have proposed a fully decentralized and reconfigurable Inter-Grid framework for resource sharing among such distributed and autonomous Grid systems (Rao et al. in ICCSA, [2006]). The specific problem that underlies in such a collaborating Grids system is scheduling of resources as there is very little knowledge about availability of the resources due to the distributed and autonomous nature of the underlying Grid entities. In this paper, we propose a probabilistic and adaptive scheduling algorithm using system-generated predictions for Inter-Grid resource sharing keeping collaborating Grid systems autonomous and independent. We first use system-generated job runtime estimates without actually submitting jobs to the target Grid system. Then this job execution estimate is used to predict the job scheduling feasibility on the target system. Furthermore, our proposed algorithm adapted itself to the actual resource behavior and performance. Simulation results are presented to discuss the correctness and accuracy of our proposed algorithm.

An adaptive and trustworthy software testing framework on the grid

Grid computing, which is characterized by large-scale sharing and collaboration of dynamic distributed resources has quickly become a mainstream technology in distributed computing and is changing the traditional way of software development. In this article, we present a grid-based software testing framework for unit and integration test, which takes advantage of the large-scale and cost-efficient computational grid resources to establish a testbed for supporting automated software test in complex software applications. Within this software testing framework, a dynamic bag-of-tasks model using swarm intelligence is developed to adaptively schedule unit test cases. Various high-confidence computing mechanisms, such as redundancy, intermediate value checks, verification code injection, and consistency checks are employed to verify the correctness of each test case execution on the grid. Grid workflow is used to coordinate various test units for integration test. Overall, we expect that the grid-based software testing framework can provide efficient and trustworthy services to significantly accelerate the testing process with large-scale software testing.

A hierarchical reliability-driven scheduling algorithm in grid systems

In a Grid computing system, many distributed scientific and engineering applications often require multi-institutional collaboration, large-scale resource sharing, wide-area communication, etc. Applications executing in such systems inevitably encounter different types of failures such as hardware failure, program failure, and storage failure. One way of taking failures into account is to employ a reliable scheduling algorithm. However, most existing Grid scheduling algorithms do not adequately consider the reliability requirements of an application. In recognition of this problem, we design a hierarchical reliability-driven scheduling architecture that includes both a local scheduler and a global scheduler. The local scheduler aims to effectively measure task reliability of an application in a Grid virtual node and incorporate the precedence constrained tasks’ reliability overhead into a heuristic scheduling algorithm. In the global scheduler, we propose a hierarchical reliability-driven scheduling algorithm based on quantitative evaluation of independent application reliability. Our experiments, based on both randomly generated graphs and the graphs of some real applications, show that our hierarchical scheduling algorithm performs much better than the existing scheduling algorithms in terms of system reliability, schedule length, and speedup.     

GAMESH: A grid architecture for scalable monitoring and enhanced dependable job scheduling

Grid computing is a largely adopted paradigm to federate geographically distributed data centers. Due to their size and complexity, grid systems are often affected by failures that may hinder the correct and timely execution of jobs, thus causing a non-negligible waste of computing resources. Despite the relevance of the problem, state-of-the-art management solutions for grid systems usually neglect the identification and handling of failures at runtime. Among the primary goals to be considered, we claim the need for novel approaches capable to achieve the objectives of scalable integration with efficient monitoring solutions and of fitting large and geographically distributed systems, where dynamic and configurable tradeoffs between overhead and targeted granularity are necessary. This paper proposes GAMESH, a Grid Architecture for scalable Monitoring and Enhanced dependable job ScHeduling. GAMESH is conceived as a completely distributed and highly efficient management infrastructure, concentrating on two crucial aspects for large-scale and multi-domain grid environments: (i) the scalable dissemination of monitoring data and (ii) the troubleshooting of job execution failures. GAMESH has been implemented and tested in a real deployment encompassing geographically distributed data centers across Europe. Experimental results show that GAMESH (i) enables the collection of measurements of both computing resources and conditions of task scheduling at geographically sparse sites, while imposing a limited overhead on the entire infrastructure, and (ii) provides a failure-aware scheduler able to improve the overall system performance, even in the presence of failures, by coordinating local job schedulers at multiple domains.     

A grid computing based approach for the power system dynamic security assessment

This paper addresses the problem of parallel dynamic security assessment applications from static homogeneous cluster environment to dynamic heterogeneous grid environment. Functional parallelism and data parallelism are supported by each of the message passing interface model and TCP/IP model. To consider the differences in heterogeneous computing resources and complexity of large-scale power system communities, a kernel-based multilevel algorithm is proposed for network partitioning. Since the bottleneck in distributed computation is low speed network communication, a bi-level latency exploitation technique is introduced for numerically solving system differential equations. The proposed grid-based implementation includes the core simulation engine, grid computing middleware, a Python interface and Python front-end utilities. Tests for a 39-bus network, a 4000-bus network and a 10,000-bus network are reported, and the results of these experiments demonstrate that the proposed scheme is able to execute the distributed simulations on computational grid infrastructure and provide efficient parallelism.     

Service-oriented middleware for distributed data mining on the grid

Distribution of data and computation allows for solving larger problems and executing applications that are distributed in nature. The grid is a distributed computing infrastructure that enables coordinated resource sharing within dynamic organizations consisting of individuals, institutions, and resources. The grid extends the distributed and parallel computing paradigms allowing for resource negotiation and dynamical allocation, heterogeneity, open protocols, and services. Grid environments can be used both for compute-intensive tasks and data intensive applications by exploiting their resources, services, and data access mechanisms. Data mining algorithms and knowledge discovery processes are both compute and data intensive, therefore the grid can offer a computing and data management infrastructure for supporting decentralized and parallel data analysis. This paper discusses how grid computing can be used to support distributed data mining. Research activities in grid-based data mining and some challenges in this area are presented along with some promising future directions for developing grid-based distributed data mining.     

A clustering method to distribute a database on a grid

Clusters and grids of workstations provide available resources for data mining processes. To exploit these resources, new distributed algorithms are necessary, particularly concerning the way to distribute data and to use this partition. We present a clustering algorithm dubbed Progressive Clustering that provides an “intelligent” distribution of data on grids. The usefulness of this algorithm is shown for several distributed datamining tasks.     

网格计算的安全性研究与技术实现

网格计算环境必须以现有的Internet为通信支撑平台,由于Internet本身的开放性和异构性,决定了网格计算面临着各种各样的安全威胁,因此网格安全已成为网格计算环境中的一个核心问题。该文简述了网格安全需求,分析了网格安全技术,并给出了基于Globus项目中网格安全的主要技术手段。     

A high level data structure—The grid

Grids are arrays that can have any shape: grid elements need not be connected to each other (e.g., a grid may be pyramid-like or two disjoint rectangular pieces). Programs using grids are smaller, semantically clearer, more general and easier to modify than programs that simulate non array-like shapes. We present a notation for specifying grids in PASCAL. Grids have been implemented as an extension to FORTRAN.     

基于遗传算法的网格计算资源调度策略

如何将网格这个复杂环境中的计算资源进行有效调度,是一个NP问题。遗传算法被证明是解决这类问题的有效算法,同时遗传算法有“早熟”和慢速收敛等缺点。为了克服其缺点,提出一种新的并行遗传算法,采取避免近亲繁殖的交叉策略和保护优秀个体的方法,提高算法搜索能力和收敛速度。仿真结果表明该算法能有效地解决网格计算资源分配问题。     

Optimizing grid computing configuration and scheduling for geospatial analysis: An example with interpolating DEM

Many geographic analyses are very time-consuming and do not scale well when large datasets are involved. For example, the interpolation of DEMs (digital evaluation model) for large geographic areas could become a problem in practical application, especially for web applications such as terrain visualization, where a fast response is required and computational demands exceed the capacity of a traditional single processing unit conducting serial processing. Therefore, high performance and parallel computing approaches, such as grid computing, were investigated to speed up the geographic analysis algorithms, such as DEM interpolation. The key for grid computing is to configure an optimized grid computing platform for the geospatial analysis and optimally schedule the geospatial tasks within a grid platform. However, there is no research focused on this. Using DEM interoperation as an example, we report our systematic research on configuring and scheduling a high performance grid computing platform to improve the performance of geographic analyses through a systematic study on how the number of cores, processors, grid nodes, different network connections and concurrent request impact the speedup of geospatial analyses. Condor, a grid middleware, is used to schedule the DEM interpolation tasks for different grid configurations. A Kansas raster-based DEM is used for a case study and an inverse distance weighting (IDW) algorithm is used in interpolation experiments.     

A survey of life sciences applications on the grid

The availability of powerful microprocessors and improvements in the performance of networks has enabled high performance computing on wide-area, distributed systems. Computational grids, by integrating diverse, geographically distributed and essentially heterogeneous resources provide the infrastructure for solving large-scale problems. However, heterogeneity, on the one hand allows for scalability, but on the other hand makes application development and deployment for such an environment extremely difficult. The field of life sciences has been an explosion in data over the past decade. The data acquired needs to be processed, interpreted and analyzed to be useful. The large resource needs of bioinformatics allied to the large number of data-parallel applications in this field and the availability of a powerful, high performance, computing grid environment lead naturally to opportunities for developing grid-enabled applications. This survey, done as part of the Life Sciences Research Group (a research group belonging to the Global Grid Forum) attempts to collate information regarding grid-enabled applications in this field. Arun Krishnan, Ph.D.: He did his undergraduate in Electrochemical Engineering in the Central Electrochemical Research Institute in India and went on to do his Ph.D. in Advanced Process Control from the University of South Carolina. He then worked in the control and high performance computing industries for about 3 years before moving to the Bioinformatics Institute in Singapore. He is currently a Young Investigator for the Distributed Computing in Biomedicine Group at BII. His research interests include parallel and distributed computing with special emphasis on grid computing and its application to the biomedical area. He is also interested in developing parallel algorithms for sequence analysis and protein structure prediction.     

A user-friendly platform for developing and accessing grid services

This article presents an enhanced platform that provides a friendly environment of developing grid services and accessing grid services over Globus Toolkit 3 (GT3). This platform includes a class of functions for processing parameters input from a developer via GUI, a class of functions for generating files required for defining a grid service specified, and a class of functions for creating client program and facilitating accesses of the deployed services. As a result, the development and access of grid services requires less special expert knowledge of a developer at the server side and users at the client side, the efficiency of developing and accessing grid services can be improved. This paper describes our design ideas, necessary functions, and implementations. The comparisons with other related toolkits are given and the extended version of the platform on top of the web service environment rather than GT3.