GMarte: Grid middleware to abstract remote task execution

Grid computing technologies are now being largely deployed with the widespread adoption of the Globus Toolkit as the industrial standard Grid middleware. However, its inherent steep learning curve discourages the use of these technologies for non‐experts. Therefore, to increase the use of Grid computing, it is important to have high‐level tools that simplify the process of remote task execution. In this paper we introduce a middleware, developed on top of the Java Commodity Grid, which offers an object‐oriented, user‐friendly application programming interface, from the Java language, which eases remote task execution for computationally intensive applications. Copyright © 2006 John Wiley & Sons, Ltd.     

A software tool for semi-automatic gridification of resource-intensive Java bytecodes and its application to ray tracing and sequence alignment

Computational Grids deliver the necessary computational infrastructure to perform resource-intensive computations such as the ones that solve the problems scientists are facing today. Exploiting Computational Grids comes at the expense of explicitly adapting the ordinary software implementing scientific problems to take advantage of Grid resources, which unavoidably requires knowledge on Grid programming. The recent notion of “gridifying” ordinary applications, which is based on semi-automatically deriving a Grid-aware version from the compiled code of a sequential application, promises users to be relieved from the requirement of manual usage of Grid APIs within their source codes. In this paper, we describe a novel gridification tool that allows users to easily parallelize Java applications on Grids. Extensive experiments with two real-world applications - ray tracing and sequence alignment - suggest that our approach provides a convenient balance between ease of gridification and Grid resource exploitation compared to manually using Grid APIs for gridifying ordinary applications.     

A middleware‐transparent approach to developing distributed applications

An innovative middleware‐transparent approach to developing distributed applications is presented. The approach uses an aspect‐oriented software development technique to separate an application's middleware‐independent functionality from its middleware‐specific functionality. Application elements that are specific to the middleware are localized in aspects that can be seamlessly integrated into middleware‐independent application designs. The middleware‐transparent approach is used to decouple business functionality from middleware‐specific functionality. The decoupling allows developers to change middleware application elements without significantly modifying business functionality. Middleware technologies such as Java Remote Method Invocation (RMI), Jini, Simple Object Access Protocol (SOAP) remote procedure call (RPC) and .Net are used as examples to illustrate the approach. Copyright © 2005 John Wiley & Sons, Ltd.     

Quality of Service on Grid: architectural and methodological issues

The rapid evolution of Grid Computing and the development of new middleware services make Grid platforms increasingly used not only for best effort scientific jobs but also in industrial and business applications. This has taken to the growing demand of Quality of Service (QoS) support. However, the QoS issue on Grid is quite difficult, as Grid has been originally designed without any QoS support, and it is a complex system. During the previous years some solutions have been proposed to supply QoS for specific classes of applications. This results in a focused and a heterogeneous approach, so that it is hard to evaluate its sufficiency and its robustness with respect to the large spectrum of possible Grid applications. In this context, our contribution concerns three points: first, we analyze the current approach to QoS on Grid as a relationship among QoS features, applications, and architectures; second, we evaluate the QoS requirements of two recent QoS‐demanding applications on Grid, namely Massive Multiplayer Online Games and Urgent Computing, comparing these requirements with the support provided by current QoS architectures; and third, we propose an alternative approach to QoS provision on Grid based on the definition of a dedicated QoS‐management layer to overcome the limitations of the current methodologies. Copyright © 2010 John Wiley & Sons, Ltd.     

Scalable Grid‐wide capacity allocation with the SweGrid Accounting System (SGAS)

The SweGrid Accounting System (SGAS) allocates capacity in collaborative Grid environments by coordinating enforcement of Grid‐wide usage limits as a means to offer usage guarantees and prevent overuse. SGAS employs a credit‐based allocation model where Grid capacity is granted to projects via Grid‐wide quota allowances that can be spent across the Grid resources. The resources collectively enforce these allowances in a soft, real‐time manner. SGAS is built on service‐oriented principles with a strong focus on interoperability and Web services standards. This article covers the SGAS design and implementation, which, besides addressing inherent Grid challenges (scale, security, heterogeneity, decentralization), emphasizes generality and flexibility to produce a customizable system with lightweight integration into different middleware and scheduling system combinations. We focus the discussion around the system design, a flexible allocation model, middleware integration experiences and scalability improvements via a distributed virtual banking system, and finally, an extensive set of testbed experiments. The experiments evaluate the performance of SGAS in terms of response times, request throughput, overall system scalability, and its performance impact on the Globus Toolkit 4 job submission software. We conclude that, for all practical purposes, the quota enforcement overhead incurred by SGAS on job submissions is not a limiting factor for the job‐handling capacity of the job submission software. Copyright © 2008 John Wiley & Sons, Ltd.     

Babylon: middleware for distributed,parallel, and mobile Java applications

Babylon is a collection of tools and services that provide a 100% Java‐compatible environment for developing, running and managing parallel, distributed and mobile Java applications. It incorporates features such as object migration, asynchronous method invocation, and remote class loading, while providing an easy‐to‐use interface. Additionally, Babylon enables Java applications to seamlessly create and interact with remote objects, while protecting those objects from other applications by implementing access restrictions and separate namespaces. The implementation of Babylon centers around dynamic proxies, a feature first available in Java 1.3, that allow proxy objects to be created at runtime. Dynamic proxies play a key role in achieving the goals of Babylon. The potential cluster computing benefits of the system are demonstrated with experimental results, which show that sequential Java applications can achieve significant performance benefits from using Babylon to parallelize their work across a cluster of workstations. Copyright © 2008 John Wiley & Sons, Ltd.     

Introduce: An Open Source Toolkit for Rapid Development of Strongly Typed Grid Services

Service-oriented architectures and applications have gained wide acceptance in the Grid computing community. A number of tools and middleware systems have been developed to support application development using Grid Services architectures. Most of these efforts, however, have focused on low-level support for management and execution of Grid services, management of Grid-enabled resources, and deployment and execution of applications that make use of Grid services. Simple-to-use service development tools, which would allow a Grid service developer to leverage Grid technologies without needing to know low-level details, are becoming increasingly important for wider application of the Grid. In this paper, we describe an open-source, extensible toolkit, called Introduce, that supports easy development and deployment of Web Services Resource Framework (WSRF) compliant services. Introduce is designed to reduce the service development and deployment effort by hiding low level details of the Globus Toolkit and to enable the implementation of strongly typed services. In strongly typed services, a service produces and consumes data types that are well-defined and published in the Grid. This enables data-level syntactic interoperability so that clients and services can access and consume data elements programmatically and correctly. We expect that enabling strongly typed Grid services while lowering the difficulty of entry to the Grid via toolkits like Introduce will have a major impact to the success of the Grid and its wider adoption as a viable technology of choice in the commercial sector as well as in academic, medical, and government research.     

Ontology‐based Grid resource management

Grid resources are typically diverse in nature with respect to their software and hardware configurations, resource usage policies and the kind of application they support. Aggregating and monitoring these resources, and discovering suitable resources for the applications become a challenging issue. This is partially due to the representation of Grid metadata supported by the existing Grid middleware which offers limited scope for matching the job requirements that directly affect scheduling decisions. This paper proposes a semantic component in conventional Grid architecture to support ontology‐based representation of Grid metadata and facilitate context‐based information retrieval that complements Grid schedulers for effective resource management. Web Ontology language is used for creating Grid resource ontology and Algernon inference engine has been used for resource discovery. This semantic component has been integrated with conventional Grid schedulers. Several experiments have also been carried out to investigate the performance overhead that arises while integrating this component with Grid schedulers. Copyright © 2009 John Wiley & Sons, Ltd.     

ICENI: Optimisation of component applications within a Grid environment

Effective exploitation of Computational Grids can only be achieved when applications are fully integrated with the Grid middleware and the underlying computational resources. Fundamental to this exploitation is information. Information about the structure and behaviour of the application, the capability of the computational and networking resources, and the availability and access to these resources by an individual, a group or an organisation.

In this paper we describe Imperial College e-Science Networked Infrastructure (ICENI), a Grid middleware framework developed within the London e-Science Centre. ICENI is a platform-independent framework that uses open and extensible XML derived protocols, within a framework built using Java and Jini, to explore effective application execution upon distributed federated resources. We match a high-level application specification, defined as a network of components, to an optimal combination of the currently available component implementations within our Grid environment, by using composite performance models. We demonstrate the effectiveness of this architecture through the high-level specification and solution of a set of linear equations by automatic and selection of optimal resources and implementations.     

Workflow Concepts of the Java CoG Kit

Many scientific simulations and experiments require the coordination of numerous tasks posed by interdisciplinary research teams. Grids can provide access to the necessary high-end resources to conduct such tasks. The complex tasks and their interactions must be supported through convenient tools. To address this issue, we introduce a number of Grid abstractions that make the development of Grid middleware-independent tools possible and allow for the integration of a number of commodity tools. Our vision is implemented through an integrated approach based on a layered architecture that bridges the gap between Grid middleware and scientific applications. Our abstractions include specialized services, a Grid workflow engine and language, and Gridfaces – graphical abstractions that can be employed in science portals and standalone applications.     

Enhancing grid security by fine-grained behavioral control and negotiation-based authorization

Nowadays, Grid has become a leading technology in distributed computing. Grid poses a seamless sharing of heterogeneous computational resources belonging to different domains and conducts efficient collaborations between Grid users. The core Grid functionality defines computational services which allocate computational resources and execute applications submitted by Grid users. The vast models of collaborations and openness of Grid system require a secure, scalable, flexible and expressive authorization model to protect these computational services and Grid resources. Most of the existing authorization models for Grid have granularity to manage access to service invocations while behavioral monitoring of applications executed by these services remains a responsibility of a resource provider. The resource provider executes an application under a local account, and acknowledges all permissions granted to this account to the application. Such approach poses serious security threats to breach system functionality since applications submitted by users could be malicious. We propose a flexible and expressive policy-driven credential-based authorization system to protect Grid computational services against a malicious behavior of applications submitted for the execution. We split an authorization process into two levels: a coarse-grained level that manages access to a computational service; and a fine-grained level that monitors the behavior of applications executed by the computational service. Our framework guarantees that users authorized on a coarse-grained level behave as expected on the fine-grained level. Credentials obtained on the coarse-grained level reflect on fine-grained access decisions. The framework defines trust negotiations on coarse-grained level to overcome scalability problem, and preserves privacy of credentials and security policies of, both, Grid users and providers. Our authorization system was implemented to control access to the Globus Computational GRAM service. A comprehensive performance evaluation shows the practical scope of the proposed system.

SchedSP: a Grid‐based application service provider of scheduling solutions

In this paper, SchedSP, a middleware framework for providing scheduling solutions as services over the Internet, is presented. Emphasis is given on creating a reusable framework that facilitates the development of specialized clients for the input, output and control interfaces of the various scheduling applications. SchedSP manages the task of preparing and running the required processes and allows the application interface developer to focus on the functionality and efficiency of the interface. The Internet‐based scheduling applications created are competitive in all aspects with traditional locally executed applications. In this paper, detailed architecture and implementation details of the SchedSP framework prototype are presented. In addition, the methodology for creating specific case studies based on the SchedSP middleware framework is presented. Copyright © 2005 John Wiley & Sons, Ltd.     

CORBA request portable interceptors: analysis and applications

Interceptors are an emerging middleware technology enabling the addition of specific network‐oriented capabilities to distributed applications. By exploiting interceptors, developers can register code within interception points, extending the basic middleware mechanisms with specific functionality, e.g. authentication, flow control, caching, etc. Notably, these extensions can be achieved without modifying either the application or the middleware code. In this paper we report the results of our experiences with CORBA request portable interceptors. In particular, we point out (i) the basic mechanisms implementable by these interceptors, i.e. request redirection and piggybacking and (ii) we analyze their limitations. We then propose a proxy‐based technique to overcome the interceptors' limitations. Successively, we present a performance analysis carried out on three Java‐CORBA platforms currently implementing the portable interceptors specification. Finally, we conclude our work with a case study in which portable interceptors are used to implement the fault‐tolerant CORBA client invocation semantic without impacting on the client application code and on the CORBA ORB. We also release fragments of Java code for implementing the described techniques. Copyright © 2003 John Wiley & Sons, Ltd.     

Achieving efficiency, quality of service and robustness in multi-organizational Grids

Scalability, flexibility, quality of service provisioning, efficiency and robustness are the desired characteristics of most computing systems. Although the emerging Grid computing paradigm is scalable and flexible, achieving both efficiency and quality of service provisioning in Grids is a challenging task but is necessary for the wide adoption of Grids. Grid middleware should also be robust to uncertainties such as those in user-estimated runtimes of Grid applications. In this paper, we present a complete middleware framework for Grids that achieves user satisfaction by providing QoS guarantees for Grid applications, cost effectiveness by efficiently utilizing resources and robustness by intelligently handling uncertain runtimes of applications.     

A secure and efficient public auditing scheme using RSA algorithm for cloud storage

Cloud storage is widely used by both individual and organizational users due to the many benefits, such as scalability, ubiquitous access, and low maintenance cost (and generally free for individual users). However, there are known security and privacy issues in migrating data to the cloud. To ensure or verify data integrity, a number of cloud data integrity checking schemes with different properties have been presented in the literature. Most existing schemes were subsequently found to be insecure or have high computation and communication costs. More recently in 2016, Yu et al. (Future Gener Comput Syst 62:85–91, 2016) proposed an identity-based auditing scheme for checking the integrity of cloud data. However, in this paper, we reveal that the scheme is vulnerable to data recovery attack. We also present a new identity-based public auditing scheme and formally prove the security of the scheme under the RSA assumption with large public exponents in the random oracle model. We then evaluate the performance of our proposed scheme and demonstrate that in comparison with Yu et al.’s scheme, our proposal is more practical in real-world applications.     

JASAG: a gridification tool for agricultural simulation applications

The Grid Computing paradigm aims to create a ‘virtual’ and powerful single computer with many distributed resources to solve resource intensive problems. The term ‘gridification’ involves the process of transforming a conventional application to run in a Grid environment. In that sense, the more automatic this process is, the easier is for developers with low expertise in parallel and distributed computing to take advantage of these resources. To date, many semiautomatic gridifiers were built to support different gridification approaches and application code structures or anatomies. Furthermore, agricultural simulation applications have a particular common anatomy based on biophysical entities, such as animals, crops, and pastures, which are updated by actions, such as growing animals, growing crops, and growing pastures, along simulation execution. However, this anatomy is not fully supported by any of the existing gridifiers. Thus, this paper presents Agricultural Simulation Applications Gridifier (ASAG), a method for easy gridification of agricultural simulation applications, and its Java implementation, named Java ASAG (JASAG). The main design drivers of JASAG are middleware independence, separation of business logic and Grid behavior, and performance increase. An experimental evaluation showing the feasibility of the gridification method and its implementation is also reported, which resulted in speedups of up to 25 by using a real agricultural simulation application. Copyright © 2014 John Wiley & Sons, Ltd.     

Studying protein folding on the Grid: experiences using CHARMM on NPACI resources under Legion

One benefit of a computational Grid is the ability to run high‐performance applications over distributed resources simply and securely. We demonstrated this benefit with an experiment in which we studied the protein‐folding process with the CHARMM molecular simulation package over a Grid managed by Legion, a Grid operating system. High‐performance applications can take advantage of Grid resources if the Grid operating system provides both low‐level functionality as well as high‐level services. We describe the nature of services provided by Legion for high‐performance applications. Our experiences indicate that human factors continue to play a crucial role in the configuration of Grid resources, underlying resources can be problematic, Grid services must tolerate underlying problems or inform the user, and high‐level services must continue to evolve to meet user requirements. Our experiment not only helped a scientist perform an important study, but also showed the viability of an integrated approach such as Legion's for managing a Grid. Copyright © 2004 John Wiley & Sons, Ltd.     

EMPEROR: An OGSA Grid Meta-Scheduler Based on Dynamic Resource Predictions

Scheduling constitutes an integral feature of Grid computing infrastructures, being also a key to realizing several of the Grid promises. In particular, scheduling can maximize the resources available to end users, accelerate the execution of jobs, while also supporting scalable and autonomic management of the resources comprising a Grid. Grid scheduling functionality hinges on middleware components called meta-schedulers, which undertake to automatically distribute jobs across the dispersed heterogeneous resources of a Grid. In this paper we present the design and implementation of a Grid meta-scheduler, which we call EMPEROR. EMPEROR provides a framework for implementing scheduling algorithms based on performance criteria. In implementing a particular instantiation of this framework, we have devised models for predicting host load and memory resources, and accordingly for estimating the running time of a task. These models hinge on time series analysis techniques and take into account results of the cluster computing literature. Apart from incorporating these models, EMPEROR provides fully fledged Grid scheduling functionality, which complies with OGSA standards as the later are reflected in the Globus toolkit. Specifically, EMPEROR interfaces to Globus middleware services (i.e., GSI, MDS, GRAM) towards discovering resources, implementing the scheduling algorithm and ultimately submitting jobs to local scheduling systems. By and large, EMPEROR is one of the few standards based meta-schedulers making use of dynamic scheduling information.     

Service selection and workflow mapping for Grids: an approach exploiting quality‐of‐service information

The advent of heterogeneous and distributed environments, such as Grid environments, made feasible the solution to computational‐intensive problems in a reliable and cost‐effective manner. In parallel, workflows with increased complexity that require specialized systems to deal with them are emerging, so as to carry out more composite and mission‐critical applications. In that rationale, quality‐of‐service (QoS) issues need to be tackled in order to ensure that each application satisfies the corresponding user requirements. Therefore, considering the quality provision aspect as fundamental for enabling Grid applications to become QoS compliant, we present an approach for service selection using QoS criteria. The latter is achieved with a suite of components that allow the different mappings of application workflow processes to Grid services that not only meet the user goals and requirements but also maximize his/her benefit in terms of the offered QoS level. We also demonstrate the operation of the aforementioned suite of components and evaluate its performance and effectiveness using a Grid scenario, based on a 3D image rendering application. Copyright © 2008 John Wiley & Sons, Ltd.     

Middleware infrastructure for parallel and distributed programming models in heterogeneous systems

We introduce a middleware infrastructure that provides software services for developing and deploying high-performance parallel programming models and distributed applications on clusters and networked heterogeneous systems. This middleware infrastructure utilizes distributed agents residing on the participating machines and communicating with one another to perform the required functions. An intensive study of the parallel programming models in Java has helped identify the common requirements for a runtime support environment, which we used to define the middleware functionality. A Java-based prototype, based on this architecture, has been developed along with a Java object-passing interface (JOPI) class library. Since this system is written completely in Java, it is portable and allows executing programs in parallel across multiple heterogeneous platforms. With the middleware infrastructure, users need not deal with the mechanisms of deploying and loading user classes on the heterogeneous system. Moreover, details of scheduling, controlling, monitoring, and executing user jobs are hidden, while the management of system resources is made transparent to the user. Such uniform services are essential for facilitating the development and deployment of scalable high-performance Java applications on clusters and heterogeneous systems. An initial deployment of a parallel Java programming model over a heterogeneous, distributed system shows good performance results. In addition, a framework for the agents' startup mechanism and organization is introduced to provide scalable deployment and communication among the agents.