Middleware infrastructure for parallel and distributed programming models in heterogeneous systems

We introduce a middleware infrastructure that provides software services for developing and deploying high-performance parallel programming models and distributed applications on clusters and networked heterogeneous systems. This middleware infrastructure utilizes distributed agents residing on the participating machines and communicating with one another to perform the required functions. An intensive study of the parallel programming models in Java has helped identify the common requirements for a runtime support environment, which we used to define the middleware functionality. A Java-based prototype, based on this architecture, has been developed along with a Java object-passing interface (JOPI) class library. Since this system is written completely in Java, it is portable and allows executing programs in parallel across multiple heterogeneous platforms. With the middleware infrastructure, users need not deal with the mechanisms of deploying and loading user classes on the heterogeneous system. Moreover, details of scheduling, controlling, monitoring, and executing user jobs are hidden, while the management of system resources is made transparent to the user. Such uniform services are essential for facilitating the development and deployment of scalable high-performance Java applications on clusters and heterogeneous systems. An initial deployment of a parallel Java programming model over a heterogeneous, distributed system shows good performance results. In addition, a framework for the agents' startup mechanism and organization is introduced to provide scalable deployment and communication among the agents.     

运载火箭测发控软件的中间件技术应用

由于不同型号硬件平台和软件平台的多样性,跨型号大规模融合应用系统通常要求在拥有多种软硬件平台(异构平台)的分布式环境下运行。借助中间件(Middleware)技术改进软件重用形式,提高软件重用程度,以实现支持分布式应用有效开发、部署、运行和管理的目的。本文在对现有火箭测发控系统软件的子系统功能聚类和程序架构分析基础上,开发基于中间件模型的通用火箭测发控系统软件。     

Managing complexity: middleware explained

Traditionally, most definitions seeking to characterize middleware suggest that it is the software that facilitates remote database access and systems transactions. More recently, the term has come to be associated-somewhat limitingly-with distributed platforms such as the Open Software Foundation's Distributed Computing Environment (DCE) and the Object Management Group's Common Object Request Broker Architecture (CORBA). And some have loosely applied it to systems as diverse as workflow support environments and even to the Web itself. We believe the essential role of middleware is to manage the complexity and heterogeneity of distributed infrastructures and thereby provide a simpler programming environment for distributed-application developers. It is therefore most useful to define middleware as any software layer that is placed above the distributed system's infrastructure-the network OS and APIs-and below the application layer     

Secure Sharing of Tuple Spaces in Ad Hoc Settings

Security is emerging as a growing concern throughout the distributed computing community. Typical solutions entail specialized infrastructure support for authentication, encryption and access control. Mobile applications executing over ad hoc wireless networks present designers with a rather distinct set of security requirements. A totally open setting and limited resources call for lightweight and highly decentralized security solutions. In this paper we propose an approach that relies on extending an existing coordination middleware for mobility (Lime). The need to continue to offer a very simple model of coordination that assures rapid software development led to limiting extensions solely to password protected tuple spaces and per tuple access control. Password distribution and security are relegated to the application realm. Host level security is ensured by the middleware design and relies on standard support provided by the Java system. Secure interactions among agents across hosts are accomplished by careful exploitation of the interceptor pattern and the use of standard encryption. The paper explains the design strategy used to add security support in Lime and its implications for the development of mobile applications over ad hoc networks.     

Utilizing architectural styles to enhance the adaptation support of middleware platforms

ContextModern middleware platforms provide the applications deployed on top of them with facilities for their adaptation. However, the level of adaptation support provided by the state-of-the-art middleware solutions is often limited to dynamically loading and off-loading of software components. Therefore, it is left to the application developers to handle the details of change such that the system’s consistency is not jeopardized.ObjectiveWe aim to change the status quo by providing the middleware facilities necessary to ensure the consistency of software after adaptation. We would like these facilities to be reusable across different applications, such that the middleware can streamline the process of achieving safe adaptation.MethodOur approach addresses the current shortcomings by utilizing the information encoded in a software system’s architectural style. This information drives the development of reusable adaptation patterns. The patterns specify both the exact sequence of changes and the time at which those changes need to occur. We use the patterns to provide advanced adaptation support on top of an existing architectural middleware platform.ResultsOur experience shows the feasibility of deriving detailed adaptation patterns for several architectural styles. Applying the middleware to adapt two real-world software systems shows the approach is effective in consistently adapting these systems without jeopardizing their consistency.ConclusionWe conclude the approach is effective in alleviating the application developers from the responsibility of managing the adaptation process at the application-level. Moreover, we believe this study provides the foundation for changing the way adaptation support is realized in middleware solutions.     

An overview of the Real-Time CORBA specification

A growing class of real-time systems require end-to-end support for various quality-of-service (QoS) aspects, including bandwidth, latency, jitter and dependability. Applications include command and control, manufacturing process control, videoconferencing, large-scale distributed interactive simulation, and testbeam data acquisition. These systems require support for stringent QoS requirements. To meet this challenge, developers are turning to distributed object computing middleware, such as the Common Object Request Broker Architecture, an Object Management Group (OMG) industry standard. In complex real-time systems, DOC middleware resides between applications and the underlying operating systems, protocol stacks and hardware. CORBA helps decrease the cycle time and effort required to develop high-quality systems by composing applications using reusable software component services rather than building them entirely from scratch. The Real-Time CORBA specification includes features to manage CPU, network and memory resources. The authors describe the key Real-Time CORBA features that they feel are the most relevant to researchers and developers of distributed real-time and embedded systems     

Arcademis: a framework for object‐oriented communication middleware development

This paper presents Arcademis, a Java‐based framework for communication middleware development. Arcademis consists of a set of abstract classes, interfaces and concrete components that define the general architecture of middleware systems. Its main objective is to support the implementation of non‐monolithic and easily configurable middleware platforms. Arcademis can be used by middleware developers to deploy systems that meet the requirements of a particular network or technology. Instances of Arcademis can also be customized by distributed systems engineers to meet the requirements of a particular application. For example, new transport protocols, connection management policies, authentication algorithms or invocation semantics can be easily configured in middleware platforms derived from Arcademis. In order to illustrate the use of the framework, the paper describes the RME system, a middleware derived from Arcademis that adds a remote method invocation service to the CLDC configuration of Java 2 Micro Edition (J2ME). Copyright © 2006 John Wiley & Sons, Ltd.     

Transparent autonomization in CORBA

Increasingly, software systems are constructed by integrating and composing multiple existing applications. The resulting complexity increases the need for self-management of the system. However, adding autonomic behavior to composite systems is difficult, especially when the constituent components are heterogeneous and they were not originally designed to support such interactions. Moreover, entangling the code for self-management with the code for the business logic of the original applications may actually increase the complexity of the systems, counter to the desired goal. In this paper, we address autonomization of composite systems that use CORBA, one of the first widely used middleware platforms introduced more than 17 years ago that is still commonly used in numerous systems. We propose a model, called Adaptive CORBA Template (ACT), that enables autonomic behavior to be added to CORBA applications automatically and transparently, that is, without requiring any modifications to the code implementing the business logic of the original applications. To do so, ACT uses “generic” interceptors, which are added to CORBA applications at startup time and enable autonomic behavior to be introduced later at runtime. We have developed ACT/J, a prototype of ACT in Java. We describe a case study in which ACT/J is used to introduce three types of autonomic behavior (self-healing, self-optimization, and self-configuration) to a distributed surveillance application.     

Middleware challenges ahead

Middleware research and development has reached the end of its first major phase, and new requirements are arising that are so fundamentally different that they will lead to new-generation middleware systems. Facing dynamic modifications in distributed systems technology, middleware developers are striving to support applications that meet the technical challenges of ubiquitous computing     

Wireless middleware: glue for the mobile infrastructure

The need for participants to communicate across so many platforms has led to the rise in wireless middleware's importance. Middleware is a software layer that resides between programs, OSs, hardware platforms, and communications protocols. Middleware lets different applications and systems, including clients and servers on different platforms, work together. Wireless middleware eases the transformation of markup languages, the delivery of content and data, protocol and device recognition, the incorporation and routing of business logic through enterprise systems, and the adaptation of data formats for compatibility with multiple databases.     

Principles for designing data-/compute-intensive distributed applications and middleware systems for heterogeneous environments

The nature of distributed systems is constantly and steadily changing as the hardware and software landscape evolves. Porting applications and adapting existing middleware systems to ever changing computational platforms has become increasingly complex and expensive. Therefore, the design of applications, as well as the design of next generation middleware systems, must follow a set of guiding principles in order to insure long-term “survivability” without costly re-engineering. From our practical experience, the key determinants to success in this endeavor are adherence to the following principles: (1) Design for change; (2) Provide for storage subsystem I/O coordination; (3) Employ workload partitioning and load balancing techniques; (4) Employ caching; (5) Schedule the workload; and (6) Understand the workload. In order to support these principles, we have collected extensive experimental results comparing three middleware systems targeted at data- and compute-intensive applications implemented by our research group during the course of the last decade, on a single data- and compute-intensive application. The main contribution of this work is the analysis of a level playing field, where we discuss and quantify how adherence to these guiding principles impacts overall system throughput and response time.     

A lightweight and distributed middleware to provide presence awareness in mobile ubiquitous systems

Several researchers have identified the need to count on presence awareness in ubiquitous systems that support mobile activities, particularly when these systems are used to perform loosely-coupled mobile work. In such a work style, mobile users conduct face-to-face on-demand interactions, therefore counting on awareness information about the position and availability of potential collaborators becomes mandatory for these applications. Most proposed solutions that provide user presence awareness involve centralized components, have reusability limitations, or simply address a part of that service. This article presents a lightweight and fully distributed middleware named Moware, which allows developers to embed presence awareness services in mobile ubiquitous systems in a simple way. The article also describes the Moware architecture, its main components and strategies used to deal with several aspects of the presence awareness support. These design strategies can be reused by software designers to provide presence awareness capabilities into middleware and specific software applications. Moware services were embedded in a mobile ubiquitous system that supports inspectors during the construction inspection process. The preliminary results indicate that the middleware was easy to use for developers, and its services were useful for the end-users.     

MidHDC: Advanced Topics on Middleware Services for Heterogeneous Distributed Computing. Part 1

Middleware solutions for Heterogeneous Distributes System aim to respond to high requirements of large scale distributed applications related to performance, flexibility, extensibility, portability, availability, reliability, safety, security, trust, and scalability, in the context of high number of users, and large geographic distribution of heterogeneous hardware and software resources. The solutions used in the design, implementation, and deployment of systems with such capabilities are based on monitoring, scheduling, optimization, sharing, balancing, discovery, and synchronization methods and techniques that are continuously improved.This special issue presents advances in virtual machine management solutions in Clouds, object storage platforms, HPC heterogeneous platforms, middleware for Android systems and reliability and performances in large scale distributed applications.     

Configurable Middleware for Distributed Real-Time Systems with Aperiodic and Periodic Tasks

Different distributed real-time systems (DRS) must handle aperiodic and periodic events under diverse sets of requirements. While existing middleware such as Real-Time CORBA has shown promise as a platform for distributed systems with time constraints, it lacks flexible configuration mechanisms needed to manage end-to-end timing easily for a wide range of different DRS with both aperiodic and periodic events. The primary contribution of this work is the design, implementation, and performance evaluation of the first configurable component middleware services for admission control and load balancing of aperiodic and periodic event handling in DRS. Empirical results demonstrate the need for, and the effectiveness of, our configurable component middleware approach in supporting different applications with aperiodic and periodic events, and providing a flexible software platform for DRS with end-to-end timing constraints.     

一种面向服务的权限管理模型

面向服务的体系结构(Service-Oriented Architecture,SOA)是设计和构建松耦合软件系统的方法,它可将基于中间件开发的分布式应用共享为Internet环境下的软件服务．传统中间件的用户权限系统具有较好的灵活性,基本满足封闭系统的安全需求．但在SOA模式下,难以满足不同节点和系统互相请求服务和共享资源过程中的授权．该文提出了一个面向服务的权限管理模型,通过支持用户之间的代理和提供一定的推理能力,为应用开发者提供了更完善的权限管理机制,并扩展了中间件跨越组织共享资源和服务的能力．该模型在一个J2EE应用服务器上被实现和验证．实验证明,该模型具有良好的灵活性和可扩展性,并且性能影响在合理的范围．     

一个MDA支撑工具的设计与实现

MDA是OMG提出的用于解决中间件集成问题的软件开发方法。MDA开发方法以系统模型作为软件开发的主线。在这样的开发过程中,强有力的模型转换支撑工具是MDA思想能够得到广泛应用的关键。而从平台无关模型PIM到平台相关模型PSM的转换工具尤其重要。描述了一个从PIM自动转换到PSM的工具的设计思想。工具使用EDOC profile的一个子集作为PIM的描述方法,使用J2EE作为目标平台。基于一组从PIM到J2EE平台上软件模型的转换规则,工具实现了从PIM到J2EE平台上的PSM的自动转换。     

Design of scalable Java message-passing communications over InfiniBand

This paper presents ibvdev a scalable and efficient low-level Java message-passing communication device over InfiniBand. The continuous increase in the number of cores per processor underscores the need for efficient communication support for parallel solutions. Moreover, current system deployments are aggregating a significant number of cores through advanced network technologies, such as InfiniBand, increasing the complexity of communication protocols, especially when dealing with hybrid shared/distributed memory architectures such as clusters. Here, Java represents an attractive choice for the development of communication middleware for these systems, as it provides built-in networking and multithreading support. As the gap between Java and compiled languages performance has been narrowing for the last years, Java is an emerging option for High Performance Computing (HPC). The developed communication middleware ibvdev increases Java applications performance on clusters of multicore processors interconnected via InfiniBand through: (1) providing Java with direct access to InfiniBand using InfiniBand Verbs API, somewhat restricted so far to MPI libraries; (2) implementing an efficient and scalable communication protocol which obtains start-up latencies and bandwidths similar to MPI performance results; and (3) allowing its integration in any Java parallel and distributed application. In fact, it has been successfully integrated in the Java messaging library MPJ Express. The experimental evaluation of this middleware on an InfiniBand cluster of multicore processors has shown significant point-to-point performance benefits, up to 85% start-up latency reduction and twice the bandwidth compared to previous Java middleware on InfiniBand. Additionally, the impact of ibvdev on message-passing collective operations is significant, achieving up to one order of magnitude performance increases compared to previous Java solutions, especially when combined with multithreading. Finally, the efficiency of this middleware, which is even competitive with MPI in terms of performance, increments the scalability of communications intensive Java HPC applications.     

Trends in big data analytics

One of the major applications of future generation parallel and distributed systems is in big-data analytics. Data repositories for such applications currently exceed exabytes and are rapidly increasing in size. Beyond their sheer magnitude, these datasets and associated applications’ considerations pose significant challenges for method and software development. Datasets are often distributed and their size and privacy considerations warrant distributed techniques. Data often resides on platforms with widely varying computational and network capabilities. Considerations of fault-tolerance, security, and access control are critical in many applications (Dean and Ghemawat, 2004; Apache hadoop). Analysis tasks often have hard deadlines, and data quality is a major concern in yet other applications. For most emerging applications, data-driven models and methods, capable of operating at scale, are as-yet unknown. Even when known methods can be scaled, validation of results is a major issue. Characteristics of hardware platforms and the software stack fundamentally impact data analytics. In this article, we provide an overview of the state-of-the-art and focus on emerging trends to highlight the hardware, software, and application landscape of big-data analytics.     

2005 EDOC: meeting the challenges of enterprise computing

An increasing demand for interoperable applications exists, sparking the real-time exchange of data across borders, applications, and IT platforms. To perform these tasks, enterprise computing now encompasses a new class of groundbreaking technologies such as Web services and service-oriented architecture (SOA); business process integration and management; and middleware support, like that for utility, grid, peer-to-peer, and autonomic computing. Enterprise computing also influences the processes for business modeling, consulting, and service delivery; it affects the design, development, and deployment of software architecture, as well as the monitoring and management of such architecture. As enterprises demand increasing levels of networked information and services to carry out business processes, IT professionals need conferences like EDOC to discuss emerging technologies and issues in enterprise computing. For these reasons, what started out as the Enterprise Distributed Object Computing (EDOC) conference has come to encompass much more than just distributed objects. So this event now used the name International EDOC Enterprise Computing Conference, to recognize this broader scope yet also retain the initial conference's name recognition.