Architecture for the next generation system management tools

To get more results or greater accuracy, computational scientists execute their applications on distributed computing platforms such as clusters, grids, and clouds. These platforms are different in terms of hardware and software resources as well as locality: some span across multiple sites and multiple administrative domains, whereas others are limited to a single site/domain. As a consequence, in order to scale their applications up, the scientists have to manage technical details for each target platform. From our point of view, this complexity should be hidden from the scientists, who, in most cases, would prefer to focus on their research rather than spending time dealing with platform configuration concerns.In this article, we advocate for a system management framework that aims to automatically set up the whole run-time environment according to the applications’ needs. The main difference with regards to usual approaches is that they generally only focus on the software layer whereas we address both the hardware and the software expectations through a unique system. For each application, scientists describe their requirements through the definition of a virtual platform (VP) and a virtual system environment (VSE). Relying on the VP/VSE definitions, the framework is in charge of (i) the configuration of the physical infrastructure to satisfy the VP requirements, (ii) the set-up of the VP, and (iii) the customization of the execution environment (VSE) upon the former VP. We propose a new formalism that the system can rely upon to successfully perform each of these three steps without burdening the user with the specifics of the configuration for the physical resources, and system management tools. This formalism leverages Goldberg’s theory for recursive virtual machines (Goldberg, 1973 6]) by introducing new concepts based on system virtualization (identity, partitioning, aggregation) and emulation (simple, abstraction). This enables the definition of complex VP/VSE configurations without making assumptions about the hardware and the software resources. For each requirement, the system executes the corresponding operation with the appropriate management tool.As a proof of concept, we implemented a first prototype that currently interacts with several system management tools (e.g., OSCAR, the Grid’5000 toolkit, and XtreemOS) and that can be easily extended to integrate new resource brokers or cloud systems such as Nimbus, OpenNebula, or Eucalyptus, for instance.