A survey on resource allocation in high performance distributed computing systems

An efficient resource allocation is a fundamental requirement in high performance computing (HPC) systems. Many projects are dedicated to large-scale distributed computing systems that have designed and developed resource allocation mechanisms with a variety of architectures and services. In our study, through analysis, a comprehensive survey for describing resource allocation in various HPCs is reported. The aim of the work is to aggregate under a joint framework, the existing solutions for HPC to provide a thorough analysis and characteristics of the resource management and allocation strategies. Resource allocation mechanisms and strategies play a vital role towards the performance improvement of all the HPCs classifications. Therefore, a comprehensive discussion of widely used resource allocation strategies deployed in HPC environment is required, which is one of the motivations of this survey. Moreover, we have classified the HPC systems into three broad categories, namely: (a) cluster, (b) grid, and (c) cloud systems and define the characteristics of each class by extracting sets of common attributes. All of the aforementioned systems are cataloged into pure software and hybrid/hardware solutions. The system classification is used to identify approaches followed by the implementation of existing resource allocation strategies that are widely presented in the literature.     

Straddling the crevasse: A review of microservice software architecture foundations and recent advancements

Microservice architecture style has been gaining wide impetus in the software engineering industry. Researchers and practitioners have adopted the microservices concepts into several application domains such as the internet of things, cloud computing, service computing, and healthcare. Applications developed in alignment with the microservices principles require an underlying platform with management capabilities to coordinate the different microservice units and ensure that the application functionalities are delivered to the user. A multitude of approaches has been proposed for the various tasks in microservices-based systems. However, since the field is relatively young, there is a need to organize the different research works. In this study, we present a comprehensive review of the research approaches directed toward microservice architectures and propose a multilevel taxonomy to categorize the existing research. The study also discusses the different distributed computing paradigms employing microservices and identifies the open research challenges in the domain.     

Network resource management mechanisms in SDN enabled WSNs: A comprehensive review

Wireless technologies usually have very limited computing, memory, and battery power that require the optimal management of network resources to increase network performance. The optimization of these network resources provides an efficient network topology, traffic control, routing, and data aggregation. This study presents a qualitative and quantitative investigation to evaluate the efficient network resource management mechanisms for software defined wireless sensor networks (SDN-enabled WSNs) from the beginning of network design to reliable data delivery. In this paper, a taxonomy of network resource management research studies is proposed. A detailed analysis of SDN-enabled WSNs architecture, SDN controllers, topology discovery, routing approaches, flow rules installation, and data aggregation is also discussed. Furthermore, the comparative analysis of resource provisioning methods along with various simulation tools is presented. Moreover, this review outlines open research challenges and prospective future directions for network resource management in SDN-enabled WSNs.     

A taxonomy of scheduling in general-purpose distributed computingsystems

One measure of the usefulness of a general-purpose distributed computing system is the system's ability to provide a level of performance commensurate to the degree of multiplicity of resources present in the system. A taxonomy of approaches to the resource management problem is presented in an attempt to provide a common terminology and classification mechanism necessary in addressing this problem. The taxonomy, while presented and discussed in terms of distributed scheduling, is also applicable to most types of resource management     

Quality-of-service differentiation on the Internet: A taxonomy

Internet applications and clients have very diverse service expectations, demanding for provisioning of different levels of quality of service (QoS) to multiple traffic classes on the Internet. To meet this demand, many different approaches and performance metrics have been proposed in an attempt to achieve per-class QoS differentiation in the network core, network edges, proxy and end servers. However, due to the wide variety of approaches to the problem, it is difficult to meaningfully compare different approaches since there is no uniform means for quantitatively or qualitatively evaluating them. Thus, it is hard to build upon existing work or identify areas worthy of additional efforts without understanding of the relationships between existing efforts.In this paper, a taxonomy of approaches to per-class QoS differentiation is presented in an attempt to provide a common terminology and classification mechanism necessary in addressing this problem. The taxonomy categorizes state-of-the-art QoS differentiation approaches in three dimensions. The first categorization dimension is the locations where the approaches are deployed, i.e., server, proxy and network sides. The QoS differentiation approaches are secondly categorized according to their policies, i.e., admission control, resource management, and content adaptation. The third categorization dimension is their implementation layer, i.e., application level, and kernel level. Representative QoS differentiation approaches in each category are reviewed.     

A Taxonomy of Workflow Management Systems for Grid Computing

With the advent of Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such application scenarios require means for composing and executing complex workflows. Therefore, many efforts have been made towards the development of workflow management systems for Grid computing. In this paper, we propose a taxonomy that characterizes and classifies various approaches for building and executing workflows on Grids. We also survey several representative Grid workflow systems developed by various projects world-wide to demonstrate the comprehensiveness of the taxonomy. The taxonomy not only highlights the design and engineering similarities and differences of state-of-the-art in Grid workflow systems, but also identifies the areas that need further research.     

Hierarchical architectures in structured peer-to-peer overlay networks

Distributed Hash Tables (DHT) are presently used in several large-scale systems in the Internet and envisaged as a key mechanism to provide identifier-locator separation for mobile hosts in Future Internet. Such P2P-based systems become increasingly complex serving popular social networking, resource sharing applications, and Internet-scale infrastructures. Hierarchy is a standard mechanism for coping with heterogeneity and scalability in distributed systems. To address the shortcomings of flat DHT designs, many hierarchical P2P designs have been proposed over recent years. The last generation is hierarchical DHTs (HDHTs) where nodes are organized onto layers and groups. This article discusses hierarchical architectures applied in structured P2P overlay networks, focusing on HDHT designs. We introduce a framework consisting of conceptual models of network hierarchy, multi-layer hierarchical DHT architectures, principles affecting the design choices, and cost models for system tradeoff analysis, performance evaluation, and scalability estimation. Based on the framework we provide a taxonomy and survey more than 20 hierarchical HDHT proposals.     

网管中结合可移动代理技术的瘦客户解决方案

实际使用的网络管理结构缺乏分布式、分散性和可变性 .近来出现了两种新的方案 .一种是用瘦客户的方案 ,另外就是使用可移动代码的委托管理方案 .本文提出了一种新的能有效地结合上面两种技术的方案 ,可以弥补它们各自的不足之处 ,提高网络管理的效率 .     

Market-oriented Grids and Utility Computing: The State-of-the-art and Future Directions

Traditional resource management techniques (resource allocation, admission control and scheduling) have been found to be inadequate for many shared Grid and distributed systems, that consist of autonomous and dynamic distributed resources contributed by multiple organisations. They provide no incentive for users to request resources judiciously and appropriately, and do not accurately capture the true value, importance and deadline (the utility) of a user’s job. Furthermore, they provide no compensation for resource providers to contribute their computing resources to shared Grids, as traditional approaches have a user-centric focus on maximising throughput and minimising waiting time rather than maximising a providers own benefit. Consequently, researchers and practitioners have been examining the appropriateness of ‘market-inspired’ resource management techniques to address these limitations. Such techniques aim to smooth out access patterns and reduce the chance of transient overload, by providing a framework for users to be truthful about their resource requirements and job deadlines, and offering incentives for service providers to prioritise urgent, high utility jobs over low utility jobs. We examine the recent innovations in these systems (from 2000–2007), looking at the state-of-the-art in price setting and negotiation, Grid economy management and utility-driven scheduling and resource allocation, and identify the advantages and limitations of these systems. We then look to the future of these systems, examining the emerging ‘Catallaxy’ market paradigm. Finally we consider the future directions that need to be pursued to address the limitations of the current generation of market oriented Grids and Utility Computing systems.     

A survey on resource discovery mechanisms,peer-to-peer and service discovery frameworks

Service and resource discovery has become an integral part of modern networked systems. In this survey we give an overview of the existing solutions for service and resource discovery for a wide variety of network types. We cover techniques used in existing systems, as well as recent developments from the research front. We also provide taxonomy for discovery systems and architectures, and review the various algorithms and search methods applicable for such systems. Peer-to-peer overlays are discussed in detail and solutions for non-IP-based networks are also included in the review. We also specifically comment on issues related to wireless networks, and give an overview of the various issues and complications that should be considered in future work in this domain.     

A survey of fault tolerance mechanisms and checkpoint/restart implementations for high performance computing systems

In recent years, High Performance Computing (HPC) systems have been shifting from expensive massively parallel architectures to clusters of commodity PCs to take advantage of cost and performance benefits. Fault tolerance in such systems is a growing concern for long-running applications. In this paper, we briefly review the failure rates of HPC systems and also survey the fault tolerance approaches for HPC systems and issues with these approaches. Rollback-recovery techniques which are most often used for long-running applications on HPC clusters are discussed because they are widely used for long-running applications on HPC systems. Specifically, the feature requirements of rollback-recovery are discussed and a taxonomy is developed for over twenty popular checkpoint/restart solutions. The intent of this paper is to aid researchers in the domain as well as to facilitate development of new checkpointing solutions.     

Attribute-based data access control in mobile cloud computing: Taxonomy and open issues

With the thriving growth of the cloud computing, the security and privacy concerns of outsourcing data have been increasing dramatically. However, because of delegating the management of data to an untrusted cloud server in data outsourcing process, the data access control has been recognized as a challenging issue in cloud storage systems. One of the preeminent technologies to control data access in cloud computing is Attribute-based Encryption (ABE) as a cryptographic primitive, which establishes the decryption ability on the basis of a user’s attributes. This paper provides a comprehensive survey on attribute-based access control schemes and compares each scheme’s functionality and characteristic. We also present a thematic taxonomy of attribute-based approaches based on significant parameters, such as access control mode, architecture, revocation mode, revocation method, revocation issue, and revocation controller. The paper reviews the state-of-the-art ABE methods and categorizes them into three main classes, such as centralized, decentralized, and hierarchal, based on their architectures. We also analyzed the different ABE techniques to ascertain the advantages and disadvantages, the significance and requirements, and identifies the research gaps. Finally, the paper presents open issues and challenges for further investigations.     

面向HPC和DC的可重构光互连网络体系结构综述

互连网络是高性能计算系统和数据中心的核心组件之一,也是决定其系统整体性能的全局性基础设施。随着高性能计算、云计算和大数据技术的迅速发展,传统的电互连网络在性能、能耗和成本等方面无法满足高性能计算应用和数据中心业务的大规模可扩展通信需求,面临着严峻的挑战。为此,近年来相关研究者提出了多种面向高性能计算和数据中心的可重构的光互连网络结构。首先阐明了光互连网络相对于电互连网络的优势;然后介绍了几种典型的可重构光互连网络体系结构,并对其特点进行了分析比较;最后探讨了可重构光互连网络的发展趋势。     

Network virtualization in energy-efficient office environments

The rising costs of energy and world-wide desire to reduce CO₂ emissions has led to an increased concern over the energy efficiency of information and communication technology. Whilst much of this concern has focused on data centres, office environments (and the computing equipment that they contain) have also been identified as a significant consumer of energy. Office environments offer great potential for energy savings, given that computing equipment often remains powered for 24 h per day, and for a large part of this period is underutilised or idle. This paper proposes an energy-efficient office management approach based on resource virtualization, power management, and resource sharing. Evaluations indicate that about 75% energy savings are achievable in office environments without a significant interruption of provided services. A core element of this office management is a peer-to-peer network that interconnects office hosts, achieves addressing and mediation, and manages energy efficiency within the office environment. Several peer-to-peer approaches are suggested and discussed in this paper. Two of the approaches are evaluated, based on a discrete-event simulation.     

Resource management framework for collaborative computing systems over multiple virtual machines

A resource management framework for collaborative computing systems over multiple virtual machines (CCSMVM) is presented to increase the performance of computing systems by improving the resource utilization, which has constructed a scalable computing environment for resource on-demand utilization. We design a resource management framework based on the advantages of some components in grid computing platform, virtualized platform and cloud computing platform to reduce computing systems overheads and maintain workloads balancing with the supporting of virtual appliance, Xen API, applications virtualization and so on. The content of collaborate computing, the basis of virtualized resource management and some key technologies including resource planning, resource allocation, resource adjustment and resource release and collaborative computing scheduling are designed in detail. A prototype is designed, and some experiments have verified the correctness and feasibility of our prototype. System evaluations show that the time in resource allocation and resource release is proportional to the quantity of virtual machines, but not the time in the virtual machines migrations. CCSMVM has higher CPU utilization and better performance than other systems, such as Eucalyptus 2.0, Globus4.0, et al. It is concluded that CCSMVM can accelerate the execution of systems by improving average CPU utilization from the results of comparative analysis with other systems, so it is better than others. Our study on resource management framework has some significance to the optimization of the performance in virtual computing systems.     

Scheduling in distributed systems: A cloud computing perspective

Scheduling is essentially a decision-making process that enables resource sharing among a number of activities by determining their execution order on the set of available resources. The emergence of distributed systems brought new challenges on scheduling in computer systems, including clusters, grids, and more recently clouds. On the other hand, the plethora of research makes it hard for both newcomers researchers to understand the relationship among different scheduling problems and strategies proposed in the literature, which hampers the identification of new and relevant research avenues. In this paper we introduce a classification of the scheduling problem in distributed systems by presenting a taxonomy that incorporates recent developments, especially those in cloud computing. We review the scheduling literature to corroborate the taxonomy and analyze the interest in different branches of the proposed taxonomy. Finally, we identify relevant future directions in scheduling for distributed systems.     

A taxonomy of network computing systems

Rapid advances in networking and microprocessor technologies have led to the emergence of Internet-wide distributed computing systems ranging from simple LAN-based clusters to planetary-scale networks. As these network computing systems evolve by combining the best features of existing systems, differences among NCs are blurring. To address this problem, researchers have proposed formal taxonomies of NC systems. We propose a new taxonomy that is both broad enough to encompass all NC systems and simple enough to be widely used.     

基于云计算的资源管理系统的研究

随着IT行业的迅猛发展,网络共享已经不能满足当前的资源管理。云计算已经成为开发各种软件系统的重要工具,利用云计算对资源进行管理,可以有效地解决当前海量资源的管理问题。本文通过分析云计算技术,然后在该技术的基础上设计资源管理系统,将整个系统分为三层,最后并给出了部分模块的核心代码。     

Cloud resource allocation schemes: review,taxonomy, and opportunities

Cloud computing has emerged as a popular computing model to process data and execute computationally intensive applications in a pay-as-you-go manner. Due to the ever-increasing demand for cloud-based applications, it is becoming difficult to efficiently allocate resources according to user requests while satisfying the service-level agreement between service providers and consumers. Furthermore, cloud resource heterogeneity, the unpredictable nature of workload, and the diversified objectives of cloud actors further complicate resource allocation in the cloud computing environment. Consequently, both the industry and academia have commenced substantial research efforts to efficiently handle the aforementioned multifaceted challenges with cloud resource allocation. The lack of a comprehensive review covering the resource allocation aspects of optimization objectives, design approaches, optimization methods, target resources, and instance types has motivated a review of existing cloud resource allocation schemes. In this paper, current state-of-the-art cloud resource allocation schemes are extensively reviewed to highlight their strengths and weaknesses. Moreover, a thematic taxonomy is presented based on resource allocation optimization objectives to classify the existing literature. The cloud resource allocation schemes are analyzed based on the thematic taxonomy to highlight the commonalities and deviations among them. Finally, several opportunities are suggested for the design of optimal resource allocation schemes.     

The challenge of networked enterprises for cloud computing interoperability

Manufacturing enterprises have to organize themselves into effective system architectures forming different types of Networked Enterprises (NE) to match fast changing market demands. Cloud Computing (CC) is an important up to date computing concept for NE, as it offers significant financial and technical advantages beside high-level collaboration possibilities. As cloud computing is a new concept the solutions for handling interoperability, portability, security, privacy and standardization challenges have not been solved fully yet. The paper introduces the main characteristics of future Internet-based enterprises and the different CC models. An overview is given on interoperability and actual standardization issues in CC environments. A taxonomy on possible connecting forms of networked enterprises and cloud-based IT systems with reference on interoperability is introduced, parallel presenting four use cases as well. Finally, an example of connecting cloud and NE is presented as an effective application of cloud computing in manufacturing industry.