Design and implementation of economics-based resource management system in ad hoc grid

The ad hoc grid is a spontaneous organization of cooperating heterogeneous nodes into a logical community without a fixed infrastructure and with only minimal administrative requirements. Resource management for ad hoc grids is challenging due to the participation of heterogeneous, dynamic, autonomous and ephemeral grid nodes. The paper proposes an ad hoc grid resource management system, the producers and consumers of ad hoc grid resource are modeled as the self-interested decision-makers described in microeconomic theory. All market participants in the ad hoc grid environment including grid resources and services can be represented as agents. We apply economic agents to build ad hoc grid resource management, where ad hoc grid resource consumers and providers can buy and sell ad hoc grid resource based on an underlying economic architecture. The main processes involved in ad hoc grid resource management are resource registration, discovery, and resource allocation. The experiments are conducted to compare ad hoc grid resource allocation algorithm with other ad hoc grid resource allocation algorithm. Simulation results show that our proposed algorithm is more efficient than compared allocation scheme.     

基于市场竞标机制的网格资源管理模型

网格资源管理是网格计算首要解决的核心问题。现行的网格资源管理模型通常需要在网格系统中建立一个高于其他节点的自身庞大的管理体系,以掌握全网格内资源动态信息,实现全局资源调配,这种模式的主要问题是管理成本高且资源配置效率低。文章基于市场经济中的招投标机理,提出了一种新的网格资源管理模型,本模型的资源分配机制是根据网格内资源供需关系和资源的市场价格通过招投标方式来决定。网格内站点是否参与网格计算交易活动,完全由站点自主地根据自身资源现状和资源的市场价格来确定而不需要一个高于自己的管理机构,本模型是一个完全的市场经济的资源分配模型。     

基于Docker Swarm集群的调度策略优化算法

Swarm是一种对集群中Docker镜像和容器进行管理的工具,其在计算节点权值时可能会得到若干个相同权值的节点.现有的Swarm调度策略只是将这些节点随机分配,由于相同权值节点的资源负载情况并不相同,所以将会造成节点负载不均衡.针对上述问题,本文提出一种动态调度算法对Swarm调度策略进行优化.通过实验,证明增加动态调度算法能够使集群中节点负载更加均衡,同时提高集群的整体资源利用率.     

Market mechanisms for resource allocation in pervasive sensor applications

This paper describes the use of market mechanisms for resource allocation in pervasive sensor applications to maximize their Value of Information (VoI), which combines the objectively measured Quality of Information (QoI) with the subjective value assigned to it by the users. The unique challenge of pervasive sensor applications that we address is the need for adjusting resource allocation in response to the changing application requirements and evolving sensor network conditions. We use two market mechanisms: auctions at individual sensor nodes to optimize routing, and switch options to optimize dynamic selection of sensor network services as well as switching between modes of operation in pervasive security applications. We also present scenarios of transient congestion management and home security system to motivate the proposed techniques.     

Xen虚拟机资源调度优化算法

针对Xen虚拟化平台中虚拟机资源分配不合理的问题,提出了两种资源调度优化算法,即细粒度优化算法和粗粒度优化算法.细粒度优化算法主要解决单个物理节点上虚拟机资源分配不合理问题,能够根据物理节点上运行的各虚拟机的资源利用情况来调整资源分配量,适当增加利用率较高的虚拟机的资源,减少资源利用率低的虚拟机的资源,从而优化资源分配,提高资源利用效率,避免不必要的虚拟机迁移.粗粒度优化算法是针对集群中多个物理节点之间虚拟机负载不均衡问题而提出的.该算法结合粒子群优化技术,选择将集群系统中热点物理机上的部分虚拟机迁移到最适合的冷点物理机上,从而避免高载物理机宕机.实验结果表明,这两种资源调度优化算法能够有效解决虚拟机资源分配不合理的问题,具有较好的适用性和应用前景.     

集群环境中微服务容器资源特征分析及优化

在集群环境中部署微服务已经成为微服务部署的重要方式.由于不同种类服务对于CPU、内存、磁盘等资源的需求不同,导致集群中的节点产生资源碎片、出现资源消耗倾斜.如何提高集群资源利用率、降低集群能耗,成为继保障服务级别协议(service level agreement, SLA)之后的重大挑战.本文以阿里巴巴集团2021年发布的近两万个微服务的详细跟踪为数据样本,从容器资源使用情况、节点部署特征和资源消耗偏好等多个维度出发,分析其集群资源消耗特征,发现集群中出现了资源消耗倾斜的情况.通过进一步分析节点中容器部署情况发现容器资源分配不合理加剧了这一现象.基于此我们提出了一种使用深度双Q网络的模型,依据上游服务资源需求的实时变化,对容器资源分配进行优化.对比实验结果表明该方法可以在保证服务SLA的情况下有效提高容器资源利用率,改善节点资源消耗倾斜的情况.     

LUNF--基于节点失效特征的机群作业调度策略

良好的可扩展性使得人们可通过扩大机群系统的规模来达到所需要的计算能力,但随着机群系统节点数目的增多,节点失效对机群系统性能的影响已经成为大规模机群系统使用过程中一个不可回避的问题．机群作业调度作为机群操作系统软件的重要组成部分,完成高效资源管理与合理作业调度,机群作业调度系统功能上可分为作业选取策略和节点分配策略两部分．结合机群系统节点失效的特征,提出了正常运行时间最长节点优先(longest uptime node first,LUNF)的节点分配策略．仿真结果表明,相对于节点随机分配策略,LUNF策略的作业平均响应时间与作业平均slowdown降低10％左右．     

一种WSN的传感器管理结构及仿真分析

无线传感器网络的传感器节点能量十分有限,且数量巨大,因而传统的传感器管理方法不再适用于无线传感器网络。针对无线传感器网络提出了一个分层的传感器管理结构,将网络的传感器管理分为最高层-簇间传感器管理、中间层-簇内传感器管理和最低层-单个传感器管理,并在最低层对传感器节点实现了采样间隔的自适应,仿真结果表明在保证一定跟踪精度的条件下,大大节省了传感器节点的能量。     

基于Gold的气象中心网格平台资源管理系统构建及应用

在介绍成都区域气象中心网格计算平台基础上,为了保证资源的合理分配与网格节点负载均衡,构建了基于Gold的网格资源管理系统,并将其应用到成都区域中心网格计算平台中,通过典型应用实例分析结果表明,该系统功能丰富。大大提高了网格平台资源管理的效率。     

Reliable and economic resource allocation in an unreliable flow network

In this paper a reliable and economic resource allocation strategy under cost constraints is developed for an unreliable multi-source multi-sink flow network. A multi-source multi-sink flow network is composed of nodes, characterized as source nodes, sink nodes and intermediate nodes, as well as directed arcs, joining pairs of nodes. In the presence of unreliable intermediate nodes and arcs, it is important to distribute resources at source nodes in a reliable and cost-effective manner so that resources are more likely to be transmitted successfully to sink nodes through the flow network. By modelling the capacities of intermediate nodes and arcs as statistically independent random variables, an integrated approach is proposed in this study that combines the existing methodologies to determine a reliability-maximizing resource allocation strategy which meets demand at sink nodes and a predetermined transmission cost requirement. Examples of cyclic and acyclic flow networks with unreliable intermediate nodes and arcs are given to illustrate the application of the proposed approach.     

一种基于市场竞拍机制的网格资源管理分配方法

提出了一种基于市场机制的网格资源分配方法：以均衡理论和第二价格竞拍机制为基础, 依靠市场机制, 实现计算网格资源的优化分配。描述了基于市场的资源分配框架, 它包括三个层次: 资源层、市场层和消费者层;采用第二价格竞拍规则作为资源分配策略,满足了网格对分配系统可扩展性和全局最优调度的需求。     

基于网格环境的资源管理模型研究

针对目前网格环境下资源管理存在的问题和现状，提出了一种基于市场机制的资源管理模型。该模型以一般均衡理论为基础，计算出资源的市场价格，依靠市场机制，实现计算网格资源的优化分配。     

The catallaxy approach for decentralized economic-based allocation in Grid resource and service markets

Efficient resource allocation in dynamic large-scale environments is one of the challenges of Grids. In centralized economic-based allocation approaches, the user requests can be matched to the fastest, cheapest or most available resource. This approach, however, shows limitations in scalability and in dynamic environments. In this paper, we explore a decentralized economic approach for resource allocation in Grid markets based on the Catallaxy paradigm. Catallactic agents discover selling nodes in the resource and service Grid markets, and negotiate with each other maximizing their utility by following a strategy. By means of simulations, we evaluate the behavior of the approach, its resource allocation efficiency and its performance with different demand loads in a number of Grid density and dynamic environments. Our results indicate that while the decentralized economic approach based on Catallaxy applied to Grid markets shows similar efficiency to a centralized system, its decentralized operation provides greater advantages: scalability to demand and offer, and robustness in dynamic environments.     

List-based Data Structures for Efficient Management of Advance Reservations

Complex eScience and other sophisticated applications in the field of HPC imply new demands that queuing based resource management systems cannot meet. To guarantee Quality of Service and co-allocation in the Grid, planning based resource management systems implementing advance reservation are needed. These systems face new challenges as a planning based management system has to keep track of the jobs and reservations in the future. Additionally, during the negotiation process of incoming reservations, a good overview of the remaining, not-yet reserved capacity is needed—not only for the current allocation, but also for the whole book-ahead time. Therefore, the resource management problem becomes a two dimensional problem for advance reservations in this field. In this paper different data structures are investigated and discussed in order to fit to planning based resource management. As a result the benefits of using lists of resource allocation or free blocks are exposed. This general idea widely used to manage continuous resources is extended to cover not only the resource dimension but also the time dimension. The list of blocks approach is evaluated in a Grid level and a local resource management system for a computing cluster. The extensive simulations showed a better runtime and higher reservation success rate compared with the currently favored approach of a slotted time and the more sophisticated approach based on AVL trees.     

A self-adaptive Model based on Multi-Objective Programming for Grid Resource Management

Abstract : The growing computational power requirements of grand challenge applications have positioned computational grid as promising next generation computing platform.However, resource management and application with varied requirements in grid environment continue to be a complex undertaking. In order to address complex resource management issues, we provide a self-adaptive model, which is based on multi-objective programming. The model make use of virtues of market mechanism efficiently, meanwhile, the shortcomings of market mechanism, such as too frequent fluctuations of price,are avoided by means of the method of changing prices after trading. Through using atom allocation of resource group, the cooperating allocation is improved, and some problems, such as deadlock of resource and inefficiently occupying resource, are solved. What' s more important, efficiently using various resources in grid system is guaranteed through importing multi-objective programming mechanism in our resource management solution. A frame of resource allocation is given at first, then, the mathematical model of the method is constructed. An algorithm is proposed to get the approximate solution in this paper.     

Design and implementation of an adaptive job allocation strategy for heterogeneous multi‐cluster computing systems

Cluster computing is an attractive approach to provide high‐performance computing for solving large‐scale applications. Owing to the advances in processor and networking technology, expanding clusters have resulted in the system heterogeneity; thus, it is crucial to dispatch jobs to heterogeneous computing resources for better resource utilization. In this paper, we propose a new job allocation system for heterogeneous multi‐cluster environments named the Adaptive Job Allocation Strategy (AJAS), in which a self‐scheduling scheme is applied in the scheduler to dispatch jobs to the most appropriate computing resources. Our strategy focuses on increasing resource utility by dispatching jobs to computing nodes with similar performance capacities. By doing so, execution times among all nodes can be equalized. The experimental results show that AJAS can improve the system performance. Copyright © 2011 John Wiley & Sons, Ltd.     

Resource allocation in decentralised computational systems: an evolutionary market-based approach

We present a novel market-based method, inspired by retail markets, for resource allocation in fully decentralised systems where agents are self-interested. Our market mechanism requires no coordinating node or complex negotiation. The stability of outcome allocations, those at equilibrium, is analysed and compared for three buyer behaviour models. In order to capture the interaction between self-interested agents, we propose the use of competitive coevolution. Our approach is both highly scalable and may be tuned to achieve specified outcome resource allocations. We demonstrate the behaviour of our approach in simulation, where evolutionary market agents act on behalf of service providing nodes to adaptively price their resources over time, in response to market conditions. We show that this leads the system to the predicted outcome resource allocation. Furthermore, the system remains stable in the presence of small changes in price, when buyers’ decision functions degrade gracefully.     

一种基于Bayes均衡的网格资源调度算法

针对经济网格提出了一种基于市场机制的资源分配方法:以Bayes均衡理论为基础,依靠市场机制,实现网格资源的优化分配。首先给出了资源交易的经济模型:双边叫价拍卖模型;接着给出了网格资源分配的市场模型:基于Bayse均衡的市场模型;最后引入了网格资源管理的调度流程和算法。     

Joint Quality Enhancement and Power Control for Wireless Visual Sensor Networks based on the Nash Bargaining Solution

We propose a cooperative method for resource allocation with power control in a multihop Direct Sequence Code Division Multiple Access Wireless Visual Sensor Network (WVSN). Typical multihop WVSNs consist of visual sensors that record different scenes and relay nodes that retransmit video data until the base station is reached. The error prone wireless environment contributes to the end-to-end video quality degradation. Moreover, the limited battery life span of the network nodes poses challenges on the management of power consumption. The different resource requirements of the WVSN nodes necessitate a quality-driven and power-aware resource allocation mechanism. We formulate the joint Quality Enhancement and Power Control problem based on a utility function that reflects both the benefit in terms of video quality and the cost in terms of transmission power. This function is employed by the Nash Bargaining Solution, which achieves higher fairness in terms of end-to-end video quality among all nodes. For the fairness assessment, a new metric is introduced. The experiments demonstrate the effectiveness of the proposed approach and explain the video quality-power consumption tradeoff as well as the resulting fairness-power consumption tradeoff.     

面向云环境的自适应集群调整方法

应用服务器集群是平台即服务(platform as a service,PaaS)模式的主要运行环境。针对云环境下动态变化的用户负载和共享、异构的集群环境,提出一种自适应集群调整方法,根据集群负载状况实现资源按需供给。该方法建立了PaaS平台的性能分析模型,并据此提出自适应的资源供给机制和负载均衡机制。实验结果表明,通过调节集群节点逻辑资源池的大小和集群节点数量,配合自适应负载均衡方法,达到了资源按需供给的目的。