Regression-based resource provisioning for session slowdown guarantee in multi-tier Internet servers

Autonomous management of a multi-tier Internet service involves two critical and challenging tasks, one understanding its dynamic behaviors when subjected to dynamic workloads and second adaptive management of its resources to achieve performance guarantees. We propose a statistical machine learning based approach to achieve session slowdown guarantees of a multi-tier Internet service. Session slowdown is the relative ratio of a session’s total queueing delay to its total processing time. It is a compelling performance metric of session-based online transactions because it directly measures user-perceived relative performance and it is independent of the session length. However, there is no analytical model for session slowdown on multi-tier servers. We first conduct training to learn the statistical regression models that quantitatively capture an Internet service’s dynamic behaviors as relationships between various service parameters. Then, we propose a dynamic resource provisioning approach that utilizes the learned regression models to efficiently achieve session slowdown guarantee under dynamic workloads. The approach is based on the combination of offline training and online monitoring of the Internet service behavior. Simulations using the industry standard TPC-W benchmark demonstrate the effectiveness and efficiency of the regression based resource provisioning approach for session slowdown oriented performance guarantee of a multi-tier e-commerce application.     

Autonomic Performance and Power Control on Virtualized Servers: Survey,Practices, and Trends

Modern datacenter servers hosting popular Internet services face significant and multi-facet challenges in performance and power control. The user-perceived performance is the result of a complex interaction of complex workloads in a very complex underlying system. Highly dynamic and bursty workloads of Internet services fluctuate over multiple time scales, which has a significant impact on processing and power demands of datacenter servers. High-density servers apply virtualization technology for capacity planning and system manageability. Such virtuMized computer systems are increasingly large and complex. This paper surveys representative approaches to autonomic performance and power control on virtualized servers, which control the quality of service provided by virtualized resources, improve the energy efficiency of the underlying system, and reduce the burden of complex system management from human operators. It then presents three designed self-adaptive resource management techniques based on machine learning and control for percentile-based response time assurance, non-intrusive energy-efficient performance isolation, and joint performance and power guarantee on virtualized servers. The techniques were implemented and evaluated in a testbed of virtualized servers hosting benchmark applications. Finally, two research trends are identified and discussed for sustainable cloud computing in green datacenters.     

URL: A unified reinforcement learning approach for autonomic cloud management

Cloud computing is emerging as an increasingly important service-oriented computing paradigm. Management is a key to providing accurate service availability and performance data, as well as enabling real-time provisioning that automatically provides the capacity needed to meet service demands. In this paper, we present a unified reinforcement learning approach, namely URL, to automate the configuration processes of virtualized machines and appliances running in the virtual machines. The approach lends itself to the application of real-time autoconfiguration of clouds. It also makes it possible to adapt the VM resource budget and appliance parameter settings to the cloud dynamics and the changing workload to provide service quality assurance. In particular, the approach has the flexibility to make a good trade-off between system-wide utilization objectives and appliance-specific SLA optimization goals. Experimental results on Xen VMs with various workloads demonstrate the effectiveness of the approach. It can drive the system into an optimal or near-optimal configuration setting in a few trial-and-error iterations.     

PAPMSC: Power-Aware Performance Management Approach for Virtualized Web Servers via Stochastic Control

As green computing is becoming a popular computing paradigm, the performance of energy-efficient data center becomes increasingly important. This paper proposes power-aware performance management via stochastic control method (PAPMSC), a novel stochastic control approach for virtualized web servers. It addresses the instability and inefficiency issues due to dynamic web workloads. It features a coordinated control architecture that optimizes the resource allocation and minimizes the overall power consumption while guaranteeing the service level agreements (SLAs). More specifically, due to the interference effect among the co-located virtualized web servers and time-varying workloads, the relationship between the hardware resource assignment to different virtual servers and the web applications’ performance is considered as a coupled Multi-Input-Multi-Output (MIMO) system and formulated as a robust optimization problem. We propose a constrained stochastic linear-quadratic controller (cSLQC) to solve the problem by minimizing the quadratic cost function subject to constraints on resource allocation and applications’ performance. Furthermore, a proportional controller is integrated to enhance system stability. In the second layer, we dynamically manipulate the physical frequency for power efficiency using an adaptive linear quadratic regulator (ALQR). Experiments on our testbed server with a variety of workload patterns demonstrate that the proposed control solution significantly outperforms existing solutions in terms of effectiveness and robustness.     

VM3: Measuring,modeling and managing VM shared resources

With cloud and utility computing models gaining significant momentum, data centers are increasingly employing virtualization and consolidation as a means to support a large number of disparate applications running simultaneously on a chip-multiprocessor (CMP) server. In such environments, contention for shared platform resources (CPU cores, shared cache space, shared memory bandwidth, etc.) can have a significant effect on each virtual machine’s performance. In this paper, we investigate the shared resource contention problem for virtual machines by: (a) measuring the effects of shared platform resources on virtual machine performance, (b) proposing a model for estimating shared resource contention effects, and (c) proposing a transition from a virtual machine (VM) to a virtual platform architecture (VPA) that enables transparent shared resource management through architectural mechanisms for monitoring and enforcement. Our measurement and modeling experiments are based on a consolidation benchmark (vConsolidate) running on a state-of-the-art CMP server. Our virtual platform architecture experiments are based on detailed simulations of consolidation scenarios. Through detailed measurements and simulations, we show that shared resource contention affects virtual machine performance significantly and emphasize that virtual platform architectures is a must for future virtualized datacenters.     

视频点播服务器的服务质量控制框架

视频点播服务器工作于开放的网络环境中,系统负载难以预测,其运行时资源瓶颈依赖于实时服务类型.基于性能模型和实时负荷估计方法,本文提出视频点播服务器的服务质量控制框架,提供系统负荷监控、准入控制和区分服务等三种服务质量控制.实际系统的验证实验表明,本文的方法可以有效的对系统负荷进行监控,确保系统负荷稳定在服务提供商希望的水平线以下,向不同级别的用户提供不同质量级别的服务.     

基于自适应控制方法的云计算服务器性能管理

随着云计算技术的广泛使用,如何对采用虚拟化技术的云计算服务器的性能进行有效管理,是云计算研究的热点问题之一.论文提出了一种基于自适应控制理论的动态资源控制策略（DRC）,该控制策略在保证服务级别协议的前提下,对运行在服务器上的各个虚拟机进行优化配置,使服务器的硬件资源得到最大化的利用.同时设计了一种新型的自适应线性二次高斯控制器,来应对具有Web应用所面对的动态负载.在基于Xen技术搭建的实验平台上,对服务器的性能在不同工作负载的情况下进行了测试,并与未采用DRC策略的服务器性能进行了对比.实验结果表明,在动态工作负载下,与为采用DRC策略的服务器相比,DRC控制策略能够有效保证不同Web应用的响应时间稳定在设定的参考值.     

Towards Automated Provisioning and Emergency Handling in Renewable Energy Powered Datacenters

Designing eco-friendly system has been at the forefront of computing research. Faced with a growing concern about the server energy expenditure and the climate change, both industry and academia start to show high interest in computing systems powered by renewable energy sources. Existing proposals on this issue mainly focus on optimizing resource utilization or workload performance. The key supporting hardware structures for cross-layer power management and emergency handling mechanisms are often left unexplored. This paper presents GreenPod, a research framework for exploring scalable and dependable renewable power management in datacenters. An important feature of GreenPod is that it enables joint management of server power supplies and virtualized server workloads. Its interactive communication portal between servers and power supplies allows dataeenter operators to perform real-time renewable energy driven load migration and power emergency handling. Based on our system prototype, we discuss an important topic： virtual machine （VM） workloads survival when facing extended utility outage and insufficient onsite renewable power budget. We show that whether a VM can survive depends on the operating frequencies and workload characteristics. The proposed framework can greatly encourage and facilitate innovative research in dependable green computing.     

面向应用服务级目标的虚拟化资源管理

在虚拟环境中实现应用服务级目标,是当前数据中心系统管理的关键问题之一.解决该问题有两个方面的要求:一方面,在虚拟化层次和范围内,能够动态和分布式地按需调整虚拟机资源分配;另一方面,在虚拟化范围之外,能够控制由于虚拟机对非虚拟化资源的竞争所导致的性能干扰,实现虚拟机性能隔离.然而,已有工作不适用于虚拟化数据中心场景.提出一种面向应用服务级目标的虚拟化资源管理方法.首先,该方法基于反馈控制理论,通过动态调整虚拟机资源分配来实现每个应用的服务器目标;同时,还设计了一个两层结构的自适应机制,使得应用模型能够动态地捕捉虚拟机资源分配与应用性能的时变非线性关系;最后,该方法通过仲裁不同应用的资源分配请求来控制虚拟机在非虚拟化资源上的竞争干扰.实验在基于Xen的机群环境中检验了该方法在RUBiS系统和TPC-W基准上的效果.实验结果显示,该方法的应用服务级目标实现率比两种对比方法平均高29.2%,而应用服务级目标平均偏离率比它们平均低50.1%.另一方面,当RUBiS系统和TPC-W基准竞争非虚拟化的磁盘I/O资源时,该方法通过抑制TPC-W基准28.7%的处理器资源需求来优先满足RUBiS系统的磁盘I/O需求.     

一种基于节的视频服务器允许接纳控制算法

提出了一种基于“服务器节”的支持压缩多媒体流的服务器中CPU、磁盘、网络和内存等资源管理的方法和允许接纳控制算法。“服务器节”概念定义了一组客户视频服务特性,如播放、快进、慢进和暂停等,并且确定了视频服务所需资源的分配参。一个“服务器节”包括视频服务器、磁盘设备、网络设备和允许接纳控制。它不但能优化使用单个资源,对于给定系统支持最大数量的客户端,保证其服务质量（QoS),而且其允许接纳控制算法能根据系统所有资源的状况,在不影响原有的视频服务基础上,确定对客户端新提出的视频服务是否接受。     

Designing an overload control strategy for secure e-commerce applications

Uncontrolled overload can lead e-commerce applications to considerable revenue losses. For this reason, overload prevention in these applications is a critical issue. In this paper we present a complete characterization of secure e-commerce applications scalability to determine which are the bottlenecks in their performance that must be considered for an overload control strategy. With this information, we design an adaptive session-based overload control strategy based on SSL (Secure Socket Layer) connection differentiation and admission control. The SSL connection differentiation is a key factor because the cost of establishing a new SSL connection is much greater than establishing a resumed SSL connection (it reuses an existing SSL session on the server). Considering this big difference, we have implemented an admission control algorithm that prioritizes resumed SSL connections to maximize the performance in session-based environments and dynamically limits the number of new SSL connections accepted, according to the available resources and the current number of connections in the system, in order to avoid server overload. Our evaluation on a Tomcat server demonstrates the benefit of our proposal for preventing server overload.     

Resource pool management: Reactive versus proactive or let’s be friends

The consolidation of multiple workloads and servers enables the efficient use of server and power resources in shared resource pools. We employ a trace-based workload placement controller that uses historical information to periodically and proactively reassign workloads to servers subject to their quality of service objectives. A reactive migration controller is introduced that detects server overload and underload conditions. It initiates the migration of workloads when the demand for resources exceeds supply. Furthermore, it dynamically adds and removes servers to maintain a balance of supply and demand for capacity while minimizing power usage. A host load simulation environment is used to evaluate several different management policies for the controllers in a time effective manner. A case study involving three months of data for 138 SAP applications compares three integrated controller approaches with the use of each controller separately. The study considers trade-offs between: (i) required capacity and power usage, (ii) resource access quality of service for CPU and memory resources, and (iii) the number of migrations. Our study sheds light on the question of whether a reactive controller or proactive workload placement controller alone is adequate for resource pool management. The results show that the most tightly integrated controller approach offers the best results in terms of capacity and quality but requires more migrations per hour than the other strategies.     

New scoring formula to rank hypervisors’ performance complementing with statistical analysis using DOE

Hypervisors enable cloud computing model to provide scalable infrastructures and on-demand access to computing resources as they support multiple operating systems to run on one physical server concurrently. This mechanism enhances utilization of physical server thus reduces server count in the data center. Hypervisors also drive the benefits of reduced IT infrastructure setup and maintenance costs along with power savings. It is interesting to know different hypervisors’ performance for the consolidated application workloads. Three hypervisors ESXi, XenServer, and KVM are carefully chosen to represent three categories full virtualized, para-virtualized, and hybrid virtualized respectively for the experiment. We have created a private cloud using CloudStack. Hypervisors are deployed as hosts in the private cloud in the respective clusters. Each hypervisor is deployed with three virtual machines. Three applications web server, application server, and database servers are installed on three virtual machines. Experiments are designed using Design of Experiments (DOE) methodology. With concurrently running virtual machines, each hypervisor is stressed with the consolidated real-time workloads (web load, application load, and OLTP) and important system information is gathered using SIGAR framework. The paper proposes a new scoring formula for hypervisors’ performance in the private cloud for consolidated application workloads and the accuracy of the results are complemented with sound statistical analysis using DOE.     

Joint computation offloading and resource provisioning for edge-cloud computing environment: A machine learning-based approach

In recent years, the usage of smart mobile applications to facilitate day-to-day activities in various domains for enhancing the quality of human life has increased widely. With rapid developments of smart mobile applications, the edge computing paradigm has emerged as a distributed computing solution to support serving these applications closer to mobile devices. Since the submitted workloads to the smart mobile applications changes over the time, decision making about offloading and edge server provisioning to handle the dynamic workloads of mobile applications is one of the challenging issues into the resource management scope. In this work, we utilized learning automata as a decision-maker to offload the incoming dynamic workloads into the edge or cloud servers. In addition, we propose an edge server provisioning approach using long short-term memory model to estimate the future workload and reinforcement learning technique to make an appropriate scaling decision. The simulation results obtained under real and synthetic workloads demonstrate that the proposed solution increases the CPU utilization and reduces the execution time and energy consumption, compared with the other algorithms.     

An efficient job management of computing service using integrated idle VM resources for high-performance computing based on OpenStack

In recent years, various studies on OpenStack-based high-performance computing have been conducted. OpenStack combines off-the-shelf physical computing devices and creates a resource pool of logical computing. The configuration of the logical computing resource pool provides computing infrastructure according to the user’s request and can be applied to the infrastructure as a service (laaS), which is a cloud computing service model. The OpenStack-based cloud computing can provide various computing services for users using a virtual machine (VM). However, intensive computing service requests from a large number of users during large-scale computing jobs may delay the job execution. Moreover, idle VM resources may occur and computing resources are wasted if users do not employ the cloud computing resources. To resolve the computing job delay and waste of computing resources, a variety of studies are required including computing task allocation, job scheduling, utilization of idle VM resource, and improvements in overall job’s execution speed according to the increase in computing service requests. Thus, this paper proposes an efficient job management of computing service (EJM-CS) by which idle VM resources are utilized in OpenStack and user’s computing services are processed in a distributed manner. EJM-CS logically integrates idle VM resources, which have different performances, for computing services. EJM-CS improves resource wastes by utilizing idle VM resources. EJM-CS takes multiple computing services rather than single computing service into consideration. EJM-CS determines the job execution order considering workloads and waiting time according to job priority of computing service requester and computing service type, thereby providing improved performance of overall job execution when computing service requests increase.

     

SLA based resource allocation policies in autonomic environments

Nowadays, large service centers provide computational capacity to many customers by sharing a pool of IT resources. The service providers and their customers negotiate utility based Service Level Agreement (SLA) to determine the costs and penalties on the base of the achieved performance level. The system is often based on a multi-tier architecture to serve requests and autonomic techniques have been implemented to manage varying workload conditions. The service provider would like to maximize the SLA revenues, while minimizing its operating costs. The system we consider is based on a centralized network dispatcher which controls the allocation of applications to servers, the request volumes at various servers and the scheduling policy at each server. The dispatcher can also decide to turn ON or OFF servers depending on the system load. This paper designs a resource allocation scheduler for such multi-tier autonomic environments so as to maximize the profits associated with multiple class SLAs. The overall problem is NP-hard. We develop heuristic solutions by implementing a local-search algorithm. Experimental results are presented to demonstrate the benefits of our approach.     

Dynamic real-time scheduling strategies for interactive continuous media servers

In this paper, we propose and study a dynamic approach to schedule real-time requests in a video-on-demand (VOD) server. Providing quality of service in such servers requires uninterrupted and on-time retrieval of motion video data. VOD services and multimedia applications further require access to the storage devices to be shared among multiple concurrent streams. Most of the previous VOD scheduling approaches use limited run-time,0 information and thus cannot exploit the potential capacity of the system fully. Our approach improves throughput by making use of run-time information to relax admission control. It maintains excellent quality of service under varying playout rates by observing deadlines and by reallocating resources to guarantee continuous service. It also reduces start-up latency by beginning service as soon as it is detected that deadlines of all real-time requests will be met. We establish safe conditions for greedy admission, dynamic control of disk read sizes, fast initial service, and sporadic services. We conduct thorough simulations over a wide range of buffer capacities, load settings, and over varying playout rates to demonstrate the significant improvements in quality of service, throughput and start-up latency of our approach relative to a static approach.     

Dynamic CPU provisioning for self-managed secure web applications in SMP hosting platforms

Overload control mechanisms such as admission control and connection differentiation have proven effective for preventing overload of application servers running secure web applications. However, achieving optimal results in overload prevention is only possible when some kind of resource management is considered in addition to these mechanisms.In this paper we propose an overload control strategy for secure web applications that brings together dynamic provisioning of platform resources and admission control based on secure socket layer (SSL) connection differentiation. Dynamic provisioning enables additional resources to be allocated to an application on demand to handle workload increases, while the admission control mechanism avoids the server’s performance degradation by dynamically limiting the number of new SSL connections accepted and preferentially serving resumed SSL connections (to maximize performance on session-based environments) while additional resources are being provisioned.Our evaluation demonstrates the benefit of our proposal for efficiently managing the resources and preventing server overload on a 4-way multiprocessor Linux hosting platform, especially when the hosting platform is fully overloaded.     

A low-overhead networking mechanism for virtualized high-performance computing systems

The use of virtualized parallel and distributed computing systems is rapidly becoming the mainstream due to the significant benefit of high energy-efficiency and low management cost. Processing network operations in a virtual machine, however, incurs a lot of overhead from the arbitration of network devices between virtual machines, inherently by the nature of the virtualized architecture. Since data transfer between server nodes frequently occurs in parallel and distributed computing systems, the high overhead of networking may induce significant performance loss in the overall system. This paper introduces the design and implementation of a novel networking mechanism with low overhead for virtualized server nodes. By sacrificing isolation between virtual machines, which is insignificant in distributed or parallel computing systems, our approach significantly reduces the processing overhead in networking operations by up to 29% of processor load, along with up to 36% of processor cache miss. Furthermore, it improves network bandwidth by up to 8%, especially when transmitting large packets. As a result, our prototype enhances the performance of real-world workloads by up to 12% in our evaluation.     

Performance evaluation of Web server workloads in Xen‐based virtualized computer system: analytical modeling and experimental validation

Recent developments in the field of virtualization technologies have led to renewed interest in performance evaluation of these systems. Nowadays, maturity of virtualization technology has made a fuss of provisioning IT services to maximize profits, scalability and QoS. This pioneer solution facilitates deployment of datacenter applications and grid and Cloud computing services; however, there are challenges. It is necessary to investigate a trade‐off among overall system performance and revenue and to ensure service‐level agreement of submitted workloads. Although a growing body of literature has investigated virtualization overhead and virtual machines interference, there is still lack of accurate performance evaluation of virtualized systems. In this paper, we present in‐depth performance measurements to evaluate a Xen‐based virtualized Web server. Regarding this experimental study; we support our approach by queuing network modeling. Based on these quantitative and qualitative analyses, we present the results that are important for performance evaluation of consolidated workloads on Xen hypervisor. First, demands of both CPU intensive and disk intensive workloads on CPU and disk are independent from the submitted rate to unprivileged domain when dedicated core(s) are pinned to virtual machines. Second, request response time not only depends on processing time at unprivileged domain but also pertains to amount of flipped pages at Domain 0. Finally, results show that the proposed modeling methodology performs well to predict the QoS parameters in both para‐virtualized and hardware virtual machine modes by knowing the request content size. Copyright © 2015 John Wiley & Sons, Ltd.