Resource allocation optimization for quantitative service differentiation on server clusters

The goal of service differentiation is to provide different service quality levels to meet changing system configuration and resource availability and to satisfy different requirements and expectations of applications and users. In this paper, we investigate the problem of quantitative service differentiation on cluster-based delay-sensitive servers. The goal is to support a system-wide service quality optimization with respect to resource allocation on a computer system while provisioning proportionality fairness to clients. We first propose and promote a square-root proportional differentiation model. Interestingly, both popular delay factors, queueing delay and slowdown, are reciprocally proportional to the allocated resource usage. We formulate the problem of quantitative service differentiation as a generalized resource allocation optimization towards the minimization of system delay, defined as the sum of weighted delay of client requests. We prove that the optimization-based resource allocation scheme essentially provides square-root proportional service differentiation to clients. We then study the problem of service differentiation provisioning from an important relative performance metric, slowdown. We give a closed-form expression of the expected slowdown of a popular heavy-tailed workload model with respect to resource allocation on a server cluster. We design a two-tier resource management framework, which integrates a dispatcher-based node partitioning scheme and a server-based adaptive process allocation scheme. We evaluate the resource allocation framework with different models via extensive simulations. Results show that the square-root proportional model provides service differentiation at a minimum cost of system delay. The two-tier resource allocation framework can provide fine-grained and predictable service differentiation on cluster-based servers.     

Harmonic proportional bandwidth allocation and scheduling for service differentiation on streaming servers

To provide ubiquitous access to the proliferating rich media on the Internet, scalable streaming servers must be able to provide differentiated services to various client requests. Recent advances of transcoding technology make network-I/O bandwidth usages at the server communication ports controllable by request schedulers on the fly. In this article, we propose a transcoding-enabled bandwidth allocation scheme for service differentiation on streaming servers. It aims to deliver high bit rate streams to high priority request classes without overcompromising low priority request classes. We investigate the problem of providing differentiated streaming services at application level in two aspects: stream bandwidth allocation and request scheduling. We formulate the bandwidth allocation problem as an optimization of a harmonic utility function of the stream quality factors and derive the optimal streaming bit rates for requests of different classes under various server load conditions. We prove that the optimal allocation, referred to as harmonic proportional allocation, not only maximizes the system utility function, but also guarantees proportional fair sharing between classes with different prespecified differentiation weights. We evaluate the allocation scheme, in combination with two popular request scheduling approaches, via extensive simulations and compare it with an absolute differentiation strategy and a proportional-share strategy tailored from relative differentiation in networking. Simulation results show that the harmonic proportional allocation scheme can meet the objective of relative differentiation in both short and long timescales and greatly enhance the service availability and maintain low queueing delay when the streaming system is highly loaded.     

Performance evaluation of e-commerce requests in wireless cellular networks

Recent technological advances in mobile devices and wireless networks enable mobile users to order goods in an anywhere and anytime fashion. Quality of Service (QoS) provision is one of the most challenging issues in the heterogeneous wireless network-based e-commerce systems. Such e-commerce systems enable users to roam between different wireless networks operators and geographical areas while providing interactive broadband services and seamless connectivity. Due to movement of users during e-commerce requests, one of the most important QoS factors for successful completion of users' requests is related to handover of request from one cell to another. A handover could fail due to unavailability of sufficient bandwidth in the destination cell. Such failure of ongoing e-commerce requests is highly undesirable and can cause serious problems to the e-commerce users and the service providers. This paper proposes an enhanced priority queuing based handover scheme in order to ensure a seamless connectivity of e-commerce requests. It focuses on the performance anaylsis of the proposed scheme. Experimental study demonstrates that the proposed scheme provides QoS with low connection failure and mean response time for handover of e-commerce requests.     

异构Web集群中的比例伸展因子区分服务

Web集群服务器已被广泛用来提高Web服务器的性能。如何保证Web服务的服务质量（QoS）是一个迫切需要解决的问题。区分服务已成为QoS研究领域中的一个焦点。本文分析了Web请求服务时间的数字特性,并使用M／G／1 FCFS排队模型对Web服务器及Web集群服务器进行了建模。在对模型进行分析的基础上,设计并实现了一种异构Web集群服务器中的比例伸展因子区分服务的方案,并提出了基于概率空间的请求调度算法。请求被分成多个类,无论系统的负载如何,系统确保每类请求的平均伸展因子与事先给定的参数成正比。实际测试表明,所设计的方案满足相对区分服务的可预测性和可控制性的要求。     

分配器上支持QoS的设计与实现

为了让Web集群服务器支持QoS，在分配器上实现了一些QoS的机制，包括区分服务、性能隔离、服务器动态划分、接纳控制和内容自适应。对于高优先级的请求，系统确保其服务质量满足事先商定的服务质量协议；对于低优先级的请求，系统提供尽力而为的服务。尤其，当服务器重载时，分配器不是简单地靠丢弃请求，而是采用内容自适应机制来防止服务器过载。实际测试表明，系统达到了所有的设计要求。     

基于QoS延时参数的服务接纳控制算法

为给并发访问的多个请求提供QoS保证,Web Service必须借助某些算法对请求加以控制。该文分析QoS延时参数的构成,提出基于QoS延时参数的服务接纳控制算法。该算法可在满足既有服务的QoS延时要求的条件下,控制接纳新进入的服务。否则即让其在队列中等待直到被服务器接纳,或服务器直接抛弃这个请求,由客户端在超时后自动重发请求。     

QoS-Broker for transactional workloads

Typical request processing systems, such as web servers and database servers, try to accommodate all requests as fast as possible, which can be described as a Best-Effort approach. However, different application items may have different quality-of-service (QoS) requirements, and this can be viewed as an orthogonal concern to the basic system functionality. In this paper we propose the QoS-Broker, a middleware for delivering QoS over servers and applications. We show its architecture to support contracts over varied targets including queries, transactions, services or sessions, also allowing expressions on variables to be specified in those targets. We also discuss how the QoS-Broker implements basic strategies for QoS over workloads. Our experimental results illustrate the middleware by applying priority and weighted- fair-queuing based differentiation over clients and over transactions, and also admission control, using a benchmark as a case-study.     

基于QoS的磁盘调度策略

随着视频点播、视频会议、视频监控、数字图书馆等流媒体应用的普及，流媒体服务器存储资源管理成为制约服务质量的瓶颈之一。根据多媒体服务器的性能要求，提出了一种支持QoS的磁盘调度策略。它由三个主要部分组成：探测模块、负载监测模块和自适应管理模块。探测模块，负责判断当前的资源情况能否满足服务请求；自适应模块，根据负载监删模块检测到的负载变化情况，动态调整服务周期在实时请求和尽力服务请求之间的分配。实验表明此磁盘调度策略能在保证实时请求无抖动执行的同时，明显减少了非实时请求的响应时间。     

VoD系统中基于优先级的准入控制和带宽分配策略

本文针对VoD系统中不同客户对视频服务质量的不同要求,提出一种基于优先级的准入控制和带宽动态分配策略。在准入控制时,综合考虑请求的优先级和并发流占用的实际带宽等因素,在保证为高优先级请求预留较多固定带宽的同时提高并发流个数;在服务过程中,根据优先级和网络状况动态调整每个流的带宽,使丢包率低于一定的阈值,并保证在相同的网络状态下为高优先级请求提供较高的视频服务质量。     

基于区分Web QoS的负载均衡集群模型

随着电子商务的应用逐步深入,用户访问量的激增且服务请求多样．如何实现对所有请求的快速响应是当前解决的问题．针对此问题,本文提出采用基于区分WebQoS的负载均衡技术．建立了基于区分WebQoS的负载均衡的集群模型．根据请求类型和用户权限划分服务等级,高服务的请求具有高优先调度权,在集群当中通过动态反馈技术均衡调度到某个节点,从而达到区分WebQoS服务的目的,同时也保证集群服务器的负载均衡．该模型在网络环境及硬件环境相同的条件下,与常用的负载均衡技术进行了比较,实验结果证明本文提出的方法效果显著．     

Automatic QoS control

User sessions, usually consisting of sequences of consecutive requests from customers, comprise most of an e-commerce site's workload. These requests execute e-business functions such as browse, search, register, login, add to shopping cart, and pay. Once we properly understand and characterize a workload, we must assess its effect on the site's quality of service (QoS), which is defined in terms of response time, throughput, the probability that requests will be rejected, and availability. We can assess an e-commerce site's QoS in many different ways. One approach is by measuring the site's performance, which we can determine from a production site using a real workload or from a test site using a synthetic workload (as in load testing). Another approach consists of using performance models. I look at the approach my colleagues at George Mason and I took that uses performance models in the design and implementation of automatic QoS controller for e-commerce sites.     

支持QoS确保的存储服务器服务资源分配算法

在电子商务、工业制造领域中,具有海量存储与高I/O吞吐能力的数据存储服务器得到了广泛应用。数据存储服务器提供QoS支持对应用需求而言很重要,但是实践中却很少有数据存储服务器提供了QoS支持。提出了一种支持服务质量区分并对高优先级用户提供服务质量保证的存储服务区服务资源分配算法,由于对高优先级用户请求与低优先级用户请求采用了可变服务质量区分因子,因而算法在提供服务优先级区分与确保高优先级请求服务质量的同时,减小了低优先级请求的请求丢弃率,同时最大化了服务资源利用率。     

Comparison of scheduling schemes for on-demand IaaS requests

Infrastructure-as-a-service (IaaS) is one of emerging powerful cloud computing services provided by IT industry at present. This paper considers the interaction aspects between on-demand requests and the allocation of virtual machines in a server farm operated by a specific infrastructure owner. We formulate an analytic performance model of the server farm taking into account the quality of service (QoS) guaranteed to users and the operational energy consumption in the server farm. We compare several scheduling algorithms from the aspect of the average energy consumption and heat emission of servers as well as the blocking probabilities of on-demand requests. Based on numerical results of a comparison of different allocation strategies, a saving on the energy consumption is possible in the operational range (where on-demand requests do not face unpleasant blocking probability) with the allocation of virtual machines to physical servers based on the priority.     

RDRP: Reward-Driven Request Prioritization for e-Commerce web sites

Meeting client Quality-of-Service (QoS) expectations proves to be a difficult task for the providers of e-Commerce services, especially when web servers experience overload conditions, which cause increased response times and request rejections, leading to user frustration, lowered usage of the service and reduced revenues. In this paper, we propose a server-side request scheduling mechanism that addresses these problems. Our Reward-Driven Request Prioritization (RDRP) algorithm gives higher execution priority to client web sessions that are likely to bring more service profit (or any other application-specific reward). The method works by predicting future session structure by comparing its requests seen so far with aggregated information about recent client behavior, and using these predictions to preferentially allocate web server resources. Our experiments using the TPC-W benchmark application with an implementation of the RDRP techniques in the JBoss web application server show that RDRP can significantly boost profit attained by the service, while providing better QoS to clients that bring more profit.     

Dynamic Scheme Transition Adaptable to Variable Video Popularity in a Digital Broadcast Network

To provide on-demand video streaming services through the network, video objects with both high and low client request rates must be served efficiently. In this paper, we propose a dynamic scheme transition to provide on-demand streaming services efficiently regardless of video popularity. This approach can maintain quality-of-service (QoS) by transitioning the service scheme according to the request rate. The server provides services by heuristically broadcasting video segments when the video popularity is low and by a Periodic Broadcast when the video popularity is high. The server identifies the variations in client request rates from the number of service channels and determines transitions to more efficient service schemes autonomously. We evaluated this scheme transition for various parameters and traces from a company providing streaming services. The results show that the performance of scheme transition is very efficient.      

Web服务器集群系统QoWS研究

请求负载的增加常常导致Web服务器系统性能降低,用户期望的服务质量得不到保证,这是服务级Web系统所面临和必须解决的问题。文章提出了一种Web服务器集群环境下的负载分配策略,通过对用户请求分类、将不同类别的请求进行响应性能隔离、优先为高级别请求提供服务以及请求许可控制等手段,对不同类别的Web请求提供不同质量的服务,保证了服务级用户的服务质量。同时采用最迟分配原则,改善系统的负载均衡能力,缩短系统平均响应时间。最后通过仿真实验,验证了该策略的正确性和有效性。     

利用RSVP隧道高效实现INTERNET的服务质量请求

在ＩＮＴＥＲＮＥＴ上应用ＲＳＶＰ时,新增通信量和时空开销问题是ＲＳＶＰ研究的关键问题之一。文中提出了利用ＲＳＶＰ隧道和ＱｏＳ请求聚合体解决此问题的方案,核心内容是：ＱｏＳ请求聚合体代表多个具有指定特征的ＲＳＶＰ会话消息,隧道段的路由器的ＲＳＶＰ模块对聚合体会话消息设置软状态而不理会构成聚合体的单个会话消息、构成聚合体的每个一ＲＳＶＰ会话消息被封装后以普通数据包形式通过隧道段。     

Web server QoS models: applying scheduling rules from production planning

Most web servers, in practical use, use a queuing policy based on the Best Effort model, which employs the first-in-first-out (FIFO) scheduling rule to prioritize web requests in a single queue. This model does not provide Quality of Service (QoS). In the Differentiated Services (DiffServ) model, separate queues are introduced to differentiate QoS for separate web requests with different priorities. This paper presents web server QoS models that use a single queue, along with scheduling rules from production planning in the manufacturing domain, to differentiate QoS for classes of web service requests with different priorities. These scheduling rules are Weighted Shortest Processing Time (WSPT), Apparent Tardiness Cost (ATC), and Earliest Due Date. We conduct simulation experiments and compare the QoS performance of these scheduling rules with the FIFO scheme used in the basic Best Effort model with only one queue, and the basic DiffServ model with two separate queues. Simulation results demonstrate better QoS performance using WSPT and ATC, especially when requested services exceed the capacity of a web server.     

Power and performance control of soft real-time web server clusters

This paper presents a novel way to control power consumption and performance in a multi-tier server cluster designed for e-commerce applications. The requests submitted to these server systems have a soft real-time constraint, given that although some can miss a pre-defined deadline, the system can still meet an agreed upon performance level. Clusters of servers are extensively used nowadays and, with the steep increase in the total power consumption in these systems, economic and environmental problems have been raised. We present ways of decreasing power expenditure, and show the implementation of a SISO (Single Input Single Output) controller that acts on the speed of all server nodes capable of dynamic voltage and frequency scaling (DVFS), with QoS (Quality of Service) being the reference setpoint. For QoS, we use the request tardiness, defined as the ratio of the end-to-end response time to the deadline, rather than the usual metric that counts missed deadlines. We adjust the servers operating frequencies to guarantee that a pre-defined p-quantile of the tardiness probability distribution of the requests meet their deadlines. Doing so we can guarantee that the QoS will be statistically p. We test this technique in a prototype multi-tier cluster, using open software, commodity hardware, and a standardized e-commerce application to generate a workload close to that of the real world. The main contribution of this paper is to empirically show the robustness of the SISO controller, presenting a sensibility analysis of its parameters. Experimental results show that our implementation outperforms other published state-of-the-art cluster implementations.     

Regression-based resource provisioning for session slowdown guarantee in multi-tier Internet servers

Autonomous management of a multi-tier Internet service involves two critical and challenging tasks, one understanding its dynamic behaviors when subjected to dynamic workloads and second adaptive management of its resources to achieve performance guarantees. We propose a statistical machine learning based approach to achieve session slowdown guarantees of a multi-tier Internet service. Session slowdown is the relative ratio of a session’s total queueing delay to its total processing time. It is a compelling performance metric of session-based online transactions because it directly measures user-perceived relative performance and it is independent of the session length. However, there is no analytical model for session slowdown on multi-tier servers. We first conduct training to learn the statistical regression models that quantitatively capture an Internet service’s dynamic behaviors as relationships between various service parameters. Then, we propose a dynamic resource provisioning approach that utilizes the learned regression models to efficiently achieve session slowdown guarantee under dynamic workloads. The approach is based on the combination of offline training and online monitoring of the Internet service behavior. Simulations using the industry standard TPC-W benchmark demonstrate the effectiveness and efficiency of the regression based resource provisioning approach for session slowdown oriented performance guarantee of a multi-tier e-commerce application.