Online machine learning for auto-scaling in the edge computing

The evolution of edge computing devices has enabled machine intelligence techniques to process data close to its producers (the sensors) and end-users. Although edge devices are usually resource-constrained, the distribution of processing services among several nodes enables a processing capacity similar to cloud environments. However, the edge computing environment is highly dynamic, impacting the availability of nodes in the distributed system. In addition, the processing workload for each node can change constantly. Thus, the scaling of processing services needs to be rapidly adjusted, avoiding bottlenecks or wasted resources while meeting the applications’ QoS requirements. This paper presents an auto-scaling subsystem for container-based processing services using online machine learning. The auto-scaling follows the MAPE-K control loop to dynamically adjust the number of containers in response to workload changes. We designed the approach for scenarios where the number of processing requests is unknown beforehand. We developed a hybrid auto-scaling mechanism that behaves reactively while a prediction online machine learning model is continuously trained. When the prediction model reaches a desirable performance, the auto-scaling acts proactively, using predictions to anticipate scaling actions. An experimental evaluation has demonstrated the feasibility of the architecture. Our solution achieved fewer service level agreement (SLA) violations and scaling operations to meet demand than purely reactive and no scaling approaches using an actual application workload. Also, our solution wasted fewer resources compared to the other techniques.     

一种面向云多层应用的费用高效的资源管理方法

由于部署方便、维护简单并且不需要搭建自己的私有机房,云数据中心正成为大多数互联网公司 尤其是初创公司和中小规模公司 部署应用程序的首选。在以基础设施为服务的云环境里,互联网公司可以根据应用程序的需要动态租赁云基础设施,从而节省预算开支,并保证应用性能。然而,在现有的业界实践中,云服务提供商提供的负载均衡和资源伸缩服务只能监控虚拟机的使用状态,并不能监控应用程序的运行状态,因此无法准确根据应用程序的服务需求自适应变换资源规模。同时,现有的文献和实践中,也很少有 研究从云基础设施使用者的角度出发,为使用者节省基础设施租赁费用或高效使用已租赁基础设施。据此提出了一种面向基础设施云环境下多层应用的费用高效的资源管理方法,其在降低用户费用的同时,还能充分利用所花费用提高应用程序性能。通过仿真对所提方法业界实际使用的方法 进行比较,结果表明所提方法不仅能够提高应用程序的服务质量和服务性能,也能较大地降低公司在基础设施租赁方面的费用。     

Non-Intrusive Elastic Query Processing in the Cloud

Cloud computing is a very promising paradigm of service-oriented computing. One major benefit of cloud computing is its elasticity, i.e., the system＇s capacity to provide and remove resources automatically at runtime. For that, it is essential to design and implement an efficient and effective technique that takes full advantage of the system＇s potential flexibility. This paper presents a non-intrusive approach that monitors the performance of relational database management systems in a cloud infrastructure, and automatically makes decisions to maximize the efficiency of the provider＇s environment while still satisfying agreed upon ＂service level agreements＂ （SLAs）. Our experiments conducted on Amazon＇s cloud infrastructure, confirm that our technique is capable of automatically and dynamically adjusting the system＇s allocated resources observing the SLA.     

A methodology for the evaluation of high response time on E-commerce users and sales

The widespread adoption of high speed Internet access and it’s usage for everyday tasks are causing profound changes in users’ expectations in terms of Web site performance and reliability. At the same time, server management is living a period of changes with the emergence of the cloud computing paradigm that enables scaling server infrastructures within minutes. To help set performance objectives for maximizing user satisfaction and sales, while minimizing the number of servers and their cost, we present a methodology to determine how user sales are affected as response time increases. We begin with the characterization of more than 6 months of Web performance measurements, followed by the study of how the fraction of buyers in the workload is higher at peak traffic times, to then build a model of sales through a learning process using a 5-year sales dataset. Finally, we present our evaluation of high response time on users for popular applications found in the Web.     

An elasticity model for High Throughput Computing clusters

Different methods have been proposed to dynamically provide scientific applications with execution environments that hide the complexity of distributed infrastructures. Recently virtualization has emerged as a promising technology to provide such environments. In this work we present a generic cluster architecture that extends the classical benefits of virtual machines to the cluster level, so providing cluster consolidation, cluster partitioning and support for heterogeneous environments. Additionally the capacity of the virtual clusters can be supplemented with resources from a commercial cloud provider. The performance of this architecture has been evaluated in the execution of High Throughput Computing workloads. Results show that, in spite of the overhead induced by the virtualization and cloud layers, these virtual clusters constitute a feasible and performing HTC platform. Additionally, we propose a performance model to characterize these variable capacity (elastic) cluster environments. The model can be used to dynamically dimension the cluster using cloud resources, according to a fixed budget, or to estimate the cost of completing a given workload in a target time.     

Method of multiagent scheduling of resources in cloud computing environments

The paper describes a method of scheduling distributed computing resources in cloud environments for solving user tasks using a variety of software agents physically located on separate processor units connected to the cloud infrastructure and representing their interests in the process of computing. The advantages of the proposed approach are as follows: Firstly, the fast tracking of all resource changes occurring to the processing unit using agents and real time correction of the computing process taking into account these changes, which in turn makes it possible to use computing resources with a dynamically changing performance in the cloud environment (e.g., personal privately owned computers), and, secondly, a cost reduction for the cloud infrastructure because there is no need to introduce expensive dedicated nodes that perform service functions into its structure.     

A dynamically self-configurable service process engine

The performance of a process engine is one of the key factors that contribute to the successful deployment of systems, based on a service-oriented architecture. A novel service process engine that can be self-configured dynamically is introduced in the paper. It is based on the Jini platform, and leverages of Jini services to provide key functionalities. It automatically maintains the global performance by performing load balancing and configuring the system structure dynamically. A heuristic algorithm is applied to indicate every client’s request with a workload tag after a service process model is designed. Based on workload tags of client requests and the status of available services in the engine, a controller allocates the requests to appropriate services and dynamically reconfigures the engine based on fuzzy control algorithms. Algorithms and the architecture for the engine are discussed in detail; in addition, performance experiments are performed to show the effectiveness and feasibility of the proposed approach.     

A hybrid cloud controller for vertical memory elasticity: A control-theoretic approach

Web-facing applications are expected to provide certain performance guarantees despite dynamic and continuous workload changes. As a result, application owners are using cloud computing as it offers the ability to dynamically provision computing resources (e.g., memory, CPU) in response to changes in workload demands to meet performance targets and eliminates upfront costs. Horizontal, vertical, and the combination of the two are the possible dimensions that cloud application can be scaled in terms of the allocated resources. In vertical elasticity as the focus of this work, the size of virtual machines (VMs) can be adjusted in terms of allocated computing resources according to the runtime workload. A commonly used vertical resource elasticity approach is realized by deciding based on resource utilization, named capacity-based. While a new trend is to use the application performance as a decision making criterion, and such an approach is named performance-based. This paper discusses these two approaches and proposes a novel hybrid elasticity approach that takes into account both the application performance and the resource utilization to leverage the benefits of both approaches. The proposed approach is used in realizing vertical elasticity of memory (named as vertical memory elasticity), where the allocated memory of the VM is auto-scaled at runtime. To this aim, we use control theory to synthesize a feedback controller that meets the application performance constraints by auto-scaling the allocated memory, i.e., applying vertical memory elasticity. Different from the existing vertical resource elasticity approaches, the novelty of our work lies in utilizing both the memory utilization and application response time as decision making criteria. To verify the resource efficiency and the ability of the controller in handling unexpected workloads, we have implemented the controller on top of the Xen hypervisor and performed a series of experiments using the RUBBoS interactive benchmark application, under synthetic and real workloads including Wikipedia and FIFA. The results reveal that the hybrid controller meets the application performance target with better performance stability (i.e., lower standard deviation of response time), while achieving a high memory utilization (close to 83%), and allocating less memory compared to all other baseline controllers.     

Self-scalable services in service oriented software for cost-effective data farming

Software maintenance is one of the major concerns in service oriented ecosystem with an ever-increasing importance. In many cases, the cost of software maintenance is higher than the cost of software development. In particular, long-lasting services, which operate in a dynamically changing environment, require continuous management and administration. One of the important administration actions is scaling management. The problem lies in responding to workload changes of the hosted services as fast as possible. This is especially important in regard to (but not limited to) cloud environments where unnecessary resource usage leads to unnecessary costs. In this paper, we are introducing the self-scalable services and scaling rules, which are intended to support development of self-scalable systems based on Service Oriented Architecture. We propose a design of a self-scalable service based on some of the well-known software development practices along with a definition of scaling rules, which express scaling policy for the service. Both concepts were evaluated in the context of a massively scalable platform for data farming. The evaluation demonstrates advantages of utilizing the proposed concepts to manage the platform in comparison with traditional platform management strategies based on fulfilling peak load.     

Complexity Model Based Proactive Dynamic Voltage Scaling for Video Decoding Systems

Significant power savings can be achieved on voltage/ frequency configurable platforms by dynamically adapting the frequency and voltage according to the workload (complexity). Video decoding is one of the most complex tasks performed on such systems due to its computationally demanding operations like inverse filtering, interpolation, motion compensation and entropy decoding. Dynamically adapting the frequency and voltage for video decoding is attractive due to the time-varying workload and because the utility of decoding a frame is dependent only on decoding the frame before the display deadline. Our contribution in this paper is twofold. First, we adopt a complexity model that explicitly considers the video compression and platform specifics to accurately predict execution times. Second, based on this complexity model, we propose a dynamic voltage scaling algorithm that changes effective deadlines of frame decoding jobs. We pose our problem as a buffer-constrained optimization and show that significant improvements can be achieved over the state-of-the-art dynamic voltage scaling techniques without any performance degradation. Index     

Enabling cost-aware and adaptive elasticity of multi-tier cloud applications

Elasticity (on-demand scaling) of applications is one of the most important features of cloud computing. This elasticity is the ability to adaptively scale resources up and down in order to meet varying application demands. To date, most existing scaling techniques can maintain applications’ Quality of Service (QoS) but do not adequately address issues relating to minimizing the costs of using the service. In this paper, we propose an elastic scaling approach that makes use of cost-aware criteria to detect and analyse the bottlenecks within multi-tier cloud-based applications. We present an adaptive scaling algorithm that reduces the costs incurred by users of cloud infrastructure services, allowing them to scale their applications only at bottleneck tiers, and present the design of an intelligent platform that automates the scaling process. Our approach is generic for a wide class of multi-tier applications, and we demonstrate its effectiveness against other approaches by studying the behaviour of an example e-commerce application using a standard workload benchmark.     

Cooperative game-based distributed resource allocation in horizontal dynamic cloud federation platform

Distributed resource allocation is a very important and complex problem in emerging horizontal dynamic cloud federation (HDCF) platforms, where different cloud providers (CPs) collaborate dynamically to gain economies of scale and enlargements of their virtual machine (VM) infrastructure capabilities in order to meet consumer requirements. HDCF platforms differ from the existing vertical supply chain federation (VSCF) models in terms of establishing federation and dynamic pricing. There is a need to develop algorithms that can capture this complexity and easily solve distributed VM resource allocation problem in a HDCF platform. In this paper, we propose a cooperative game-theoretic solution that is mutually beneficial to the CPs. It is shown that in non-cooperative environment, the optimal aggregated benefit received by the CPs is not guaranteed. We study two utility maximizing cooperative resource allocation games in a HDCF environment. We use price-based resource allocation strategy and present both centralized and distributed algorithms to find optimal solutions to these games. Various simulations were carried out to verify the proposed algorithms. The simulation results demonstrate that the algorithms are effective, showing robust performance for resource allocation and requiring minimal computation time.     

Hierarchical Particle Swarm Optimization with Ortho-Cyclic Circles

Cloud computing is an emerging technology which deals with real world problems that changes dynamically. The users of dynamically changing applications in cloud demand for rapid and efficient service at any instance of time. To deal with this paper proposes a new modified Particle Swarm Optimization (PSO) algorithm that work efficiently in dynamic environments. The proposed Hierarchical Particle Swarm Optimization with Ortho Cyclic Circles (HPSO-OCC) receives the request in cloud from various resources, employs multiple swarm interaction and implements cyclic and orthogonal properties in a hierarchical manner to provide the near optimal solution. HPSO-OCC is tested and analysed in both static and dynamic environments using seven benchmark optimization functions. The proposed algorithm gives the best solution and outperforms in terms of accuracy and convergence speed when compared with the performance of existing PSO algorithms in dynamic scenarios. As a case study, HPSO-OCC is implemented in remote health monitoring application for optimal service scheduling in cloud. The near optimal solution from HPSO-OCC and Dynamic Round Robin Scheduling algorithm is implemented to schedule the services in healthcare.     

CloudsStorm: A framework for seamlessly programming and controlling virtual infrastructure functions during the DevOps lifecycle of cloud applications

The infrastructure-as-a-service (IaaS) model of cloud computing provides virtual infrastructure functions (VIFs), which allow application developers to flexibly provision suitable virtual machines' (VM) types and locations, and even configure the network connection for each VM. Because of the pay-as-you-go business model, IaaS provides an elastic way to operate applications on demand. However, in current cloud applications DevOps (software development and operations) lifecycle, the VM provisioning steps mainly rely on manually leveraging these VIFs. Moreover, these functions cannot be programmatically embedded into the application logic to control the infrastructure at runtime. Especially, the vendor lock-in issue, which different clouds provide different VIFs, also enlarges this gap between the cloud infrastructure management and application operation. To mitigate this gap, we designed and implemented a framework, CloudsStorm, which enables developers to easily leverage VIFs of different clouds and program them into their cloud applications. To be specific, CloudsStorm empowers applications with infrastructure programmability at design-level, infrastructure-level, and application-level. CloudsStorm also provides two infrastructure controlling modes, ie, active and passive mode, for applications at runtime. Besides, case studies about operating task-based and big data applications on clouds show that the monetary cost is significantly reduced through the seamless and on-demand infrastructure management provided by CloudsStorm. Finally, the scaling and recovery operation evaluations of CloudsStorm are performed to show its controlling performance. Compared with other tools, ie, “jcloud” and “cloudinit.d”, the scaling and provisioning performance evaluations demonstrate that CloudsStorm can achieve at least 10% efficiency improvement in our experiment settings.     

An efficient resource provisioning approach for analyzing cloud workloads: a metaheuristic-based clustering approach

With the recent advancements in Internet-based computing models, the usage of cloud-based applications to facilitate daily activities is significantly increasing and is expected to grow further. Since the submitted workloads by users to use the cloud-based applications are different in terms of quality of service (QoS) metrics, it requires the analysis and identification of these heterogeneous cloud workloads to provide an efficient resource provisioning solution as one of the challenging issues to be addressed. In this study, we present an efficient resource provisioning solution using metaheuristic-based clustering mechanism to analyze cloud workloads. The proposed workload clustering approach used a combination of the genetic algorithm and fuzzy C-means technique to find similar clusters according to the user’s QoS requirements. Then, we used a gray wolf optimizer technique to make an appropriate scaling decision to provide the cloud resources for serving of cloud workloads. Besides, we design an extended framework to show interaction between users, cloud providers, and resource provisioning broker in the workload clustering process. The simulation results obtained under real workloads indicate that the proposed approach is efficient in terms of CPU utilization, elasticity, and the response time compared with the other approaches.

     

Active rule base development for dynamic vertical partitioning of multimedia databases

Currently, vertical partitioning has been used in multimedia databases in order to take advantage of its potential benefits in query optimization. Nevertheless, most vertical partitioning algorithms are static; this means that they optimize a vertical partitioning scheme (VPS) according to a workload, but if this workload suffers changes, the VPS may be degraded, which would result in long query response time. This paper presents a set of active rules to perform dynamic vertical partitioning in multimedia databases. First of all, these rules collect all the information that a vertical partitioning algorithm needs as input. Then, they evaluate this information in order to know if the database has experienced enough changes to trigger a performance evaluator. In this case, if the performance of the database falls below a threshold previously calculated by the rules, the vertical partitioning algorithm is triggered, which gets a new VPS. Finally, the rules materialize the new VPS. Our active rule base is implemented in the system DYMOND, which is an active rule-based system for dynamic vertical partitioning of multimedia databases. DYMOND’s architecture and workflow are presented in this paper. Moreover, a case study is used to clarify and evaluate the functionality of our active rule base. Additionally, authors of this paper performed a qualitative evaluation with the aim of comparing and evaluating DYMOND’s functionality. The results showed that DYMOND improved query performance in multimedia databases.     

A complex event routing infrastructure for distributed systems

With the growing number of mega services and cloud computing platforms, industrial organizations are utilizing distributed data centers at increasing rates. Rather than the request/reply model, these centers use an event-based communication model. Traditionally, the event-based middleware and the Complex Event Processing (CEP) engine are viewed as two distinct components within a distributed system’s architecture. This division adds additional system complexity and reduces the ability for consuming applications to fully utilize the CEP toolset. This article will address these issues by proposing a novel event-based middleware solution. We introduce Complex Event Routing Infrastructure (CERI), a single event-based infrastructure that serves as an event bus and provides first class integration of CEP. An unstructured peer-to-peer network is exploited to allow for efficient event transmission. To reduce network flooding, superpeers and overlay network partitioning are introduced. Additionally, CERI provides each client node the capability of local complex query evaluation. As a result, applications can offload internal logic to the query evaluation engine in an efficient manner. Finally, as more client nodes and event types are added to the system, the CERI can scale up. Because of these favorable scaling properties, CERI serves as a foundational step in bringing event-based middleware and CEP closer together into a single unified infrastructure component.     

A workload-driven approach to database query processing in the cloud

This paper is concerned with data provisioning services (information search, retrieval, storage, etc.) dealing with a large and heterogeneous information repository. Increasingly, this class of services is being hosted and delivered through Cloud infrastructures. Although such systems are becoming popular, existing resource management methods (e.g. load-balancing techniques) do not consider workload patterns nor do they perform well when subjected to non-uniformly distributed datasets. If these problems can be solved, this class of services can be made to operate in more a scalable, efficient, and reliable manner. The main contribution of this paper is a approach that combines proprietary cloud-based load balancing techniques and density-based partitioning for efficient range query processing across relational database-as-a-service in cloud computing environments. The study is conducted over a real-world data provisioning service that manages a large historical news database from Thomson Reuters. The proposed approach has been implemented and tested as a multi-tier web application suite consisting of load-balancing, application, and database layers. We have validated our approach by conducting a set of rigorous performance evaluation experiments using the Amazon EC2 infrastructure. The results prove that augmenting a cloud-based load-balancing service (e.g. Amazon Elastic Load Balancer) with workload characterization intelligence (density and distribution of data; composition of queries) offers significant benefits with regards to the overall system’s performance (i.e. query latency and database service throughput).     

DynamicCloudSim: Simulating heterogeneity in computational clouds

Simulation has become a commonly employed first step in evaluating novel approaches towards resource allocation and task scheduling on distributed architectures. However, existing simulators fall short in their modeling of the instability common to shared computational infrastructure, such as public clouds. In this work, we present DynamicCloudSim which extends the popular simulation toolkit CloudSim with several factors of instability, including inhomogeneity and dynamic changes of performance at runtime as well as failures during task execution. As a validation of the introduced functionality, we simulate the impact of instability on scientific workflow scheduling by assessing and comparing the performance of four schedulers in the course of several experiments both in simulation and on real cloud infrastructure. Results indicate that our model seems to adequately capture the most important aspects of cloud performance instability. The source code of DynamicCloudSim and the examined schedulers is available at https://code.google.com/p/dynamiccloudsim/.     

A task-level adaptive MapReduce framework for real-time streaming data in healthcare applications

Healthcare scientific applications, such as body area network, require of deploying hundreds of interconnected sensors to monitor the health status of a host. One of the biggest challenges is the streaming data collected by all those sensors, which needs to be processed in real time. Follow-up data analysis would normally involve moving the collected big data to a cloud data center for status reporting and record tracking purpose. Therefore, an efficient cloud platform with very elastic scaling capacity is needed to support such kind of real time streaming data applications. The current cloud platform either lacks of such a module to process streaming data, or scales in regard to coarse-grained compute nodes.In this paper, we propose a task-level adaptive MapReduce framework. This framework extends the generic MapReduce architecture by designing each Map and Reduce task as a consistent running loop daemon. The beauty of this new framework is the scaling capability being designed at the Map and Task level, rather than being scaled from the compute-node level. This strategy is capable of not only scaling up and down in real time, but also leading to effective use of compute resources in cloud data center. As a first step towards implementing this framework in real cloud, we developed a simulator that captures workload strength, and provisions the amount of Map and Reduce tasks just in need and in real time.To further enhance the framework, we applied two streaming data workload prediction methods, smoothing and Kalman filter, to estimate the unknown workload characteristics. We see 63.1% performance improvement by using the Kalman filter method to predict the workload. We also use real streaming data workload trace to test the framework. Experimental results show that this framework schedules the Map and Reduce tasks very efficiently, as the streaming data changes its arrival rate.