ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores

Transcription

1 ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores Ahmad Al-Shishtawy KTH Royal Institute of Technology Stockholm, Sweden Doctoral School Day in Cloud Computing Louvain-la-Neuve, Belgium, 20 Nov 2012

2 Outline Introduction 1 Introduction ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 2/33

3 Outline Introduction Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control 1 Introduction Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 3/33

4 Dealing with Complexity Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Problem All computing systems need to be managed ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 4/33

5 Dealing with Complexity Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Problem All computing systems need to be managed ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 4/33

6 Dealing with Complexity Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Problem Computing systems are getting more and more complex ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 4/33

7 Dealing with Complexity Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Problem Complexity means higher administration overheads ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 4/33

8 Dealing with Complexity Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Problem Complexity poses a barrier on further development ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 4/33

9 Dealing with Complexity Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Solution The Autonomic Computing initiative by IBM ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 4/33

10 Dealing with Complexity Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Solution Self-Management: Systems capable of managing themselves ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 4/33

11 Dealing with Complexity Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Solution Use Autonomic Managers Autonomic Manager Autonomic Manager Autonomic Manager Autonomic Manager Analyze Plan Analyze Plan Analyze Plan Analyze Plan Monitor Knowledge Execute Monitor Knowledge Execute Monitor Knowledge Execute Monitor Knowledge Execute ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 4/33

12 Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control The Autonomic Computing Architecture A Generic Solution Managed Resource Touchpoint (Sensors & Actuators) Autonomic Manager Monitor Analyze Plan Execute Manager Interface Touch Point Autonomic Manager Analyze Plan Monitor Knowledge Execute Touch Point Knowledge Source Communication Manager Interface Managed Resource Managed Resource ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 5/33

13 Self-* Properties Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Inspired by the autonomic nervous system of the human body Feedback control from Control Theory Self-management along four main axes (self-* properties): self-configuration self-optimization self-healing self-protection ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 6/33

14 Web 2.0 Applications Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Increasing popularity of Web 2.0 applications (e.g., wikis, social networks, blogs) Pose new challenges on the underlying provisioning infrastructure Scale Highly dynamic workload Data centric Traditional solutions does not work anymore Vertical vs. horizontal scalability ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 7/33

15 Cloud Computing Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Provide the illusion of infinite resources (e.g., VMs) Pay-as-you-go pricing model Elastic services (Horizontal scalability) High load: Allocate more VMs to improve performance Low load: Release VMs to save money ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 8/33

16 The Need for Elasticity Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Web 2.0 applications frequently experience high workloads A service can become popular in just an hour The high level load does not last for long and keeping resources in the Cloud costs money This has led to Elastic Computing Ability of a system to grow and shrink at run-time in response to changes in workload Cloud computing allows on-the-fly requesting and releasing VMs Meet SLOs at a minimal cost ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 9/33

17 Elasticity versus Static Provisioning Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 10/33

18 Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Elasticity can be controlled either manually (by the sys-admin) or automatically (by a autonomic manager) Automation of elasticity can be achieved by providing an Elasticity Controller Helps to avoid SLO violations while keeping the cost low Automatically adds/removes VMs (servers, service instances) in response to changes in some SLO metrics, e.g., request latency, caused by changes in workload Can be built using elements of Control Theory and/or Machine Learning Feedback-loop (a.k.a. closed-loop) control Model Predictive Control (MPC) ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 11/33

19 Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Pay-as-you-go + Elastic services + Web 2.0 dynamic workload ==> Elasticity Controller ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 12/33

20 Feedback Control Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control Desired Value Error Feedback Control System + System - Controller Input Output The system s output (e.g., response time) is being monitored Controller calculates the control error Controller changes the control input (e.g., number of servers to add or remove) according to the amount and sign of the control error Advantage: controller can adapt to disturbance Disadvantages: oscillation, overshoot, possible instability ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 13/33

21 Feedforward Control Autonomic Computing Cloud Computing and Elastic Services Feedback versus Feedforward Control System State Feedforward Controller Control Input System System Output The system s output is not monitored Other system states and variables are monitored Controller relies on a model of the system to calculate necessary change Advantages: faster and avoids oscillation and overshoot Disadvantages: sensitive to unexpected disturbances that are not modeled ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 14/33

22 Outline Introduction Problem Definition Challenges ElastMan 1 Introduction 2 Problem Definition Challenges ElastMan 3 4 ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 15/33

23 Target System Problem Definition Challenges ElastMan Multi-Tier Web 2.0 Application Public / Private Cloud Environment P Presentation Tier P P P D A D P D A D A D Application Tier D A Data Tier D D A A Elasticity Controller Deployed in a Cloud Environment A P C Virtual Machine Hosting a Server D P A D D A A Physical Machine D P Horizontal Scalability (add more servers) Each server is a Virtual Machine running on a physical machine in a Cloud environment ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 16/33

24 Minimum Requirements Problem Definition Challenges ElastMan Horizontal scalability Sensors to monitor workload and performance (e.g., read latency) Actuators to add/remove storage servers Rebalancing ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 17/33

25 Elasticity Controller Problem Definition Challenges ElastMan Objective Control the elasticity of Cloud-based key-value stores by adding/removing resources in order to meet SLOs at a minimal cost SLO examples Average read latency in one minute interval is less than 10ms 99% of reads in one minute interval are performed in less than 10ms per read ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 18/33

26 Problem Definition Challenges ElastMan Challenges to Design an Elasticity Controller for Storage Actuator delays due to data movement (rebalancing) Interference with applications and sensor measurements Discrete storage units Nonlinearity due to diminishing reward of adding a storage unit with increasing scale 99th percentile is a relatively noisy signal VM performance is difficult to model and predict Highly dynamic workload that is composed of both gradual (diurnal) and sudden (spikes) variations ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 19/33

27 Problem Definition Challenges ElastMan ElastMan: Combining Feedback and Feedforward Control Feedback Monitor 99th percentile Classical PI controller We use it to handle diurnal workloads Can tolerate and adapt to modeling errors Feedforward Monitor workload A binary classifier using logistic regression We use it to handle spikes in workload Allows us to smooth the noisy 99th percentile signal Changes in workload can be: Slow day-night changes (diurnal) Rapid changes (spikes) ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 20/33

28 Modeling the Store Problem Definition Challenges ElastMan Typical approach Workload (noise) Control Input Number of Servers Key-Value Store System Output 99th percentile of reads Non linear (1+1 vs ) Workload treated as noise ElastMan approach Control Input Average Workload per Server Key-Value Store System Output 99th percentile of reads Control the number of servers indirectly by controlling the average workload per server Relies on near linear scalability of key-value stores ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 21/33

29 Problem Definition Challenges ElastMan Feedback and Feedforward Controllers SLO (Desired 99th Percentile of Read Latency) + _ Error New Average PI Throughput Controller per Server Actuator New Number of Nodes Measured 99th Percentile of Read Latency Key-Value Store Target System Smoothing Filter Measured Average Throughput per Server FF Binary Classifier New Average Throughput per Server Actuator New Number of Nodes Key-Value Store Measured 99th Percentile of Read Latency Target System ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 22/33

30 Binary Classifier Problem Definition Challenges ElastMan Training Data and Model Write Throughput (ops/second/server) Violate SLO Satisfy SLO Model Read Throughput (ops/second/server) ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 23/33

31 Combined Feedback and Feedforward Flow Chart Start Controller Measure Average Throughput per server (tp) and 99 percentile of read latency (R99p) Filter rp99 fr99p=f(r99p) Error in dead-zone Yes Do nothing! No Rebalancing No Yes If storage supports rebalance restart then use Feedforward controller designed for the store in rebalance mode FFR(tp) Large change in Throughput Yes Use Feedforward FF(tp) Binary Classifier No Use Feedback FB(fp99) PID Controller Calculate number of new VMs new_vms = total_throughput/new_throughput_per_server subject to: replication_degree <= new_vms <= max_vms Start rebalance instance rebalance(n) End

32 Outline Introduction 1 Introduction ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 25/33

33 Implemented ElastMan Elasticity Controller Evaluated with LinkedIn s Voldemort key-value store Deployed in our private Cloud based on OpenStack ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 26/33

34 Elasticlity Controller for the Voldemort Key-Value Store ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 27/33

35 Voldemont performance with fixed number of servers (18) 10 8 Elasticity Controller Read p99 (ms) Desired VMs Total Throughput (K requests/sec) Number of Servers

36 Voldemont performance with fixed number of servers (18) 10 8 Elasticity Controller Read p99 (ms) Desired VMs Total Throughput (K requests/sec) Number of Servers Diurnal Workload Spikes

37 ElastMan controller performance under gradual (diurnal) workload 10 8 Elasticity Controller Read p99 (ms) Desired VMs Total Throughput (K requests/sec) Number of Servers

38 ElastMan controller performance with rapid changes (spikes) in workload 10 8 Elasticity Controller Read p99 (ms) Desired VMs Total Throughput (K requests/sec) Number of Servers

39 Outline Introduction 1 Introduction ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 31/33

40 Conclusions ElastMan addresses the challenges of the variable performance of Cloud VMs, dynamic workload, and stringent performance requirements ElastMan combines and leverages the advantages of both feedback and feedforward control feedforward control quickly respond to rapid changes in workload feedback controller handle diurnal workload and to correct modeling errors in the feedforward control results show the feasibility of our approach ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 32/33

41 Future Work Investigate the controllers needed to control all tiers of a Web 2.0 application and the orchestration of the controllers in order to correctly achieve their goals Provide a feedforward controller for the store during rebalancing Distributed version of ElastMan ElastMan: Autonomic Elasticity Manager for Cloud-Based Key-Value Stores (A. Al-Shishtawy) 33/33