Dynamic Resource Allocation for Database Servers Running on Virtual Storage

Transcription

1 Dynamic Resource Allocation for Database Servers Running on Virtual Storage Gokul Soundararajan, Daniel Lupei, Saeed Ghanbari, Adrian Daniel Popescu, Jin Chen, Cristiana Amza University of Toronto 1

2 Multi-tier Resource Allocation Application-A Application-B Composed of several tiers Web Server Application Server Share resources in each tier Can lead to interference Database Server Consolidated Environment 2

3 Our Focus: Storage Hierarchy Application-A Application-B Disk is a bottleneck for Database Apps Database Network Buffer Pool Want to use all resources efficiently Storage Cache Storage Disk Bandwidth 3

4 State of the Art Previous work studied resources in isolation - Memory Partitioning: MRC [ASPLOS 04] - Disk Bandwidth: Facade [FAST 03], Argon [FAST 07], etc and many more Want to use the storage hierarchy efficiently However, performance depends on all layers - Interdependency between resources - E.g., Increasing buffer pool reduces number of storage accesses 4

5 Motivating Scenario Cache Friendly 1 Outstanding I/O Cache Un-Friendly 10 Outstanding I/O Small Large Using Oracle ORION I/O tool Buffer Pool Storage Cache Disk Bandwidth 5

6 Motivating Scenario 7.5 Normalized Latency Benefits cachefriendly workload Avoids disk interference Has best performance Shared Cache Disk Cache & Disk Small Large 6

7 Contributions Build performance models dynamically - Account for interdependencies between resources - Lightweight but still accurate Multi-level Resource Allocator - Uses performance models to guide resource allocation - Corrects model errors through runtime sampling - Uses global utility (SLOs) to partition resources - Minimize sum of application latencies 7

8 Approach Build performance models - One per application - Derive function to predict application latency given configuration L avg = f(ρ c, ρ s, ρ d ) Find resource partitioning setting - Minimize sum of application latencies - Find best setting using hill climbing 8

9 Outline Online Performance Models - What are they? - Why are they hard to build? Multi-level Resource Allocator Prototype Implementation Experimental Results Conclusions 9

10 Avg. Latency One-Level Cache Model Allocate in 32MB chunks Choose 512MB m=cachesize/chunksize =1GB/32MB=32 choices G 32 choices Cache size 10

11 Miss-Ratio MRC Cache Model Computes miss-ratio given an I/O trace Multiply by I/O latency gets Avg. Latency G Cache size 11

12 Two-Level Cache Model Performance affected by - DB Buffer Pool Size (m choices) - Storage Cache (n choices) Changes the I/O trace at storage Performance model - Needs to consider all parameters (m*n choices) - 1GB caches allocated in 32MB chunks - m = 1GB/32MB = 32 settings - m*n = 1024 distinct settings 12

13 Two-Level Cache Model 2 caches create a 3D surface 32x32=1024 data points! High Latency Avg. Latency 32 data points 256 Storage Cache Size Buffer Pool Size 512 Buffer Pool Size (MB) Storage Cache Size Low Latency 13

14 Overall Performance Model Application ρ c ρ s ρ d Buffer Pool Storage Cache Disk Bandwidth Needs 32x32x10=10240 samples 15 mins/sample takes 3 months! 14

15 Outline Online Performance Models Multi-level Resource Allocator - Building performance models - Allocating resources using models Prototype Implementation Experimental Results Conclusions 15

16 Key Observations Known cache replacement policies - Most cache replacement algorithms are LRU - Only as effective as the largest cache (cache inclusiveness) Disk is a closed loop system - Rate of responses is same as rate of requests - Performance proportional to the disk bandwidth fraction 16

17 Cache Inclusiveness LRU 8 LRU 8 I/Os: 01 17

18 Cache Inclusiveness LRU Storage cache includes data in the buffer pool LRU I/Os: 6 18

19 Cache Inclusiveness LRU LRU Buffer pool includes data in the storage cache I/Os: 6 19

20 Approximate Single Cache Model (LRU) LRU M c (max[ρ c, ρ s ]) LRU I/Os: 6 Same Number of I/Os 20

21 Cache Model (DEMOTE) Maintain cache exclusiveness - E.g., using DEMOTEs [USENIX 02] - Every block brought into buffer pool is not cached below - Only evictions from buffer pool cached in storage cache Approximate performance using single cache - M c (ρ c + ρ s ) 21

22 Latency Latency Find Best Partitioning Setting Storage Cache Size App-1 Buffer Pool Size Find Best Resource Allocation Setting Minimize sum of application latencies Storage Cache Size App-2 Buffer Pool Size 22

23 Disk Model Observation: Closed loop system - Rate of responses same as rate of requests - Use interactive response time law Performance proportional to disk bandwidth fraction - Measure base disk latency: L d (1) - Predict latency for smaller bandwidth fractions L d (ρ d ) = L d(1) ρ d 23

24 Putting it All Together Application Buffer Pool Approximate Single-Level Cache Storage Cache Can now be solved using MRC M c (ρ c )M s (ρ c, ρ s )N = M c (max[ρ c, ρ s ])N 24

25 Putting it all Together Application H c (ρ c )L c M c (ρ c )H s (ρ c, ρ s )L net M c (ρ c )M s (ρ c, ρ s )L d (ρ d ) 25

26 Inaccuracies in the Model Cache Model - Approximations to LRU, i.e., CLOCK - Large fraction of writes in the workload Disk Model - Using Quanta-based scheduler [Wachs et. al, FAST 07] - Interference due to disk seeks at small quanta Inaccuracies localized in known regions - E.g., Small disk quanta 26

27 Iterative Refinement Build model - Use trace collected at the database buffer pool Refine the model - Use cross-validation to measure quality - Selectively sample where error is high - Interpolate computed and measured samples - Using regression (SVM) 27

28 Virtual Storage Prototype MySQL Storage Buffer Pool NBD Cache Quanta Block Layer Linux Network Block Layer Linux SCSI NBD SCSI Disk Disk Disk Disk CLIENT SERVER 28

29 Experimental Setup Benchmarks - UNIFORM (microbenchmark), TPC-W and TPC-C LAMP Architecture - Linux, Apache 1.3, MySQL/InnoDB 5.0, and PHP 5.0 Cache Configuration - MySQL buffer pool = 1GB - Storage cache = 1GB - Using InnoDB cache replacement in MySQL, CLOCK in storage cache 29

30 Our Algorithms GLOBAL - Gather trace at the buffer pool - Measure base disk latency - Compute performance using performance model GLOBAL+ - Run GLOBAL - Evaluate model accuracy - Refine model using runtime samples 30

31 Algorithms for Comparison MRC - Partition cache (independently) using miss-ratio curves DISK - Partition caches equally, determine best disk quanta MRC+DISK - Run MRC then DISK IDEAL* - Build model with SVM using 16*16*5=1280 sampled configurations 31

32 Roadmap of Results Multi-level cache allocator - Using LRU and DEMOTE cache replacement policies Multi-level cache and disk Accuracy of computed models 32

33 Miss-Ratio Curves Miss Ratio (%) TPC-W TPC-C UNIFORM Buffer Pool Size (MB) 33

34 Multi-Level Caching (LRU) 2 TPC-W Instances 1G Optimal HeatMap Lighter: Better Darker: Worse Buffer Pool Size (A) Optimal 0 Storage Cache Size (A) 1G 34

35 Multi-Level Caching (DEMOTE) 2 TPC-W Instances 1G Buffer Pool Size (A) Optimal 0 Storage Cache Size (A) 1G 35

36 Roadmap of Results Multi-level cache allocator Multi-level cache and disk - Using two identical applications - Using different applications Accuracy of computed models 36

37 UNIFORM/UNIFORM Allocate caches to 50/50 Average Latency (ms) Matches GLOBAL GLOBAL GLOBAL+ MRC DISK MRC+DISK IDEAL* UNIFORM UNIFORM 37

38 TPC-W/UNIFORM 7.5 Allocate caches to 50/50 Average Latency (ms) Not enough buffer pool to UNIFORM Compensate for MRC settings GLOBAL GLOBAL+ MRC DISK MRC+DISK IDEAL* TPC-W UNIFORM 38

39 TPC-W/TPC-C Average Latency (ms) Corrects model at runtime TPC-C allocated more in both Corrects imbalance in MRC GLOBAL GLOBAL+ MRC DISK MRC+DISK IDEAL* TPC-W TPC-C 39

40 Roadmap of Results Multi-level cache allocator Multi-level cache and disk Accuracy of computed models - Cache model - Disk model 40

41 Cache Model Accuracy (TPC-W) Storage Cache Size (MB) Localized in the middle Error (%) Buffer Pool Size (MB) 41

42 Disk Model Accuracy (TPC-W) Measured Computed Latency (ms) Disk Quota 42

43 Conclusions Problem - Need to consider resources on multiple tiers - Independent cache/disk allocators are not sufficient Dynamic allocation of cache hierarchy and disk - Build performance models dynamically - Iteratively refine (if necessary) - Use models for global resource partitioning Performance up to 2.9 better than single resource allocators 43

44 Thank you. 44