Ensuring Quality of Service in High Performance Servers

Transcription

1 Ensuring Quality of Service in High Performance Servers YAN SOLIHIN Fei Guo, Seongbeom Kim, Fang Liu Center of Efficient, Secure, and Reliable Computing (CESR) North Carolina State University

2 Motivation Two trends Utility/On-demand computing A server shared by programs from many clients Programs need Quality of Service (QoS) guarantee Multicore chips (Chip Multi-Processor/CMP) Building blocks for utility computing servers Do not naturally support QoS Due to shared platform resources (caches, bandwidth) Goal Build software and architecture support for providing QoS 2

3 Shared Cache in CMP Server P P P P IBM Power5 Now L1 L1 L1 L1 Near future L2 Cache L2 Cache L3 Cache Shared resources present new performance and fairness challenges 3

4 mcf's L2 Cache Misses Impact of Cache Space Contention 400% 350% 300% 250% 200% 150% 100% 50% 0% Alone 512-KB shared L2 cache mcf+art mcf+swim mcf+mst mcf+gzip mcf's Normalized IPC 100% 80% 60% 40% 20% 0% Application-specific and Coschedule-specific Significant: Up to 4X cache misses, 65% IPC reduction Other benchmarks affected, too Alone mcf+art mcf+swim mcf+mst CMP-based server needs to provide Quality of Service mcf+gzip 4

5 Current Multicore Chip (CMP) Node CMP Job done Job QoS Request: - OS Priority Scheduler Job Queue Problems: - OS Priority does not provide guarantee - OS Priority does not control platform resources - The same priority yields different performance under different load 5

6 Current Server Node Job QoS Request: - # procs - mem size - max time Node Global Scheduler Problems: - Too coarse: - Oblivious of CMP boundary - Exclude platform resources - QoS not precise (deadline and load management) Node... 6

7 Quality of Service Models Job QoS Request: - # procs - mem size - max time - Platform Resources (cache size, bandwidth) - Resource Performance - Overall Performance Resource Usage Monitoring CMP QoS Support Job done Admission Control Accept QoS Scheduler Job Queue Reject Negotiate QoS 7

8 Quality of Service Models Predict Cache Contention, Predict Cache Performance Resource Usage Monitoring Ensure Fairness, Enforce resource partitioning CMP Qos Support Job done Admission Control Accept Job Queue QoS Scheduler Reject Negotiate QoS Define QoS Models, Construct Satisfiable Schedule Monitor & adjust schedule, Maximize Throughput 8

9 QoS Models Multiple Models Resource Usage Model (RUM) Cache size, bus bandwidth, etc. Resource Performance Model (RPM) e.g. miss rate Overall Performance Model (OPM) e.g. exec time, TPS Multiple Service Tiers, e.g. for RUM: Gold: 16GB Mem, 4MB Cache, 6GB/sec bandwidth Silver: 4GB Mem, 2MB Cache, 3GB/sec bandwidth Standard: 2GB Mem, 1MB Cache, 2GB/sec bandwidth 9

10 Cache Performance Prediction model App A Profiling App B Profiling Contention Prediction Model Miss rate of each app in each co-schedule A+B A s MR B s MR App C Profiling A+C B+C A s MR B s MR C s MR C s MR Validated against detailed CMP simulation Average error of 3.9% Correctly identifies all hyper contention cases 10

11 QoS Support in CMP Enforce Resource Partitioning Policies Cache and Bandwidth Partitioning Enforce Fairness in Resource Usage How is cache fairness measured? Cache fairness metrics How is fairness improved? Static and dynamic fair caching policies What benefits fairness gives? Improved fairness, but also improved throughput! 11

12 Dynamic Fair Caching Results 2 Normalized 1.5 Combined 1 IPC 0.5 (higher is better) apsi+art gzip+art swim+gzip tree+mcf AVG18 Base Fair Normalized 1 Fairness 0.5 (lower is better) 0 apsi+art gzip+art swim+gzip tree+mcf AVG18 Fair improves both fairness and throughput 12

13 Dynamic Fair Caching Results Normalized 1.5 Combined 1 IPC 0.5 (higher is better) apsi+art gzip+art swim+gzip tree+mcf AVG18 Base Fair Normalized 1 Fairness 0.5 (lower is better) 0 apsi+art gzip+art swim+gzip tree+mcf AVG18 Better fairness better throughput (16/18 cases) 13

14 Dynamic Fair Caching Results Normalized 1.5 Combined 1 IPC 0.5 (higher is better) apsi+art gzip+art swim+gzip tree+mcf AVG18 Base Fair Normalized 1 Fairness 0.5 (lower is better) 0 apsi+art gzip+art swim+gzip tree+mcf AVG18 Better fairness better throughput (2/18 cases) 14

15 State of the Project Completed Performance Monitoring and Modeling Cache Space Contention in CMP [HPCA 2005] Replacement Policy Performance Model [Sigmetrics 2006] Performance impact of OS activities [ISPASS 2007] QoS Models and Architecture Fair Caching, Cache partitioning [PACT 2004] QoS Models Impact on Performance [Sigmetrics 2007] Prototype of OS with QoS Support [dascmp 2007] Ongoing Job Admission Control and Scheduler Broader Impact HPCA 2007 Tutorial Software: Analytical Cache Performance Prediction (ACAPP) Toolset Industry Collaboration with Intel and RedHat 15

16 System Integration of QoS Components 1.40E E E E E E E E E E E E E E E+0 Wall-clock time of 6 jobs in different mode Alone QoS-hard OS-default 16

17 Acknowledgement Collaborators: Ravi Iyer, Intel Li Zhao, Intel Will Cohen, RedHat Students: Seongbeom Kim, PhD (VMWare) Fei Guo, PhD Fang Liu, PhD Radha Venkatagiri, MST Other ARPERS (Architecture Research for PErformance, Reliability, and Security) Helper Threads Secure Architecture and Operating System Support for Software Reliability 17

18 Q & A Thank you 18