The Impact of Memory Subsystem Resource Sharing on Datacenter Applications. Lingia Tang Jason Mars Neil Vachharajani Robert Hundt Mary Lou Soffa

Transcription

1 The Impact of Memory Subsystem Resource Sharing on Datacenter Applications Lingia Tang Jason Mars Neil Vachharajani Robert Hundt Mary Lou Soffa

2 Introduction Problem Recent studies into the effects of memory resource sharing among threads concentrated on common multithread workload benchmarks (SPEC, etc). These studies do not represent the workloads that have emerged in datacenter applications. Real-world datacenter applications

3 Introduction Demonstrates the impact of memory sharing in datacenter applications Importance of a good Thread-to-Core (TTC) mapping Evaluate the impact of co-locating threads Optimal TTC mappings changes with co-located applications Identify characteristics that impact performance in various TTC mappings Amount of sharing between threads Required memory bandwidth Application s cache footprint

4 Background and Motivation Memory Resource Sharing Sharing Configurations can vary within the same set of cores

5 Background and Motivation Datacenter Job Scheduling Global job scheduler selects appropriate machine Machine s OS then selects TTC mappings Memory resource sharing is not considered in scheduling decisions Job Priority and Co-location Latency-sensitive applications take a higher priority High priority jobs are scheduled with lower-priority jobs for efficiency Managing QoS priorities on multicores remains a challenge

6 Intra-application Sharing Experiment Methodology Platform: Intel Clovertown (Xeon E5345) dual-socket Linux kernel with a custom GCC Latency-sensitive applications run alone or with a batch application A load generator tests peak behavior with real-world query traces Experiments show performance variability among different TTC mappings Three experimental runs of a fixed load executing across four cores Measured application s specified performance metric

7 Intra-application Sharing Measurement and Findings Performance impact of resource sharing is significant Although each has a different sharing preference bigtable actually performs better when sharing resources!!

8 Intra-application Sharing Performance Variability Last Level Cache Misses Normalized to {X.X.X.X.} Fixed code section Results are consistent with performance trend More LLC misses leads to performance degradation

9 Intra-application Sharing Performance Variability FSB Bandwidth Consumption Normalized to {X.X.X.X.} Fixed code section Results consistent with performance trend More LLC misses increases the number of bus requests

10 Intra-application Sharing Performance Variability Data Sharing Five States of L2 Requests Prefetch, Modified, Exclusive, Shared, and Invalid Purple shows amount of sharing between threads Observations consistent with performance trend

11 Intra-application Sharing Performance Variability Summary LLC sharing has significant positive or negative performance effects Can add up to 10% variability Bus contention also has significant effects Can add another 10% of variability Applications with a high level of data sharing will benefit significantly from sharing the LLC and FSB The results demonstrate the importance of a good TTC mapping that mimics the applications inherent data sharing pattern

12 Inter-application Sharing Experiment Design Same setup as the previous experiment, except this experiment considers the impact of co-located applications with varying TTC mappings Performance of contentanalyzer, bigtable, and websearch are measured while sharing resources with co-running batch applications Co-running applications are stitcher and protobuf Each is a batch, low-priority application

13 Inter-application Sharing Measurement and Findings Normalized to solo-performance under the corresponding TTC mapping The TTC mappings have different effects when a co-runner (*) is present {X*X*X*X*} shares LLC, FSB, and memory subsystem {XX**XX**} shares the FSB and memory subsystem {XXXX****} shares only the memory subsystem (unavoidable)

14 Inter-application Sharing Measurement and Findings Normalized to solo-performance under the TTC mapping {X.X.X. X.} The TTC mappings have different effects when a co-runner (*) is present {X*X*X*X*} shares LLC, FSB, and memory subsystem {XX**XX**} shares the FSB and memory subsystem {XXXX****} shares only the memory subsystem (unavoidable)

15 Inter-application Sharing Measurement and Findings Summary The impact of sharing the LLC and FSB are effected by the specific co-running application Performance swings between best and worst TTC mapping can be very significant The optimal TTC mapping for an application may change with the specific co-running application chosen An intelligent TTC mapping system will account for underlying resources and sharing configurations

16 Varying Thread Count and Architecture Varying the Number of Threads Latency-sensitive applications run with 2 threads Co-located batch applications run with 6 threads Normalized to applications running solo with TTC mapping {X X } Results are consistent with 4-thread behavior

17 Varying Thread Count and Architecture Varying the Number of Threads Latency-sensitive applications run with 6 threads Co-located batch applications run with 2 threads Normalized to applications running solo with TTC mapping {X X } Results are consistent with 4-thread behavior

18 Varying Thread Count and Architecture Varying Architecture Experimented using Intel s Westmere (Xeon X5660) dual-socket platform Each socket has six cores with a 12MB shared LLC Shared memory controller with 3 channels of 8.5 GB/s bus to DIMM Key application running solo with 2 threads Normalized to the TTC mapping {X...X...}

19 Varying Thread Count and Architecture Varying Architecture Experimented using Intel s Westmere (Xeon X5660) dual-socket platform Each socket has six cores with a 12MB shared LLC Shared memory controller with 3 channels of 8.5 GB/s bus to DIMM Key application running solo with 6 threads Normalized to the TTC mapping {XXX...XXX...}

20 Varying Thread Count and Architecture Varying Architecture Experimented using Intel s Westmere (Xeon X5660) dual-socket platform Each socket has six cores with a 12MB shared LLC Shared memory controller with 3 channels of 8.5 GB/s bus to DIMM Key application running with 6 threads Batch application running with 6 threads Normalized to the TTC mapping {XXX...XXX...}

21 Thread-To-Core Mapping Memory Bandwidth Usage Hypothesis: Is the amount of bus bandwidth usage an indicator for its proper FSB sharing configuration?? Yes, FSB demand correlates with the optimal TTC mapping found High FSB demand applications significantly impact performance

22 Thread-To-Core Mapping Data Sharing The percentage of an application s cache lines in the share state can indicate an applications level of data sharing Measured LLC misses, shared LLC references, and other LLC hits

23 Thread-To-Core Mapping Cache Footprint Contention occurs when the total size of multiple threads cache footprints is larger than the cache itself LLC miss-rate is a good indicator of the footprint size Predicts potential performance degradation for co-runners

24 Thread-To-Core Mapping A Heuristic Approach to TTC Mapping An algorithm to predict optimal TTC mappings based on resource usage characteristics of an application (bus and cache usage, and data sharing) Avoids co-locating threads with the same resource bottlenecks while maximizing potential benefits from sharing resources

25 Thread-To-Core Mapping A Heuristic Approach to TTC Mapping Evaluating the Heuristics Limitations: Algorithm is hardware specific Characteristic profiles necessary for each application Characteristic metrics may not accurately represent the application Advantages: Effective and requires simple runtime support Evaluation:

26 Thread-To-Core Mapping An Adaptive Approach to TTC Mapping AToM Adaptive Thread-to-core Mapping Adaptive learning implementation that uses a competition heuristic Searches for optimal TTC assignment for a given set of threads Learning Phase Various mappings compete for the best performance Execution Phase Winner of learning phase is executed for some time period

27 Thread-To-Core Mapping An Adaptive Approach to TTC Mapping Evaluating AToM Applications configured to run on 4 cores Learning phase for 3 runs of 10 minutes each Execution phase for 1 run of 2 hours Performance normalized to performance under TTC mapping {X...X...}

28 Conclusion Outlined the importance of intelligent TTC Mapping decisions The optimal TTC mapping varies with the co-running application Presented key characteristics that impact TTC mappings Presented heuristics for TTC mapping using characteristics Presented a more attractive adaptive approach Simple to implement and very effective for long-running applications Discovered a performance swing of up to 40% A 1% improvement could save a datacenter millions $$ Using adaptive approach Datacenter workloads could improve 22% Perform within 3% of optimal mapping on average

29 Discussion Any questions???

30 Related Work Similar datacenter application studies All study effects of application characteristics On datacenter system design In combination with the underlying architecture On database workloads Architecture Studies investigate memory resource sharing and contention New architecture supports cache and memory bus management Software (Operating systems) Techniques focus on co-scheduling to avoid resource contention Control execution rate using hardware features or a runtime system Cache and contention aware schedulers and compilers