Runtime Tuning of Dynamic Memory Management For Mitigating Footprint-Fragmentation Variations

Transcription

1 Runtime Tuning of Dynamic Memory Management For Mitigating Footprint-Fragmentation Variations Sotirios Xydis, Ioannis Stamelakos, Alexandros Bartzas, Dimitrios Soudris School of Electrical and Computer Engineering National Technical University of Athens {sxydis, gstamelakos, alexis, Partially supported by 2PARMA Project FP7-ICT

2 INTRODUCTION AND MOTIVATION 2

3 Application paradigm Multi-Threaded applications are becoming increasingly prevalent for emerging systems Unpredictable input data and multiple usecase scenarios lead to unexpected memory footprint variations Dynamic memory services cope with the increased dynamism malloc/free functions in C 3

4 Dynamic memory Impact of Dynamic Memory Management (DMM): DMM forms a performance bottleneck for multithreaded applications (Berger et al., ASPLOS 2000) Impact of DMM on embedded computing domain: DMM heavily affects memory footprint and energy consumption of the embedded device 4

5 Basic concepts Heap pool of memory available for allocation or deallocation of arbitrarily-sized blocks in arbitrary order that will live an arbitrary amount of time Dynamic Memory (DM) Manager or Allocator used to request or return memory blocks to the heap aware only of the size of a memory block, not its type and value implements the various DMM policies and strategies 5

6 Need for runtime tuning To tackle runtime variation problems designers adopt over-designed solutions based on worst-case estimates There is a need for extending towards adaptive implementations Software-controlled runtime tuning will be the main future approach to system designs due to the excessive variations (Yalamanchili et al., 2007) 6

7 State-of-Art in DMM General-purpose computing: Efficient DM managers for multi-threaded applications i.e. Hoard (Berger et al.), Ptmalloc (POSIX), LKmalloc (Larson et al.), Lea (Lea) etc. General DMM implementations not tailored to the application Not so efficient for the embedded domain Embedded computing field: Efficient customization methodologies for application-specific DM managers regarding memory footprint and fragmentation reduction (Atienza et al., TODAES 2006) Exist only for single-threaded applications 7

8 DMM parameter space DYNAMIC MEMORY MANAGEMENT DESIGN SPACE 8

9 Effective DMM Solution Pruning Through Inter-Dependency Analysis There is no need to incorporate free-block movement services in case of serial heap DMMs In case of DMMs with ownership then enable that functionality in the intra-thread Influences the order of de-allocation 9

10 Static vs. Adaptive DMM Static DMM Allocator I/F malloc()/free() F F.2 96 F.2 any H.1 G.2 LM G.2 SM I.1 FIFO J.1 K.1 I.1 H.2 F G.1 SLL F.1 SLL E.4 Header 10

11 Runtime tunable DMM parameters All the parameters that required recompilation of the DMM source code are characterized as DMM non-tunable Indicated two classes of DMM decisions The numerical DMM tunable parameters The algorithmic DMM tunable parameters 11

12 Inter-Heap MTh-DMM Design Space A. Architectural Scheme Decisions B. Data Coherency Decisions 1. Heap Architecture 2. Global Heap 1. Synchronization Mechanisms Single Heap Multiple Heaps Pure Private Heaps Yes No No Synch. Blocking Non-Blocking Serial Heap... Concurrent Heap One Many Mutexes Spin Locks Confict Detection Guard Mech. C. Inter-Heap Allocation Decisions D. Inter-Heap Deallocation Decisions E. Inter-Heap Fragmentation (Memory Blowup) 1. Thread-Heap Mapping 1. Free-Block Movement Strategy 2. Destination Heap 1. Threshold Decisions 2. Number of Moved Free Blocks Each Thread One Heap Mixed All Threads in one Heap w/o Ownership w/ Ownership Source Global Other w/o w/ One... All Hash Mapping Function Static Assignment 1% % RT-TUNABLE MTh-DMM DECISIONS/PARAMETERS 12

13 Intra-Heap MTh-DMM Design Space (1) Intra-Thread Level Design Space E. Block Structure Decisions Block Sizes Block Tags One Size Many Sizes Non-Specific None Header Boundary Tags Block Recorded Info Block Stucture Size Status Pointers SLL DLL Dynamic Array F. Pool Organization Decisions Pool Structure Based on Block Size Pool Structure Based on Blocks Order Pool Structure Based on Block Address One Pool per Size Single Pool One Pool per Block Order Single Pool One Pool per Block Address Single Pool Pool Stucture SLL PARMA 2011, DLLComo, Italy Dynamic Array 13

14 Intra-Heap MTh-DMM Design Space G. Block Allocation Decisions (2) Allocation Search Order Allocation Fit Algorithms FIFO LIFO Address Size First Fit Exact Fit Worst Fit Best Fit H. Block Deallocation Decisions Order Blocks within Pools Destination Pool FIFO LIFO Address Size Source Pool Other Pool J. Splitting Decisions Splitting Frequency Splitting Triggering Criterion Always Often Never Always at deallocation Based on internal fragmentation Low workload Min Block Size Destination Pool One fixed Size Many Fixed Sizes No specific size Source Pool Other Pool K. Coalescing Decisions Coalescing Frequency Coalescing Triggering Criterion Always Often Never Always at deallocation Based on external fragmentation Low workload Block Size Destination Pool One fixed Size Many Fixed Sizes No specific size Source Pool Other Pool 14

15 Runtime tunable DMM parameters Thread-to-Heap mapping Inter-Heap Emptiness Threshold Inter-Heap Percentage of Moved Blocks Allocation Fit Algorithms Best Fit Searching Percentage Splitting Threshold MinSize Coalescing Threshold MaxSize 15

16 Runtime tunable DMM parameters Thread-to-Heap Mapping Inter-Heap Emptiness Threshold Inter-Heap Percentage of Moved Blocks THR_1 THR_2 Heap1 Heap2 10% 10% THR_N HeapK 90% 100% 90% 100% Allocation Fit Algorithms BestFit Searching Percentage FirstFit 10% BestFit NextFit 90% 100% Allocation Search Algorithms Splitting Threshold MinSize Coalescing Thresh. MaxSize FIFO LIFO

17 Static vs. Adaptive DMM Adaptive DMM Allocator I/F malloc()/free() F F.2 96 F.2 any H.1 Tuning Knobs G.2 LM G.2 SM I.1 FIFO J.1 K.1 I.1 H.2 DMM Monitors F G.1 SLL F.1 SLL E.4 Header Controller Platform Monitors 17

18 Runtime DMM monitoring Through software monitors, runtime statistics are collected to provide to the controller useful information at each moment for: Total Memory Footprint Requested memory per time-slot Actual memory per time-slot Per heap memory 18

19 Runtime DMM controller Handles runtime fragmentation-footprint variations The designer specifies the upper bounds of a valid DMM operational range, regarding metrics such as fragmentation and footprint Lower bounds are provided for the targeted DMM metrics Upper bounds are used for adapting the DM manager towards more aggressive fragmentation-footprint optimization knobs in order to avoid a system crisis Lower bounds are used for adapting the DM manager towards more aggressive performance optimizations and less aggressive fragmentation-footprint optimizations 19

20 Fragm. > Max. Fragm. > Max. Fragm. > Max. Runtime DMM controller Fragm. < Max. Footp. < Max. Fragm. < Min. Footp. < Min. Fragm. < Max. Footp. < Max. Fragm. < Min. Footp. < Min. Fragm. < Max. Footp. < Max. Fragm. < Min. Footp. < Min. Fragm. < Max. Footp. < Max. Fragm. > Max. Initial DMM configuration Footp. > Max. Fragm. < Min. Footp. < Min. Fragm. > Max. Fragm. < Max. Footp. > Max. Footp. < Max. BestFit, x% Coalesce, y Split, z Emptiness, k Moved Blk, w% Fragm. < Max. Footp. < Max. BestFit, x++% Coalesce, y++ Split, z-- Fragm. > Max. Fragm. > Max. Fragm. > Max. Footp. > Max. Footp. > Max. Emptiness, k-- Moved Blk, w++% Footp. > Max..... Fragm. < Max. Footp. < Max. Footp. > Max. Footp. > Max. BestFit, Max% Coalesce, ymax Split, zmin Emptiness, kmin Moved Blk, w max % Footp. > Max. Fragm. < Min. Footp. < Min. Fragm. < Min. Footp. < Min. 20

21 EVALUATION 21

22 Application Larson benchmark (Larson and Krishnan, IWMM 1998) Simulates the workload for a multi-threaded server The application repeatedly generates a number of threads that allocate and free memory blocks ranging from 8 to 2000 bytes in a random order The thread that performs the de-allocation is usually spawned after the termination of the thread that performed the allocation 22

23 DM managers used A static performance-optimized DMM (StaticDMM1) First fit policy w/o splitting/coalescing mechanisms A static footprint-fragmentation optimized DMM (StaticDMM2) Best fit policy with 100% searching percentage and splitting/coalescing mechanisms with minimum split size 300 bytes and maximum coalesce size 2000 bytes, A runtime tunable footprint-fragmentation optimized DMM (AdaptiveDMM) 23

24 Fragmentation Runtime fragmentation trace (example) Fragmentation Runtime Evolution in Two DMM Adaptation Points Max. Fragmentation 1,2 1 Adaptation Point 1 Adaptation Point 2 0,8 0,6 0, ,2 0 State: S1 Fit: FirstFit M insplitsize: No split M axcoalsize:no coal. State: S2 Fit: BestFit 20% M insplitsize: 1200 M axcoalsize: 400. Execution DMM Events time State: S3 Fit: BestFit 40% M insplitsize: 1000 M axcoalsize:

25 Fragmentation Fragmentation around worst-case execution windows Comparative Study Between Static and Adaptive DMM Around their Worst Fragmentation Time Windows 2,5 2 Worst Fragm. StaticDMM1 StaticDMM1 StaticDMM2 AdaptiveDMM 1,5 Worst Fragm. AdaptiveDMM 1 Worst Fragm. StaticDMM2 0,5 0 Execution DMM Events time 25

26 Footprint (Bytes) Fragmentation Footprint and fragmentation evaluation Average Footprint Fragmentation , ,1% + 69,9% 0,7 0, ,6% 0,5 0, , ,5% 0,2 0,1 0 StaticDMM1 StaticDMM2 AdaptiveDMM 0 Footprint Fragmentation 26

27 Latency and code size evaluation Overheads Evaluation Execution Latency (sec) DMM Code Size (KB) 1, ,2% 25 0, ,2% 0,6-29,8% 15 0, ,3% 0,2 0 StaticDMM1 StaticDMM2 AdaptiveDMM 5 0 DMM DMM DMM + Control StaticDMM1 StaticDMM2 AdaptiveDMM 27

28 CONCLUSIONS AND FUTURE WORK 28

29 Conclusions Design methodology for runtime tunable dynamic memory management services in order to tackle fragmentation-footprint variations that cannot be efficiently handled only through with design-time decisions Generic enough and can be applied both to generalpurpose and application-specific implementations of DM managers Future work will focus toward: Development of more advanced runtime management schemes Automated generation of the RT-DMM controller 29

30 More info: Alex Bartzas THANK YOU! QUESTIONS? 30