Scheduling Support for Heterogeneous Hardware Accelerators under Linux

Transcription

1 Scheduling Support for Heterogeneous Hardware Accelerators under Linux Tobias Wiersema University of Paderborn Paderborn, December / 24 Tobias Wiersema Linux scheduler extension for accelerators

2 Introduction Motivation Basics Concept Implementation Time sharing Migration Efficiency 2 / 24 Tobias Wiersema Linux scheduler extension for accelerators

3 Motivation Basics Accelerator-based heterogeneous systems Accelerators: General-purpose GPUs, FPGAs, ClearSpeed boards,... Accelerated systems increasingly important Off-the-shelf components Hybrid processor/accelerator designs (Cell BE, Virtex II Pro, Excalibur) Supercomputing (TSUBAME, TianHe-1a) New programming models (data-parallel execution) Cost-efficient and power-efficient designs Problem: Development of accelerated software 3 / 24 Tobias Wiersema Linux scheduler extension for accelerators

4 Current situation user space Accelerated applications kernel space Runtime library Driver Scheduler hardware Accelerator CPUs

5 Operating system integration Motivation Basics Integrate into OS kernel Hardware abstraction Master-Slave-relationship Communication Synchronization Scheduling: Global decisions 5 / 24 Tobias Wiersema Linux scheduler extension for accelerators

6 Operating system integration Motivation Basics Integrate into OS kernel Hardware abstraction Master-Slave-relationship Communication Synchronization Scheduling: Global decisions 5 / 24 Tobias Wiersema Linux scheduler extension for accelerators

7 Time Sharing Introduction Motivation Basics Historic predecessor: Multiprogramming Enhance processor utilization Time sharing Take fair turns on processor Shared usage of sought-after resources Tasks Users Today mostly relies on preemption 6 / 24 Tobias Wiersema Linux scheduler extension for accelerators

8 Motivation Basics Completely Fair Scheduler Current Linux scheduler: CFS Previous: O(n), O(1) Fairness O(log n): Red-black-tree Virtual time 7 / 24 Tobias Wiersema Linux scheduler extension for accelerators

9 Concept Implementation Introduction Motivation Basics Concept Implementation Time sharing Migration Efficiency 8 / 24 Tobias Wiersema Linux scheduler extension for accelerators

10 Current situation user space Accelerated applications kernel space Runtime library Driver Scheduler hardware Accelerator CPUs

11 Scheduling possibilities user space Accelerated applications kernel space Runtime library Driver Scheduler hardware Accelerator CPUs

12 Concept Implementation Possible approaches Interrupt path: Application accelerator 1. Application runtime library 2. Runtime library driver Pro and contra + Transparent scheduling + Unchanged applications Accelerator specific No time sharing No task migration 11 / 24 Tobias Wiersema Linux scheduler extension for accelerators

13 Concept Implementation Possible approaches Interrupt path: Application accelerator 1. Application runtime library 2. Runtime library driver Pro and contra + Transparent scheduling + Unchanged applications Accelerator specific No time sharing No task migration 11 / 24 Tobias Wiersema Linux scheduler extension for accelerators

14 Approach: user space Accelerated applications kernel space Runtime library Driver Scheduler extension Scheduler hardware Accelerator CPUs

15 Concept Implementation Kernel extension and programming model with Cooperative multitasking Checkpointing Pro and contra No transparent scheduling No unchanged applications + Not accelerator specific + Time sharing without preemption + Task migration 13 / 24 Tobias Wiersema Linux scheduler extension for accelerators

16 Concept Implementation Kernel extension and programming model with Cooperative multitasking Checkpointing Pro and contra No transparent scheduling No unchanged applications + Not accelerator specific + Time sharing without preemption + Task migration 13 / 24 Tobias Wiersema Linux scheduler extension for accelerators

17 Scheduling granularity Concept Implementation time spent on accelerator a) granularity copy execution allocation re-request re-request free time spent on accelerator b) granularity copy execution allocation re-request free 14 / 24 Tobias Wiersema Linux scheduler extension for accelerators

18 Concept Implementation Challenges / Design decisions Schedulable entity: Thread Ghost threads Static affinity model Problem: Affinity inversion 15 / 24 Tobias Wiersema Linux scheduler extension for accelerators

19 Integration into Linux kernel Concept Implementation Once One per computing unit One per task struct computing_units struct computing_unit_info struct cfs_rq rq struct hardware_properties hp struct task_struct struct sched_entity se struct sched_entity hwse struct hardware_properties struct sched_entity struct meta_info mi struct cfs_rq list tasks rb_tree tasks_timeline struct meta_info mi 16 / 24 Tobias Wiersema Linux scheduler extension for accelerators

20 Time sharing Migration Efficiency Introduction Motivation Basics Concept Implementation Time sharing Migration Efficiency 17 / 24 Tobias Wiersema Linux scheduler extension for accelerators

21 Time sharing of an accelerator Time sharing Migration Efficiency tasks on one GPU searched strings in billions seconds 18 / 24 Tobias Wiersema Linux scheduler extension for accelerators

22 Heterogeneous task migration Time sharing Migration Efficiency tasks on either CPU or GPU searched strings in billions seconds 19 / 24 Tobias Wiersema Linux scheduler extension for accelerators

23 Time sharing Migration Efficiency Task switching overhead tasks on one GPU seconds execution time average turnaround time sec 1sec 2sec 4sec fcfs granularity setting 20 / 24 Tobias Wiersema Linux scheduler extension for accelerators

24 Time sharing Migration Efficiency Load balancer 100% 90% 80% 70% 60% 50% 40% 30% load balancing MD5 computation PF computation 20% 10% 0% concurrent tasks 21 / 24 Tobias Wiersema Linux scheduler extension for accelerators

25 Conclusions of the kernel extension Kernel-controlled time sharing Accelerator sharing Fairness Heterogeneous task migration Speedup Load balancing Exploit heterogeneity Enhance accelerator utilization 22 / 24 Tobias Wiersema Linux scheduler extension for accelerators

26 Future work Enhance load balancer Combine with enhanced runtime libraries Automate programming model Enhance ghost threads Communication Synchronization 23 / 24 Tobias Wiersema Linux scheduler extension for accelerators

27 Thank you for your attention! Do you have questions? 24 / 24 Tobias Wiersema Linux scheduler extension for accelerators

28 Completely Fair Scheduler Current Linux scheduler: CFS Previous: O(n), O(1) Fairness O(log n): Red-black-tree Virtual clock No time slices 25 Tobias Wiersema Linux scheduler extension for accelerators

29 Notation of time sharing task timings turnaround time a) A1 B1 A2 A3 A1 B1 A2 latency execution time latency execution time A3 b) A1 B1 A2 B1 A3 B1 A3 A2 B1 A1 turnaround time 26 Tobias Wiersema Linux scheduler extension for accelerators

30 Task migration with true time sharing searched strings in billions a) b) seconds 27 Tobias Wiersema Linux scheduler extension for accelerators

31 Task migration with QL5 and no fixed set of tasks searched strings in billions a) seconds 28 Tobias Wiersema Linux scheduler extension for accelerators

32 Runqueue of an accelerator Running Waiting... Runqueue with ghost threads... Execution units Accelerator 29 Tobias Wiersema Linux scheduler extension for accelerators

33 Affinity Inversion example Step 1 Step 2 Step 3 Step 4 Unit 1 A A A C C Unit 2 B B B 30 Tobias Wiersema Linux scheduler extension for accelerators

34 Overview of data structures Once One per computing unit One per task struct computing_units list list_of_cus[type] current_id access_mutex struct computing_unit_info id type struct cfs_rq rq struct hardware_properties hp struct task_struct struct sched_entity se struct sched_entity hwse struct hardware_properties concurrent_kernels bandwitdth struct cfs_rq load min_vruntime list tasks rb_tree tasks_timeline count maxcount access_mutex struct sched_entity load semaphore_up need_migrate task_granularity_nsec current_affinity offerer offered_affinity struct meta_info mi 31 Tobias Wiersema Linux scheduler extension for accelerators

35 Task states if using an accelerator cooperatively Non-accelerated computation Accelerated computation No computing unit assigned free allocate computing unit assignment invalid finished computation or denied re-request Computing unit assignment valid successful re-request 32 Tobias Wiersema Linux scheduler extension for accelerators

36 Turnaround times with infinite granularity 75 Tasks (25 MD5, 50 PF), started at the same time seconds tasks 33 Tobias Wiersema Linux scheduler extension for accelerators

37 Turnaround times with four second granularity 75 Tasks (25 MD5, 50 PF), started at the same time seconds tasks 34 Tobias Wiersema Linux scheduler extension for accelerators