Linux Process Scheduling. sched.c. schedule() scheduler_tick() hooks. try_to_wake_up() ... CFS CPU 0 CPU 1 CPU 2 CPU 3

Transcription

1 Linux Process Scheduling sched.c schedule() scheduler_tick() try_to_wake_up() hooks RT CPU 0 CPU 1 CFS CPU 2 CPU 3

2 Linux Process Scheduling 1. Task Classification 2. Scheduler Skeleton 3. Completely Fair Scheduler (CFS) sched.c schedule() scheduler_tick() try_to_wake_up() hooks RT 4. Real Time Scheduling (RT) 5. Load Balancing CFS CPU 0 CPU 1 CFS 6. Load Balancing RT CPU 2 CPU 3

3 1. Task Classification Task Types Real Time vs Normal/Other CPU bound vs I/O bound Efficient vs Responsive Server vs Desktop vs HPC

4 1. Task Classification Scheduling Classes include/linux/sched.h struct sched_class kernel/sched_rt.c rt_sched_class.next = &fair_sched_class const struct sched_class *next (*enqueue_task) (*dequeue_task) (*check_preempt_curr) (*pick_next_task) (*select_task_rq) (*pick_next_task).enqueue_task = &enqueue_task_rt.dequeue_task = &dequeue_task_rt kernel/sched_fair.c fair_sched_class.next = &idle_sched_class.enqueue_task = &enqueue_task_fair.dequeue_task = &dequeue_task_fair

5 1. Task Classification Scheduling Class Priorities stop rt fair idle per CPU stop task real time tasks normal tasks per CPU idle task Highest Lowest Priority

6 2. Scheduler Skeleton Scheduler Entry Point Per CPU runqueue with sub runqueues for CFS and RT Remove or put back to runqueue different task selected than before? Pull tasks from other CPUs kernel/sched.c schedule() Look for highest priority class with available tasks

7 2. Scheduler Skeleton Calling the Scheduler 1. Timer Interrupt 2. Currently running task goes to sleep 3. Sleeping task wakes up

8 2. Scheduler Skeleton Calling the Scheduler - Timer kernel/sched.c scheduler_tick() { sched_class->task_tick() } CPU 0 CPU 1 CPU 2 CPU 3 kernel/sched_x.c task_tick_x() { } X_sched_class update task s scheduling entity is it another task s turn yet? Yes Invoke scheduling

9 2. Scheduler Skeleton Calling the Scheduler - Sleep

10 2. Scheduler Skeleton Calling the Scheduler Wake up kernel/sched.c try_to_wake_up() { sched_class->enqueue_task() sched_class->check_preempt_curr() task.state = TASK_RUNNING } CPU 0 CPU 1 CPU 2 CPU 3 kernel/sched_x.c enqueue_task_x() { } check_preempt_curr_x() { } X_sched_class put task into runqueue Awoken task higher prio than current task? Yes Invoke scheduling

11 3. Completely Fair Scheduler Concept Virtual Runtime CFS tries to maintain an equal virtual runtime for each task in a CPU s runqueue at all time.

12 3. Completely Fair Scheduler Virtual Runtime Same Priority Target Latency 20ms 10ms 10ms Priorities Time elapses slower for higher priorities 5ms 5ms 5ms 5ms Minimum Granularity 1ms 1ms 1ms 1ms 1ms How does it work out for I/O bound and CPU bound Tasks?

13 3. Completely Fair Scheduler Runqueue - Red-Black-Tree Red-Black-Tree self balancing binary search tree O(logN) operations vruntime used as key in the RB-Tree gravest need for CPU 7 25 NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL virtual runtime

14 3. Completely Fair Scheduler Fair Group Scheduling Without Group Scheduling User 1 User 2 With Group Scheduling User 1 User 2 T0 T1 T2 T3 T0 T1 T2 T3 G1 50% G2 50% 25% 25% 25% 25% 16.7% 16.7% 16.7% 50% CPU CPU First, be fair to groups then, be fair to tasks.

15 4. Soft-Real-Time Scheduling Scheduling Modes Scheduling of tasks with strict timing requirements. RT scheduling is reliable but kernel does not guarantee that deadlines will be met. SCHED_FIFO no time slices until termination or yielding CPU T1 T2 T3 SCHED_RR fixed time slice round robin scheduling T1 T2 T3

16 4. Soft-Real-Time Scheduling Runqueue Priority Queues Priority Bitmap Allows finding the highest priority task in O(1) 0 0 T2 FIFO 7 T0 RR 9 T1 RR 7 T4 RR 9 T5 RR 9 99 Priority Queues One task queue for each priority 99 T3 FIFO 12

17 5. Load Balancing CFS Scheduling Domains CFS load balancing is used to offload task from busy CPUs to less busy or idle ones on SMP Systems. Domain Specific Balancing Policy how often to do balancing how far to move tasks how long before cache cools down Scheduling Domains Handle the topology variety of modern processor.

18 5. Load Balancing CFS Active and Idle Balancing kernel/sched_fair.c Balancing walks up the domain hierarchy and looks for the busiest group Active Balancing regularly by scheduler_tick() pulls tasks over from busiest group per domain Idle Balancing as soon as CPU goes idle in schedule() checks if average idle time is larger than migration cost pulling tasks like active balancing

19 5. Load Balancing CFS Where To Put a New Task kernel/sched.c select_task_rq(flag) { sched_class->select_task_rq(flag) } kernel/sched_fair.c fair_sched_class select_task_rq_fair(flag) { } SD_BALANCE_EXEC used in sched_exec() upon starting a new task with exec() SD_BALANCE_FORK Returns optimal CPU to put the task on used in wake_up_new_task() upon a fork command Looks for idlest CPU considering only those domains that have flag set in their balancing policy SD_BALANCE_WAKE used in try_to_wake_up() upon waking up a task that already ran

20 6. Load Balancing RT Root Domains and CPU Pri RT load balancing aims to make sure that the N highest priority tasks on the system are running at all time where N is the number of CPUs. struct root_domain struct cpupri overload mask CPU Pri CPU 0 CPU 1 CPU 2 CPU 3 Root Domain scope for RT scheduling decisions overall overload and priority state

21 6. Load Balancing RT Push A Low Priority Task is Pushed to Another CPU If post_schedule() lower prio task wakes up on CPU with higher prio task running higher prio task wakes up on CPU and preempts lower prio task T2 needs to be pushed away get CPU Lowest Mask represents the lowest priority tasks in the system Elect best target CPU based on cache affinity and topology CPU 4 94 CPU 0 84 CPU 2 87 T2 93 CPU 7 84

22 6. Load Balancing RT Pull A High Priority Task is Pulled from Another CPU If pre_schedule() priority of task to be scheduled would be lower than previously one s T0 89 prev CPU 0 85 CPU 1 90 T1 85 T2 90 T3 87 T4 83 next Pulled over

23 Linux Process Scheduling Thanks, Questions? sched.c schedule() scheduler_tick() try_to_wake_up() hooks RT CPU 0 CPU 1 CFS CPU 2 CPU 3