Using Promethee Methods for Multicriteria Pull-based scheduling in DCIs

Size: px

Start display at page:

Download "Using Promethee Methods for Multicriteria Pull-based scheduling in DCIs"

Rodger Jones
10 years ago
Views:

1 Using Promethee Methods for Multicriteria Pull-based scheduling in DCIs Mircea MOCA Babeş-Bolyai University Cluj-Napoca România Gilles FEDAK LIP, INRIA Universite de Lyon France

Cluj-Napoca România Mircea.Moca@econ.ubbcluj.

2 Agenda Scheduling context, pull mechanism Key concepts, challenge Background: the Promethee model Simulator Scenarios Results Current work Conclusions & future work 2 escience 2012, Chicago

3 Why pull? Scalability Host config. intimacy, abstraction Deal with different types of DCI Job/task scheduling Scheduler key component Use Promethee to make scheduling decisions Concepts: Host, resource: CP, Price, ErrorRate Specific to a DCI type Pulling host H pull Work unit ex., task/job 3 escience 2012, Chicago Scheduling - context (1) List of tasks (2) Ranking

component Use Promethee to make scheduling decisions Concepts: Host, resource: CP, Price,

4 Promethee II Context: - the scheduler = the decision maker - choose the task (from the set of alternatives) that best fits to the characteristics of the pulling host (Pow, Price, ) - what task to choose for this particular pulling host? 4 escience 2012, Chicago

the characteristics of the pulling host (Pow, Price, ) - what

5 Promethee II Promethee: A multi-criteria decision aid based on pair-wise comparisons of the alternatives (Promethee I and II, J.P. Brans, 1982) Prerequisites: - List of criteria that characterize tasks host (resource) independent: NOI host dependent: ECT, Cost, Error Rate - Weights of importance for the criteria - (allows a degree of subjectivity), -Target per criterion: min/max (final ranking), - Preference function (allows a degree of subjectivity) 5 escience 2012, Chicago

Brans, 1982) Prerequisites: - List of criteria that characterize tasks host (resource) independent: NOI host dependent:

6 Promethee II how it works 1 Example: Host: Pow(H pull ) = 5, Price(H pull ) = 0.2 Algorithm:? 1. Set of alternatives -> build the evaluation matrix - N c x N t T 1 NOI=100 T 2 NOI=200 T 3 NOI=400 ECT Cost escience 2012, Chicago

7 Jean-Pierre Brans, Bertrand Mareschal, Multiple Criteria Decision Analysis State of the Art Surveys 7 T 1 T 2 T 3 Promethee II how it works 2 2. For each criterion, calculate preference relations for all pairs of tasks T 1 NOI=100 T 2 NOI=200 T 3 NOI=400 ECT Cost T 1 T 2 T 3 0 P( d(t 1,T 2 ) ) P( d(t 2,T 1 ) ) 0 0 escience 2012, Chicago

For each criterion, calculate preference relations for all pairs of tasks T 1 NOI=100 T

8 Promethee II how it works 3 3. For each task & criterion, calculate positive and negative outranking flows 4. Calculate net flow T 0 T 1 T 2 T 0 T 1 T 2 T 0 0 P( d(t 0,T 1 ) ) ECT T 1 P( d(t 1,T 0 ) ) 0 Cost T 0 T 1 T 2 T escience 2012, Chicago

9 Promethee II how it works 4 4. Calculate the net (and aggregated) flow T 0 T 1 T 2 ECT + ECT (ECT) Aggregated net flow + T 0 T 1 T 2 (Cost) Cost 9 escience 2012, Chicago Cost Compute final ranking, select the top-most one

10 Promethee II - main strenghts Yields a complete ranking of the alternatives the top-most ranked task is the best Employs pair-wise comparisons among the evaluations of the tasks within a criterion The decision maker can assign weights of importance for each criterion allows building user-aware scheduling policies Choose (define) a preference function The user can define prioritizing policies in the scheduler 10 escience 2012, Chicago

can assign weights of importance for each criterion allows building user-aware scheduling policies Choose

11 Research approach Hypothesis: -The Promethee MCD model can be used to efficiently schedule jobs in DCIs. - Using the MCD model, the scheduler can adapt its scheduling strategies in order to better respond to user s aims (within a bag of work units execution). Methodology: - Develop the pull-based & Promethee-inspired scheduler method - Develop an event and trace based simulator - Experiment with specific, relevant scenarios: compare the Promethee-based approach with FCFS and ideal (sufferage heuristic) methods. 11 escience 2012, Chicago

12 Event-based simulator running on real traces. Failure trace archive. Available: Pull-based, MCD scheduler simulator (FTA) Simulated DCI environment 12 escience 2012, Chicago Priority queue Reading traces: -configurable: no. of hosts, time period/trace Events: -type: HJ, HLE, WKUA, SCH, RES - attributes: -Timestamp -Host -Task Experiment: -The run of the simulator for one wk.-load Execution: - Run the simulator -> complete a wk.load.

13 Experimentation setup DCIs: Idg: BOINC, 691 hosts, 18 months, from 2010 cloud: Amazon EC2 si, 1754 hosts, 6 months, 2011 beg: Grid5000, 2256 hosts, Bordeaux, Grenoble, Lille and Lyon, 12 months, 2011 Criteria: ECT, Price Metrics: makespan, cost 13 escience 2012, Chicago

Grid5000, 2256 hosts, Bordeaux, Grenoble, Lille and Lyon, 12

14 Results 14 escience 2012, Chicago

$Performance - makespan Scenarios: in time: all hosts return results exactly at ECT delay: a fraction of the hosts return results with certain delays (effect:$

15 Performance - makespan Scenarios: in time: all hosts return results exactly at ECT delay: a fraction of the hosts return results with certain delays (effect: reschedule, repl., cost) fail: a fraction of the tasks never yield results (effect: reschedule, repl., cost) 15 Makespan escience 2012, for 3 Chicago scenarios scenarios.

$, cost) fail: a fraction of the tasks never yield results (effect: , cost) 15$

16 2 Criteria ECT & Price Makespan and cost for 2 criteria scenario, variate the importance weights of criteria. 16 escience 2012, Chicago

17 Failure scenario Makespan for various tasks failure degrees. 17 escience 2012, Chicago Makespan difference between Promethee and FCFS.

18 Current work -Passing to hybrid DCIs -Choosing the preference function -2 criteria -> 3 criteria: ECT, Cost, Error rate -user-based scenarios 18 escience 2012, Chicago

19 Choosing a preference function Challenge: Finding an efficient preference function 19 escience 2012, Chicago

20 Real execution times for different preference functions Scheduling improvement mechanisms bound task queue 20 escience 2012, Chicago

21 Setting p and q 21 escience 2012, Chicago

22 Tuning the Level preference function Makespan for different values of p and q. 22 escience 2012, Chicago

23 Conclusions and future work The proposed approach can be successfully used in scheduling tasks in DCIs Allows true multi-criteria scheduling decisions that can lead to a customized execution -> user oriented, allows prioritization policies Proves to decrease makespan up to 32% for IDG in fail scenario Finding optimal weights for the criteria can be hard Hard to analyze behavior for more criteria Validation process for Hybrid DCIs ECT, Cost, Error Rate Parallelization of the scheduling method Larger wk. loads Integrate the proposed scheduling mechanism in XtremeWeb 23 escience 2012, Chicago

Volunteer Computing, Grid Computing and Cloud Computing: Opportunities for Synergy. Derrick Kondo INRIA, France

Volunteer Computing, Grid Computing and Cloud Computing: Opportunities for Synergy Derrick Kondo INRIA, France Outline Cloud Grid Volunteer Computing Cloud Background Vision Hide complexity of hardware