SimGrid Cloud Broker: Simulation of Public and Private Clouds

Transcription

1 SimGrid Cloud Broker: Simulation of Public and Private Clouds Jonathan Rouzaud-Cornabas CNRS CC-IN2P3 / LIP (UMR 5668) J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 1 / 2

2 SimGrid Cloud Broker SimGrid simulates many different distributed systems: clusters, wide-area and local-area networks, peers over DSL connexions, data centers, etc has models are theoretically and experimentally assessed is scalable exists for 13 years Simulation of Clouds From an user point of view From a cloud provider point of view Evaluates application(s) running on Clouds Evaluates different policies in Cloud middleware Multi-Clouds (private and public) J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 2 / 2

3 Architecture J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 3 / 2

4 AWS Amazon Web Services S3 / EC2 API All the instance types All the regions of AWS On-demand and Spot Instances 3 types of storage: local, EBS and S3 Accounting of network, computing and storage resources Models: Spot instance prices: smart random, file and prediction model Life cycle of VMs Storage (3 models) Compute Network: VM inter/intra-regions and S3 J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 4 / 2

5 Amazon AWS Platforms AWS J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 5 / 2

6 Private Cloud IaaS Platforms Simulation J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 6 / 2

7 Private Cloud IaaS Computing Simulation Basic compute API (loosely based on OpenStack) Eucalyptus and OpenStack models (in progress) J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 7 / 2

8 Easy-to-use Easy-to-use simulator Easy to use platform description One XML file to describe all the Clouds SimGrid users: only few modifications to migrate a SimGrid code to SGCB code Easy to replace and extend: VM image deployment and VM to PM mapping policies Modular approaches for all the models J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 8 / 2

9 Scalability Studying the scalability of SGCB Tested on the simulated AWS platform with all the models activated Mid-term goal: 1 millions VMs on 1, Hosts 4 types of scenario (1 N 5) 1 N instances m1.large in eu 1 2 N instances of each instance types in eu 1 3 N instances m1.large in all regions 4 N instances of each instance types in all regions J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 9 / 2

10 Scalability of SGCB Scalability 15 SimTime NBVM J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 1 / 2

11 Scalability 12 9 Duration 6 State Boot Deploy Running Shutdown AMISIZE J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 11 / 2

12 Scalability Simple bag of tasks Simple Bag Of Tasks scenario BoT Characteristics 1, Tasks Input data: 1GB Output data: 5MB Storage Input and/or Output: Local, EBS ou S3 1 master (CeS): m2.2xlarge 5 slaves (CoS): m1.large J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 12 / 2

13 Scalability Simple bag of tasks Scenario si_s3_s3 si_s3_local si_s3_ebs si_local_s3 si_local_local si_local_ebs si_ebs_s3 si_ebs_local si_ebs_ebs od_s3_s3 od_s3_local od_s3_ebs od_local_s3 od_local_local od_local_ebs od_ebs_s3 od_ebs_local od_ebs_ebs State Start RequestVM SendTasks ReceiveResults TerminateVM End e+ 5e+5 Duration 1e+6 J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 13 / 2

14 Scalability Simple bag of tasks Scenario si_s3_s3 si_s3_local si_s3_ebs si_local_s3 si_local_local si_local_ebs si_ebs_s3 si_ebs_local si_ebs_ebs od_s3_s3 od_s3_local od_s3_ebs od_local_s3 od_local_local od_local_ebs od_ebs_s3 od_ebs_local od_ebs_ebs State SendInput ReceiveInput WriteInput ExecuteTask ReadOutput SendOutput ReceiveOutput 5 Duration 1 15 J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 14 / 2

15 Scalability Simple bag of tasks Type S3 EBS Scenario od_ebs_ebs od_ebs_local od_ebs_s3 od_local_ebs od_local_local od_local_s3 od_s3_ebs od_s3_local od_s3_s3 si_ebs_ebs si_ebs_local si_ebs_s3 si_local_ebs si_local_local si_local_s3 si_s3_ebs si_s3_local si_s3_s3 5 GBMonth 1 J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 15 / 2

16 Scalability Simple bag of tasks Type Network S3 Intranet Network VM Internet Network VM Scenario od_ebs_ebs od_ebs_local od_ebs_s3 od_local_ebs od_local_local od_local_s3 od_s3_ebs od_s3_local od_s3_s3 si_ebs_ebs si_ebs_local si_ebs_s3 si_local_ebs si_local_local si_local_s3 si_s3_ebs si_s3_local si_s3_s3 e+ 2e+11 GB 4e+11 6e+11 J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 16 / 2

17 Scalability Simple bag of tasks Scenario si_s3_s3 si_s3_local si_s3_ebs si_local_s3 si_local_local si_local_ebs si_ebs_s3 si_ebs_local si_ebs_ebs od_s3_s3 od_s3_local od_s3_ebs od_local_s3 od_local_local od_local_ebs od_ebs_s3 od_ebs_local od_ebs_ebs Type EBS Network S3 Network VM S3 VM OD VM SI 5 1 Price 15 2 J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 17 / 2

18 Scalability Simple bag of tasks m1.small m1.large m1.xlarge m2.xlarge m2.2xlarge m2.4xlarge c1.medium c1.xlarge 1.5 Price 1..5 Region brasil_1 asia_2 asia_1 eu_1 usa_east_1 usa_west_1 usa_west_ Hour J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 18 / 2

19 Use case SGCB usage in the Inria Avalon team Study different resource reservation and task allocation algorithms for Bag Of Tasks on Clouds Multi-regions applications and inter-region migration on AWS (Jose Luis Lucas UCM) Scientific Workflows on Clouds (Dao Van Toan Master internship) Applications composed of multiple VMs with security requirements (Arnaud Lefray Doctorant) Component based applications on federated PaaS (FP7 PaaSage) J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 19 / 2

20 Future work Future work Validation of the simulation results with experiment on AWS Finer grain models for AWS Running the same experimentation on a private Cloud Study the different between OpenStack and OpenNebula on same hardware and applications Models for private Clouds Integrate works around live-migration Multi-core for VMs models in SimGrid Increase the scalability J. Rouzaud-Cornabas (CNRS) SimGrid Cloud Broker 2 / 2