Con PaaS. ConPaaS: a Platform as a Service for Multi-clouds. Guillaume Pierre Rennes 1 University, France

Transcription

1 ConPaaS: a Platform as a Service for Multi-clouds Guillaume Pierre Rennes 1 University, France Con PaaS ConPaaS: a Platform as a Service for Multi-clouds 1 / 47

2 Table of Contents 1 Introduction 2 ConPaaS 3 Architecture 4 Multi-cloud ConPaaS 5 Conclusion ConPaaS: a Platform as a Service for Multi-clouds Introduction 2 / 47

3 Lucas Cranach the Elder, ConPaaS: a Platform as a Service for Multi-clouds Introduction 3 / 47

4 The Cloud of Eden Lucas Cranach the Elder, ConPaaS: a Platform as a Service for Multi-clouds Introduction 3 / 47

5 Don't touch this proprietary API! The Cloud of Eden Lucas Cranach the Elder, ConPaaS: a Platform as a Service for Multi-clouds Introduction 3 / 47

6 Proprietary Cloud APIs ConPaaS: a Platform as a Service for Multi-clouds Introduction 4 / 47

7 Proprietary cloud APIs Some cloud services are (becoming) well-standardized VM instance creation/management/destruction Data storage But all the high-level services are cloud-specic Amazon: RDS, DynamoDB, AutoScaling, CloudWatch,... Google: APIs to access lots of Google services Etc. Why is this so? It is impossible to innovate while respective standard APIs Commercial lock-in strategy ConPaaS: a Platform as a Service for Multi-clouds Introduction 5 / 47

8 Wanted: cloud provider independence Example: if you want DynamoDB you must use the Amazon cloud Just because it was invented by Amazon The choice of a set of high-level services should not lock users with a single provider Think of mobile phone operator, electricity providers, etc. How can we oer the same high-level services in multiple clouds? Option 1: standardize APIs (mostly impossible for the moment) Option 2: let third-parties develop cloud-independent services You also want seamless application migration from one cloud to another ConPaaS: a Platform as a Service for Multi-clouds Introduction 6 / 47

9 VM migration is not a solution VM migration sounds like an appealing solution VMs currently run in Cloud X Live-migrate them to cloud Y and the problem is solved! Unfortunately things are not so simple... Business considerations: No cloud provider wants to oer support to users who want to leave their system Technical considerations: Dierent clouds often use dierent, incompatible hypervisors VMs make use of local services oered by the cloud VMs cannot maintain the same IP address after migration (we would need Mobile IP) ConPaaS: a Platform as a Service for Multi-clouds Introduction 7 / 47

10 Platform-as-a-Service to the rescue Platform-as-a-Service systems are usually elastic They can dynamically change the set of resources they are using We can exploit these elasticity properties to implement application migration 1 The application runs in cloud X 2 Give new resources from cloud Y to the application 3 Replicate data to the new resources 4 Update the application's access point (e.g., using DNS) 5 Shift workload to the new resources 6 Retract old resources from cloud X ConPaaS: a Platform as a Service for Multi-clouds Introduction 8 / 47

11 ConPaaS: a Platform as a Service for Multi-clouds Introduction 9 / 47

12 Table of Contents 1 Introduction 2 ConPaaS 3 Architecture 4 Multi-cloud ConPaaS 5 Conclusion ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 10 / 47

13 Hosting a simple application in a multi-cloud environment ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 11 / 47

14 Hosting a simple application in a multi-cloud environment ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 11 / 47

15 ConPaaS takes care of your applications ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 12 / 47

16 Web hosting services Code upload (PHP, Java) ZIP le upload, git push Choose the version you want to run Seamless capacity control Add/remove processing power in one click Coordinated recongurations No service interruption even during recongurations ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 13 / 47

17 Database services Relational (MySQL) and NoSQL (Scalarix) Data upload Choose your administrator password Upload a database dump Automatic replication Add/remove replicas in one click ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 14 / 47

18 File system service Mount the le system From your client machine From other ConPaaS services Read-write data as in a local le system Full POSIX support Automatic capacity scaling Add/remove storage nodes in one click ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 15 / 47

19 High-performance computing services TaskFarming for scientic workows and massive batch executions Control the tradeo between cost and execution time MapReduce for BigData processing Based on Hadoop ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 16 / 47

20 Automated application deployment ConPaaS can deploy entire applications in one click A manifest le species all relevant information about the application List of services, code/data to upload, conguration Users can use ready-made manifests or upload their own ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 17 / 47

21 Release Timeline Oct 2010: Beginning of the project Apr 2012: ConPaaS Five services: PHP, Java, MySQL, Scalarix, MapReduce Support of Amazon EC2 and OpenNebula clouds Oct 2012: ConPaaS TaskFarming and Selenium services Secure control communications GIT-based code uploads Feb 2013: ConPaaS XtreemFS service Full control via command-line tools Jun 2013: ConPaaS Multi-cloud support IPOP support for internal application communications Automated deployment of entire applications Performance monitoring ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 18 / 47

22 Release Timeline Oct 2010: Beginning of the project Apr 2012: ConPaaS Five services: PHP, Java, MySQL, Scalarix, MapReduce Support of Amazon EC2 and OpenNebula clouds Oct 2012: ConPaaS TaskFarming and Selenium services Secure control communications GIT-based code uploads Feb 2013: ConPaaS XtreemFS service Full control via command-line tools Jun 2013: ConPaaS Multi-cloud support IPOP support for internal application communications Automated deployment of entire applications Performance monitoring ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 18 / 47

23 ConPaaS: teams ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 19 / 47

24 ConPaaS: industrial partners ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 20 / 47

25 ConPaaS: funding ConPaaS: a Platform as a Service for Multi-clouds ConPaaS 21 / 47

26 Table of Contents 1 Introduction 2 ConPaaS 3 Architecture 4 Multi-cloud ConPaaS 5 Conclusion ConPaaS: a Platform as a Service for Multi-clouds Architecture 22 / 47

27 ConPaaS Services A ConPaaS service is composed of 1 Manager and n Agents One Python class running in the Manager VM One Python class running in the Agent VM ConPaaS: a Platform as a Service for Multi-clouds Architecture 23 / 47

28 ConPaaS Services: Manager All ConPaaS manager expose the same set of standard methods... startup get_logs add_nodes remove_nodes list_nodes get_node_info shutdown... and any number of service-specic methods (here: MySQL) load_dump set_password ConPaaS: a Platform as a Service for Multi-clouds Architecture 24 / 47

29 ConPaaS Services: Agent All ConPaaS agents expose the same set of standard methods... check_agent_process... and any number of service-specic methods (here: MySQL) setup_master setup_slave load_dump set_password ConPaaS: a Platform as a Service for Multi-clouds Architecture 25 / 47

30 The ConPaaS core Writing a new service means writing two Python classes: A manager class An agent class These class inherit from the ConPaaS core IaaS HTTPS (certicate management etc.) IPOP (networking) Ganglia (monitoring)... ConPaaS: a Platform as a Service for Multi-clouds Architecture 26 / 47

31 ConPaaS Applications An application is dened as a set of ConPaaS services For example: PHP + MySQL + XtreemFS An application can be specied using a manifest: { " D e s c r i p t i o n " : " Sudoku e x a m p l e ", " S e r v i c e s " : [ { " ServiceName " : "PHP sudoku backend ", " Type " : " php ", " S t a r t " : 0, " A r c h i v e " : " h t t p : / /www. e x a m p l e. o r g / sudoku. t a r. gz " } ] } ConPaaS: a Platform as a Service for Multi-clouds Architecture 27 / 47

32 The ConPaaS Director The single entry point to the system Keeps tracks of users, credentials, services Handles the life-cycle of ConPaaS applications ConPaaS: a Platform as a Service for Multi-clouds Architecture 28 / 47

33 ConPaaS clients Clients make it easy to send commands to the Director Command-line interface Web-based GUI ConPaaS: a Platform as a Service for Multi-clouds Architecture 29 / 47

34 Table of Contents 1 Introduction 2 ConPaaS 3 Architecture 4 Multi-cloud ConPaaS 5 Conclusion ConPaaS: a Platform as a Service for Multi-clouds Multi-cloud ConPaaS 30 / 47

35 Multi-cloud increases exibility Points-of-presence closer to the user Lower latency, higher bandwidth ConPaaS: a Platform as a Service for Multi-clouds Multi-cloud ConPaaS 31 / 47

36 Multi-cloud applications are more reliable Replicate data across multiple data centers In case of failure: handover to other locations ConPaaS: a Platform as a Service for Multi-clouds Multi-cloud ConPaaS 32 / 47

37 Multi-cloud applications are (potentially) more secure Keep sensitive data in your private cloud Distribute data across clouds such that no cloud has all the information ConPaaS: a Platform as a Service for Multi-clouds Multi-cloud ConPaaS 33 / 47

38 Multi-cloud support in ConPaaS ConPaaS can use multiple underlying clouds simultaneously For example: OpenNebula in my private cluster, EC2 in Virginia, and EC2 in Oregon We can easily add support for other clouds Each virtual machine can be started in any cloud ConPaaS: a Platform as a Service for Multi-clouds Multi-cloud ConPaaS 34 / 47

39 Network issues in multi-cloud Network issues: Disjoint IP address space Firewalls, private addresses, NATs Condentiality, security Solution: IPOP, a multi-point VPN IP-over-P2P ( Single address space between VMs Elastic and easily recongurable Each application creates and manages its own VPN ConPaaS: a Platform as a Service for Multi-clouds Multi-cloud ConPaaS 35 / 47

40 IPOP multi-point VPN ConPaaS: a Platform as a Service for Multi-clouds Multi-cloud ConPaaS 36 / 47

41 Data placement issues Where should data be located? For good throughput: in a single location For good access latency: as close as possible to the users For good fault-tolerance: in as many data centers as possible For good security: in my private cloud ConPaaS will enable users to dene their placement policies XtreemFS distributed le system Scalaris distributed key-value store ConPaaS: a Platform as a Service for Multi-clouds Multi-cloud ConPaaS 37 / 47

42 Application migration issues We must keep the application in working order during the migration Add new resources in the destination cloud Migrate data/workload Remove old resources in the origin cloud ConPaaS: a Platform as a Service for Multi-clouds Multi-cloud ConPaaS 38 / 47

43 Coming soon HARNESS E.U. project: heterogeneous cloud platforms GPUs, FPGAs, OpenFlow routers... An application may have multiple implementations The platform will build performance models automatically and choose the right implementation/scale automatically SEEP: real-time event processing service Scalable Elastic Dynamically recongurable Considering a mobile phone network as a multi-cloud Data centers mobile base stations Run smartphone apps backends close to the users ConPaaS: a Platform as a Service for Multi-clouds Multi-cloud ConPaaS 39 / 47

44 Table of Contents 1 Introduction 2 ConPaaS 3 Architecture 4 Multi-cloud ConPaaS 5 Conclusion ConPaaS: a Platform as a Service for Multi-clouds Conclusion 40 / 47

45 Conclusion An integrated runtime environment for elastic cloud applications Standard services can be composed to build cloud applications Extending ConPaaS with new services is quite easy Extending ConPaaS with new types of clouds is quite easy Multi-cloud support On a permanent basis for performance, reliability and security On a temporary basis for cloud provider indendence and application migration Mature Open-source (BSD licence) ConPaaS: a Platform as a Service for Multi-clouds Conclusion 41 / 47

46 People behind ConPaaS Ismail El Helw Services, Managers, Agents ConPaaS: a Platform as a Service for Multi-clouds Conclusion 42 / 47

47 People behind ConPaaS Adriana Szekeres Security ConPaaS: a Platform as a Service for Multi-clouds Conclusion 43 / 47

48 People behind ConPaaS Francesco Allertsen Applications, Manifest ConPaaS: a Platform as a Service for Multi-clouds Conclusion 44 / 47

49 People behind ConPaaS Emanuele Rocca Director, IPOP integration ConPaaS: a Platform as a Service for Multi-clouds Conclusion 45 / 47

50 People behind ConPaaS Claudiu Gheorghe Frontend ConPaaS: a Platform as a Service for Multi-clouds Conclusion 46 / 47

51 People behind ConPaaS Yann Radenac Multi-cloud support ConPaaS: a Platform as a Service for Multi-clouds Conclusion 47 / 47