Suppor&ng So*ware Evolu&on to the Mul&- cloud with a Cross- Cloud Pla;orm

Transcription

1 Suppor&ng So*ware Evolu&on to the Mul&- cloud with a Cross- Cloud Pla;orm Mike Smit Dalhousie University, Halifax, Canada Bradley Simmons, Mark Shtern, Hongbin Lu, and Marin Litoiu York University, Toronto, Canada 1

2 A Brief Introduc&on Information People Open Information Data Sources & Usability Research Communities Technology Big Data Cloud Computing Dissemination & Discovery 2

3 The Cloud Migra&on Challenge X- cloud Applica&on Management Pla;orm Implementa&on & Case Study

4 The Cloud Migra&on Challenge 4

5 Cloud Background This slide intendonally lee blank 5

6 6

7 Availability Zones m1.small 1 ECU 1.7GB low io networking ElasDc Compute Cloud Aggregate Managed cloud Reserved Spot Instances EBS Store Instances SciMark and SPEC Service credits Profiling Sub- regions MulDcloud Image 1.6GHz CPU, 1.75GB RAM, 225GB Storage RepoPath RightScript Geo Redundant Worker roles Service Bus Alert escaladon Locally Redundant Reserved Instances Azure SQL First- generadon 512 MB, 1 vcpu 12.8% guaranteed, 10 Mbps Open cloud Managed cloud Openstack Second generadon BursDng Pricing Cloud servers Community metrics mswin Auto- benchmark 7

8 ss Services Do the clouds! The last phase, building the strategic r cloud pilot, is critical and impacts how exploit cloud to achieve business and I on investment. e IBM Difference Cloud Cloud affects all dimensions of your o and operations, to technology, organiz The Strategy and Change Services for methods to address these dimensions 8 i cloud strategy.

9 So let s look at migra&on 9

10 1. From in- house to cloud resources 10

11 2. From one provider to another 11

12 2b. or to mul&ple others IBM Global Business Services Solution Brief The IBM Difference for Cloud Only IBM can help you create new sources of business value with cloud through deep industry and strategy expertise combined with technology

13 The Migra&on Challenge Ideally, migrators are experts in: the soeware being migrated cloud deployment and management What happens when you don t have both? 13

14 X- cloud Applica&on Management Pla;orm 14

15 Goals To enable cloud rookies to deploy and manage an applicadon on the muld- cloud To replace technology- and provider- specific terminology and interfaces with applica&on- driven interfaces and language To improve cloud resources as teaching and research tools Facilitate the devops methodology 15

16 XCAMP Analyze Plan Monitor Knowledge XCAMP Execute Application Multicloud

17 Applica&on- level view Analyze Plan Monitor Knowledge XCAMP Execute Application A Application B METRICS External Data Sources 17

18 XCAMP Internals Analyze Plan Notification Notification Management Logic for Application A Execution Notification Abstraction Abstraction Plugin Abstraction Knowledge Information Aggregation Service Monitor Knowledge Store Deployment Deployment Service Service Execute METRICS External Data Sources Application A 18

19 Applica&on Developer / Deployer Analyze Plan Notification Notification Management Logic for Application A Execution Notification Abstraction Abstraction Plugin Abstraction Knowledge Information Aggregation Service Monitor Knowledge Store Deployment Deployment Service Service Execute METRICS External Data Sources Application A 19

20 Deployment Descriptor Data base Web Server 1 Web Server 2 Web Server 2 (replica) Web Server 2 (replica) Load Balance Web Server 3 Database Monitoring Server

21 PaPern- based Deployment Service PostgresSQL SSH MySQL War file Security Groups Image Keypair Tomcat OpenStack Account Apache PDS EC2 SQL Script Account Application on multi-cloud H. Lu, M. Shtern, B. Simmons, M. Smit, and M. Litoiu, Pahern- based Deployment Service for Next GeneraDon Clouds, in IEEE 9th World Congress on Services, Cloud Cup, in press, 2013.

22 Applica&on Doesn t know (or care) about details As long as its constraints are met Examples of things to not care about: IPs, subnets, number of resources, size of resource, version of OS, performance 22

23 Management Logic Integrated via web- based technology Receive input measurements via REST operadons on the URL of your management logic Input: Metrics Records about nodes (POST or GET) A Metrics Record has three main properdes source terminology from the deployment descriptor metricname name of metric being reported value Output: JSON formahed acdons to modify your applicadon deployment 23

24 Management Logic (2) Responsible for Analyze & Plan MAPE stages Interface with XCAMP is web- oriented So you write custom rules in Java EE (or other preferred web development system) or integrate with Drools, or adapdve system, and extend and grow with your comfort level. 24

25 Management Logic (3) Developers given: List of available acdons and what they do List of available Metric Records and what they mean (cpu udlizadon, free memory, io wait, ) Basic pahern: Read metric record from request Determine if state change required Output response (change request or noop) 25

26 What makes this possible? 26

27 Pla;orm Responsibili&es Analyze Plan Notification Notification Management Logic for Application A Execution Notification Abstraction Abstraction Plugin Abstraction Knowledge Information Aggregation Service Monitor Knowledge Store Deployment Deployment Service Service Execute METRICS External Data Sources Application A 27

28 Built on Stream Processing Stream = sequence of tuples Stream processors accept, aggregate, manipulate, and produce streams efficiently Storm, S4, Misure is a publish- subscribe system for streams of metrics (used by XCAMP) M. Smit, B. Simmons, and M. Litoiu, Distributed, ApplicaDon- level Monitoring of Heterogeneous Clouds using Stream Processing, Future GeneraDon Computer Systems, in press,

29 Recall: Applica&on- level view Analyze Plan Monitor Knowledge XCAMP Execute Application A Application B METRICS External Data Sources 29

30 Recall: Pla;orm Responsibili&es Analyze Plan Notification Notification Management Logic for Application A Execution Notification Abstraction Abstraction Plugin Abstraction Knowledge Information Aggregation Service Monitor Knowledge Store Deployment Deployment Service Service Execute METRICS External Data Sources Application A 30

31 Implementa&on & Case Study 31

32 Case 1 Three- Der web applicadon A hybrid public- private deployment Cloud bursdng ~100 lines of Management Logic code Wrihen by someone not involved in actual development of XCAMP implementadon Designed to keep udlizadon < 100%, > 40% 32

33 Load Average (1m) public instances private instances workload load max load target Throughput :15 02:00 01:45 01:30 01:15 01:00 00:45 00:30 00: public instances private instances workload response time Response Time Throughput 0 02:15 02:00 01:45 01:30 01:15 01:00 00:45 00:30 00:15 33

34 Management Logic Throughput 34

35 Case 2 SAVI Smart ApplicaDons on Virtual Infrastructure Pan- Canadian network of researchers: soeware, networks, systems Large, distributed testbed InvesDgates a core/edge cloud paradigm Group of students & researchers introduced to XCAMP in hands- on tutorial Despite experience gaps, generally felt that XCAMP would meet their requirements 35

36 Conclusion An architecture and a system for cloud- agnosdc applicadon deployment & management Suitable for expert developers without detailed cloud understanding MigraDon to muldcloud an important challenge Developers should not face it alone! Remember the goals 36

37 Goals To enable cloud rookies to deploy and manage an applicadon on the muld- cloud To replace technology- and provider- specific terminology and interfaces with applica&on- driven interfaces and language To improve cloud resources as teaching and research tools Facilitate the devops methodology 37

38 Thank You! ceraslabs.com mikesmit.com/publicadons 38

39 39

40 40

41 41

42 42

43 Let s do stuff on the clouds! 43

44 Do the clouds! 44