An Overview of the Open Cloud Consor4um

Transcription

1 An Overview of the Open Cloud Consor4um Robert Grossman Open Cloud Consor4um OMG Cloud Compu4ng Interoperability Workshop March 23, 2009

2 This talk represents my personal opinions and does not necessarily represent a consensus posi4on of the Open Cloud Consor4um or posi4ons of any of its members. 2

3 Part 1. Introduc4on 3

4 What is a Cloud? Clouds provide on demand resources or services over a network, ojen the Internet, with the scale and reliability of a data center. No standard defini4on. Cloud architectures are not new. What is new: Scale Ease of use Pricing model. 4

5 Varie4es of Clouds Architectural Model Compu4ng Instances vs Compu4ng Services Economic Model Elas4c, usage based pricing, lease/own, Management Model Private vs Public; Single vs Mul4ple Tenant; Programming Model Queue Service, MPI, MapReduce, Distributed UDF Computing instances vs computing services Elastic, usagebased pricing or not Private internal vs public external All combinations occur. 5

6 Instances, Services & Frameworks many instances single instance Amazon s EC2 Hadoop DFS & MapReduce Amazon s SQS Azure Services Amazon s S3 Google AppEngine VMWare Vmotion Microsoft Azure This table contains some examples but is not inclusive of the various offerings available. instance service framework operating system 6

7 OCC s Current Focus many instances single instance Amazon s EC2 Hadoop DFS & MapReduce Amazon s SQS Azure Services Google AppEngine VMWare Vmotion Microsoft Azure Amazon s Complementary to most other S3 current standardization efforts. instance service framework operating system 7

8 Architectural Model On demand resources & services over a network at the scale of a data center On demand compu&ng instances IaaS: Amazon EC2, S3, etc.; Eucalyptus supports many Web 2.0 applica4ons/users On demand compu&ng services for data intensive compu&ng GFS/MapReduce/Bigtable, Hadoop, Sector, Manage and compute with large data (say 10 TB) 8

9 Delivering On Demand Compu4ng Instances App App App App App App App App Table based Data (BigTable) Services Rela4onal Data Services (Pig, etc.) App App Cloud Compute Services (MapReduce & Generaliza4ons) App App Cloud Storage Services On demand compu4ng instances can be delivered using data center scale on demand compu4ng services 9

10 1. Supports the development of standards. 2. Supports reference implementa4ons for cloud compu4ng, preferably open source. 3. Manages a testbed for cloud compu4ng called the Open Cloud Testbed. 4. Supports the development of benchmarks. 5. Sponsors workshops and other events related to cloud compu4ng. 10

11 Organiza4on OCC is a consor4um operated by a not for profit. OCC is member driven. OCC is oriented to companies and organiza4ons, not individuals. Members set up working groups (WG). Current WG: Standards and interoperability for clouds that provide on demand compu4ng services Informa4on sharing and security working group Wide area clouds and networking working group Strongly encourages reference implementa4ons, open source if possible 11

12 Part 2. Some Roles the USG Might Play in Cloud Standards 12

13 USG & Internet (CACM 1996) USG supported short term projects, not long term programs. NSF & DARPA provided only seed funding. Emphasis on focused projects with concrete goals. Projects provided leadership. Projects established open, vendor neutral standards. Coopera4on between USG, universi4es & industry. Rela4vely few, high competent people were involved & they had complete autonomy. Extracted from: Seeding Networks: the Federal Role, Larry Press, CACM

14 Some Roles the USG Might Choose Purchase cloud services for end users from cloud service providers. Build data centers and provide private cloud services for internal USG consump4on (GOGO, GOCO, COCO); i.e. purchase cloud infrastructure. Encourage standards by adding clauses to contracts to support open standards. Support some short term projects with concrete goals. 14

15 Some Possible USG Drivers Improve produc4vity get a new desktop instance in 20 minutes Go green Reduce power & environmental impact Disaster recovery Preserve USG informa4on Informa4on sharing Health care, homeland defense, 15

16 Risks of Ac4ng Too Early Appropriate standards and designs are yet understood. Bad decision can skew the market in the wrong direc4on. 16

17 Part 3: Standards & Interoperability WG Train gauge in China is 1435 mm Train gauge in Russia is 1520 mm Change of gauge at Ussuriisk (near Vladivostok) at the Chinese Russian border How can a cloud application move from one cloud storage service to another? 17

18 Clouds for Data Intensive Compu4ng App App App App App Table based Data (BigTable) Services Rela4onal Data Services (Pig, etc.) App App Cloud Compute Services (MapReduce & Generaliza4ons) App App Cloud Storage Services 18

19 vs On Demand Compu4ng Instances App App App

20 Ini4al Focus of OCC Interoperability WG TBD Hbase Hive Casandra Pig, SQL+MR varia4ons, DryadLINQ(?),... Hadoop MapReduce Sector UDF/MR ThriJ TBD Hadoop HDFS Sector DFS KFS 20

21 Part 4. Informa4on Sharing & Security Working Group How can two HIPAA compliant clouds run by two different organiza4ons share data? How can two different USG clouds run by two different organiza4ons share data? 21

22 Informa4on Sharing & Security Mediator (Media4on Policies with alerts, audit trail & provenance) Cross Policy Services / Applica4ons Interface Policy P2 Services / Applica4ons Authoriza4ons Security Users Data Interface Policy P1 Services / Applica4ons Authoriza4ons Security Users Data Interface Policy P1 Services / Applica4ons S e r v i c e s / A p li c a ti o n s Authoriza4ons Users Security Data

23 Both Technical and Policy Issues Several Organiza&ons or Authori&es One Organiza&on or Authority????????? One Cloud Several Clouds 23

24 Driving Project Develop standards for sharing informa4on between two HIPAA compliant clouds. Goal: Replace fax, which is used ojen today to exchange informa4on between two different HIPAA environments. 24

25 Part 5. Open Cloud Testbed For reference implementa&ons For interoperability studies For benchmarks and other experimental studies 25

26 Open Cloud Testbed Phase 1 (2008) Phase 1 4 racks 120 Nodes 480 Cores 10+ Gb/s CENIC C-Wave MREN Dragon Hadoop Sector/Sphere ThriJ Eucalyptus 26

27 Open Cloud Testbed Phase 2 (2009) Phase 2 6 loca4ons 8 racks 256 Nodes 1024 Cores 10+ Gb/s + shared resources 27

28 Open Cloud Testbed

29 Part 6. Benchmarks To help inform purchase decisions To help inform the design of cloud applica4ons To help inform cloud architecture decisions 29

30 MalGen MalGen generates synthe4c data with realis4c distribu4ons. MalStone A & B stylized computa4ons that can be used as benchmarks for architectures, sojware and systems for cloud compu4ng. 30

31 Part 7. OCC Sponsored Events 31

32 Cloud Compu4ng & Its Applica4ons 08 CCA 09 being planned now. 32

33 Please Work With the OCC To learn more about the Open Cloud Consor4um Join one of our working groups. 33