OpenStack: we drink our own Champagne. Teun Docter Software developer

Transcription

1 OpenStack: we drink our own Champagne Teun Docter Software developer

2 About Bright Computing Enterprise infrastructure software company Founded in 2009 HQ in Amsterdam & San Jose Bare-metal & cloud provisioning, monitoring, management of clustered systems in the data center: HPC, Hadoop, OpenStack, storage, database and other clustered systems Customers: Boeing, ING Bank, Sanofi, NASA, DoD, DoE, Stanford, Oracle, HP, Intel, Sinopec and >400 more Worldwide resellers: Dell, Cisco, Cray, Huawei, Acer, DataDirect Networks and >50 more

3 UK resellers

4 HPC clusters storage clusters database clusters Trends - Centralization - Consolidation - Standardization - Cloud - General purpose clusters - Private clouds - Across on-premise and public cloud Hadoop clusters private clouds server farms

5 About Bright Cluster Manager Deployment Provisioning Dashboards Monitoring Visualization Health checking Workload management Cloud extension Environment specific HPC Hadoop Private cloud 5

6 About Bright Cluster Manager Deployment Provisioning Dashboards Monitoring Visualization Health checking Workload management Cloud extension Environment specific HPC Hadoop Private cloud 6

7 About Bright Cluster Manager Deployment Provisioning Dashboards Monitoring Visualization Health checking Workload management Cloud extension Environment specific HPC Hadoop Private cloud 7

8 About Bright Cluster Manager Deployment Provisioning Dashboards Monitoring Visualization Health checking Workload management Cloud extension Environment specific HPC Hadoop Private cloud 8

9 About Bright Cluster Manager Deployment Provisioning Dashboards Monitoring Visualization Health checking Workload management Cloud extension Environment specific HPC Hadoop Private cloud 9

10 About Bright Cluster Manager Deployment Provisioning Dashboards Monitoring Visualization Health checking Workload management Cloud extension Environment specific HPC Hadoop Private cloud 10

11 About Bright Cluster Manager Deployment Provisioning Dashboards Monitoring Visualization Health checking Workload management Cloud extension Environment specific HPC Hadoop Private cloud 11

12 About Bright Cluster Manager Deployment Provisioning Dashboards Monitoring Visualization Health checking Workload management Cloud extension Environment specific HPC Hadoop Private cloud node001 head node node002 node003 node006 node004 node007 node005 head node node001 node002 node003 12

13 About Bright Cluster Manager Deployment Provisioning Dashboards Monitoring Visualization Health checking Workload management Cloud extension Environment specific HPC Hadoop Private cloud 13

14 Bright Cluster Manager & OpenStack Confidential. Do not distribute beyond your organization. Copyright 2013.

15 Managing OpenStack Clusters Managing OpenStack Clusters even more difficult then other types of clusters Without proper infrastructure, OpenStack will not be able to run Setting up OpenStack manually is often extremely complex for production setups Bright Cluster Manager provides single-pane-of-glass to manage and monitor all aspects of OpenStack cluster Includes: Hardware (set up, configuration, monitoring) Operating system (provisioning, updates) OpenStack installation OpenStack configuration Bright Cluster Manager provides perfect environment for OpenStack to run on 15

16 Ease of Installation Deploying OpenStack private clouds made simple Bright pre-packages OpenStack (Icehouse) on the DVD Bright installs Linux, OpenStack & itself OpenStack can be enabled during installation or afterwards OpenStack will run as services on top of Bright cluster Bright configures, manages and monitors physical cluster (hardware + OS + other middleware) Virtual Machines are managed through OpenStack Support various network setups (VLAN, VxLAN, flat, etc.) Bright integrates with Ceph, can be used for OpenStack Configuration wizard to guide the setup process 16

17 17

18 18

19 19

20 20

21 21

22 22

23 23

24 24

25 25

26 26

27 27

28 28

29 29

30 30

31 31

32 32

33 33

34 34

35 35

36 OpenStack Configuration OpenStack configuration through roles OpenStack consists of several services to accomplish specific tasks Tasks can be assigned by assigning roles to nodes Example roles: Compute (Nova) Image (Glance) Block Storage (Cinder) Dashboard (Horizon) Auth (Keystone) Networking (Neutron) Assigning/unassigning role will: Write out config file based on role parameters Start/stop/monitor relevant services Add/remove entries to Keystone service registry OpenStack configuration is dynamically updated with cluster changes (e.g. hostname, network settings) 36

37 USE CASE: problem Developers need clusters to test their code Support engineers need clusters to reproduce issues Need to quickly spin up clusters with: Different version of Bright, including custom builds Different base distributions (eg RedHat, CentOS, SUSE, etc.) Different types of configurations Limited amount of hardware available Manual setup becomes tedious and time consuming 37

38 USE CASE: solution We drink our own Champagne Build our own private cloud Runs Bright to manage the hardware and OS Use Bright to deploy and configure OpenStack Create set of end-user tools to spin up virtual clusters Command-line driven Make use of OpenStack API s Use web based dashboards for console access Each user has an isolated environment Users have full control over their virtual clusters Virtual clusters directly accessible from office network Load is balanced automatically Capacity can now be expanded easily 38

39 39

40 40

41 41

42 42

43 USE CASE: plans & considerations Automated virtual head node installation Automated nightly installations Head node installation using volume snapshots Works in POC and allows big clusters to spin up in minutes Make sure you have a fast network DNS integration 43

44 OpenStack in HPC Improved flexibility User can manage their code, libs, etc. OpenStack Ironic also allows non-virtualized instances to be created SR-IOV allows HPC resources (e.g. InfiniBand and GPU) to be made available directly to VMs at near-native performance (work in progress) Container/docker integration in OpenStack 44

45 Conclusion Bright provides tried & tested method of cluster management Hundreds of clusters world-wide being managed using Bright Cluster Manager Inclusion of OpenStack management capabilities provides complete solution for setup, management & monitoring of OpenStack based private clouds. Single pane of glass for management of cluster & OpenStack Especially well suited for clusters that run various workloads (e.g. OpenStack, HPC compute, Hadoop) Very scalable (i.e. thousands of nodes) 45

46 Thank you! Teun Docter