CERN Cloud Infrastructure. Cloud Networking

Transcription

1 CERN Cloud Infrastructure Cloud Networking

2 Contents Physical datacenter topology Cloud Networking - Use cases - Current implementation (Nova network) - Migration to Neutron 7/16/2015 2

3 Physical network Topology restrictions - A machine has only network connectivity in a specific area of the CC Network DB Registration (LanDB) - DHCP (~2 minutes) - DNS (~10 minutes) 7/16/2015 Cloud Service 3

4 Cloud Network nova-network Provider Network VMx VMx VMx VMx VMx Project Router Project Router Private Network Private Network VMx VMx 7/16/2015 Cloud Service 4

5 Nova-network architecture Cloud Controller Public Network Compute Node(s) eth0 nova-api nova-network eth0 br0 nova-compute nova-scheduler nova-conductor VMx 7/16/2015 Cloud Service 5

6 CERN (I) Hypervisor agents - KVM Linux Bridge - Hyper-V Virtual Switch 7/16/2015 Cloud Service 6

7 CERN (II) LanDB + AD integration - SQL extra information + Code Patch - IP/MAC pairs registered beforehand - nova-api Rename LanDB device Update AD information Wait for DNS update - ec2 Choose one device and run Rename LanDB device 7/16/2015 Cloud Service 7

8 Migration to Neutron Nova-network will be deprecated Neutron as a replacement - Same end-user functionality - ML2 Plugin (Flat + Linux Bridge) - Integration with CERN services - On-demand IP registration 7/16/2015 CERN 8

9 Create instance with port 1. Machine created (nova) VMx VMx VMx VMx 2. Allocate a port (neutron) - Generate MAC address - Add port to DB - Lookup subnets on network - Allocate IP on subnet - Bind port on instance to host Subnet Subnet Subnet Provider Network 7/16/2015 Cloud Service 9

10 Create instance with CERN 1. Create instance VMx VMx VMx VMx 2. Allocate a port on a network + host - Generate MAC address - Add port to DB - Lookup subnets on network host filter - Allocate IP on subnet (LanDB) - Bind port on instance to host IP Service IP Service IP Service Provider Network 7/16/2015 Cloud Service 10

11 Integration with LanDB Register port creation/deletion - Device + Interface registration on neutron Import network restrictions from LanDB - Subnet Clusters (aggregation of Subnets) - Host Restrictions 7/16/2015 CERN 11

12 Q & A 7/16/2015 Cloud Service 12

13

14 CERN Cloud Infrastructure Backup slides 7/16/2015 CERN 14

15 Why? Nova-network will be deprecated Neutron as a replacement - Same end-user functionality - ML2 Plugin (Flat + Linux Bridge) - Integration with CERN services - On-demand IP registration 7/16/2015 CERN 15

16 Create instance with port 1. Machine created (nova) VMx VMx VMx VMx 2. Allocate a port (neutron) - Generate MAC address - Add port to DB - Lookup subnets on network - Allocate IP on subnet - Bind port on instance to host Subnet Subnet Subnet Provider Network 7/16/2015 Cloud Service 16

17 Create instance with CERN 1. Create instance VMx VMx VMx VMx 2. Allocate a port on a network + host - Generate MAC address - Add port to DB - Lookup subnets on network host filter - Allocate IP on subnet (LanDB) - Bind port on instance to host IP Service IP Service IP Service Provider Network 7/16/2015 Cloud Service 17

18 Integration with LanDB Register port creation/deletion - Device + Interface registration on neutron Import network restrictions from LanDB - Subnet Clusters (aggregation of Subnets) - Host Restrictions 7/16/2015 CERN 18

19 Improve LanDB integration Split device from interface registration Method to retrieve not available IPs - 1 Network, 1 Broadcast, 1 Gateway, 6 Reserved Avoid clustername on interface registration IPv4 IPv6 7/16/2015 CERN 19

20 Contents Overview of CERN Neutron basics and architecture Decisions Planning 7/16/2015 Cloud Service 20

21 Nova-network architecture Cloud Controller Public Network Compute Node(s) eth0 nova-api nova-network eth0 br0 nova-compute nova-scheduler nova-conductor VMx 7/16/2015 Cloud Service 21

22 CERN (I) Hypervisor agents - KVM Linux Bridge - Hyper-V Virtual Switch 7/16/2015 Cloud Service 22

23 CERN (II) LanDB + AD integration - SQL extra information + Code Patch - IP/MAC pairs registered beforehand - nova-api Rename LanDB device Update AD information Wait for DNS update - ec2 Choose one device and run Rename LanDB device 7/16/2015 Cloud Service 23

24 OpenStack Neutron Provides networking as a service (NaaS) Concepts - Networks - Subnets - Ports - Interfaces virtual network virtual port virtual interface Net /24 VM1 VM1 7/16/2015 Cloud Service 24

25 Neutron Components API Server Queue Network node Agents - L2 - L3 - DHCP - Advanced 7/16/2015 Cloud Service 25

26 Neutron Architecture

27 Neutron Topology Provider Network VMx VMx VMx VMx VMx Project Router Project Router Private Network Private Network VMx VMx 7/16/2015 Cloud Service 27

28 Neutron Networks Private Networks - User created - Not shared - Routable/Isolated Provider Networks - Created by admins - Shared among tenants - Map to existing physical networks 7/16/2015 Cloud Service 28

29 Neutron Drivers (ML2) Core Plugin (ML2) Type Manager Mechanism Manager Type Driver Mechanism Driver flat GRE VLAN VXLAN Linux Bridge OvS Hyper-V Other 7/16/2015 Cloud Service 29

30 Decisions ML2 mechanism and type drivers - KVM Linux Bridge Open vswitch - HyperV Deployment architecture - Flat (provider networks) - Routed (private networks) - Mixed (both) 7/16/2015 Cloud Service 30

31 Create instance with port 1. Machine created (nova) VMx VMx VMx VMx 2. Allocate a port (neutron) - Generate MAC address - Add port to DB - Lookup subnets on network - Allocate IP on subnet - Bind port on instance to host Subnet Subnet Subnet Provider Network 7/16/2015 Cloud Service 31

32 Create instance with CERN 1. Create instance VMx VMx VMx VMx 2. Allocate a port on a network + host - Generate MAC address skip?? - Add port to DB - Lookup subnets on network host filter - Allocate IP + MAC on subnet (LanDB) - Bind port on instance to host IP Service IP Service IP Service Provider Network 7/16/2015 Cloud Service 32

33 CERN Specifics AD integration - Where is the right point to do it? IP/MAC address handling - IP/MAC addresses registered beforehand - Scheduler awareness of host on creation Interference with APIs 7/16/2015 Cloud Service 33

34 Neutron Resources Core Networks Subnets Ports Extended Routers Floating IPs Firewalls Load Balancers 7/16/2015 CERN 34

35 Neutron Components Extensions API Core L3 FW Plugins Core Plugin L3 Plugin FW Plugin Service Plugins Agents L2 Agent (Hyper-V, LinuxBridge, ) L3 Agent 7/16/2015 CERN 35

36 Neutron CERN Extensions API Core CERN Extensions Plugins Core Plugin CERN Service Plugin Service Plugins Agents L2 Agent (LinuxBridge, Hyper-V) 7/16/2015 CERN 36

37 ML2 Plugin Core L2 Type Manager Mechanism Manager Extension Manager Type Drivers (Flat, GRE, VLAN, VXLAN) Mechanism Drivers (LinuxBridge, OVS, Hyper-V, Cisco, ) Extension Drivers 7/16/2015 CERN 37

38 ML2 CERN Core L2 Type Manager Mechanism Manager Extension Manager Flat Type Driver CERN Mechanism Drivers No Extension Drivers 7/16/2015 CERN 38

39 CERN Cloud Infrastructure Implementation 7/16/2015 CERN 39

40 Summary LanDB Custom Mechanism Drivers IP Service Clusters SubnetCluster API IP restrictions HostRestrictions API VM Metadata Security groups 7/16/2015 CERN 40

41 CERN LB Mechanism Driver Based on LinuxBridge driver. Handles LanDB registration / deletion. Queries Nova to resolve instance name and other LanDB metadata. If LanDB registration fails, port gets deleted, instance gets error state. If LanDB deletion fails, instance gets deleted anyway wasted IPs. Code is run by neutron-server. 7/16/2015 CERN 41

42 SubnetCluster API extension Defines new Cluster network resource Allows association of subnets with a clusters Neutron Subnet == CERN IP Service Provider Network Cluster X Cluster Y Subnet XX Subnet XY Subnet YX Subnet YY 7/16/2015 CERN 42

43 HostRestrictions API extension Exposes information about CERN network restrictions as part of the Neutron API. Hypervisor Allowed IP Service(s). Can choose an IP service based on different algorithms. 7/16/2015 CERN 43

44 Instance Metadata Problem: Neutron implementation depends on L3 or DHCP. Solution: Get rid of Neutron metadata. Add NAT rule to forward metadata requests on compute node. Chain PREROUTING (policy ACCEPT) DNAT tcp -- anywhere tcp dpt:http to: :8775 neutron-linuxbri-prerouting all -- anywhere anywhere Add FORWARD rule to allow packets from/to metadata server. Chain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere neutrondev.cern.ch neutron-filter-top all -- anywhere anywhere neutron-linuxbri-forward all -- anywhere anywhere Implemented as patch in IPTables Manager. Added metadata_host and metadata_port in neutron.conf 7/16/2015 CERN 44

45 Security Groups Supported with nova cells. IP Tables implementation. Default rules don t allow external ingress traffic. Cannot change default rules from API. Patch on default security group rules creation. 7/16/2015 CERN 45