SDN CONTROLLER IN VIRTUAL DATA CENTER Emil Gągała PLNOG, 30.09.2013, Kraków
INSTEAD OF AGENDA 2 Copyright 2013 Juniper Networks, Inc. www.juniper.net
ACKLOWLEDGEMENTS Many thanks to Bruno Rijsman for his contributions to the development of this technology 3 Copyright 2013 Juniper Networks, Inc. www.juniper.net
SDN IS A NEW PARADIGM AND ARCHITECTURE SDN s definition encompasses 3 elements: 1 Abstract North-Bound Interfaces Network Programming Model Centralized 2 Logically Control Plane Centralized 3 Control, Data, Management and Service Plane Separation With Standards SDN 5 Copyright 2013 Juniper Networks, Inc. www.juniper.net
NETWORK AS A SERVICE Management Plane M Service Plane S S S Control Plane C C s on x86 Control Forwarding Plane Forwarding THE SYSTEM AS A PLATFORM THE NETWORK AS A PLATFORM 6 Copyright 2013 Juniper Networks, Inc. www.juniper.net
BRIDGING PHYSICAL/VIRTUAL NETWORK A GENERAL PURPOSE SDN PLATFORM Orchestration, Automation, Analytics Open source and partner eco system of orchestrators Api and sdk for integration with OSS / BSS OSS Control Plane - Physical, Virtual Open, standards-based, federated controller Scalable and resilient Configuration manager, Automation Control Plane Control Plane Control Plane Virtual Network Overlay Multi-tenancy for private and virtual public clouds Finance HR Marketing Gateway functions - connect to virtual to physical network Service chaining (physical and virtual) Physical Network Interoperability with traditional network devices Any-to-any non-blocking low-latency fabric: Q-Fabric or Clos 8 Copyright 2013 Juniper Networks, Inc. www.juniper.net
SDN USE-CASES SP IaaS, VPC OPTIMIZED SP CORE SP UNIVERSAL EDGE ENTERPRISE PRIVATE CLOUD Managed Cloud Services L3VPN extension into DC s Global Optimization for TE Policy-based BW Allocation Network Functions Virtualization Service Orchestration and Chaining IT-as-a-Service (Internal Managed Svcs) Orchestration Intra, Inter-Domain Orchestration Multi-tenancy Automation Agility 9 Copyright 2013 Juniper Networks, Inc. www.juniper.net
ENTERPRISE PRIVATE CLOUD Manual configuration of VLANs at every switch Administration and configuration is complex and slow Dynamic resource allocation Automated configuration Dynamic service chains Scale : 4096 Tenant IDs Physical Service Appliances VLAN Virtual Services (NFV) Virtual Network Finance HR Marketing Finance HR Marketing 12 Copyright 2013 Juniper Networks, Inc. www.juniper.net
SERVICE PROVIDER CLOUD (IAAS, VPC) End-to-End Virtual Network Orchestration and Automation Public Cloud Providers, Content Providers,... Service Provider Infrastructure as a Service (IaaS) Service Provider Managed Virtual Private Cloud (VPC) Service Provider L3VPN, E-VPN Enterprise Offices Enterprise Data Center 13 Copyright 2013 Juniper Networks, Inc. www.juniper.net
SDN CONTROLLER ARCHITECTURE OVERVIEW 15 Copyright 2013 Juniper Networks, Inc. Juniper Restricted Confidential - Do not distribute externally.
ROLE OF CONTROLLER IN A VIRTUALIZED ENVIRONMENT Orchestrator, CloudStack Storage Compute Network High Level Abstraction Compute Controller "Logically Centralized, Physically Distributed" Low Level Realization Network (Physical and Virtual) Storage Server vrouter Physical Network (Fabric) Physical Network (Gateway) Physical and Virtual Network Services Server 16 Copyright 2013 Juniper Networks, Inc. www.juniper.net
BUILDING BLOCK: MULTI-TENANCY Physical Topology L3 Network L2 Network L2 Network L3 router L2 Network Logical Topology 17 Copyright 2013 Juniper Networks, Inc. www.juniper.net
BUILDING BLOCK: GATEWAY FUNCTIONS Internet Tenant VPN Data Center 1 Data Center 2 Gateway Router Gateway WAN Gateway Switch A B A C D B A D Non Virtualized Server 18 Copyright 2013 Juniper Networks, Inc. www.juniper.net
BUILDING BLOCK: SERVICE CHAINING (NFV) Tenant Network A LB Internet Tenant Network A NAT Tenant Network B Tenant Network A1 Tenant Network A2 19 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CONTRAIL MULTI-TENANCY IMPLEMENTATION Quantum SDN Controller Configuration Control Analytics Tenant Network A Virtualized Server Virtualized Server IP fabric (underlay network) Tenant Network B Data Center 20 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CONTRAIL IS BASED ON MPLS VPN TECHNOLOGY Orchestrator Network Management System (NMS) DMI IBGP Route Reflector BGP Route Reflector Config Node Control Node BGP Analytics Node Control Node SDN System XMPP CE PE P P PE MPLS over MPLS CE vrouter Underlay Switch Underlay Switch vrouter MPLS over GRE, MPLS over UDP, VXLAN, NVGRE, etc L3VPN / E-VPN 21 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CONTRAIL MULTI-TENANCY IMPLEMENTATION Quantum Orchestrator Quantum Plug-in SDN Controller Configuration Control Analytics SDN Controller Virtualized Server Virtualized Server XMPP IP fabric (underlay network) Tenant s K Hypervisor + vrouter VXLAN or MPLS/GRE or MPLS/UDP Underlay switches 22 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CONTRAIL MULTI-TENANCY IMPLEMENTATION 23 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CONTRAIL GATEWAY IMPLEMENTATION Quantum SDN Controller Configuration Analytics Control Tenant Network A Data Center Tenant Network B Virtualized Server Virtualized Server IP fabric (underlay network) Customer A L3VPN Customer B L3VPN L3VPN 24 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CONNECT PHYSICAL L3VPN TO VIRTUAL NETWORK FOR CLOUD ACCESS AND/OR SERVICE CHAINING BGP Route Reflector Controller Tenant or Service BGP BGP BGP BGP XMPP XMPP Red L3VPN Red VN WAN DC / POP Underlay Green VN Green L3VPN WAN MPLS over TE-LSP similar to "Inter-AS Option (b)" Data Center MPLS over GRE 25 Copyright 2013 Juniper Networks, Inc. www.juniper.net
OVERLAY ENCAPSULATIONS Encapsulated payload Tenant identification Transport tunnel L3 over MPLS over GRE L4-L7 IP MPLS GRE IP Ethernet L2 over MPLS over GRE L4-L7 IP Ethernet MPLS GRE IP Ethernet L2 over VXLAN L3-L7 Ethernet VXLAN UDP IP Ethernet L3 over MPLS over UDP L4-L7 IP MPLS UDP IP Ethernet L2 over MPLS over UDP L4-L7 IP Ethernet MPLS UDP IP Ethernet 26 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CONTRAIL SERVICE CHAINING IMPLEMENTATION IN THE DATA CENTER Neutron SDN Controller Configuration Analytics Control Tenant Network A Virtualized Server Virtualized Server NA T IP fabric (underlay network) Tenant Network B Virtualized Server NA T Virtualized Server Data Center All packets from VN A to VN B must pass through NAT, services 27 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CONTRAIL SERVICE CHAINING IMPLEMENTATION 28 Copyright 2013 Juniper Networks, Inc. www.juniper.net
SDN CONTROLLER DETAILED WALK-THROUGH 29 Copyright 2013 Juniper Networks, Inc. Juniper Restricted Confidential - Do not distribute externally.
LOGICAL TOPOLOGY VN G Virtual Network Virtual Firewall Tenant Virtual Machines VN R Physical Gateway Router Non-Virtualized (Bare Metal) Server PN Physical Network (Internet, L3VPN,...) 30 Copyright 2013 Juniper Networks, Inc. www.juniper.net
PHYSICAL TOPOLOGY Nova Neutron Controller Virtualized Server Hypervisor with vrouter Non-Virtualized (Bare Metal) Server Underlay Switches Gateway Router to Internet or L3VPN 31 Copyright 2013 Juniper Networks, Inc. www.juniper.net
MAPPING OF LOGICAL TO VIRTUAL TOPOLOGY Nova Neutron Controller VN G VN R L3VPN PHYSICAL 32 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
STARTING POINT EMPTY LOGICAL TOPOLOGY Nova Neutron Controller VN G VN R PN PHYSICAL 33 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CREATE GREEN TENANT CREATE VIRTUAL NETWORK "GREEN" Nova Neutron Controller Create VN G VN G VN R PN PHYSICAL 34 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CREATE GREEN TENANT CREATE VIRTUAL MACHINE "" Nova Neutron Controller Create Attach to VN G VN G Nova: Create VN R PN PHYSICAL 35 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CREATE GREEN TENANT CREATE VIRTUAL MACHINE "" Nova Neutron Controller Create Attach to VN G Neutron: Attach to VN XMPP: Create routing-instance VN G VN R PN PHYSICAL 36 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CREATE GREEN TENANT CREATE VIRTUAL MACHINE "" Nova Neutron Controller Create Attach to VN G VN G Nova: Create VN R PN PHYSICAL 37 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CREATE GREEN TENANT CREATE VIRTUAL MACHINE "" Nova Neutron Controller Create Attach to VN G Neutron: Attach to VN XMPP: Create routing-instance VN G VN R PN PHYSICAL 38 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CREATE GREEN TENANT CREATE VIRTUAL MACHINE "" Nova Neutron Controller Create Attach to VN G VN G XMPP: Exchange routes Create tunnels VN R PN PHYSICAL 39 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CREATE GREEN TENANT FORWARDING TABLES AND ENCAPSULATION Payload Inner IP header Source IP Dest IP MPLS Label GRE Outer IP header Source IP Dest IP Ethernet Source MAC Dest MAC... L2... Server S1 Server S2 Server S1 Server S2 S1 Packet S2 Green routing-instance IP FIB IP prefix Nexthop Virtual ethernet port to Push label L2 + GRE encaps to server S2 Green routing-instance IP FIB IP prefix Nexthop Push label L1 GRE encaps to server S1 Virtual ethernet port to MPLS label Global MPLS FIB Nexthop MPLS label Global MPLS FIB Nexthop L1 Pop + Green routing-instance L2 Pop + Green routing-instance IP prefix Global IP FIB Nexthop IP prefix Global IP FIB Nexthop Server S2 Physical ethernet port Server S1 Physical ethernet port 40 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CREATE GREEN TENANT CREATE VIRTUAL MACHINE "" Nova Neutron Controller Create Attach to VN G VN G VN R Nova: Create PN PHYSICAL LOGICAL 41 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CREATE GREEN TENANT CREATE VIRTUAL MACHINE "" Nova Neutron Controller Create Attach to VN G Neutron: Attach to VN VN G VN R XMPP: Create routing-instance PN PHYSICAL LOGICAL 42 Copyright 2013 Juniper Networks, Inc. www.juniper.net
CREATE GREEN TENANT CREATE VIRTUAL MACHINE "" Nova Neutron Controller Create Attach to VN G VN G XMPP: Exchange routes Create tunnels VN R PN PHYSICAL 43 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CREATE GREEN TENANT END STATE Nova Neutron Controller VN G VN R PN PHYSICAL 44 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CREATE RED TENANT SAME STEPS AS GREEN TENANT Nova Neutron Controller VN G VN R PN PHYSICAL 45 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT GREEN TO RED TENANT VIA FIREWALL CREATE VIRTUAL MACHINE FOR FIREWALL Nova Neutron Controller Create Attach to VN G Attach to VN R VN G Nova: Create VN R PN PHYSICAL 46 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT GREEN TO RED TENANT VIA FIREWALL ATTACH FIREWALL TO RED AND GREEN VIRTUAL NETWORKS Nova Neutron Controller Create Attach to VN G Attach to VN R Neutron: Attach to VNs VN G VN R XMPP: Create routing-instance PN PHYSICAL 47 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT GREEN TO RED TENANT VIA FIREWALL APPLY POLICY, EXCHANGE ROUTES, AND CREATE TUNNELS Nova Neutron Controller Apply Policy VN G VN R XMPP: Exchange routes Create tunnels VN G VN R L3VPN PHYSICAL 48 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT GREEN TO RED TENANT VIA FIREWALL END STATE Nova Neutron Controller VN G VN R L3VPN PHYSICAL 49 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT GREEN TO RED TENANT VIA FIREWALL DATA PLANE: RED GREEN TRAFFIC FORCED THROUGH THE FIREWALL Nova Neutron Controller VN G VN R L3VPN PHYSICAL 50 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT RED TENANT TO PHYSICAL L3VPN CONFIGURE L3VPN ROUTING INSTANCE Nova Neutron Controller Apply Policy VN R L3VPN VN G Netconf: Configure routing-instance VN R L3VPN PHYSICAL 51 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT RED TENANT TO PHYSICAL L3VPN EXCHANGE ROUTES WITH PHYSICAL ROUTER, CREATE TUNNELS Nova Neutron Controller Apply Policy VN R L3VPN VN G BGP: Exchange routes Create tunnels VN R L3VPN PHYSICAL 52 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT RED TENANT TO PHYSICAL L3VPN EXCHANGE ROUTES WITH VROUTERS, CREATE TUNNELS Nova Neutron Controller Apply Policy VN R L3VPN VN G XMPP: Exchange routes Create tunnels VN R L3VPN PHYSICAL 53 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT BARE METAL SERVER TO RED TENANT USE TOP-OF-RACK SWITCH AS GATEWAY Nova Neutron Controller VN G VN R L3VPN PHYSICAL 54 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT BARE METAL SERVER TO RED TENANT CREATE ROUTING INSTANCE Nova Neutron Controller Attach to VN R using switch S Netconf: Configure routing-instance VN G VN R L3VPN PHYSICAL 55 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT BARE METAL SERVER TO RED TENANT EXCHANGE ROUTES WITH PHYSICAL SWITCH, CREATE TUNNELS Nova Neutron Controller Attach to VN R using switch S VN G BGP: Exchange routes Create tunnels VN R L3VPN PHYSICAL 56 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT BARE METAL SERVER TO RED TENANT EXCHANGE ROUTES WITH VROUTERS, CREATE TUNNELS Nova Neutron Controller Attach to VN R using switch S XMPP: Exchange routes Create tunnels VN G VN R L3VPN PHYSICAL 57 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
CONNECT BARE METAL SERVER TO RED TENANT END STATE Nova Neutron Controller VN G VN R L3VPN PHYSICAL 58 Copyright 2013 Juniper Networks, Inc. www.juniper.net LOGICAL
SUMMARY 59 Copyright 2013 Juniper Networks, Inc. Juniper Restricted Confidential - Do not distribute externally.
THE IMPORTANCE OF ABSTRACTION Nova Neutron Controller PHYSICAL TOPOLOGY Complex Low level of abstraction Many vrouters Many routing-instances Many tunnels Many routes Complex to configure Complex to troubleshoot 60 Copyright 2013 Juniper Networks, Inc. www.juniper.net
THE IMPORTANCE OF ABSTRACTION LOGICAL TOPOLOGY VN G Simple High level of abstraction Simple to configure Simple to troubleshoot VN R provides abstraction Configure logical layer Mapping to physical layer "SDN as a Compiler" Analytics at physical layer Mapping to logical layer L3VPN 61 Copyright 2013 Juniper Networks, Inc. www.juniper.net
STANDARDS Overall architecture IETF NVO3 WG ETSI NFV ISG Cloud Orchestration Overlay control plane protocols: XMPP: RFC 6120, draft-ietf-l3vpn-endsystem BGP L3VPN: RFC 4364 BGP EVPN: draft-ietf-l2vpn-evpn NetConf: RFC 6241 Multicast: draft-marques-l3vpn-mcast-edge N/B REST API s WAN Gateway Control BGP Federation Controller Configuration Mgmt Control Analytics Configuration Horizontally scalable Highly available Federated BGP Federation Controller XMPP Virtualized Server Virtualized Server IP fabric (switch underlay) Agent/vRouter (K, Xen, Linux ) Underlay control plane protocols: Existing layer-2 or layer-3 protocols WAN Gateway 62 Copyright 2013 Juniper Networks, Inc. www.juniper.net Tunnel fabric - MPLS over GRE/UDP, VXLAN, NVGRE Overlay data plane encapsulation: MPLS over GRE: RFC 4797 VXLAN (encapsulation only): draft-mahalingam-duttdcops-vxlan
WANT TO TRY? www.opencontrail.org 63 Copyright 2013 Juniper Networks, Inc. www.juniper.net