BARE METAL SOFTWARE DEFINED NETWORKS, AND OPEN SOURCE. Rob Sherwood Big Switch Networks, CTO BANV, May 2014

Transcription

1 BARE METAL SOFTWARE DEFINED NETWORKS, AND OPEN SOURCE Rob Sherwood Big Switch Networks, CTO BANV, May 2014

2 OUTLINE Mo6va6on: Open Networking and Hyperscale Modern SDN Big Switch Technologies Open Source SoKware Open Network Linux OF- DPA Indigo OpenFlow agent Loxi + Floodlight Conclusions and Community 2014 BIG SWITCH NETWORKS, INC. 2

3 MOTIVATION

4 Do you?

5 THE DC NETWORKING REVOLUTION What are they doing? 1) SDN sovware 2) Bare metal switch hardware 2014 BIG SWITCH NETWORKS, INC. 5

6 ANDROMEDA 2014 BIG SWITCH NETWORKS, INC. 6

7 WHAT DOES BIG SWITCH DO? Help you get from here to there We build SDN SoVware... that runs on Bare Metal Switch Hardware to monitor networks and build cloud fabrics (to reduce complexity) (to reduce cost) (to deliver Bare Metal SDN) 2014 BIG SWITCH NETWORKS, INC. 7

8 SDN ARCHITECTURAL EVOLUTION BEFORE Automa[on Tool SDN App North- bound APIs SDN Controller OpenFlow APIs CP/DP Separa[on (Thick) NetOS Network HW OF Too many moving parts (mul[ple SDN SW vendors) OF agent SW from HW vendor! Limited access to switch ASIC access 2014 BIG SWITCH NETWORKS, INC. 8

9 SDN ARCHITECTURAL EVOLUTION BEFORE Automa[on Tool NOW Automa[on Tool SDN App North- bound APIs SDN Controller OpenFlow APIs CP/DP Separa[on North- bound APIs SDN App SDN Controller OpenFlow & Extensions Openflow is just API (Thick) NetOS Network HW OF (Thin) SDN OS Network HW Too many moving parts (mul[ple SDN SW vendors) OF agent SW from HW vendor! Limited access to switch ASIC access SDN SW solu[on from single vendor Exactly like the hypervisor/server model Full access to switch ASIC Accelerate HW/SW disaggrega[on 2014 BIG SWITCH SWITCH NETWORKS, NETWORKS, INC. INC. PROPRIETARY AND CONFIDENTIAL 9

10 SDN ARCHITECTURAL EVOLUTION Cloud Fabric Big Tap Big Switch Network s Commercial Products Open Source NOW Automa[on Tool North- bound APIs SDN App SDN Controller OpenFlow & Extensions (Thin) SDN OS Network HW OpenFlow is just an API SDN SW solu[on from single vendor Exactly like the hypervisor/server model Full access to switch ASIC Accelerate HW/SW disaggrega[on 2014 BIG SWITCH SWITCH NETWORKS, NETWORKS, INC. INC. PROPRIETARY AND CONFIDENTIAL 10

11 ISN T NETWORKING ALREADY OPEN? Faster Throughput = More commercial value = More Proprietary All Hardware Specs Public X86 Servers ß Open Binary SDK for Packet Forwarding Closed OS, Open Control Plane APIs NPU, FPGA, SoC DC 1U ToRs, Spines Closed OS, Open APIs for Policy, Stats Mul[- chassis Routers 10 Gb/s 100 Gb/s 1 Tb/s 10 Tb/s Aggregate Throughput à 2014 BIG SWITCH NETWORKS, INC. 11

12 ISN T NETWORKING ALREADY OPEN? Faster Throughput = More commercial value = More Proprietary All Hardware Specs Public X86 Servers ß Open Binary SDK for Packet Forwarding Closed OS, Open Control Plane APIs NPU, FPGA, SoC DC 1U ToRs, Spines DC 1U ToRs, Spines This talk Closed OS, Open APIs for Policy, Stats Mul[- chassis Routers 10 Gb/s 100 Gb/s 1 Tb/s 10 Tb/s Aggregate Throughput à 2014 BIG SWITCH NETWORKS, INC. 12

13 MODERN SDN

14 SDN BASICS Manage Your Network Like a Big Switch BIG SWITCH NETWORKS, INC. 14

15 SDN BASICS Manage Your Network Like a Big Switch Supervisor 1 Supervisor 2 Line Card Line Card Line Card Line Card Fabric Backplane Line Card Chassis- based Fabric 2014 BIG SWITCH NETWORKS, INC. 15

16 SDN BASICS Manage Your Network Like a Big Switch Supervisor 1 Supervisor 2 Line Card Line Card Line Card Line Card Line Card Fabric Backplane Controller 1 Controller 2 Spine Spine Spine Leaf Leaf Leaf Leaf Chassis- based Fabric Leaf- Spine Fabric 2014 BIG SWITCH NETWORKS, INC. 16

17 SDN BASICS Manage Your Network Like a Big Switch Supervisor 1 Supervisor 2 Line Card Line Card Line Card Line Card Line Card Fabric Backplane Chassis- based Fabric These designs are fundamentally the same! Controller 1 Controller 2 Spine Spine Spine Leaf- Spine Fabric Leaf Leaf Leaf Leaf 2014 BIG SWITCH NETWORKS, INC. 17

18 CLOSEDFLOW VS. OPENFLOW WWCFD? Same Packet Forwarding Architecture. For the rest of the talk, before you ask a ques6on: WWCFD What Would ClosedFlow do? Answer: OpenFlow/SDN probably does the same thing Is this really the right ques6on? SDN is a design paradigm, not a solu[on onto itself - Unless you ve got an army of programmers Customer benefits are automa[on and simplicity, not SDN Just like OO Programming, DevOps, etc B I G S W I T C H N E T W O R K S, I N C. W W W. B I G S W I T C H. C O M 18

19 DECOUPLE CONTROL FROM FORWARDING OFDatapat OFDatapath h OFDatapath OpenFlow Controller OFDatapath OFDatapat OFDatapath h Reduce number of management touchpoints Mapping from datapaths to controllers a crucial network design ques[on OpenFlow does not imply single point of failure! 2013 BIG SWITCH NETWORKS, INC. 19

20 DECOUPLE CONTROL FROM FORWARDING OpenFlow Controller OpenFlow Controller OFDatapat OFDatapath h OFDatapath OFDatapath OFDatapat OFDatapath h OFDatapat OFDatapat h OFDatapath h OFDatapat OFDatapat h OFDatapath h OpenFlow does not imply single point of failure! Allows load balancing 2013 BIG SWITCH NETWORKS, INC. 20

21 DECOUPLE CONTROL FROM FORWARDING OpenFlow Controller OpenFlow Controller OFDatapat OFDatapath h OFDatapath OFDatapath OFDatapat OFDatapath h OFDatapat OFDatapat h OFDatapath h OFDatapat OFDatapat h OFDatapath h OpenFlow does not imply single point of failure! Allows load balancing and failover 2013 BIG SWITCH NETWORKS, INC. 21

22 PACKET FORWARDING ABSTRACTION: FLOW TABLE Sequence of tables in a packet processing pipeline Flow Table Flow Table Flow Table Priority Match Ac6on List 500 IP.proto=6 TCP.dst= IP.dst= 128.8/16 TTL- -, Fwd:port 3 Queue: * DROP Exis[ng networking hardware actually very flexible Trade- off: Large + narrow versus small + wide match tables Ac[ve work in the Open Networking Founda[on to bring OpenFlow to feature parity with closed flow 2013 BIG SWITCH NETWORKS, INC. 22

23 SWITCH SCALING: UNDERSTANDING TABLES Classical OpenFlow versus Modern: Proac[ve and Mul[- table OpenFlow Processing Pipeline L2 Table: ~100K+ Entries L3 Table: ~100K+ Entries Early OpenFlow implementa[ons only used the most flexible table. Didn t scale well Scale forced reac[ve popula[on Reac[ve caused controller load and addi[on scale problems Our Modern OpenFlow implementa[on leverages all tables Scales like tradi[onal Allows proac[ve popula[on Proac[ve reduces controller load and allows for headless control VLAN Table ACL Table: ~2k Entries Egress Table OpenFlow does NOT imply Flow- based Networking 2014 BIG SWITCH NETWORKS, INC. 23

24 BIG SWITCH TECHNOLOGIES

25 WHAT S INSIDE A SWITCH? Applica[on Network OS Hardware Driver Box Silicon B I G S W I T C H N E T W O R K S, I N C. W W W. B I G S W I T C H. C O M 25

26 COMPONENT ECOSYSTEM AND BARE METAL Applica[on Network OS Driver Box Silicon Single Vendor Closed Product Tradi[onal Networking (past) Single Vendor Closed Product ODM Box ODM Chip Tradi[onal Networking (today) SDN Controller Vendor OpenFlow SDN Hardware Vendor OpenFlow Model Vendor ODM Box ODM Chip Bare Metal Vision Tradi[onal Network Stack/OS 2014 BIG SWITCH NETWORKS, INC. 26

27 SWITCH LIGHT IS SDN ON BARE METAL The OpenFlow Ecosystem Model didn t work for us Hard: one OF agent that is all things to all people OF Wire protocol is (mostly) fine, but needs work for exis[ng hardware Economic incen[ves are backwards for SDN hardware vendors Out- of- phase release cycles caused low feature velocity Switch Light Model: download OS image and OF agent from Controller Easy : Customize switch sovware to applica[on and controller version Add extensions to wire protocol to validate before standardiza[on Perfect economic incen[ves One engineering team à high feature velocity 2014 BIG SWITCH NETWORKS, INC. 27

28 BARE METAL + MERCHANT SI: LOWERS FABRIC COSTS Financial Benefit of SDN Business Agility is Increased Revenue Reduced OpEx Reduced CapEx CapEx Savings is Not Trivial Brand name network vendors are using the same components Bare metal is more than white box Name brand vs. no- name vendors Buy your network hardware just like servers Vendors already successful in the server model have advantage 2014 BIG SWITCH NETWORKS, INC. 28

29 ARCHITECTURE IS OPEN FROM THE DATA PLANE UP Open data plane Re- use exis[ng virtualiza[on formats: no proprietary protocols/new hardware Compa[ble: deployed firewalls, IDS, WAN accelerators, Server offload NICs Open control plane OF1.3: All extensions are open source and will be taken to ONF Open REST APIs is primary interface: CLI is just a REST client Integra[on with OpenStack/Quantum and custom BSS/OSS systems Open source components: sokware engineering best prac6ce Accelerates adop[on, eases por[ng Open Network Linux: bare metal Linux distribu[on in OCP Indigo Open Flow agent: many 3rd par[es have ported to their hardware B I G S W I T C H N E T W O R K S, I N C. W W W. B I G S W I T C H. C O M 29

30 FABRIC USE- CASE (1/3): CONNECTIVITY, POLICY Spine Spine Spine Leaf Leaf Leaf Leaf Server Server Server Server Server Server Router Router Most common fabric use- case Policy: connec[vity, ACLs, QoS Op[onal VM orchestra[on e.g., OpenStack, HyperV 2014 BIG SWITCH NETWORKS, INC. 30

31 FABRIC USE- CASE (2/3): MONITORING Spine Spine Spine Tap Traffic Leaf Leaf Leaf Leaf Exis[ng Produc[on Network IDS VoIP QoS Tool BigTap: Scale- out monitoring Monitoring as a Service Programmable via REST RBAC Overlapping policy support Physically isolated from prod In/out heavily over- subscribed Load balance across tools My first SDN App 2014 BIG SWITCH NETWORKS, INC. 31

32 FABRIC USE- CASE (3/3): PATCH PANEL Spine Spine Leaf Leaf DUT DUT DUT DUT Programma[cally create large virtual L1 topologies Automate complex tests without rewiring Spine Leaf Leaf DUT DUT Packet Generator Packet Generator Blindly forward packets by port Similar to Pseudo- wire Even CRC failures, runts, etc. *DUT: Device Under Test Non- product: in our Q/A labs 2014 BIG SWITCH NETWORKS, INC. 32

33 OPEN SOURCE SOFTWARE

34 SWITCH LIGHT ARCHITECTURE Legend Open Network Linux BSN Open BSN Closed 3 rd Party Closed Source ZTN Loader SSH LibC on Debian Wheezy Base Distribu[on ONL Linux Kernel Switch Light OS Fan Control NTP Syslog SNMP I2C Big Network Controllers GPIO Device Trees ASIC SDK ASIC 2014 BIG SWITCH NETWORKS, INC CLI Indigo Indigo/ASIC Driver OpenFlow Agent Loxi Switch Light is our Indigo OpenFlow Agent running on Open Network Linux on x86 or ASIC- based hardware.

35 OPEN NETWORK LINUX A Linux distribu[on for bare metal switches Open- sourced commercial code, build scripts, drivers from Switch Light Part of Open Compute Project (OCP) Project goals: Improve percep[on/de- risk bare metal Crowd source larger Switch Light HCL Released mid- January: Ini6al results Strong community response from ODMs, OCP Accton self- suppor[ng three new boxes, more ODMs interested Other vendors inves[ga[ng ONL for their own purposes Info: h{p://opennetlinux.org under construc[on 2014 BIG SWITCH NETWORKS, INC. 35

36 ONL SUPPORTED HARDWARE X86 versus PPC Discussion ONL Support is Pre- cursor to Switch Light Support Working on OCP box from Interface Masters with x86 support 2014 BIG SWITCH NETWORKS, INC. 36

37 ONL IS MULTI- PLATFORM Support many boxes from the same code- base Open Network Linux: Kernel Drivers Loader Work flow Build scripts Manage. Model X86 Arch PPC ARM? Interface Master s x86 VM others? Quanta LB9, LY2, LY5 Accton 5652 Delta, Alpha, etc.??? 2014 BIG SWITCH NETWORKS, INC. 37

38 INSTALL USING ONIE THEN BOOT ONL ~64MB uboot ENVs ONIE Free Space Boot Flash Boot Logic: 1. uboot POSTs 2. $nos_bootcmd is read from ENVs 3. run $nos_bootcmd If $nos_bootcmd returns, run ONIE On install, ONIE sets $nos_bootcmd to load ONL loader 4. Loader downloads specified SWI URL if not cached 5. Loader mounts rootfs as ramdisk with overlayfs 6. ONL loader kexec s SWI kernel ~2GB ONL Loader ONL config SWI s: ONL SWI #1 (cached) Mass Storage 2014 BIG SWITCH NETWORKS, INC. 38

39 PERSPECTIVE RELATIVE TO ONIE ONL is a Network Opera[ng System (NOS) for ONIE ~3MB ~16MB ~160 MB ONIE Normal Full- featured First boot Loader Boot Loader (w/busybox) Main Network OS Image (.swi) (w/real binaries) Github.com/ onie/onie Open Network Linux Common kernel and DTS files?? 2014 BIG SWITCH NETWORKS, INC. 39

40 2014 BIG SWITCH NETWORKS, INC. 40

41 OpenFlow Datapath Abtrac[on: OF- DPA Indigo driver OF- DPA SDK Chip open closed OF- DPA API 2014 BIG SWITCH NETWORKS, INC. 41

42 LOXI IS FULLY OF1.3.1 git://github.com/floodlight/loxigen C Backend libloci.a Indigo Single OF Wire Desc LOXI- GEN Java Backend Python Backend OpenFlow J- LOXI Pylib openflow Floodlight OFTest Wireshark Backend Wireshark Plugin (Lua) Wireshark 2014 BIG SWITCH NETWORKS, INC. 42

43 FLOODLIGHT SDN CONTROLLER 2014 BIG SWITCH NETWORKS, INC. 43

44 CONCLUSION Big Switch builds SDN solu6ons on bare metal hardware Commercial Products: BigTap, Cloud Fabric SDN has evolved significantly Mul[- table, proac[ve, bare metal Lots of Open Source projects Take the pieces or the whole: ONL Join at: opennetlinux.org/community Comments and feedback welcome 2014 BIG SWITCH NETWORKS, INC. 44