Open Network Linux. A Path to an Open Source Network OS. Rob Sherwood Big Switch Networks CTO

Transcription

1 Open Network Linux A Path to an Open Source Network OS Rob Sherwood Big Switch Networks CTO

2 Outline: Open Network Linux (ONL) Context: What is ONL? Network OS Architecture View What does ONL provide? ONIE Compatible NOS Installer Supported Hardware Basic L3 Routing with Open Route Cache Platform Abstraction Layer ONLP Server-like Management Second stage more feature-full boot loader Plans for the Future How to Get Involved

3 ISN T NETWORKING ALREADY OPEN? Faster Throughput = More commercial value = More Proprietary All Hardware Specs Public X86 Servers ß Open Binary SDK for Packet Forwarding Closed OS, Open Control Plane APIs NPU, FPGA DC 1U ToRs, Spines Closed OS, Open APIs for Policy, Stats MulT- chassis Routers 10 Gb/s 100 Gb/s 1 Tb/s 10 Tb/s Aggregate Throughput à 2014 BIG SWITCH NETWORKS, INC. 3

4 ISN T NETWORKING ALREADY OPEN? Faster Throughput = More commercial value = More Proprietary All Hardware Specs Public X86 Servers ß Open Binary SDK for Packet Forwarding Closed OS, Open Control Plane APIs NPU, FPGA DC 1U ToRs, Spines DC 1U ToRs, Spines This talk Closed OS, Open APIs for Policy, Stats MulT- chassis Routers 10 Gb/s 100 Gb/s 1 Tb/s 10 Tb/s Aggregate Throughput à 2014 BIG SWITCH NETWORKS, INC. 4

5 LOTS OF OPEN SOURCE CONTROLLERS But what are they controlling? Open Source SDN Controllers??? This talk SDN OSS Software Switches SDN OSS Hardware Switches 2014 BIG SWITCH NETWORKS, INC. 5

6 ONL is a Linux Distribution for Bare Metal Switches A collection of software packages, utilities, drivers, and abstractions to run on OCP, bare metal, brite box hardware i.e., a NOS that ONIE would install Why not use an existing Linux distribution? Does build on existing distribution Debian Wheezy Need to create ONIE installers for many platforms Need to manage switch-specific hardware (e.g.,sfps) Switches are very similar to servers, but not quite

7 ONL Example Use-Cases 1. DIY packet forwarding platform, e.g., for students Platform driver code too complicated for most hobbyists 2. Reference hardware testing platform, e.g., OCP Certification Common core in open source for verification, reference 3. Building Block for Commercial Software Pull ONL in to a larger, commercially supported solution Not a Big Switch talk, but we use ONL in our products

8 Why Use ONL? Help ecosystem focus on innovation Many annoying software details to run an OCP switch Building platform drivers not high value asset; should be common Enables a reference NOS implementation Hardware without software is not useful Package up details and best practices into one place Help bootstrap the Open ecosystem and OCP adoption Allows commercial companies and DYI-folks to build OCP-based products faster

9 Outline: Open Network Linux (ONL) Context: What is ONL? Network OS Architecture View What does ONL provide? ONIE Compatible NOS Installer Supported Hardware Basic L3 Routing with Open Route Cache Platform Abstraction Layer ONLP Server-like Management Second stage more feature-full boot loader Plans for the Future How to Get Involved

10 Background: Generic NOS Architecture Applications Environment Agent(s) Forwarding Agent Platform Platform Abstraction Layer OS Distribution (Linux kernel, etc.) ASIC SDK Hardware CPU Misc Hardware (fans, LEDs, SFP, sensors) ASIC Per Switch Stack

11 Shipping now: ONL + Open Route Cache Applications ONLP APIs SensorD ONLP Dump S N M P Quagga/Xorp/Etc. Open Route Cache Orc-brcm Driver This API looks a lot like SAI ORC Driver API Platform ONL Platform APIs ONL Distribution (Linux + stuff) BRCM SDK Closed Source Hardware CPU (x86,ppc) Misc Hardware (fans, LEDs, SFP, sensors) BRCM Per Switch Stack

12 ONL Contributors/Supporters (So Far) Adds Forwarding Agents Open Network Linux Additional Drivers Provides Platform Drivers OTHERS

13 Kernel Linux Patches Architecture Infrastructure Module (AIM) Linux Linux Specific component: `make C $ONL/components/ $arch/$component/ deb` Busybox Buildroot Etc. Open Route Cache FaultD Plaaorm- * (installers, etc.) ONLP (Plaaorm API Library) ONLP- $vendor ONL Loader Anatomy of `make onl- $arch` All components: `make C $ONL/builds/components` ONL- Specific.debs $ONL/debian/repo Generic.debs From Emdebian.org and Debian.org Via apt- cacher- ng SWI: `make C $ONL/builds/swi/$arch/all` Using MulTstrap and `onl- mkws` ONL Loader Switch Image (.swi) ONIE Installer SHAR wrapper Installer: `make C $ONL/builds/ installer/$arch/all` ONL ONIE Installer New from ONL 3 rd Party OSS Mixed

14 ONL Relative to ONIE An Installer Needs a NOS ~16MB ~3MB ~160 MB ONIE One Time Installer Normal Full- featured Boot Loader (w/busybox) Main Network OS Image (.swi) (w/real binaries) Github.com/ onie/onie Open Network Linux Common kernel and DTS files??

15 Open Network Linux Community Project has been public for ~1 year Code Website ~50K LOC (without Linux patches); 56K Makefile lines Community Traction Six platforms added by community; more coming Three companies have commercial projects w/onl ~14 code contributors from four companies

16 Outline: Open Network Linux (ONL) Context: What is ONL? Network OS Architecture View What does ONL provide? ONIE Compatible NOS Installer Supported Hardware Basic L3 Routing with Open Route Cache Platform Abstraction Layer ONLP Server-like Management Second stage more feature-full boot loader Plans for the Future How to Get Involved

17 ONL Provides an ONIE-Compatible Installer ONIE is great but needs an installer Installer packages entire NOS into a single image Installer is the glue between Platform and NOS Sets the $nos_bootcmd magic e.g., nos_bootcmd= diskboot 0x :1 ; bootm 0x Formats local storage, e.g., flash or local disk Installs ONL loader onto local storage (see next)

18 Install Using ONIE then Boot ONL ~64MB uboot ENVs ONIE Free Space Boot Flash Boot Logic: 1. uboot POSTs 2. $nos_bootcmd is read from ENVs 3. run $nos_bootcmd If $nos_bootcmd returns, run ONIE On install, ONIE sets $nos_bootcmd to load ONL loader 4. Loader downloads specified SWI URL if not cached 5. Loader mounts rootfs as ramdisk with overlayfs 6. ONL loader kexec s SWI kernel ~2GB ONL Loader ONL config SWI s: ONL SWI #1 (cached) Mass Storage

19 ONL SUPPORTED HARDWARE X86 and PPC Support 2014 BIG SWITCH NETWORKS, INC. 19

20 Basic L3 Routing With Open Route Cache (ORC) 1. ORC creates a orcxx interface for each port 2. Quagga (or Xorp, etc.) installs routes into Linux kernel 3. Linux kernel sends RT Netlink Updates to ORC 4. ORC translates to SDK calls Hardware accelerated routing! Only unicast IPv4 now No IPv6, Multicast, ECMP support 5. Linux kernel is slow path to ASIC s fast path Open Route Cache Orc-brcm Driver BRCM SDK BRCM Quagga/Xorp/Etc.

21 ONL has a Platform Abstraction Layer BIOS does not provide inventory/drivers for all devices SFP has a life cycle, like USB; F2B vs. B2F power supplies Hot Pluggable devices: PSU, Fan trays, AC/DC power Not all platforms have BIOSs Need for per-platform memory maps of LEDs, Fans, Temp ONL Platform provides an Platform Abstraction Layer Not tied to Linux subsystem (but could be integrated) Goal is for trivial driver impls; leverage existing diag code OID based; single threaded, multi-application

22 ONL Provides Server-like Management Switches have flash, not hard drives Problem 1: Maximum flash cycle time limit disk writes Problem 2: Flash and ram more limited than typical servers Fix: Use overlayfs to overlay copy-on-write ram disk over flash ONL uses full-featured binaries For size, most switch OS s use stripped binaries, e.g., busybox Bigger binaries uses additional space, but ok with overlayfs Install/use proper Debian binaries using apt-get Useful for development or operations, e.g., gcc or Chef/Puppet

23 ONL Provides A Second-Stage Loader uboot (PPC) or Grub (x86) is the first stage loader ONIE is a 2 nd -stage loader; only runs at install-time ONL 2 nd Stage Loader Runs every time before NOS boots Built on Linux enables a full shell for rescue mode Load NOS via ssh/scp/http/tftp/nfs/ftp/local flash Net boot allows centralized NOS image management NFS book is very useful (future) Initialize ASIC/front panel ports inline boot

24 Outline: Open Network Linux (ONL) Context: What is ONL? Network OS Architecture View What does ONL provide? ONIE Compatible NOS Installer Supported Hardware Basic L3 Routing with Open Route Cache Platform Abstraction Layer ONLP Server-like Management Second stage more feature-full boot loader Plans for the Future How to Get Involved

25 Next Target: ONL + OpenFlow Agent OpenFlow Controller OpenFlow Protocol: v1.0-v1.3 Applications SensorD ONLP Dump S N M P Indigo OpenFlow Agent Indigo OF-DPA Driver Indigo API Platform ONL Platform APIs ONL Distribution (Linux + stuff) OF-DPA BRCM SDK Closed Source Hardware CPU (x86,ppc) Misc Hardware (fans, LEDs, SFP, sensors) BRCM Per Switch Stack

26 OpenFlow Datapath AbtracTon: OF- DPA Indigo driver OF- DPA SDK Chip open closed OF- DPA API 2014 BIG SWITCH NETWORKS, INC. 26

27 How to Get Involved Join the mailing list and provide feedback Compile the code Try a new platform Feedback is fine, but patches are preferred J ONL is a Linux Distribution for bare metal switches Growing support for OCP and non-ocp switches Now supports ORC forwarding agent Indigo-based OpenFlow agent in progress Find out more at Documentation Videos Pre-compiled binaries