Networking for Science + Software-Defined Networking (SDN): Hype vs. Hope

Transcription

1 Networking for Science + Software-Defined Networking (SDN): Hype vs. Hope Inder Monga Chief Technologist and Area Lead HEAnet Conference 2013 Athlone, Ireland

2 Outline Energy Sciences Network Networking for Science Software-Defined Networking

4 ESnet was formed 26 years ago - 3 years after HEAnet

5 26 Years as a Mission Network Mission of DOE Office of Science: Deliver knowledge and tools for transforming our understanding of the universe. Mission of Energy Sciences Network: Accelerate research and discovery for DOE Office of Science. $5B/year for the US National Lab Complex, which includes: world's largest collection of scientific user facilities (32) supercomputers, accelerators, xray / neutron sources, electron microscopes, sequencers, fusion facilities, Energy Sciences Network >100 Nobel Prizes

6 Embedded in a US National Laboratory (Berkeley Lab) one of 3 CS Divisions at Berkeley Lab surrounded by scientific collaborations, largescale tools, Petabytes of data, 4000 researchers / staff advantages of proximity: cafeteria and hallway conversations 11/14/2013 6

7 Coupled with a Major Research University UC Berkeley is just downhill. 36,000 students, 1500 faculty hundreds with joint appointments at Berkeley Lab 11/14/2013 7

8 Our Advisory Board Cees De Laat David Clark Larry Smarr Jagdeep Singh David Foster Vint Cerf Kristin Raushenbach

9 Log scale From HEAnet Strategic Plan

10 Global Transfer Activities (LHC/ATLAS)

11 Clev. The 100G Energy Sciences Network (Fall 2013) Int l 100 Int l JGI 10 PNNL Capability to scale to 13.2 Tbps BNL Int l SUNN SNLL LLNL AMES PPPL GFDL PU Physics 10 Int l SNLA U.S. commercial peerings U.S. R&E peerings (many) ESnet routers Optical only site routers 100G 10-40G 1G 100 Metro area circuits Site provided circuits 100 Geographical representation is only approximate SF Bay Area Chicago New York Amsterdam SUNN STAR AofA AMST 100G testbed

12 276 miles 560 miles 618 miles ESnet Research Testbeds 100G Testbed High-speed protocol research Available since Jan 2012 Dedicated 100G wave from Oakland to Chicago to NYC Connects to 100G across Atlantic to Amsterdam (ANA-100G) SUNN NERSC LBNL Test% Hosts% ESnet 100G Testbed NERSC StarLight ANL Test% Hosts% 2/25/13 3 StarLight BNL MAN LAN NYC OpenFlow Testbed 10G Nationwide Footprint ESnet 10G OpenFlow Testbed Seattle HOUS LBNL Long Haul Dark Fiber Routes 12,924 miles Dark Fiber Testbed Continental-scale fiber footprint for disruptive research 284 miles Boise Albany Eugene 2/25/13 20 Cambridge 74 miles Buffalo Chepachet 204 miles Echo Springs 264 miles Stamford Pittsburgh 61 miles 336 miles South Bend Salt Lake City 152 miles Chicago New York Reno 95 miles Cleveland Sacramento Philadelphia Peoria Goodland Indianapolis 116 miles 198 miles Sunnyvale Denver 460 miles Washington DC 138 miles 422 miles 532 miles Los Angeles 551 miles 317 miles Phoenix 325 miles Silver City 119 mil es 500 miles Albuquerque 249 miles El Paso 863 miles Kansas City Tulsa 257 miles 278 miles 317 miles 205 miles St. Louis Dallas Jackson 130 miles 119 miles 228 miles 255 miles 212 miles 259 miles Louisville Nashville 150 miles Chattanooga 147 miles Seminary 22 8 miles 246 miles Selma 172 miles 27 5 mile s Atlanta 295 mil es 306 miles 212 miles Raleigh Charlotte 179 miles 13 7 miles 248 miles 275 miles Houston BayExpres Metro Fibers: 432 miles ChiExpress Metro Fibers: 167 miles NYExpress Metro Fibers: 6 miles

14 Network engineered for the Elephants 11/14/2013

15 Sensitive Elephants, Robust Mice 7.3 Gbps out / 9.8 Gbps in 1 ms RTT 1 Gbps out / 9.5 Gbps in 11 ms RTT 122 Mbps out / 7 Gbps in 51 ms RTT 60 Mbps out / 5 Gbps in 88 ms RTT How to build a lossless network service? > 80x reduction in data transfer rate at DOErelevant distances (ANL to NERSC) and speeds (10Gpbs). Infrastructure: ample network capacity Equipment: deep packet buffers ScienceDMZ: optimized end-site architecture perfsonar: automatic and continual verification of network health OSCARS: fast lanes

16 Science DMZ Design Pattern for Data Transfer Dedicated Systems for Data Transfer Network Architecture Performance Testing & Measurement Data Transfer Node high performance tuned for data transfer proper tools Science DMZ dedicated and clean location for DTN easy to deploy - no need to redesign the whole network additional info: perfsonar enables fault isolation verifies correct operation widely deployed in ESnet and other networks, as well as sites and facilities source: Eli Dart ESnet 11/14/

17 Prototype Science DMZ Border Router Enterprise Border Router/Firewall WAN 10G 10GE perfsonar Clean, High-bandwidth WAN path 10GE Site / Campus access to Science DMZ resources Science DMZ Switch/Router 10GE Site / Campus LAN 10GE source: Eli Dart ESnet High performance Data Transfer Node with high-speed storage Per-service security policy control points perfsonar 11/14/

18 Prototype Science DMZ Data Path Border Router Enterprise Border Router/Firewall WAN 10G 10GE perfsonar Clean, High-bandwidth WAN path 10GE Site / Campus access to Science DMZ resources Science DMZ Switch/Router 10GE Site / Campus LAN 10GE source: Eli Dart ESnet High performance Data Transfer Node with high-speed storage Per-service security policy control points perfsonar High Latency WAN Path Low Latency LAN Path 11/14/

19 Science DMZ is critical. Knowledgebase: Science DMZ: Security: CC-NIE: 11/14/

21 SDN is everywhere! SDN

22 What is SDN? Loose definition: separation of data-plane from control plane In essence: enables programmability programmable Network Provisioning Network Monitoring Cloud/End-user Applications [Science] Applications Network Apps [NaaS] Network Virtualization Network Controller(OS) Statistics Topology Provisioning Protocols (SNMP, TL1) Protocol(s) (OpenFlow,?) Control Software Firmware Network ASICs Network Element control control control Control Firmware control Software control Firmware Firmware Network Firmware Network Firmware ASICs Network ASICs Network ASICs Network ASICs ASICs Network Element

23 Layer 3 Layer 2 Layer 1 What is the paradigm change? Internet today: - Built-in control in each layer - Multiple management domains SDN Approach: - Network-wide cross-layer view - OpenFlow enables programmatic access to network flows Layer 3 Control? Control (Network-wide view) Layer 3 Control OpenFlow OpenFlow Layer 2 Layer 1 Control Control Management Management 10/16/13 Inder Monga 23

24 Simple programming constructs OpenFlow 1.0 standard Rule Action Stats Packet + byte counters 1. Forward packet to zero or more ports 2. Encapsulate and forward to controller 3. Send to normal processing pipeline 4. Modify Fields 5. Any extensions you add! Switch Port VLAN ID VLAN pcp MAC src MAC dst Eth type IP Src IP Dst IP ToS IP Prot L4 sport L4 dport + mask what fields to match 11/14/2013 Inder Monga Slide courtesy Srini Seetharaman 24

25 OpenFlow usage Classic model, Simple example Controller OpenFlow Alice s Rule Switch Alice s App PC Decision? OpenFlow Protocol OpenFlow Alice s Rule Switch OpenFlow Alice s Rule Switch OpenFlow offloads control intelligence to remote software 11/14/2013 Inder Monga, WLCG GDB Alice 25

26 Why SDN? Bridging the application-network divide First workshop report for ESnet on intersite networking, 1986 Network community is still struggling to meet application requirements captured in 1986! Brute force approach (add more bandwidth) is not going to meet those requirements

27 Slide from Ian Foster

28 1-ID-1 1-ID-2 1-ID-3 1-ID-4 2-BM 2-ID-B 2-ID-E 2-ID-D 3-ID-B 3-ID-C 7 8-BM 8-ID-I 8-ID-E 9 11-ID-B 11-ID-C 11-ID-D 12-BM 12-ID-B 12-ID-C/D 15-ID ID 21-ID-D 21-ID-E 21-ID-F 21-ID-G ID-D 23-ID-B ID-1 32-ID-2 34-ID Advanced Photon Source data rates Data Rate (expected in the next 5-10 years) MB/s Data Rate (current) MB/s Francesco de Carlo, APS 28 Slide from Ian Foster

29 Hope #1: Programmability will lead to greater predictability Burst movement of data using PhEDex Analysis triggered data movement (PD2P) Science applications increasingly need to deal with high performance, anyany bursts of data SDN enables multi-layer control packet and optical layer Control over individual flows ex. Route science flows around packet bottlenecks Routing non-tcp flows over WAN Many NRENs have access to fiber, optical and packet platforms.

30 Journey towards programmability Seamless multi-layer for handling elephant flows Layer123 SDN World Congress, Bad Homburg, October 2013 OTS Config Manager L0/L1 Topology Advanced Reservation System (OSCARS) Multi-Layer Topology App Multi-Layer Path Engine Multi-Layer Provisioning Multi-Layer SDN Control Layer Traffic Optimization Engine SDN Controller Floodlight OpenFlow & REST/JSON OpenFlow 1.0 Host A Host B OTS Virtualization WDM/ OTN/ Packet Infinera DTN-X Live Demo Nov 22:

31 Abstractions are important (Scott Shenker, October 2011) The ability to master complexity is not the same as the ability to extract simplicity Abstractions key to extracting simplicity SDN is defined precisely by these three abstractions Distribution: centralized vs. distributed Forwarding: programming the fabric Specification: virtualization 11/14/2013 Inder Monga OFC/NFEC,

32 What is the right abstraction for a (dynamic) collaboration? Set of (dynamic) point to point circuits Restricted & static routing policy Lots of meetings 11/14/2013 Inder Monga OFC/NFEC,

33 Hope #2: Virtualization will simplify how applications program the network App 1 App n NB API Modeled as a Virtual WAN Network Element Simple, Multipoint, Network Virtualization Programmable simple complex Network slice Network Controller(OS)

34 Journey towards programmability Real network is too complex to program for applications SRS, Ciena, SuperComputing 2012, Salt Lake City Insights Virtualization is the killer-app for SDN (Scott Shenker) complexity pushed to the network hypervisor Architectural simplicity Flow programming only needed at edges of the network, core can be legacy

35 Thought experiment: Build an N-port virtual switch for a collaboration CERN T1 mile s kms France Italy UK Netherlands Germany Spain Nordic USA New York USA - Chicago Canada BC Taiwan Source: Bill Johnston Universities/ physics groups Universities/ physics groups Universities/ physics groups The LHC Open Network Environment (LHCONE) Universities/ physics groups Universities/ physics groups Universities/ physics groups Universities/ physics groups Universities/ physics groups Universities/ physics groups Universities/ physics groups WAN Virtual Switch Universities/ physics groups Universities/ physics groups Universities/ physics groups Universities/ physics groups Universities/ physics groups Universities/ physics groups Universities/ physics groups LHC Tier 2 Analysis Centers 11/14/2013 Inder Monga OFC/NFEC,

36 SDN is about system optimization Google s B4 SDN Network Utilization When the application and network work as a system, network resource optimization is possible Without knowledge of flows, networking can only do coarse characterization Fine discrimination of flows possible with SDN, meet application needs

37 Hope #3: SDN enables an opportunistic way to leverage all bandwidth without extra investment exploits the fact In general it s much cheaper to transport data than to store it, for example, vision of a diskless Tier3 for LHC

38 Is SDN ready for operations? 2013: SDN is here The innovator s dilemma, Clayton Christensen

39 Challenges = Opportunities? Cloud/End-user Applications 5) Who do you blame? Who do you call? Who debugs? [Science] Applications Network Apps [NaaS] Network Virtualization 4) Complexity of management from operations on virtual to physical reality? 2) Single point of failure or attack? Network Controller(OS) Statistics Topology Provisioning 3) Responsive to rapid topology changes? Flapping? 7) How does this interoperate with the current IP network? Protocol(s) (OpenFlow,?) control control control Firmware control control Firmware Firmware Network Firmware Network Firmware ASICs Network ASICs Network ASICs Network ASICs ASICs 1) Communication plane can be disrupted 6) Hardware will never be simple, manage capability differences

40 Journey towards programmability How to bridge the Internet with SDN networks? Treehouse, BGP over SDN infrastructure, ONS 2013 and ongoing Insights SDN networks can now peer with existing Internet New techniques need to be developed to scale controller-based networking Baby steps Project led by Josh Bailey, Google

41 The Bigger Picture: Organizational challenge to deal with SDN Layer 10 People People (network engineers**, sysadmins, operators**) (network engineers, + sysadmins, operators) (software engineers/devops) Layer 8-9 Network Operating System (control) Management, Tools, + Measurement New tools, service plane and management Layer 0-7 Network (control (API + and data data plane) plane) ** need to develop new skills

42 SDN Take-Away SDN is a journey R&E networks have been on for a while, but recently commercially formalized Innovator s dilemma gaps between established and the new ways, industry and researcher momentum will close those gaps Maturity will still take some time Focus on the problem being solved aka hope rather than the vendor hype Plan for the SDN future skillsets, training and hiring

43 Questions?

44 Problem: Mice and Elephant flow separation OSCARS, SDN before it was called SDN Insights abstractions are key to success, regardless of the protocol can only learn by doing (lots of naysayers) Primary use will be different than the original usecase

45 Bringing it Together: A potential SDN R&E architecture (e2e resource broker) OneWan Switch OSCARS NSI NSI NRM (e2e resource broker) Service Plane Customer SDN Ctrl. OF SDN Ctrl. OF OSCARS Customer SDN Ctrl. OF Control Plane NERSC BNL FLA Router Transport ESnet SDN ORNL FLA Router FLA Router R&E Network SDN only at edges, efficient transport in core RON OF Univ. Data Plane

46 Three Inflection Points for Data-Intensive Science Abundant capacity (88 λ x 100Gbps) 2. Programmability Campus architectures newly optimized for data mobility (optimizing network architectures end-to-end) ESnet architecture (Science DMZ) + NSF grants.

47 What is common between modern networks and analog phone switches?

48 Labor-intensive, nearly static, error prone