The future of SDN: Transforming the REN in support of Big Data Oct 4th, 2013 Eric Choi Senior Product Manager, Data Centre Business Unit APAC and Japan
SDN Transforming REN Across Data Centres
Data Intensive Science Main Driver for High Capacity Research Education Networks Large Hadron Collider- ATLAS ~10s of terabytes of data transfers Data distribution across continents Distribution rates at 40+ Gbps Requirements increasing over time Genomics Data rates increasing exponentially Instrument data rate up 10x in 5 yrs Sequencing cost down 10x in 5 yrs New applications for genomics data as science improves Square Kilometer Array 100Gbps rates needed to distribute data to international collaborators Large radio telescope ~1 sq km of combined signal collection area ~2800 telescopes in array, ~2 petabytes/sec at central correlator
Uniqueness of Research Networks Commercial networks Research/collaboration network #flows Many (mouse flows) Few ( elephant flows) Flow sizes TCP effectiveness Typically follows Internet mix (IMIX) Very good Very large- up to 50% of link bandwidth in a single flow Weak: result in loss sensitivity, increased CPU load
hashing Typical 100GbE Forwarding Engine Architecture Suitable for internet traffic but not elephant flows in REN 10 wavelengths of 10Gbps 10 x10g 10 x10g 64B/66B PHY 50Gbps PFE 50Gbps PFE 10 wavelengths of 10Gbps 10 x10g 10 x10g 64B/66B PHY Lkup hashing. 100Gbps Buffering ASIC Lkup COMPROMISED NETWORK PERFORMANCE DUE TO UNEQUAL LOAD BALANCING 5 2013 Brocade Communications Systems, Inc. Company Proprietary Information
Packet Buffer for 10Ge/100Ge port Uniqueness of Research Networks 40G 25G Avg 25Gbps No Congestion? 100Ge Router 40G Avg 25Gbps 25G 120G Avg 25Gbps 40G 100GbE 75G 100GbE 25G SUFFICIENT PACKET BUFFER IS REQUIRED TO TCP SESSION BACKOFF 6 2012 Brocade Communications Systems, Inc. CONFIDENTIAL For Internal Use Only
Internet2 Requirement of On-Demand L2 Connectivity Olympia Portland Eugene Sacramento Sunnyvale San Luis Obispo Los Angeles San Diego Seattle Reno IP router node 7 Spokane Bozeman Boise Las Vegas Optical add/drop facility Missoula Salt Lake City Phoenix Tucson Billings Miles City Dickinson Denver Albuquerque El Paso Bismarck Minneapolis Kansas City San Antonio Fargo Houston (2) Chicago (3) Tulsa Dallas Madison Memphis St. Louis Jackson Baton Rouge Detroit Chattanooga Nashville Buffalo Cincinnati Louisville Cleveland Indianapolis Ashburn Atlanta Pittsburgh Charlotte Albany Philadelphia Boston New York (2) Washington DC Raleigh Jacksonville 2013 Brocade Communications Systems, Inc. Company Proprietary Information Exchange Point Internet 2 49 Custom Location Facilities 15,500 miles of dark Fiber 8.8 Tbps of Optical Capacity Hybrid Mode with protected OpenFlow traffic SOME OF THESE PROBLEMS CAN BE ADDRESSED BY PROGRAMATIC NETWORK CONTROL
OpenFlow-Enabled Device Operation How it works? The Flow Table contains Flow Entries Each Flow Entry represents a Flow, e.g., packets with a given destination IP address Incoming packets are matched against the Flow Entries in priority order Matching stops when a match occurs, and the actions for that Flow Entry are performed Unmatched packets are typically dropped Flow Entry Matching Fields Actions Flow Table Flow Entries Highest Priority Lowest Priority Stats Matching search 8 2012 Brocade Communications Systems, Inc. PROPRIETARY
SDN WAN Use Case 1 Protected Production IP Network with OpenFlow Overlay Protection Layer Features (Applications) Network Controller OpenFlow Overlay Traditional MPLS/IP Routing WAN Physical Infrastructure Hybrid port mode OpenFlow overlay runs concurrently with traditional MPLS/IP routing OpenFlow enabled on existing production network Protected OpenFlow experimentation by researchers does not affect production traffic OpenFlow/Production traffic isolation in hardware 9 2013 Brocade Communications Systems, Inc. Company Proprietary Information
SDN WAN Use Case 2 Unprotected Production IP Network with OpenFlow Overlay Hybrid port mode OpenFlow overlay runs concurrently with traditional MPLS/IP routing OpenFlow enabled on existing production network Unprotected OpenFlow allowed as an exception rule to MPLS/IP forwarding Service Provider can add new WAN Physical Infrastructure revenue-generating features on top of existing production network 2013 Brocade Communications Systems, Inc. Company Proprietary Information 10 Features (Applications) Network Controller OpenFlow Overlay Traditional MPLS/IP Routing
Hybrid Mode Operation Flow Packet flow Priority of lookup for hybrid mode ports are based on below order (high to low). Packets with Protected vlans Packets matching Openflow rules Packets with Configured unprotected vlans All other packets MAP NI 5.5 11 2013 Brocade Communications Systems, Inc. CONFIDENTIAL For Internal Use Only
Feature Reference Diagram Openflow Hybrid port MAP NI 5.5 12 2013 Brocade Communications Systems, Inc. CONFIDENTIAL For Internal Use Only
SDN Transforming REN within Data Centre
Software Defined Networking: Overlay Networking Approach: Build Overlay Logical Networks Research Team A Research Team B Research Team c Server vswitch DC Logical Network DC Physical Network Server vswitch Server vswitch East-west tunnels between software switches (vswitch) enable physical network abstraction (logical networks) Proposed tunnel technologies VxLAN, STT: ware NVGRE: Microsoft Technology benefits Supports millions of logical networks (no 4K limit as VLANs) Support s with overlapping IP/MAC addresses 2013 Brocade Communications Systems, Inc. Company Proprietary Information
External Storage Capacity by Protocol (PB) IDC: By 2017, FC remains on top IDC Worldwide Enterprise Storage Systems Forecast Update, May 2013
How can we connect them together? FW Aggregation FW Storage is still physical and non-vxlan aware SLB Ethernet Fabric SLB Some applications running on physical servers like Oracle DB etc. Existing appliances like firewalls and server loadbalancers. Access to existing L3 networks via existing routers. Physical Servers Non-VxLAN vsw 2 4 1 VxLAN vsw 3 10/7/2013 16 2013 Brocade Communications Systems, Inc. CONFIDENTIAL For Internal Use Only
Underlay Overlay Tunnel Control Network (Overlay) Virtualization Introducing VxLAN Gateway OVERLAY CONTROLLERS Tenant Virtual Subnets (Logical View) VXLAN/GRE Tunnels Any Network Infrastructure VXLAN/GRE Tunnels vswitch vswitch VTEP Gateway OS OS OS OS OS OS App App App App App App Web/DB/App Virtual (Any Hypervisor) Physical
Virtual Physical INFRASTRUCTURE Network Virtualization with VxLAN GW & ware NSX Solution Overview and Components VxLAN Tunnels VTEP VTEP Mgmt VSwitch Ethernet Fabric VTEP 3 VSwitch Server 2 VTEP 1 Storage Physical Connectivity Services 1. VXLAN Gateway Bridges virtual and physical infrastructure 2. Ware NSX: Network Provisioning and Automation 3. Ethernet Fabric: Efficient, Resilient foundation for the Network Infrastructure
Summary Two Key Technologies in SDN: Device Based(Openflow) and Overlay Tunnel based. Network programmability, openness, control/data plane separation. Near-term use cases on WAN/DC network virtualization. Hybrid port mode simplifies transition to SDN Network Virtualization using Overlay Tunnels 2013 Brocade Communications Systems, Inc. Discussed under NDA