https://indico.cern.ch/event/401680/ P2P Service Trial Session Notes Amsterdam LHCONE meeting, Oct. 28-29, 2015 1 Introduction Gerben van Malenstein, SURFnet The general activities of the service trial are to 1) Connect sites to AutoGOLE for P2P trials 2) Investigate middleware integration of the virtual circuit The situation P2P service as of the Amsterdam meeting was For sites supporting OpenFlow, Caltech has an LHCONE P2P prototype For sites using IP routers, NORDUnet has a BGP-based approach 2 Dynamic P2P Service with BGP Route Servers Magnus Bergroth, NORDUnet Current situation: P2P circuits mostly connect sites rather than applications Dynamic P2P circuits have end points that typically terminate on a site aggregation router o In other words, P2P circuits are typically used to connect two sites ebgp is configured on the interface at each site o Reachability is advertised after the P2P link is up and BGP established There are several problems with this o A full mesh of BGP sessions must be maintained o Extensive configuration is required o BGP sessions on short-lived P2P circuits are down most of the time which causes alarms Proposal: Use a Route Server based approach One BGP session per site minimizes configuration BGP session is always up Prefixes are learned via BGP only when dynamic P2P is up o Use BFD connection up/down condition to prevent BGP advertisement if condition ( up ) is not met o Put BGP with conditions in route server o Fast detection when dynamic P2P goes down If a link goes down, how long for route server to detect? MB: seconds. Route server provides one RIB per site with steering done using communities Edoardo Martelli experimented with o Creating a route server at CERN o Clint routers communicate reachability to the route server via Link-State and TE information
o Route server stores, for each peer, a Next-Hop Information Base (NHIB) giving reachability for all nexthops o Client routers can verify connectivity to other sites using BFD (Bidirectional Forwarding Detection) Several project are under way that could implement this: o BIRD Internet routing daemon a project of Charles University, Prague, CZ BIRD is fairly widely used in Europe as a route server as well as the routing table manager in some open-source routers. See https://en.wikipedia.org/wiki/bird_internet_routing_daemon for an introduction. o Quagga routing software suite http://www.nongnu.org/quagga/ o Several issues of interest to our P2P usages have been communicated to the BIRD and Quagga developers for changes 3 Progress on SDN, OVS and dynamic circuits Ramiro Voicu, Caltech Current situation: OpenFlow based P2P circuits and application to application experience Using dynamic circuits in PhEDEx allows for more deterministic workflows, useful for co-scheduling CPU with data movement o Integrating circuit awareness into the FileDownload agent o Application is backend agnostic; No modifications to PhEDEx DB o All control logic is in the FileDownload agent o Transparent for all other PhEDEx instances o The approach is to create a shim in the storage access stack for PhEDEx so that all circuit control logic is transparent to the application, including circuit failure. A testbed consisting of AutoGOLE sites interconnected by OSCARS NSI circuits Progress in OpenFlow+SDN technology Open vswitch ( OVS ) is a production quality, multilayer virtual switch o OpenFlow protocol support (1.3) o Kernel and user space forwarding engines 2
o Fine grained QoS Ingress qdisc (packet scheduling), HTB (a qdisc class that allows for packet filters), HFSC, (Hierarchical Fair Service Curve classful shaping and scheduling) /2/ o Interoperates with OpenStack (which manages a pool of hardware resources in a Cloud Computing environment) o Open Virtual Network OVN complements the existing capabilities of OVS to add native support for virtual network abstractions, such as virtual L2 and L3 overlays and security groups. /3/ Using OVS for end-host orchestration Integrating PhEDEx with Dynamic Circuits for CMS o Standard OpenFlow (or OVSDB) protocol for end-host network orchestration (no need for custom SB (switch and router control communication protocol /4/) o Simple procedure to migrate to OVS on the end-host. SDN controller not required in the initial deployment phase Traffic Shaping with Open vswitch (OVS) WAN tests over NSI o OVS 2.4 with stock kernel o NSI circuit Caltech -> UMICH (~60ms) o Very stable up to 7.5Gbps o Fairly good shaping above 8Gbps (small instabilities) 3 Open vswitch (OVS) benefits o Standard OpenFlow (and/or OVSDB) end-host orchestration o QoS SDN orchestration in non-openflow clusters o OVS works with stock SL/CentOS/RH 6.x kernel used in o HEP; works out-of-the-box on SL7/CC7 o OVS bridged interface achieved the same performance as the hardware (10Gbps) o No CPU overhead when OVS does traffic shaping on the physical port o Traffic shaping (egress) of outgoing flows may help performance in such cases when the upstream switch (or ToR) has smaller buffers Open Daylight (ODL) controlling Open vswitch via OVSDB and OpenFlow o OVSDB Open vswitch Database Management is for managing and configuring Open vswitch instances /5/ o Support as SB /4/ protocol in major SDN controllers
o Used to create the virtual bridges o Virtual bridges can use standard OF to speak with the controller o Normal routing if the controller is down There follows a discussion of o OpenFlow topology discovery in a network with non-openflow islands o OpenFlow islands over WAN & NSI circuits o Possible internetwork architecture with a single controller Glossary /1/ Qdisc, : See Components of Linux Traffic Control at http://tldp.org/howto/traffic-control- HOWTO/components.html#c-filter /2/ HFSC: see HFSC - Linux traffic shaping's best kept secret at https://gist.github.com/bradoaks/940616 /3/ OVN: see OVN, Bringing Native Virtual Networking to OVS at http://networkheresy.com/2015/01/13/ovn-bringing-native-virtual-networking-to-ovs/ /4/ SB: In a software-defined network (SDN) architecture, southbound application program interfaces (APIs) are used to communicate between the SDN Controller and the switches and routers of the network. They can be open or proprietary. See https://www.sdxcentral.com/resources/sdn/southbound-interface-api/ /5/ RFC 7047 on The Open vswitch Database Management Protocol: Open vswitch is an open-source software switch designed to be used as a vswitch (virtual switch) in virtualized server environments. A vswitch forwards traffic between different virtual machines (VMs) on the same physical host and also forwards traffic between VMs and the physical network. Open vswitch is open to programmatic extension and control using OpenFlow and the OVSDB (Open vswitch Database) management protocol. This document defines the 4
OVSDB management protocol. The Open vswitch project includes open-source OVSDB client and server implementations. The schema of the Open vswitch database is documented in [DB-SCHEMA]. This document specifies the protocol for interacting with that database for the purposes of managing and configuring Open vswitch instances. 4 GÉANT BoD Service Evolution; Introducing SDN capabilities in backbone - Mian Usman, GÉANT Current BoD implementation o JunOS Space is the device controller for BoD and does not work well enough to use in production o Deployment is in a set of waves parallel to the production network o Path can be mapped to NSI DynPaC o New service manager for introduction of new services into network Move flows to alternate paths, disaggregation, resiliency o Will migrate to ONOS /1/ controller WEJ: What does this imply for support of BGP-based hybrids that are currently the norm because very few sites are production SDN capable? Glossary /1/ ONOS: Open Network Operating System, is a newly released open-source SDN controller that is focused on service provider use-cases. Carrier organization is developing. http://sdnhub.org/tutorials/onos/ What difference between ONOS and ODL?" 1) The major differences between the two is that ODL focuses on bringing legacy (BGP, SNMP, and such) and new networks (i.e., OpenFlow and SDN) together whereas ONOS focuses on the performance aspects and the clustering to increase the availability and scalability. I heard that ODL recently started to enhance their performance and Lithium release begins to support clustering. So I think both of them can learn and benefit from each other. So I can safely say the major difference is their focuses, but they could change overtime. 2) ONOS focuses more on carrier-grade networks and many telcos are involved in their projects. ODL has more vendors like Cisco, Juniper, and NEC than ONOS. Real field deployments are not very common for both of them as far as I know. - Justin Park, SDN researcher, Daejeon, Republic of Korea. 5 Automated GOLE update Gerben van Malenstein, SURFnet AutoGOLE fabric delivers dynamic network services between GOLEs and networks o Based on NSI Connection Service v2.0 and TLS o Using DDS /1/ service between aggregators o Used for numerous projects Work in progress includes o Monitoring system for fault finding and troubleshooting 5
AutoGOLE Dashboard Prototype exists e.g. http://dashboard.lab.uvalight.net:8000/dpm o Supporting LHC Sites Supporting LHC sites that are connecting to the AutoGOLE o More redundant control plane Forwarding requests to another aggregator if a certain upa cannot be reached Future work o Getting rid of old common VLAN range Support for VLAN retagging needed in all GOLEs and networks Ultimately using ENNI /2/ ports between networks (requires a new service definition) o Further data plane improvements, e.g. full dynamic control over long-haul links o Shifting from manual/noc operation to dynamic provisioning by default Glossary /1/ DDS - Document Distribution Service: A simple peer-to-peer flooding protocol for exchange and distribution data documents between NSA within the interconnected control plane or document space. Supports both polling and subscription based notification mechanisms for exchange of documents. Documents that might be distributed: topology descriptions, NSA identities/locations, Service Definition, etc. See Network Services, Interface Document Distribution Service John MacAuley, ESnet, Open Grid Forum presentation. https://redmine.ogf.org/dmsf_files/13404?download= /2/ External Network-to-Network Interface (ENNI) for service availability Simple 2-carrier and multi-carrier service models Ethernet Virtual Connection (EVC) between UNIs spread across multiple networks Operator Virtual Connection (OVC) for each EVC segment o Key ENNI requirements Services: p2p and mp EVC types, single and multiple CoS per EVC Encapsulation: Standard S-Tag frame format Scalability: 1 and 10 Gbps PHYs ENNI Protection: 2-link LAG, active/standby, LACP o See, Developing Ethernet Services Offerings with MEF Specifications, Dmitry Dergalov, Technical Director, RAD Data Communications-Russia. http://www.rad.ru/media/21037_transnet-2011- DergalovD_RADkr.ppt 6