SURFnet7 explained SNE, 17 maart 2014 Wouter Huisman
SURFnet7: Scalable network Institute Institute Routed IP Applica'on Applica'on Eduroam Applica'on (Layer 3) Applica'on Eduroam Applica'on Applica'on Service Next Generation Ethernet (Layer 2) Service Bandwidth Optical Transport (Layer 0/1) Bandwidth 2
Transport layer SURFnet6
Juniper T-series: 4/2
Ciena OM5200: 20
Nortel MERS 8600: 20
Ciena OME6500: 100
Ciena CPL: 100
Nortel OME1010/1060: 350
What to optimize of SURFnet6? And make SURFnet7 simple
Concepts in the SURFnet7 tender Single technology end-2-end transport Remove as many OEO as possible Introduce 40G and 100G Improve the BoD solution Selected Ciena CES solution 5410, 3930, 5150 Based on PBB-TE
Tunable XFP & OTN XFP 5410 - CES CPL DWDM OADM Transponder No transponder shelves required Tunable DWDM XFP used Save on power, footprint, OAM ~25% of the links require G.709
Scalability issue Uptake lightpath services was significant Technology limitation Bandwidth claimed for 100%, even though not all used. Single service per port!!
Statistical Multiplexing a EIR CIR a b c b c a,b,c a b c 14
One or more services on a single port Single Service Port (SSP) - One service, one port Multi Service Port (MSP) - Multiple network services, one port - Efficient use of interfaces SURFinternet SURF-lightpath SURF-lightpath OnDemand lightpath Free 10 Gbit/s MSP Interface 2 Gbit/s 1 Gbit/s 2 Gbit/s 3 Gbit/s 2 Gbit/s 15
BoD in SURFnet7 NOC engineer User Applica/on NSI Inter Domain NSI GUI GUI SURFnet7 BoD MTOSI SURFnet7 NMS API Iden'ty Provider Group Provider 16 16
Ciena CESD solution
Ciena 5410 Core switch, 2 per main POP Modular switch with 400Gb/s per slot 32-port 1G 4-port 10G 10-port 10G 40-port 10G/48-port 1G 4-port 100G/2-port 40G Pluggable optics 1G SX, LX, CWDM, DWDM 10G SR, LR, CWDM, DWDM 40G LR4 CFP 100G LR4 CFP
Ciena 5150 Pizza box with extension slots 2U height 48x 1G SFP 4x 10G XFP Dual AC/DC power
Ciena 3930 Pizza box 1U height 8x 1G SFP 2x 1G/10G SFP+ Dual AC/DC power
Lightpath Transparent Guaranteed Bandwidth Minimal jitter and latency Behaves like a patch cord
Behaves like a patch cord Remote Port shutdown / Virtual Link Loss Indication (VLLI) Port down Port down Service- level 802.1ag CCMs
Behaves like a patch cord Port down Service- level 802.1ag CCMs BGP can respond to the portstate, without the BGP keepalive timer being timed out, thus very quick.
PBB-TE Carrier Ethernet technology Connection oriented Ethernet Tunnels with protection mechanism Services transported through PBB-TE tunnels Customer data is encapsulated at the edges
SURFnet7 EPL - EPL payload payload payload SA SA SA DA DA DA I- SID B- VID B- SA B- DA 802.1p / 802.1Q EPL UNI (MEF EVC UNI) 802.1ah EPL UNI (MEF EVC UNI) 802.1p / 802.1Q
PBB-TE Protection Connection Oriented Ethernet Primary & Backup Tunnels Monitored via 802.1ag CCMs CCMs configurable for 10 msec intervals MEP A MEP C Ingress PBB-TE Edge Bridge Primary Tunnel Protected Node MEP B MEP D Egress PBB-TE Edge Bridge 26
Fiber PBB-TE hierarchy from fiber to service
PBB-TE hierarchy from fiber to service Links Fiber
PBB-TE hierarchy from fiber to service Aggregates Links Fiber
PBB-TE hierarchy from fiber to service PBB- TE Tunnels Aggregates Links Fiber
PBB-TE hierarchy from fiber to service Services PBB- TE Tunnels Aggregates Links Fiber
PBB-TE hierarchy from fiber to service Services PBB- TE Tunnels Aggregates Links Fiber
Bandbreedte garantie A xgb/s protected lightpath from A to B 1. Remove all links from the map with insufficient bandwidth available 2. Find shortest path by OSPF route algorithm 3. Remove primary path 4. Find secundary shortest path by OSPF route algorithm 5. Confirm tunnel and claim bandwidth in database A B
Bandbreedte garantie A xgb/s protected lightpath from A to B 1. Remove all links from the map with insufficient bandwidth available 2. Find shortest path by OSPF route algorithm 3. Remove primary path 4. Find secundary shortest path by OSPF route algorithm 5. Confirm tunnel and claim bandwidth in database 6. Create service on the selected tunnel A B
Creating a tunnel A C Startnode B Transit node Z Transit node Endnode virtual-switch create vs TRANSIT-1334! pbt transit create pbt-transit TRANSIT-1334_503 parent-port 5/3! pbt transit add pbt-transit TRANSIT-1334_503 class-element 1 bvid 1334! virtual-switch vlan interface create vlan attach 1337 pbt-transit name TRANSIT-1337! TRANSIT-1334_503 vs TRANSIT-1334! vlan add vlan 1337 port 3.1! pbt transit vlan create add pbt-transit vlan 1337 port TRANSIT-1334_101 to_asd001a_5410_01t_l1! parent-port 1/1! pbt transit add pbt-transit TRANSIT-1334_101 class-element 1 bvid 1334! virtual-switch interface attach pbt-transit TRANSIT-1334_101 vs TRANSIT-1334!
Creating a tunnel A C Startnode B Transit node Transit node Z Endnode pbt tunnel-group create group DH5150_5410s tunnel-sync on! pbt encap-tunnel create static-encap P_DH5150_5410s tunnel-group DH5150_5410s pair-index 4 port to_asd001a_5150_02t_l1 bvid 1333 destbridge-name Asd001A_5150_02 weight 6! pbt decap-tunnel create static-decap P_DH5150_5410s tunnel-group DH5150_5410s pair-index 4 port to_asd001a_5150_02t_l1 bvid 1333 srcbridge-name Asd001A_5150_02! cfm service create static-encap P_DH5150_5410s name PBT-1333 next-mepid 1333 level 2! cfm service set service PBT-1333 ccm-interval 100ms alarm-time 0! cfm service enable service PBT-1333!
Creating a service A C Startnode B Z Endnode virtual-circuit pbt create static-vc 3669IP tunnel 1Hedr1_Asd_D egress-isid 73860 ingress-isid 73860! virtual-circuit pbt set static-vc 3669IP retain-stag yes! virtual-switch add reserved-vlan 150! virtual-switch ethernet create vs 3669IP vc 3669IP reserved-vlan 150! virtual-switch ethernet set vs 3669IP description "3669IP"! virtual-switch ethernet set vs 3669IP encap-cos-policy fixed encap-fixed-dot1dpri 3!! MTU size increased to 9190 for CN 5150 Hedr001A_5150_01 port 1.1! port set port 1.1 max-frame-size 9190! port set port 1.1 vs-ingress-filter off! virtual-switch ethernet add vs 3669IP port 1.1! port set port 1.1 acceptable-frame-type all untagged-data-vs 3669IP! aggregation set port 1.1 agg-mode manual! lldp set port 1.1 mode rx-only notification off! virtual-switch ethernet set port 1.1 vs 3669IP encap-cos-policy fixed encap-fixed-dot1dpri 3! traffic-profiling set port 1.1 mode standard-dot1dpri! traffic-profiling standard-profile create port 1.1 profile 1 name 3669IP01 cir 0 cbs 16 pir 1000000 ebs 64! traffic-profiling set port 1.1 nonconform-standard-profile 3669IP01! traffic-profiling enable port 1.1! cfm service create vs 3669IP name 3669IP next 1 level 4! cfm service set service 3669IP alarm-priority 3! cfm service set service 3669IP alarm-time 10000! cfm service set service 3669IP reset-time 3000! cfm service set service 3669IP ccm-interval 1s! cfm service enable service 3669IP!! CN 5150 Hedr001A_5150_01 port 1.1 forced to Enabled! port enable port 1.1!
Tiered Network Architecture
Tiered Network Architecture
Tiered Network Architecture
Tiered Network Architecture
Tiered Network Architecture
Fiber dilemma Opties nieuw Via Hoofd PoP naar Leiden Via Red. PoP naar Delft Gv041A Gv015A GM4 GM3 Gv017A Gv032A Project PoP redundantie Den Haag # Fase 0 Opties nieuw Gv014A GM2 Gv009A Asd001A ROC Mondriaan is afgevallen als redundante PoP voor Den Haag Gv010A Gv012A Asd002A Gv006A GM1 Gv006B Gv0037A CPL Ledn002A GM-ring A CPL Dt001B SPOF 7,5km CPL Red. PoP Gv 5410_02 TOADM Gv001A HHS 5410_01 WSS SPOF 2,5km Gv002A GM1 Gv043A Ledn007A KOG 5410_02 TOADM Regio CPL Ledn001A Snellius 5410_01 WSS SPOF 3,0km Gv008A GM2 TNO nieuw GM-ring A Ledn003A GM3 Ledn004B GM-string B Ledn004A Gv029A GM4 Gv020A Gv038A Ledn010A GM-string C GM4 Gv040A GM3 Gv016A Gv013A GM2 Gv007A GM1 Gv004A Via Red. PoP naar Delft Via Hoofd PoP naar Leiden Page no. 1 PoP redundantie/faseplan-den-haag-versie-0.60 Alex Kip - NWD
Protected services
Protected services
Protected services
Protected services
Protected services
Protected services
wouter.huisman[at]surfnet.nl W www.surfnet.nl +31 30 2 305 305 Creative Commons Attribution license: http://creativecommons.org/licenses/by/3.0/