StarPlane take-off. Cees de Laat. SURFnet BSIK. University of Amsterdam SARA TI TNO

Transcription

1 StarPlane take-off Cees de Laat SURFnet BSIK EU University of Amsterdam SARA TI TNO NCF

2 History - 1 DAS = Distributed ASCI Supercomputer Project DAS-1 started in 1997 by Andrew Tanenbaum To prove distributed clusters were as effective as super clusters connected via high speed links DAS-1 -> 6 Mbit/s full mesh ATM DAS-2 -> Gbit/s L3 DAS-3 -> StarPlane DAS-1 ran BSD, changed to Linux (Andrew... :-) DAS-1 and 2 uniform architecture, not so in DAS-3

3 DAS 1-2 Cluster head node To local University and wide area interconnect Ethernet Local interconnect Fast interconnect 32 compute nodes

4 History - 2 SURFnet6 Architecture discussions photonic backbone (L2 and) L3 services NORTEL Static Summer 2004 K&C NWO-GLANCE StarPlane PHD-PD-SP R Start 1-feb-06, Li Xu, Jan Philip Velders, Jason Maasen Henri Bal, Paola Grosso, Herbert Bos, CdL, SN-folks. R ref: cdl ucl-opt.ppt R

5 3 ms 2 ms

6 Groningen1 Middenmeer1 Den Helder Emmeloord Harlingen Subnetwork 4: Blue Azur Lelystad2 Leeuwarden Dwingeloo1 BT Assen1 Beilen1 Hoogeveen1 Meppel1 NLR Beilen1 IBG1 & IBG2 Emmen1 Common Photonic Layer Haarlem1 Alkmaar1 Amsterdam2 Amsterdam1 BT DLO Lelystad1 NLR Subnetwork 3: Red Zwolle1 BT (CPL) in SURFnet6 Leiden1 Apeldoorn1 3XLSOP Subnetwork 1: Green Breukelen1 Schiphol-Rijk Hilversum1 Arnhem Enschede1 >5300 km Zutphen1 DenHaag Utrecht1 Wageningen1 Nijmegen1 Rotterdam4 Zierikzee Middelburg Delft1 Bergen-op- Zoom Rotterdam1 Dordrecht1 Breda1 Subnetwork 2: Dark blue Ede Nieuwegein1 Den Bosch1 Eindhoven1 Maasbracht1 Venlo1 Heerlen1 Subnetwork 5: Grey Vlissingen Krabbendijke Tilburg1 Maastricht1 Geleen1 Heerlen2 Heerlen1

7 StarPlane DWDM R CPU s backplane CPU s R R CPU s SURFnet NOC university WS+AAA SURFnet WS+AAA NOC C P U s s w it c h CPU s R CPU s R CdL

8 Module Operation input and output fibres diffraction grating common port 1 collimating lens > this schematic shows several input fibres and one output fibre light is focused and diffracted such that each channel lands on a different MEMS mirror the MEMS mirror is electronically controlled to tilt the reflecting surface the angle of tilt directs the light to the correct port > in this example: channel 1 is coming in on port 1 (shown in red) when it hits the MEMS mirror the mirror is tilted to direct this channel from port 1 to the common only port 1 satisfies this angle, therefore all other ports are blocked! 1! n MEMS mirror array (1 pixel per channel) ref Eric Bernier, NORTEL

9 DAS-3 Cluster head node To local University To SURFnet 10 Gbit/s Ethernet lanphy 1 Gbit/s Ethernet Local interconnect Fast interconnect 32 compute nodes

10 Existing ring 1 to be upgraded Existing Ring 1 G6 G6 G4 G4 CMD CMD CMD CMD CMD CMD G9 Leiden Hilversum G9 Den Haag Delft Utrecht G3 G3 G1 G1 Hoogwaardig internet voor hoger onderwijs en onderzoek

11 WSS Upgrade Day 1 creating the static mesh Amsterdam 2 Amsterdam 1 WSS -> select WSS <- WSS -> select WSS <- WSS -> select WSS <- WSS -> select WSS <- Band 5 (up to 8 channels) added at all participating nodes Wavelength Selective Switches (WSS) added for reconfigurability support G6 G6 Del G6 ft G6 Leiden Amsterdam 3 spur hardwired connections G9 G9 Hilversum G9 G9 Spur to connect VU Full photonic mesh now possible between DAS-3 sites G4 G4 G4 Den Haag G4 Delft Delft G3 G3 G3 G3 Utrecht Delft G1 G1 G1 G1 This architecture is supported now Hoogwaardig internet voor hoger onderwijs en onderzoek

12 Day 2 set-up: branching out Amsterdam 2 Amsterdam 1 WSS -> select WSS -> select WSS -> select WSS -> select Add WSSes at Amsterdam sites WSS <- WSS <- select WSS -> WSS <- WSS <- WSS <- select WSS -> WSS <- Is NOT supported in March 2006 Full reconfigurability achieved G6 G6 Delft G6 G6 Leiden branch instead of spur Amsterdam 3 Adding WSSes increases reconfigurability G9 G9 Hilversum G9 G9 Only limits are Presence of card Wavelength blocking No changes to basic static mesh G4 G4 G4 G4 Delft G3 Delft G3 Delft G1 Utrecht G1 Den Haag G3 G3 G1 G1 Hoogwaardig internet voor hoger onderwijs en onderzoek

13 Day 2 detail Leiden L-V1, L-A2, L- A3, L-D4 V-L1, A-L2, A -L3, D-L WSS -> select WSS <- V-L1, V-A 5, V-A 6, VD7 WS S <- Amsterdam 2 Does this require CMDs on all internal patches?? L-V 1, L-A 2, L-A3, L-D4 L-V1, L-A2, L-A 3, L-D4 sele ct WS S -> WSS -> select WSS <- A -L2, A-L3, D-L4, A -V 5, A-V 6, D-V 7 L-A2, L-A3, L-D4, V-A 5, V-A 6, V-D7 A-L2, A-L3, D-L4, A-V 5, A-V 6, D-V 7 WSS -> select WSS <- A -D1, A-L2, A -L3, A-D4, A -V5, A-V 6, W SS <- Amsterdam 1 Does this require CMDs on all internal patches?? L-A 2, L-A3, L-D4, V-A5, V-A6, V-D7 spl it L-A2, L-A3, L-D4, V-A 5, V-A 6, V-D7 A-D1, A-L2, A -L3, A-V5, A -V 6,A -D8 se lect W SS -> D-A 1, D-L4, D-V7, D-A8 WSS -> select WSS <- Delft A -D1, L-D4, V-D7, A-D8 TO DELFT D-A 1, D-L4, D-V7, D-A Wavelength assignment remains no external changes Adding WSSes allows redirecting wavelengths from/to VU and AMS V -L1, V-A 5, V-A6, VD 7 L-V1, A-V 5, A -V6, D-V7 A -D1, A-L2, A -L3, A-V5, A -V6, A-D8 D-A 1, L-A 2, L-A 3, V-A 5, V-A 6, D-A Amsterdam 3 - VU Hoogwaardig internet voor hoger onderwijs en onderzoek

14 Day 2 black box reconfigurability DAS-3 Switch Amsterdam DAS-3 Switch Hilversum Compared to day 1 now four instead of one possible redirection DAS-3 Switch Delft AMS_VU + 1 AMS-LE + 1 AMS_DE + 0 VU-LE + 0 VU-DE + 1 LE-DE Amsterdam 1 VU Leiden DAS-3 Switch Redirection only limited by presence of cards and internal wavelength blocking DAS-3 Switch Hoogwaardig internet voor hoger onderwijs en onderzoek

15 Day 2 increased reconfigurability - adding cards Delft Amsterdam Hilversum Leiden Adding two cards allows to create more connectivity between ALL sites! AMS_VU + 2 AMS-LE + 2 AMS_DE + 1 VU-LE + 1 VU-DE + 2 LE-DE Amsterdam 4 VU Some sites can connectivity threefold (from 10 Gb/s to 30 Gb/s) Hoogwaardig internet voor hoger onderwijs en onderzoek

16 StarPlane Goals 1. fast, application-driven allocation of the photonic network resources 2. application-specific composition of the protocol stack that controls the resources 3. low-level resource partitioning (and, hence, no interference) 4. high-level requests (whereby policies and inference are used to assist the user).. To achieve and validate these goals the project will deliver: 1. the implementation of the StarPlane management infrastructure 2. the implementation of an intelligent broker service to handle highlevel requests 3. the modification of a set of real applications to exploit the functionality of such a management plane 4. a library of standard components (protocols, middleware) to support and build new applications

17 GRID-Colocation problem space CPU DATA Extensively under research Lambda s New!

18

19

20 Key issue #1: how to describe such networks?

21 UvA/SARA LightHouse A joint network research lab of the University of Amsterdam and SARA. Connects end resources to NetherLight. Proof of concept e.g. tier 0/1, webservices, GSP

22 Semantic web a universal medium for the exchange of data where data can be shared and processed by automated tools as well as by people The Resource Description Framework (RDF) uses XML as an interchange syntax. Data is described by triplets: Subject Predicate Object

23 NDL - Network Description Language A way to describe network resources using RDF. Parser can use the data to: - generate network maps - provide information to schedulers <ndl:device rdf:about="#vangogh3"> <ndl:name>vangogh3</ndl:name> <rb:isoftype>computingelement</rb:isoftype> <ndl:locatedat rdf:resource="#lighthouse"/> <ndl:hasinterface rdf:resource="#vangogh3:eth2"/> </ndl:device>

24 NetherLight in RDF <?xml version="1.0" encoding="utf-8"?> <rdf:rdf xmlns:rdf=" xmlns:ndl=" <!-- Description of Netherlight --> <ndl:location rdf:about="#netherlight"> <ndl:name>netherlight Optical Exchange</ndl:name> </ndl:location> <!-- TDM3.amsterdam1.netherlight.net --> <ndl:device rdf:about="#tdm3.amsterdam1.netherlight.net"> <ndl:name>tdm3.amsterdam1.netherlight.net</ndl:name> <ndl:locatedat rdf:resource="#amsterdam1.netherlight.net"/> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:501/1"/> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:501/3"/> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:501/4"/> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:503/1"/> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:503/2"/> <!-- all the interfaces of TDM3.amsterdam1.netherlight.net --> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:503/3"/> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:503/4"/> <ndl:interface rdf:about="#tdm3.amsterdam1.netherlight.net:501/1"> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:504/1"/> <ndl:name>tdm3.amsterdam1.netherlight.net:pos501/1</ndl:name> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:504/2"/> <ndl:connectedto rdf:resource="#tdm4.amsterdam1.netherlight.net:5/1"/> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:504/3"/> </ndl:interface> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:504/4"/> <ndl:interface rdf:about="#tdm3.amsterdam1.netherlight.net:501/2"> <ndl:hasinterface rdf:resource="#tdm3.amsterdam1.netherlight.net:501/2"/> <ndl:name>tdm3.amsterdam1.netherlight.net:pos501/2</ndl:name> <ndl:connectedto rdf:resource="#tdm1.amsterdam1.netherlight.net:12/1"/> </ndl:interface>

25 RDF -> Picture

26 Key issue #2: How to book resources on such networks?

27 Web services Web services interfaces provide the API for the reservation framework: <wsdl:operation name="getresourceinformation"> <wsdl:operation name="getresourcelist"> <wsdl:operation name="gettypelist"> <wsdl:operation name="getresourcesoftype"> <wsdl:operation name="reservepath"> <wsdl:operation name="getpossiblepaths"> <wsdl:operation name="ispathavailable"> <wsdl:operation name="confirmpathreservation"> <wsdl:operation name="cancelpathreservation">

28

29 Questions? Credits: Leon Gommans, Paola Grosso, Marten Hoekstra, Arie Taal, Freek Dijkstra, Bert Andree, Jeroen van der Ham, Hans Blom, Yuri Demchenko, Fred Wan, Karst Koymans, Martijn Steenbakkers Jaap van Ginkel, Li Xu SURFnet / GigaPort, Kees Neggers, Erik-Jan Bos, et al! NORTEL: Franco Travostino, Kim Roberts, Rod Wilson SARA: Anwar Osseryan, Paul Wielinga, Pieter de Boer, Ronald van der Pol, teams Joe Mambretti, Bill starnaud, GLIF community Tom & Maxine & Larry, Laurin, OptIPuter, OnVector team!!!!