Data transport in radio astronomy Arpad Szomoru, JIVE
Some acronyms EVN: European VLBI Network Consortium of radio telescopes Involving 14 different organizations around the world: Europe, China, Puerto Rico, South Africa, Russia, Korea Operational approximately 70 days/year JIVE: Joint Institute for VLBI in Europe Promote and advance the use of VLBI for astronomy Founded in 1993 Base budget from partners in 9 countries: China, France, Germany, Italy, Spain, Sweden, United Kingdom, the Netherlands, South Africa About to become an ERIC
Radio vs. Optical Astronomy Radio waves with λ of 0.7mm to 90cm Compared to optical light 400 700 nm
Radio Astronomy and VLBI Radio emission from astrophysical sources can be detected against the sky with telescopes larger than a few meters Resolution scales with size and wavelength: Solution: build larger telescopes! Only goes so far... θ λ D Or: combine series of telescopes into radio-interferometer Not necessarily on same continent Or planet
Astronomy in motion A resolution which is high enough to see things move at cosmological distances Like quasars at the edge of the Universe
Exploding stars in other galaxies supernova in M81 (1993)
PRIDE PRIDE: a multidisciplinary enhancement of the mission science return with minimum onboard instrumentation Planetary Radio Interferometry & Doppler Experiment
VLBI data storage/transport Several PB transported per year Since the late nineties Big data?
e-vlbi. Why? Rapid turn-around Rapid response Instant feedback Robustness, reliability Logistics: several PB on 100s of disk packs in transit No (less) dependence on magnetic media Cheaper (?) Upgrade path for higher bandwidth New science
First steps
January 2002: Proof-of-Concept e-vlbi over GÉANT ES UK BE NL DE SE CH HU IT 2.5 G GÉANT GEANT 0 FR GR 310 M CZ AT PT SI PL HR 34 M 45 M LU IE RO EVN telescope LV BG CY LT IL SK EE
From Proof of Concept to reality: EXPReS EXPReS proposal to Research Infrastructures Call - Communication and Network Developments Project kicks off 2006 Retrofit correlator to work real-time Help solve last mile problem at telescopes Work with NRENs on robust connectivity Push to 1024 Mbps limit Bring in the big telescopes
First European transfer tests Mbps 800 600 400 Tr-Dw 200 0 Dw-Bo max min
A truly global result Connections work great! Dedicated lightpaths, VPNs, routed connections Optimized transport protocol Instant feedback: very robust
Re-cycling of telco dishes Many telecom dishes are being scrapped Some functional, well maintained But need major refurbishing Interest in astronomical community: UK, New Zealand, South Africa, South America, the Azores Up to 26 dishes spread across the African continent: immediate connection to European VLBI
The future. Just about happening.. Higher bandwidth, higher sensitivity: 2 Gb/s, 4 Gb/s,.. Bandwidth-on-Demand to be implemented by NRENs (?) 100 Gb/s technology rolled out No more shipping of magnetic media Optimized e-shipping using home-grown software (which works!) More telescopes, higher sensitivity 65m-class Shanghai and Sardinia telescopes operational Clocks, frequency standards via public networks 32 Gb/s/telescope VLBI with event horizon telescope VLBI with new and developing telescope arrays ALMA, MeerKAT, African VLBI Array, SKA,
Current challenges: LOFAR and ILT 240 Gb/s from stations to correlator On bespoke network 6 PB/year growth of archive Needs to be shipped for (re-)processing
Future challenges: SKA Three telescope arrays: Mid: 256 dishes, 90 Gb/s/dish Survey: 96 dishes RF over fibre (core) 21 long baselines 864 Gb/s/dish Low: 1024 stations RF over fibre (core) 45 long baselines 10 Gb/s/station Equivalent to ~120 Tb/s (?) 100 days operation/year ~130 EB/year of raw data ~130 PB/year of processed data?