Network issues on FR cloud. Eric Lançon (CEA-Saclay/Irfu)
|
|
|
- Norma Johnson
- 10 years ago
- Views:
Transcription
1 Network issues on FR cloud Eric Lançon (CEA-Saclay/Irfu)
2 Network Usage Data distribution MC production Analysis Distributed storage
3 Network used for Data distribution, 2 components : Analysis Pre-placed data (a la MONARC) Retrieving results Dynamic data distribution (popular data to available sites) MC production Within a given cloud Across clouds Small data sets Distributed storage To optimize resources Simplify data management The old computing model is dying Rencontre LCG-France, SUBATECH Nantes, septembre
4 Experience of two years of LHC running USE THE NETWORK The ATLAS Data Model has changed Moved away from the historical model 4 recurring themes: Flat(ter) hierarchy: Any site can replicate data from any other site Multi Cloud Production Need to replicate output files to remote Tier-1 Dynamic data caching: Analysis sites receive datasets from any other site on demand based on usage pattern Possibly in combination with pre-placement of data sets by centrally managed replication of datasets Remote data access: local jobs accessing data stored at remote sites ATLAS is now heavily relying on multi-domain networks and needs decent e2e network monitoring Introduction/Summary Rencontre LCG-France, SUBATECH Nantes, septembre
5 ATLAS sites and connectivity ATLAS computing model has (will continue to) changed More experience More tools and monitoring New category of sites : Direct T2s (T2Ds) Primary hosts for datasets (analysis) and for group analysis T2D: revising the criteria New criteria - under evaluation All transfers from the candidate T2D to 9/12 T1s for big files ( L ) must be above 5 MB/s during the last week and during 3 out of the 5 last weeks. All transfers from 9/12 T1s to the candidate T2D for big files must be above 5 MB/s during the last week and during 3 out of the 5 last weeks FR-cloud T2Ds : BEIJING, GRIF- LAL, GRIF-LPNHE, IN2P3-CPPM, IN2P3-LAPP, IN2P3-LPC, IN2P3- LPSC Get and send data from different clouds Participate in cross cloud production Rencontre LCG-France, SUBATECH Nantes, septembre 2012
6 Network performance monitoring Networking accounting : Organized (FTS) file transfers : not for direct transfers by users (dq2-get) ATLAS sonar : Calibrated file transfers by ATLAS Data Distribution system, from storage to storage : bourricot.cern.ch/dq2/ftsmon/ > 1 GB file transfers used to monitor and validate T2Ds perfsonar (PS) : Network performance (throughput, latency) : Located as close as possible to storage at site and with similar connection hardware Rencontre LCG-France, SUBATECH Nantes, septembre
7 T0 exports over a year 16,336 TB to T1s Over 1GB/s Better LHC efficiency and higher trigger rate LHC winter stop 1.2 GB/s 2,186 TB to CCIN2P3 150 MB/s 78% to T1s Rencontre LCG-France, SUBATECH Nantes, septembre
8 T1 T1 23,966 TB 1GB/s Popularity based Pre-placement Group data Cross-cloud MC production User requests Rencontre LCG-France, SUBATECH Nantes, septembre
9 T1 T2 T2 T1 54,491 TB 16,487 TB 2 GB/s 1 GB/s Reconstruction in T2s User subscriptions! Dynamic data placement > pre-defined Not in original computing model Rencontre LCG-France, SUBATECH Nantes, septembre
10 T2 T2 Dynamic data placement > pre-defined 9,050 TB 0.5 GB/s Network mesh tests User subscriptions Group data at some T2s + Outputs of analysis Rencontre LCG-France, SUBATECH Nantes, septembre
11 Data volume: T1s 60% of sources ALL together 131,473 TB 5 GB/s Activity: T2s 40% of sources Rencontre LCG-France, SUBATECH Nantes, septembre
12 ALL together Data volume: pre-placement ~2 times dynamic placement room for improvements Activity: users ~30% of transfers Rencontre LCG-France, SUBATECH Nantes, septembre
13 Cross-cloud MC production The easy part No need to be a T2D Only connection to remote T1 needed Example : NL cloud 65! sites contributing Rencontre LCG-France, SUBATECH Nantes, septembre 2012
14 The French cloud The most exploded cloud of ATLAS 4 Romanians sites at the far end of GEANT 2 sites in far east Beijing & Tokyo connected to CCIN2P3 via different paths
15 T1 : Lyon The French cloud T2s : 14 sites Annecy Clermont Grenoble Grif (3 sites) Lyon Marseille Beijing Romania x4 Tokyo Rencontre LCG-France, SUBATECH Nantes, septembre
16 Imports : 9,9 PB Data exchanges for FR-cloud CA CERN DE ES IT ND NL TW UK US French cloud exchanges 0 1,000 2,000 3,000 IMPORT [TB] EXPORT [TB] [Sep Sep ] Rencontre LCG-France, SUBATECH Nantes, septembre % of exchanges with EU clouds CATW ND 4% 3% ES5% 5% IT 8% NL 8% UK 10% DE 13% NL US 6% 6% CA 6% ES 8% IT 10% CERN 5% ND 2% TW 19% CERN 27% US 17% Exports : 7,5 PB DE 19% UK 19% cross-cloud production 16
![- 2012] Rencontre LCG-France, SUBATECH Nantes, septembre 2012 70-75% of exchanges with EU clouds CATW ND 4%](/docs-images/42/2256846/images/page_16.jpg "3% ES5% 5% IT 8% NL 8% UK 10% DE 13% NL US 6% 6% CA 6% ES 8% IT 10% CERN 5% ND 2% TW 19% CERN 27% US 17%")
17 Imports 400 MB/s Popularity based Pre-placement T0 export User requests Exports 300 MB/s Group data Cross-cloud MC production Rencontre LCG-France, SUBATECH Nantes, septembre
18 Data volume transferred to French T2s not proportional to number physicist nor CPUs Rencontre LCG-France, SUBATECH Nantes, septembre
19 LAPP & LPSC performance measurements sensitive to ATLAS activity Connectivity within French cloud (ATLAS sonar) T2s T1 T1 T2s 5 MB/s Beijing Rencontre LCG-France, SUBATECH Nantes, septembre
20 Origin of data transferred to 2 GRIF sites LAL : T2D connected to LHCONE From everywhere IRFU : T2D connected to LHCONE 80% from CCIN2P3 Rencontre LCG-France, SUBATECH Nantes, septembre
21 User & group analysis Most of users run final analysis at their local site : delivery of data or analysis job outputs to users very sensitive (the last transferred file determines the efficiency) 2 ways to get (reduced format) data The majority : Let PanDA decide where to run the jobs ; where data are (in most of the cases). Outputs stored on SCRATCHDISK (buffer area) at remote sites have to be shipped back to user local site Get limited volume of data at local site to run locally analysis Both imply data transfers from remote site to local site Direct transfers for T2Ds Through T1 for other T2s Rencontre LCG-France, SUBATECH Nantes, septembre
22 User + Group transfers to FR-cloud sites ⅓ of data transfers 1,828 TB (not data volume) used for final analysis from T2 sites outside FRcloud Not allowed in original model Rencontre LCG-France, SUBATECH Nantes, septembre
23 Data come from 52 T2 sites Rencontre LCG-France, SUBATECH Nantes, septembre
24 ⅓ from non EU T2s 1/4 from UK (not on LHCONE) Rencontre LCG-France, SUBATECH Nantes, septembre
25 Issues with distant T2s
26 Beijing Beijing : Connected to Europe via GEANT/TEIN3 (except CERN : GLORIAD/KREONET) RTT ~190 ms Tokyo : Connected to Europe via GEANT/MANLAN/SINET4 RTT ~ 300 ms Several network operators on the path (Nationals, GEANT, ) Rencontre LCG-France, SUBATECH Nantes, septembre
27 The difficulty to solve network issues Sites Network SINET 10Gbps providers (National & International) Network Switch 10GE File Server Projects (experiments) Disk Array 8G- FC File System 8G- FC File System Disk Array x15 to x17 File System 8G- FC File System Various approaches Backbone topology as at and March GÉANT is complementary operated by DANTE on behalf of Europe s NRENs. tools needed Rencontre LCG-France, SUBATECH Nantes, septembre
28 perfsonar dashboard of FR-cloud Being expanded as sites install perfsonar Rencontre LCG-France, SUBATECH Nantes, septembre
29 ATLAS transfers to Beijing since beg Beijing CCIN2P3 CCIN2P3 Beijing Performances changed over last year Asymmetry in transfer rate : why? Asymmetry reversed Each event explained sometime after some delay... Rencontre LCG-France, SUBATECH Nantes, septembre
30 ATLAS transfers to Tokyo since beg Tokyo CCIN2P3 CCIN2P3 Tokyo Performances changed over last year Asymmetry in transfer rate : why? Asymmetry reversed Each event explained sometime after some delay... Rencontre LCG-France, SUBATECH Nantes, septembre
31 Beijing from/to EU T1s Each T1 is different Beijing EU better for most T1s Rencontre LCG-France, SUBATECH Nantes, septembre
32 Tokyo from/to EU T1s Each T1 is different EU Tokyo better for most T1s Rencontre LCG-France, SUBATECH Nantes, septembre
33 Beijing - T1s asymmetry T1->Beijing Beijing->T1 Throuhput [Mb/s] perfsonar (July) 0 IN2P3 KIT PIC CNAF RAL ND TRIUMF-LCG2 INFN-T1 TAIWAN-LCG2 FZK-LCG2 PIC BNL-OSG2 IN2P3-CC NDGF-T1 NIKHEF-ELPROD SARA-MATRIX CERN-PROD RAL-LCG2 FTSmon (last 5 weeks) asymetry [MB/s] Beijing -> T1s > T1s -> Beijing 33
34 Tokyo EU as seen by perfsonar Site -> Tokyo Tokyo -> Site Last 3 months (MB/s) perfsonar CC-IN2P3 CNAF DESY LAL RO SARA Rencontre LCG-France, SUBATECH Nantes, septembre
35 US (LHCONE) GRIF (LHCONE) US LAL > US LPNHE BNL LPNHE US > LAL US SLAC Rencontre LCG-France, SUBATECH Nantes, septembre
36 DISTRIBUTED STORAGE / REMOTE ACCESS Better used of storage resources (disk prices!) Simplification of data management Eventually remote access (with caching at both ends); direct reading or file copy Bandwidth and stability needed On going projects Storage : dcache, dpm,... Protocol : Xrootd, HTTP/WebDAV
37 Existing distributed dcache systems : 2 examples MWT2 (Chicago) NORDUGRID Distributed dcache OPN EMI INFSO-RI Rencontre LCG-France, SUBATECH Nantes, septembre
38 Xrootd federation project in US REMOTE ACCESS R&D Activity to Production 2011 R&D project FAX (Federating ATLAS data stores using Xrootd) was deployed over US Tier 1, Tier 2s and some Tier3s Feasibility testing monitoring, site integrations In June 2012 extended effort to European sites as an ATLAS-wide project 2 BNL Tier 1 AGLT2 (Tier 2) MWT2 (Tier 2) SWT2 (Tier 2) SLAC (Tier 2) ANL (Tier 3) BNL (Tier 3) Chicago (Tier 3) Duke(Tier 3) OU (Tier 3) SLAC (Tier 3) UTA (Tier 3) NET (Tier 2) EU federation tests Four levels of redirection: site-cloud-zone-global Start locally - expand search as needed 15 Rencontre LCG-France, SUBATECH Nantes, septembre
39 Topology validation REMOTE ACCESS Launch jobs to every site, test reading of site-specific files at every other site Parse client logs to infer resulting redirection US regional result from Ilija Vukotic UK regional DE regional US-central regional XRD redirector federating site EU regional 16 WAN Read Tests (basis for cost matrix ) CERN to MWT2 (140 ms rtt) MWT2 federation internal (<5 ms) SLAC to MWT2 (50 ms) Spread grows with network latency Time to read file Overall WAN performance adequate for a class of use cases 17 from Ilija Vukotic Rencontre LCG-France, SUBATECH Nantes, septembre
40 REMOTE ACCESS HTTP/WebDAV Storage Federations using standard web protocols Project with CERN DM under the umbrella of EMI but not limited to the EMI funding period. Definition of TEG: Collection of disparate storage resources managed by co-operating but independent administrative domains transparently accessible via a common name space We do it with standard HTTP/WebDAV Atlas WS 2012, CERN dcache.org 10 Sep Use http urls for input files DDM enabled web redirection service generic url including dataset and lfn redirects to http turl in dcache/dpm storage Tfile::Open( mc11_7tev j6_pythia_jetjet.merge.ntup_jetmet.e815_s1310_s1300_r3334_r3063_p886/ NTUP_JETMET _ root.1 ) atlas-data.cern.ch Webservice: DDM lookup, choose best replica redirect to http turl ETMET _ root.1 Redirection works only in recent Root versions Https not yet supported redirect DESY dcache Webdav door Disk pool 2 Rencontre LCG-France, SUBATECH Nantes, septembre
41 Summary Thanks to the high performances of networks ATLAS computing model has changed significantly: simplification of data and workflow management Would have been impossible to handle current data volume (LHC performing beyond expectations) and LHC running extension up to spring 2013 with initial model More efficient use of storage resources (reduce replica counts; direct sharing of replicas across sites) Ongoing projects (distributed storage, remote access) will further change the landscape Rencontre LCG-France, SUBATECH Nantes, septembre
42 BACKUP
43 ATLAS Computing Model: T0 Data Flow 200Hz RAW: ~1.6MB/evt Event Summary Data (ESD): ~1 MB/evt Analysis Object Data (AOD): ~100 kb/evt derived data (desd, daod, NTUP,...) distributed over the Grid Calibration CERN Analysis Facility Tier-1 Tier-0 Data Recording to tape First Pass Processing French cloud Tier-1 10 Tier-1 centers RAW data copy on tape Analysis data on disk Reprocessing 38 Tier-2 centers Analysis data on disk User Analysis Tier-1 Tier-2 Tier-2 Tier-2 Tier-2 Tier-2 Tier-2 Tier-2 Static environment, fear of the networks! Tier-2 Tier-2 Tiered site/cloud architecture ATLAS Operations - Ueda I. (ISGC2012, Taipei, ) Pre-planned data distribution Jobs-to-data brokerage 43
44 ATLAS Computing Model: MC Data Flow CERN Analysis Facility Tier-1 Tier-2 Tier-2 Tier-2 10 Tier-1 centers + CERN Aggregation of produced data Source for further distribution t job u inp Tier-1 Tier-2 Tier-2 Tier-2 38 Tier-2 centers MC Simulation Group/User Analysis ATLAS Operations - Ueda I. (ISGC2012, Taipei, ) Tier-1 Tier-2 Tier-2 Tier-2 44
45 Data Processing Model Revised CERN Analysis Facility Tier-1 10 Tier-1 centers + CERN Aggregation of produced data Source for further distribution t job u inp Tier-1 Tier-2 Tier-2 Tier-2-D Tier-2 Tier-2 Tier-2-D Tier-1 Tier-2 Tier-2 Tier-2 Tier-2-Direct : Super Tier-2s ATLAS Operations - Ueda I. (ISGC2012, Taipei, ) 45
46 T1s -> IN2P3-CPPM June 25th connected to 10 Gb/s 5 MB/s TRIUMF 46
47 IN2P3-CPPM -> T1s 5 MB/s TW TRIUMF 47
48 TRIUMF <-> FR T2Ds ~None of T2Ds ever reaches the 5 MB/s canonical parameter value 5 MB/s 48
49 T2-T2 destination T2-T2 source Rencontre LCG-France, SUBATECH Nantes, septembre
50 Network throughput measured with perfsonar CCIN2P3 Tokyo Tokyo CCIN2P3 5% No so stable Last 3 months better by ~5% for CCIN2P3 Tokyo Rencontre LCG-France, SUBATECH Nantes, septembre
51 CCIN2P3 Exports To Lyon Rencontre LCG-France, SUBATECH Nantes, septembre
52 T2s Exports T2s Import Rencontre LCG-France, SUBATECH Nantes, septembre
53 destination on FR cloud Rencontre LCG-France, SUBATECH Nantes, septembre
54 trans-siberian route GEANT/TEIN3 At the Heart of Global Research Networking GÉANT2 GÉANT2 is an advanced pan-europ backbone network that interconnec Research and Education Networks across Europe. With an estimated 3 research and education users in 34 across the continent connected via GÉANT2 offers unrivalled geograph coverage, high bandwidth, innovat networking technology and a range user-focused services, making it th advanced international network in Together with the NRENs it connec has links totalling more than 50,00 length and its extensive geographic interconnects networks in other wo to enable global research collabora Europe s academics and researcher exploit dedicated GÉANT2 point-to links, creating optical private netw connect specific research centres. GÉANT Coverage ALICE2-RedCLARA Network EUMEDCONNECT2 Network TEIN3 Network BSI Network AfricaConnect - UbuntuNet Alliance AfricaConnect - WACREN CAREN Network CERNET China Education and Research Net (CERNET), the nationwide academic is the largest non-profit network in funded by the Chinese Government by the Ministry of Education (MOE constructed and operated by Tsingh University. Currently, CERNET has 10 regional 38 provincial nodes, and the nation is located at Tsinghua University. M 2,000 research and education insti connect to CERNET reaching an est 20 million end users. GÉANT and sister networks enabling user collaboration across the globe April th workshop of the France China Particle Physics Laboratory CERNET was established in 1994 an first backbone research and educat in China. In 2007, the bandwidth o CERNET backbone was between Gbps, with regional bandwidth c of 155 Mbps and 2.5 Gbps reaching 200+ cities distributed in 31 provin China. The bandwidth interconnect with other global networks reaches of 15 Gbps. As an important national education research infrastructure, CERNET su many key national network applica including on-line enrollment for un students, distance learning, digital GRID for education and research an 54 an outstanding contribution to edu information for China.
55 Tokyo is far from CCIN2P3 : ~300 ms RTT (Round Trip Time) Throughput ~ 1 / RTT Data are transferred from site to site through a lot of networks (multi-hop) and software layers ideal view The reality Rencontre LCG-France, SUBATECH Nantes, septembre
56 Rencontre LCG-France, SUBATECH Nantes, septembre
57 Rencontre LCG-France, SUBATECH Nantes, septembre
58 Backbone topology as at March GÉANT is operated by DANTE on behalf of Europe s NRENs. Rencontre LCG-France, SUBATECH Nantes, septembre
59 Rencontre LCG-France, SUBATECH Nantes, septembre
60 Rencontre LCG-France, SUBATECH Nantes, septembre
61 Disk Array SINET 10Gbps Network Switch 10GE File Server 8G- FC 8G- FC File System File System Disk Array x15 to x17 8G- FC File System File System Rencontre LCG-France, SUBATECH Nantes, septembre
62 T1s -> IN2P3-LPSC Heavy transfers from IN2P3-CC (T1) interference with FTS monitoring IN2P3-CC -> IN2P3-LPSC 62
63 Rencontre LCG-France, SUBATECH Nantes, septembre
64 IN2P3-CC Beijing as seen by perfsonar Beijing -> IN2P3-CC IN2P3-CC -> Beijing Link unstable Asymmetry 64
65 IN2P3-CC Tokyo as seen by perfsonar IN2P3-CC-> Tokyo Tokyo -> IN2P3-CC 65
Network monitoring with perfsonar. Duncan Rand Imperial College London
Network monitoring with perfsonar Duncan Rand Imperial College London A little history: Gridmon Cast your minds back to GridPP16 at QMUL about 6 years ago; we saw a not too dissimilar network monitoring
(Possible) HEP Use Case for NDN. Phil DeMar; Wenji Wu NDNComm (UCLA) Sept. 28, 2015
(Possible) HEP Use Case for NDN Phil DeMar; Wenji Wu NDNComm (UCLA) Sept. 28, 2015 Outline LHC Experiments LHC Computing Models CMS Data Federation & AAA Evolving Computing Models & NDN Summary Phil DeMar:
Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft. Holger Marten. Holger. Marten at iwr. fzk. de www.gridka.de
Tier-2 cloud Holger Marten Holger. Marten at iwr. fzk. de www.gridka.de 1 GridKa associated Tier-2 sites spread over 3 EGEE regions. (4 LHC Experiments, 5 (soon: 6) countries, >20 T2 sites) 2 region DECH
Grid Computing in Aachen
GEFÖRDERT VOM Grid Computing in Aachen III. Physikalisches Institut B Berichtswoche des Graduiertenkollegs Bad Honnef, 05.09.2008 Concept of Grid Computing Computing Grid; like the power grid, but for
The dcache Storage Element
16. Juni 2008 Hamburg The dcache Storage Element and it's role in the LHC era for the dcache team Topics for today Storage elements (SEs) in the grid Introduction to the dcache SE Usage of dcache in LCG
Status and Evolution of ATLAS Workload Management System PanDA
Status and Evolution of ATLAS Workload Management System PanDA Univ. of Texas at Arlington GRID 2012, Dubna Outline Overview PanDA design PanDA performance Recent Improvements Future Plans Why PanDA The
The CMS analysis chain in a distributed environment
The CMS analysis chain in a distributed environment on behalf of the CMS collaboration DESY, Zeuthen,, Germany 22 nd 27 th May, 2005 1 The CMS experiment 2 The CMS Computing Model (1) The CMS collaboration
Managed Storage @ GRID or why NFSv4.1 is not enough. Tigran Mkrtchyan for dcache Team
Managed Storage @ GRID or why NFSv4.1 is not enough Tigran Mkrtchyan for dcache Team What the hell do physicists do? Physicist are hackers they just want to know how things works. In moder physics given
Tier3 Network Issues. Richard Carlson May 19, 2009 [email protected]
Tier3 Network Issues Richard Carlson May 19, 2009 [email protected] Internet2 overview Member organization with a national backbone infrastructure Campus & Regional network members National and International
CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT
SS Data & Storage CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT HEPiX Fall 2012 Workshop October 15-19, 2012 Institute of High Energy Physics, Beijing, China SS Outline
LHC schedule: what does it imply for SRM deployment? [email protected]. CERN, July 2007
WLCG Service Schedule LHC schedule: what does it imply for SRM deployment? [email protected] WLCG Storage Workshop CERN, July 2007 Agenda The machine The experiments The service LHC Schedule Mar. Apr.
High Availability Databases based on Oracle 10g RAC on Linux
High Availability Databases based on Oracle 10g RAC on Linux WLCG Tier2 Tutorials, CERN, June 2006 Luca Canali, CERN IT Outline Goals Architecture of an HA DB Service Deployment at the CERN Physics Database
Deploying distributed network monitoring mesh
Deploying distributed network monitoring mesh for LHC Tier-1 and Tier-2 sites Phil DeMar, Maxim Grigoriev Fermilab Joe Metzger, Brian Tierney ESnet Martin Swany University of Delaware Jeff Boote, Eric
The GÉANT Network & GN3
The GÉANT Network & GN3 Tom Fryer, DANTE CLARA-TEC San José, Costa Rica Tuesday, 11 th August 2009 What is GÉANT? The pan-european research and education backbone network A high-capacity internet reserved
perfsonar MDM updates for LHCONE: VRF monitoring, updated web UI, VM images
perfsonar MDM updates for LHCONE: VRF monitoring, updated web UI, VM images Domenico Vicinanza DANTE, Cambridge, UK perfsonar MDM Product Manager [email protected] LHCONE Meeting Oslo 20-21
Report from SARA/NIKHEF T1 and associated T2s
Report from SARA/NIKHEF T1 and associated T2s Ron Trompert SARA About SARA and NIKHEF NIKHEF SARA High Energy Physics Institute High performance computing centre Manages the Surfnet 6 network for the Dutch
Tier0 plans and security and backup policy proposals
Tier0 plans and security and backup policy proposals, CERN IT-PSS CERN - IT Outline Service operational aspects Hardware set-up in 2007 Replication set-up Test plan Backup and security policies CERN Oracle
Agenda. NRENs, GARR and GEANT in a nutshell SDN Activities Conclusion. Mauro Campanella Internet Festival, Pisa 9 Oct 2015 2
Agenda NRENs, GARR and GEANT in a nutshell SDN Activities Conclusion 2 3 The Campus-NREN-GÉANT ecosystem CAMPUS networks NRENs GÉANT backbone. GÉANT Optical + switching platforms Multi-Domain environment
perfsonar: End-to-End Network Performance Verification
perfsonar: End-to-End Network Performance Verification Toby Wong Sr. Network Analyst, BCNET Ian Gable Technical Manager, Canada Overview 1. IntroducGons 2. Problem Statement/Example Scenario 3. Why perfsonar?
LHCONE Site Connections
LHCONE Site Connections Michael O Connor [email protected] ESnet Network Engineering Asia Tier Center Forum on Networking Daejeon, South Korea September 23, 2015 Outline Introduction ESnet LHCONE Traffic Volumes
Data storage services at CC-IN2P3
Centre de Calcul de l Institut National de Physique Nucléaire et de Physique des Particules Data storage services at CC-IN2P3 Jean-Yves Nief Agenda Hardware: Storage on disk. Storage on tape. Software:
LHCOPN and LHCONE an introduction
LHCOPN and LHCONE an introduction APAN workshop Nantou, 13 th August 2014 [email protected] CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it 1 Summary - WLCG - LHCOPN - LHCONE - L3VPN
Computing at the HL-LHC
Computing at the HL-LHC Predrag Buncic on behalf of the Trigger/DAQ/Offline/Computing Preparatory Group ALICE: Pierre Vande Vyvre, Thorsten Kollegger, Predrag Buncic; ATLAS: David Rousseau, Benedetto Gorini,
Trial of the Infinera PXM. Guy Roberts, Mian Usman
Trial of the Infinera PXM Guy Roberts, Mian Usman LHC Workshop Recap Rather than maintaining distinct networks, the LHC community should aim to unify its network infrastructure Traffic aggregation on few
Software Defined Networking for big-data science
Software Defined Networking for big-data science Eric Pouyoul Chin Guok Inder Monga (presenting) TERENA Network Architects meeting, Copenhagen November 21 st, 2012 ESnet: World s Leading Science Network
Campus Network Design Science DMZ
Campus Network Design Science DMZ Dale Smith Network Startup Resource Center [email protected] The information in this document comes largely from work done by ESnet, the USA Energy Sciences Network see
Global Server Load Balancing
White Paper Overview Many enterprises attempt to scale Web and network capacity by deploying additional servers and increased infrastructure at a single location, but centralized architectures are subject
GÉANT Open Service Description. High Performance Interconnectivity to Support Advanced Research
GÉANT Open Service Description High Performance Interconnectivity to Support Advanced Research Issue Date: 20 July 2015 GÉANT Open Exchange Overview Facilitating collaboration has always been the cornerstone
Software Defined Networking for big-data science
Software Defined Networking for big-data science Eric Pouyoul Chin Guok Inder Monga (presenting) SRS presentation November 15 th, Supercomputing 2012 Acknowledgements Many folks at ESnet who helped with
Disaster-Resilient Backbone and Access Networks
The Workshop on Establishing Resilient Life-Space in the Cyber-Physical Integrated Society, March. 17, 2015, Sendai, Japan Disaster-Resilient Backbone and Access Networks Shigeki Yamada ([email protected])
An Evaluation of Peering and Traffic Engineering in the Pan- African Research and Education Network
An Evaluation of Peering and Traffic Engineering in the Pan- African Research and Education Network *Clifford Sibanda (National University of Science and Technology- Bulawayo) *Josiah Chavula (Mzuzu University
Network & HEP Computing in China. Gongxing SUN CJK Workshop & CFI
Network & HEP Computing in China Gongxing SUN CJK Workshop & CFI Outlines IPV6 deployment SDN for HEP data transfer Dirac Computing Model on IPV6 Volunteer Computing Future Work IPv6@IHEP-Deployment Internet
Maurice Askinazi Ofer Rind Tony Wong. HEPIX @ Cornell Nov. 2, 2010 Storage at BNL
Maurice Askinazi Ofer Rind Tony Wong HEPIX @ Cornell Nov. 2, 2010 Storage at BNL Traditional Storage Dedicated compute nodes and NFS SAN storage Simple and effective, but SAN storage became very expensive
Operating Systems and Networks Sample Solution 1
Spring Term 2014 Operating Systems and Networks Sample Solution 1 1 byte = 8 bits 1 kilobyte = 1024 bytes 10 3 bytes 1 Network Performance 1.1 Delays Given a 1Gbps point to point copper wire (propagation
Steven Newhouse, Head of Technical Services
Challenges at EMBL-EBI Steven Newhouse, Head of Technical Services European Bioinformatics Institute Outstation of the European Molecular Biology Laboratory International organisation created by treaty
SURFsara Data Services
SURFsara Data Services SUPPORTING DATA-INTENSIVE SCIENCES Mark van de Sanden The world of the many Many different users (well organised (international) user communities, research groups, universities,
Service Challenge Tests of the LCG Grid
Service Challenge Tests of the LCG Grid Andrzej Olszewski Institute of Nuclear Physics PAN Kraków, Poland Cracow 05 Grid Workshop 22 nd Nov 2005 The materials used in this presentation come from many sources
DSS. High performance storage pools for LHC. Data & Storage Services. Łukasz Janyst. on behalf of the CERN IT-DSS group
DSS High performance storage pools for LHC Łukasz Janyst on behalf of the CERN IT-DSS group CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it Introduction The goal of EOS is to provide a
ATLAS GridKa T1/T2 Status
ATLAS GridKa T1/T2 Status GridKa TAB, FZK, 19 Oct 2007 München GridKa T1/T2 status Production and data management operations Computing team & cloud organization T1/T2 meeting summary Site monitoring/gangarobot
Deploying a distributed data storage system on the UK National Grid Service using federated SRB
Deploying a distributed data storage system on the UK National Grid Service using federated SRB Manandhar A.S., Kleese K., Berrisford P., Brown G.D. CCLRC e-science Center Abstract As Grid enabled applications
Running SAP Solutions in the Cloud How to Handle Sizing and Performance Challenges. William Adams SAP AG
Running SAP Solutions in the Cloud How to Handle Sizing and Performance Challenges William Adams SAP AG Agenda What Types of Cloud Environments we are talking about Private Public Critical Performance
Patrick Fuhrmann. The DESY Storage Cloud
The DESY Storage Cloud Patrick Fuhrmann The DESY Storage Cloud Hamburg, 2/3/2015 for the DESY CLOUD TEAM Content > Motivation > Preparation > Collaborations and publications > What do you get right now?
Designing HP SAN Networking Solutions
Exam : HP0-J65 Title : Designing HP SAN Networking Solutions Version : Demo 1 / 6 1.To install additional switches, you must determine the ideal ISL ratio. Which ISL ratio range is recommended for less
GridKa: Roles and Status
GridKa: Roles and Status GmbH Institute for Scientific Computing P.O. Box 3640 D-76021 Karlsruhe, Germany Holger Marten http://www.gridka.de History 10/2000: First ideas about a German Regional Centre
Scientific Storage at FNAL. Gerard Bernabeu Altayo Dmitry Litvintsev Gene Oleynik 14/10/2015
Scientific Storage at FNAL Gerard Bernabeu Altayo Dmitry Litvintsev Gene Oleynik 14/10/2015 Index - Storage use cases - Bluearc - Lustre - EOS - dcache disk only - dcache+enstore Data distribution by solution
Load Balancing in Distributed Web Server Systems With Partial Document Replication
Load Balancing in Distributed Web Server Systems With Partial Document Replication Ling Zhuo, Cho-Li Wang and Francis C. M. Lau Department of Computer Science and Information Systems The University of
Network performance monitoring Insight into perfsonar
Network performance monitoring Insight into perfsonar Szymon Trocha, Poznań Supercomputing and Networking Center E-infrastructure Autumn Workshops, Chisinau, Moldova 9 September 2014 Agenda! Network performance
Testing ARES on the GTS framework: lesson learned and open issues. Mauro Femminella University of Perugia mauro.femminella@unipg.
Testing ARES on the GTS framework: lesson learned and open issues Mauro Femminella University of Perugia [email protected] Outline What is ARES What testing on GTS? Our solution Performance evaluation
Global Server Load Balancing
White Paper Global Server Load Balancing APV Series Application Delivery Controllers May 2011 Global Server Load Balancing Access. Security. Delivery. Introduction Scalability, high availability and performance
Scalable Internet Services and Load Balancing
Scalable Services and Load Balancing Kai Shen Services brings ubiquitous connection based applications/services accessible to online users through Applications can be designed and launched quickly and
ATLAS Petascale Data Processing on the Grid: Facilitating Physics Discoveries at the LHC
ATLAS Petascale Data Processing on the Grid: Facilitating Physics Discoveries at the LHC Wensheng Deng 1, Alexei Klimentov 1, Pavel Nevski 1, Jonas Strandberg 2, Junji Tojo 3, Alexandre Vaniachine 4, Rodney
The Compact Muon Solenoid Experiment. Conference Report. Mailing address: CMS CERN, CH-1211 GENEVA 23, Switzerland
Available on CMS information server CMS CR -2012/114 The Compact Muon Solenoid Experiment Conference Report Mailing address: CMS CERN, CH-1211 GENEVA 23, Switzerland 23 May 2012 CMS Data Transfer operations
Big data management with IBM General Parallel File System
Big data management with IBM General Parallel File System Optimize storage management and boost your return on investment Highlights Handles the explosive growth of structured and unstructured data Offers
Software-defined Storage Architecture for Analytics Computing
Software-defined Storage Architecture for Analytics Computing Arati Joshi Performance Engineering Colin Eldridge File System Engineering Carlos Carrero Product Management June 2015 Reference Architecture
ANI Network Testbed Update
ANI Network Testbed Update Brian Tierney, ESnet, Joint Techs, Columbus OH, July, 2010 ANI: Advanced Network Initiative Project Start Date: September, 2009 Funded by ARRA for 3 years Designed, built, and
2. Research and Development on the Autonomic Operation. Control Infrastructure Technologies in the Cloud Computing Environment
R&D supporting future cloud computing infrastructure technologies Research and Development on Autonomic Operation Control Infrastructure Technologies in the Cloud Computing Environment DEMPO Hiroshi, KAMI
Globule: a Platform for Self-Replicating Web Documents
Globule: a Platform for Self-Replicating Web Documents Guillaume Pierre Maarten van Steen Vrije Universiteit, Amsterdam Internal report IR-483 January 2001 Abstract Replicating Web documents at a worldwide
Scala Storage Scale-Out Clustered Storage White Paper
White Paper Scala Storage Scale-Out Clustered Storage White Paper Chapter 1 Introduction... 3 Capacity - Explosive Growth of Unstructured Data... 3 Performance - Cluster Computing... 3 Chapter 2 Current
irods at CC-IN2P3: managing petabytes of data
Centre de Calcul de l Institut National de Physique Nucléaire et de Physique des Particules irods at CC-IN2P3: managing petabytes of data Jean-Yves Nief Pascal Calvat Yonny Cardenas Quentin Le Boulc h
Designing a Cloud Storage System
Designing a Cloud Storage System End to End Cloud Storage When designing a cloud storage system, there is value in decoupling the system s archival capacity (its ability to persistently store large volumes
Riverbed WAN Acceleration for EMC Isilon Sync IQ Replication
PERFORMANCE BRIEF 1 Riverbed WAN Acceleration for EMC Isilon Sync IQ Replication Introduction EMC Isilon Scale-Out NAS storage solutions enable the consolidation of disparate pools of storage into a single
CMS Tier-3 cluster at NISER. Dr. Tania Moulik
CMS Tier-3 cluster at NISER Dr. Tania Moulik What and why? Grid computing is a term referring to the combination of computer resources from multiple administrative domains to reach common goal. Grids tend
A Physics Approach to Big Data. Adam Kocoloski, PhD CTO Cloudant
A Physics Approach to Big Data Adam Kocoloski, PhD CTO Cloudant 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Solenoidal Tracker at RHIC (STAR) The life of LHC data Detected by experiment Online
Comparison of the Frontier Distributed Database Caching System with NoSQL Databases
Comparison of the Frontier Distributed Database Caching System with NoSQL Databases Dave Dykstra [email protected] Fermilab is operated by the Fermi Research Alliance, LLC under contract No. DE-AC02-07CH11359