NT1: An example for future EISCAT_3D data centre and archiving?

Transcription

1 March 10, NT1: An example for future EISCAT_3D data centre and archiving? John White NeIC

2 xx March 10, Introduction High Energy Physics and Computing Worldwide LHC Computing Grid Nordic Tier 1 EISCAT_3D Computing Future

3 Large Hadron Collider (LHC) March 10,

4 March 10,

5 March 10,

6 March 10, High Energy Physics (HEP) and Computing At L = cm 2 s beam crossings per second. 20 events per beam crossing events per second. Each general purpose detector 100 M channels. 80PB/s. L1 trigger 75,000 events/s. L2 trigger 2,000 events/s. Event Filter selects 200 interesting events/s. At 1.6MB/event: 320MB/s or 3.2(5.5)PB/y. Overall LHC (ALICE,ATLAS,CMS,LHCb) produces 30 PB/y. Large collaborations O1000 members, worldwide distribution. We turn to the Worldwide LHC Computing Grid (WLCG). Flexible system, accessing and analysing data in near real-time. Seamless access to data storage capacity, processing power. Users make job requests from one of the entry points to system. WLCG does user AuthN/AuthZ, sends job to available site(s). Tier computing model.

7 Worldwide LHC Computing Grid (WLCG) March 10,

8 Each LHC detector events written to Tier-0 (CERN) at MB/s. Raw data replicated (3x) to 12 Tier-1 sites worldwide at 10GB/s. At Tier-1 sites, raw data is re-processed (reconstruction, selections). Event catalogues generated. Tier-2 sites are generally University departments, Institutes. Analysis jobs are submitted back into the WLCG from Tier-2 sites. Brokering algorithm determines CPU availability/data location. Results delivered to storage element. Data replicated, no single point of failure. Access independent of geographical location. Computer centres in multiple time zones: Eases round-the-clock monitoring; Availability of expert support. Resources distributed across the world, for funding and sociological reasons. March 10,

9 WLCG Data Transfers March 10,

10 March 10, Nordic Tier-1 (NT1) Nordic Tier 1. Distributed site. Bergen,Olso,Umeå Umeå,Uppsala,Linköping Lund,Göteborg,Stockholm Copenhagen,Helsinki Project hosted within NeIC. 12 people. 50% average. Since 2010, part of EGI. Serves ALICE, ATLAS and CMS. Compute and Storage resources.

11 NT1 Compute Resources March 10,

12 NT1 Compute Resources Large amounts of computing power available. More HTC rather than HPC. Clusters of many CPUs. Weather forecast/climate simulation cluster (Linköping). Computing jobs are run in batch mode. Jobs enter a cluster through a Computing Element (CE). Dispatched through LRMS to worker nodes (WNs). Large amounts of storage (disk and tape) managed. 12 PB online and nearline storage. Data managed by dcache 1. System for storing and retrieving huge amounts of data. Single virtual filesystem tree. Access: NFSv4.1, https (webdav), ldap/kerberos, native and HEP-specific protocols. 1 March 10,

13 March 10,

14 EISCAT_3D Computing March 10,

15 EISCAT_3D Computing EISCAT_3D RX arrays HEP sub-detectors. EISCAT_3D on-site computing HEP detector trigger levels. Reduces data rate, forms the beams TFLOPS computing at each RX site. EISCAT_3D operations centre WLCG Tier-0: 500TFLOPS computing power. Controls the measurements; Monitors the production of standard data products from RX sites; Generates non-standard and multi-static data products; Transfers data products to the data centre; EISCAT_3D data centre(s) WLCG Tier-1: User access to data, EISCAT_3D archive; Some computing capacity for analysis. EISCAT_3D institutes/users WLCG Tier-2: EISCAT_3D users access data through portal or API.March 10,

16 ATLAS EISCAT_3D Detector to Tier-0 320MB/s (2.6Gb/s) 12Gb/s Tier-1 data/year 5.5 PB est. 2 PB 100 Gbit/s network from RX to operations centre? Location of on-site process computing (n 55 TFLOPS)? Can the operation centre process jobs in batch mode? EISCAT_3D will need resources ON DEMAND Take advantage of virtualization stacks. Automated management of resource usage. e.g. Resource purging/vm migration. Co-locate resources within NT1. HEP processes can backfill resources. Take advantage of cost-sharing and on-site expertise. User portal... continue with Madrigal? Or other? EISCAT_3D user survey(s) required. Requirements gathering. dcache good candidate access to data through portalṁarch 10,

17 March 10, Thank you Questions?

18 March 10,

19 March 10,

20 March 10,

21 Worldwide LHC Computing Grid (WLCG) March 10,