WLCG and Grid computing at CERN

Transcription

1 WLCG and Grid computing at CERN Andrea Sciabà Visit of the Korean National Health Insurance Service 18 March 2016, CERN, Switzerland

2 What is the LHC The LHC is a particle accelerator 27 km of superconducting magnets Beams of protons or heavy ions colliding head-on at very high energies producing thousands of particles Purpose of the LHC To study fundamental particles and forces at the smallest scale To discover new phenomena: Higgs boson, extra dimensions, dark matter... 2

3 The experiments Matter-antimatter asymmetry General purpose Discovery of new physics: Higgs, Supersymmetry Exploration of a new energy frontier in particle physics State of matter of early universe 3

4 What is the data? 150 million sensors deliver data 40 million times per second Up to 6 GB/s to be permanently stored after filtering 4

5 From collisions to discoveries Accelerator Experiment Data analysis (Grid, Cloud) Discovery Proton bunches collide every 25 ns producing hundreds of TB/s of information Detector electronics and computer farms to keep only the interesting events (few GB/s) Data processed, replicated, stored and analysed at hundreds of processing sites Analysed data used to make measurements and possibly discover new particles and phenomena 5

6 LHC: a computing challenge Data volume 30+ Petabytes of new data/year All data must be stored on tape, much of it on disk Computing power 300,000 cores to process all data Distributed resources and funding CERN can provide only up to 20-30% of the resources Large user base 5000 scientists 500 institutes Tens of countries 6

7 What is a grid? In our case, resources need to be distributed Several technologies used in different computing centres all around the world Many of these centres are used also by other communities The Grid provides a way to integrate these resources so they can be accessed in a uniform way by our users You don t need to know where the resources you are using are located WLCG (Worldwide LHC Computing Grid) is the Grid for LHC By itself a federation of Grids (EGI, OSG, NorduGrid) 7

8 How do Grids work? The middleware is the layer that makes multiple data centres to look like a single system Data management Processing management Security Monitoring The complexity is hidden, but still there Independent sites Different systems Many user communities sharing resources 8

9 Grid security Security is traditionally based on X.509 certificates, with short-lived proxy certificates for any interaction with Grid services Still the case in WLCG Allows fine-grained access control to data Grid-wide, centrally managed by each virtual organisation Strong point for Grids vs. Clouds The downside is it s not user-friendly, and mostly commandline based tools In the past, services were developed for the medical community Cryptographic solutions based on Hydra, a distributed key store, and data encryption Not used in WLCG though 9

10 The WLCG universe An International collaboration to distribute and analyse LHC data Integrates computer centres worldwide that provide computing and storage resource into a single infrastructure accessible by all LHC physicists Tier-0 (CERN): data recording, reconstruction and distribution 14 Tier-1: permanent storage, re-processing, analysis ~150 Tier-2: Simulation, end-user analysis 170+ sites 40 countries ~300,000 cores 500 PB of storage 2 million jobs/day Gb links 10

11 WLCG in Korea KISTI Tier cores 1.5 PB of tape 1.5 PB of disk KNU Tier cores 0.8 PB of disk 11

12 Data distribution and processing Raw and reconstructed data is distributed to the Tier-1 sites for archival and further processing Data in a more compact format (AOD) is produced AOD are distributed to the Tier-2 sites for end-user analysis Simulated events (Monte Carlo) are generated at Tier-2 sites and archived at Tier-1 sites Databases are used to store calibration data needed for event processing 12

13 Impact of WLCG Impact of WLCG WLCG has been leveraged on both sides of the Atlantic, to benefit the wider scientific community Europe: USA: European Grid Infrastructure (EGI) Open Science Grid (OSG) Many scientific applications Archeology Astronomy Astrophysics Civil Protection Comp. Chemistry Earth Sciences Finance Fusion Geophysics High Energy Physics Life Sciences Multimedia Material Sciences 13

14 Extending WLCG to clouds What is cloud computing? Computing resources off-site that can be rented and used transparently How to use cloud resources? Grid sites can add them to their local resources, or Experiments can use them as virtual sites Why to use them? Can be easier to manage Can be provided instantly Leverage on work from other strong communities 14

15 WLCG in ten years Data: ~25 PB/yr 400 PB/yr CMS ATLAS ALICE LHCb Run 1 Run 2 Run 3 Run GRID ATLAS CMS LHCb ALICE Compute: Growth > x Historical growth of 25%/year 20 0 Run 1 Run 2 Run 3 Run 4 15

16 Evolution and challenges Easily integrate other sources of resources Grids, clouds, desktops Publicly funded and from commercial providers Use more efficiently the modern computer architectures Invent more efficient ways to use the resources to close the gap between the expected amount of data to be processed and the resources that will be available 16

17 Conclusions Grid computing is essential for the LHC physics programme Allows all countries involved to contribute resources WLCG is highly successful in delivering rapidly data to physicists Will need to further evolve to meet the expected scale in the future 17