HAMBURG ZEUTHEN. DESY Tier 2 and NAF. Peter Wegner, Birgit Lewendel for DESY-IT/DV. Tier 2: Status and News NAF: Status, Plans and Questions

Transcription

1 DESY Tier 2 and NAF Peter Wegner, Birgit Lewendel for DESY-IT/DV Tier 2: Status and News NAF: Status, Plans and Questions

2 Basics T2: 1.5 average Tier 2 are requested by CMS-groups for Germany Desy commitment: 1 av. Tier 2 for CMS av. Tier 2 for Atlas av. Tier 2 for LHCb Aachen commitment: 0.5 average Tier 2 Desy s Tier 2 is distributed between Hamburg and Zeuthen Birgit Lewendel 2

3 The Tier 2 is part of larger Grid@DESY Managed via Virtual Organisations (VOs) see: grid.desy.de H1/Zeus Grids Lattice Data Grid LHC-Tier 2 others Ice Cube ILC/Calice Birgit Lewendel 3

4 Grid Infrastructure and T2 ssh UI JDL output VO exclusive RB WMS LFC BDII LFC certs $HOME/.globus/ VOMS RB CE SE CE Batch GRIS GIIS GRIS SE world /etc/grid-security/grid-mapfile WN WN WN WN R-GMA HMS DESY Site2 Site Birgit Lewendel 4

5 Counts, Developments, Limits ~580 CPU cores (~770kSpecInt), 900GB, ~250TB disk All cores on SL4, 32bit (grid-ce3, grid-batch3) Several dcache SE, 1 dedicated for CMS (default SE), 1 for Atlas ~30 boxes, core services - steering/management (RB(4), WMS, VOMS, LFC,...) Grid@DESY/ZN 180 CPU cores (~280kSpecInt) 100 cores on SL3, 32bit, 80 cores on SL4, 64bit ~45TB disk dcache SE Birgit Lewendel 5

6 Counts, Developments, Limits (2) networking - wide area 1 Gb/s connecticity to XWIN officially 10 Gb/s in place (XWIN) 10 Gb/s VPN to Zeuthen pool accounts as deployed by yaim and EGEE (special treatment for CMS software deployment needed?) 2GB/core RAM, 5GB/job disk VO user support by CMS Birgit Lewendel 6

7 DESY T2 Usage 2007 Wallclock time HH+ZN lhcb dteam 6% ops icecube atlas 16% calice 7% dech hone 2 ilc 2% dcms desy dgtest geant4 cms 28% zeus 21% geant 1% hermes cms zeus hone atlas calice dteam ilc geant dgtest ops geant4 lhcb dech icecube dcms desy hermes Birgit Lewendel 7

8 DESY T2 Usage 2007 CPU time HH+ZN calice 7% ilc 3% dteam dgtest geant 1% atlas 15% hone 29% geant4 icecube dech dcms desy lhcb cms 2 hermes ops zeus 25% cms zeus hone atlas calice dteam ilc geant dgtest ops geant4 lhcb dech icecube dcms desy hermes Birgit Lewendel 8

9 DESY T2 Usage 2007 Wallclock/CPU time (h) HH+ZN Wallclock CPU cms zeus hone atlas calice dteam ilc geant dgtest ops geant4 lhcb dech icecube dcms desy hermes ATLAS/CMS: Relative high I/O fraction Birgit Lewendel 9

10 DESY T2 Usage 2007 Total Wallclock/CPU time (days) HERA vs LHC WallClock CPU atlas+cms hone+zeus Birgit Lewendel 10

11 DESY T2 Usage 2007 Wallclock time ZN ilc 2% icecube lhcb ops zeus 8% atlas calice cms dech desy dteam ghep hone 29% atlas 53% geant4 ghep hone geant4 1% dteam desy dech calice 7% cms icecube ilc lhcb ops zeus Birgit Lewendel 11

12 DESY T2 Usage 2007 CPU time ZN ilc 3% lhcb ops zeus 8% atlas icecube calice cms dech desy CMS VO Supported (!) atlas 45% dteam geant4 hone 36% ghep hone icecube ilc geant4 1% ghep dteam desy dech cms calice 7% lhcb ops zeus Birgit Lewendel 12

13 Overview National Analysis Facility (NAF) architecture (simplified) Certificate based login gsissh Batch Access Local Grid?? Interactive Batch (Sun Grid Engine SGE) proof AFS Cluster (parallel) FS dcache SRM Tier 2 grid-access Birgit Lewendel 13

14 Overview NAF (2) Distributed at DESY Zeuthen and Hamburg for users from german institutes from CMS, Atlas, LHCb and ILC. Hardware Computing: interactive server WN -> 2008 proof same blades to keep number flexible 1. order: blades with several hundred cores, 2GB/core, 2*150GB disk/blade, Infiniband Hardware Storage: probably SUN thumpers (17.5TB/box at raid 6) for dcache pools, for afs home and group space, high-performance I/O Back-up: more user input needed Birgit Lewendel 14

15 NAF Worker Node Architecture CPUs today Quad-Core Intel CPUs 5435 ( Clovertown ), if appropriate Dual-Core (5160 Woodcrest ) Per core - 2 GB main memory - 20 GB disk - 20 MB/s bandwidth for I/O applications (PROOF MB/s?) - n(?) MB/s bandwidth for disk I/O LO Management incl. digital KVM, IPMI at the minimum, service processor, interface for automatic upgrades, diagnostics RedHat certified redundant power supply (not clear whether necessary) good performance/watt ratio Service Level: 3-years On-side service NBD Format: Blade Birgit Lewendel 15

16 NAF: Org., Plans, Schedule Def. of work packages, each WP with 3-5 people and a convener OS I/O file system batch montoringinteractive software operation procurement SL4+ shared cluster file system for high-performance access SGE grid tools +qsub common afs cell afs home, group dirs login with proxy WGS env. Root, UI no desktop env. Soon (October?) meeting with users from all groups to review the plans. Pilot system should be ready to use end of November (Helmholtz Alliance kick-off meeting Dec 3 rd -5 th ) Finances: Helmholtz Alliance and DGrid Sonderinvestition Birgit Lewendel 16

17 Summary Grid Infrastructure and T2 in good shape and ready for ramp up heavily in use by several VOs shows good performance in CMS tests (thanks to UniHH T2 monitoring shift team) National Analysis Facility in preparation many open questions, items user feedback meeting soon Birgit Lewendel 17

18 Back-up Birgit Lewendel 18

19 Revised hardware resources plan (C=CTDR, N=New) for a average CMS Tier 2 (Ass.: 25 Tier 2 s) CPU [ksi2k] 300 N 600 N 1000 N 1810 N 400 C 900 C 1400 C 2300 C Disk [TB] 50 N 170 N 340 N 530 N 100 C 200 C 400 C 700 C Tape [TB] (?) Birgit Lewendel 19

20 Revised hardware resources plan ramp up (total for DESY) CPU [ksi2k] Disk [TB] Tape [TB] (?) Birgit Lewendel 20