DCMS Tier 2/3 prototype infrastructure 1 Anja Vest, Uni Karlsruhe DCMS Meeting in Aachen, Overview LCG Queues/mapping set up Hardware capacities Supported software Summary
DCMS overview CMS DCMS: Tier 1 GridKa Collaboration of the German institutes in AC, HH and KA participating in the CMS experiment LCG VO 'dcms' 'federated' Tier 2 DESY / Aachen Aims: more independence exchange of experience in analysis (on the grid) Tier 3 Aachen Tier 3 Hamburg Tier 3 Karlsruhe sharing of recources prioritisation of dcmsusers at the DCMS sites DCMS 2
DCMS communication 3 RWTH Aachen DESY / Uni Hamburg EKP Karlsruhe DCMS has about 90 members Two mailing lists: cms-germany-grid@cern.ch: Grid related topics (41 members) cms-germany-prs@cern.ch: Physics related topics (67 members) For subscription, go to http://listboxservices.web.cern.ch/listboxservices (or mail to anja.vest@cern.ch) VRVS Video Meeting every third Wednesday a month Internet portals: DCMS Wiki: https://www-flc.desy.de/dcms (B. Hegner, C. Rosemann) Topics: ORCA, data, grid, MC-production, meetings... VisaHEP: http://visahep.home.cern.ch/visahep/ (S. Kappler) Seminars, organisation, documentation, meetings...
DCMS VO's/LCG admins 4 DCMS sites support at least 4 VOs: cms, dcms (CMS collaboration) dteam (LCG deployment team), dech (EGEE federation DE/CH) participation in GridKa School '05 with testbed (V. Büge, C. Jung, A. Vest) VO dcms hosted at DESY (no software manager 'dcmssgm') DESY runs a RB/BDII/PXY for dcms DESY provides the VO server and the RLS; VOMS LCG-site administration: Aachen: A. Nowack, (M. Kirsch) Hamburg: A. Gellrich, M. de Riese Karlsruhe: V. Büge, C. Jung, A. Vest Up to now: one VRVS LCG-Admin meeting
DCMS LCG clusters 5 All sites are running LCG 2.6.0 (since ~August) and SLC 3.0.5 on the LCG components (UI, CE, WN, SE etc.) The Worker Nodes are controlled by PBS/TORQUE batch systems using Maui/Torque At Karlsruhe: local Linux cluster successfully integrated into LCG, installed with yaim (EKP-Note in preparation) Developing homepage http://www-ekp.physik.uni-karlsruhe.de/~lcgadmin At Aachen: LCG cluster installed with Quattor Cluster monitoring with Lemon A. Nowack At Hamburg: separate LCG cluster installed with Quattor/yaim
DCMS LCG monitoring 6 Grid status of German LCG sites: http://goc.grid.sinica.edu.tw/gstat/germanyswitzerland.html DESY-HH RWTH-Aachen ekplcg2 LCG Monitoring: http://goc03.grid-support.ac.uk/googlemaps/lcg2.html
DCMS LCG Monitoring 7
DCMS LCG Monitoring 8
DCMS LCG Monitoring 9
DCMS LCG Monitoring 10
DCMS LCG queues 11 ekp-lcg-ui:> edg-job-list-match -vo dcms testjob.jdl Selected Virtual Organisation name (from --vo option): dcms Connecting to host grid-rb.desy.de, port 7772 **************************************************************** COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* ekp-lcg-ce.physik.uni-karlsruhe.de:2119/jobmanager-lcgpbs-dcms ekp-lcg-ce.physik.uni-karlsruhe.de:2119/jobmanager-lcgpbs-cms grid-ce.desy.de:2119/jobmanager-lcgpbs-dcms grid-ce.physik.rwth-aachen.de:2119/jobmanager-lcgpbs-cms grid-ce.physik.rwth-aachen.de:2119/jobmanager-lcgpbs-short grid-ce.physik.rwth-aachen.de:2119/jobmanager-lcgpbs-dcms grid-ce.desy.de:2119/jobmanager-lcgpbs-cms **************************************************************** ekp-lcg-ui:>
DCMS queues / mapping: set up 12 Queue properties Aachen Karlsruhe Hamburg Published # CPUs 18/20 20/36 70 Queues for DCMS dcms,cms,short dcms,cms dcms,cms Max. # of jobs running 500 15 10, 5 Max. # of jobs per user - 9 - Max. CPU time 24 h 24 h 48 h Max. wallclock time 48 h 36 h 72 h Nice value 15 10 - The queue lengths are normalised to the CPU capacities Each CMS member coming to a DCMS site (VO cms or dcms) is mapped to a dedicated account depending on its affiliation (e.g. dcms001or cms001) dcms-members can be prioritised by e.g. different fairshare targets of their user groups Setup @ EKP CMS members DCMS members EKP CMS members Mapping account cms001 cms050 dcms001 dcms050 local account Groups cmsgrid dcms + cmsgrid cms + dcms + cmsgrid Fairshare low priority medium priority high priority Common queue/mapping policies in DCMS?
Hardware capacities 13 Aachen Karlsruhe Hamburg Sum SE classic 1.6 TB + 1.9 TB 220 GB (+ 2 TB) 5.7 TB will be: 1.6 TB (3.8 TB) SE dcache 1.9 TB + 1.2 TB 10 TB 13.1 TB will be: (1.9 +1.9 + 1.2 +...) TB + 20 (30) TB in Nov. (45 TB) CPU's 20 x AMD Opteron 246 5 x AMD Athlon XP 1600+ 70 x Dual Xeon 2.8/3.07 GHz 110 13 x AMD Athlon T-Bird (126) 1 x AMD Opteron 242 8 x AMD Opteron 246 Storage Elements: ~ 19 TB (39-49 TB in November) Worker Nodes: 110 CPU's published (126 can be used)
Supported software Grid monitoring dcms (V. Büge): http//www-ekp.physik.uni-karlsruhe.de/~vbuege/web/cmsmon.html Software installed with XCMSI K. Rabbertz Homogeneous software environment Status 30/09/2005 Software Aachen Hamburg Karlsruhe FZK CMKIN_4_3_1 X X X X CMKIN_5_0_0 X X X X CMKIN_5_1_0 X X X X CMKIN_5_1_1 X X X X OSCAR_3_6_5 X X OSCAR_3_7_0 X X X X ORCA_8_7_1 X X ORCA_8_7_3 X X X X ORCA_8_7_4 X X X X ORCA_8_7_5 X X X X ORCA_8_10_1 X X X FAMOS_1_2_0 X X X X FAMOS_1_3_1 X X X X slc3_ia32_gcc323 X X X X LCG 2.6.0 X X X X R-GMA X X X X geant42ndprod X X X X 14
Datasets / file transfers PhEDEx Tier 1 GridKa Tier 1... DESY Aachen Hamburg Karlsruhe File transfers with PhEDEx J. Rehn Hits, Digis, DSTs Publishing/zipping ongoing DCMS data manager? Tier 1 CNAF Datasets (not exhaustive): bt03_ttbb_tth bt03_ttjj_tth bt03_tth120_6j1l eg03_tt_2l_topr eg03_wt_2l_toprex hg03b_wt_tauqq_toprex mu03_dy2mu mu03_w1mu jm03b_ttbar_incl jm03b_qcd jm03b_wjets jm03b_zjets jm03b_ww_inclusive jm03b_zw_inclusive jm03b_zz_inclusive Sum: ~ 15-20 TB 15
Summary: Analysis within DCMS 16 Communication: Internet portal(s), mailing lists, meetings, phone... Hardware capacities: ok sharing of recources via LCG VO dcms CMS related software: ok Datasets: File transfers are running tth, QCD, Pileups... Analysis tools: PAX (C++ toolkit for advanced physics analyses) S. Kappler ExRootAnalysis (writes root trees) CRAB (submitting a job on the Grid using official datasets) Tutorial; C. Rosemann Do analyses, write theses, papers, (P)TDR and conference contributions...