Anar Manafov, GSI Darmstadt. GSI Palaver,

Transcription

1 Anar Manafov, GSI Darmstadt

2 HEP Data Analysis Implement algorithm Run over data set Make improvements Typical HEP analysis needs a continuous algorithm refinement cycle 2

3 PROOF Storage File Catalog Query PROOF cluster Scheduler CPUs PROOF Query: data file list, myselector.c Feedback, merged final output Master 3

4 PROOF PROOF cluster as extension of a local PC, same macro and syntax as in local ROOT session, more dynamic use of resources, real-time feedback, automatic splitting and merging. 4

5 Dynamic cluster User can entirely control it, can setup and use it on demand, can reserve desired amount of workers, can select a preferable master host, doesn t need admins to take an action, doesn t interact with other users. 5

6 6

7 PoD v2.1.x pod-console PoDWorker PoD server management PROOF workers monitor PoD utilities and command line tools Job Manager plug-in system glite LSF PBS SSH Configuration files (PoD, xrootd, PROOF) pod-user-defaults glite-api-wrapper (GAW) pod-agent glite Grid API LSF API PBS (torque) API ssh/scp xrootd/xproof plug-in 7

8 8

9 Resource management system User workspace 8

10 Resource management system User workspace 8

11 Resource management system User workspace 8

12 Worker node workspace Resource management system Firewall User workspace User workspace 8

13 Worker node workspace Resource management system Firewall pod-agent server User workspace xrootd User workspace 8

14 Resource management system PoDWorker job #1 Worker node workspace xrootd pod-agent worker #1 Firewall pod-agent server User workspace xrootd User workspace 8

15 Worker node workspace Resource management system PoDWorker job #1 xrootd PoDWorker job #2 pod-agent worker #1 pod-agent worker #2 Firewall pod-agent server User workspace xrootd User workspace 8

16 Worker node workspace Resource management system PROOF worker #1 PROOF worker #2 PoDWorker job #1 xrootd PoDWorker job #2 pod-agent worker #1 pod-agent worker #2 Firewall pod-agent server User workspace xrootd User workspace PROOF master 8

17 Worker node workspace Resource management system PROOF worker #1 PROOF worker #2 PoDWorker job #1 xrootd PoDWorker job #2 pod-agent pod-agent xrootd User workspace PROOF master pod-agent server User workspace 9

18 LIVE DEMO 10

19 Key features Easy to use GUI & Command-line Different job managers Multiuser/-core environment Native PROOF connections Packet-forwarding User defaults - configuration 11

20 PoD development PoD is about 85% C++, 13% Bash, 2% (Python and Perl) VCS is git: Continues integration system is buildbot: Development environment: Xcode and Eclipse 12

21 PoD Development Build system CMake. Documentation systems DocBook (user manual and web site Doxygen (source code doc.). All docs. and web site are generated automatically every night.

22 PoD at GSI Dedicated LSF queue preemptive, max. 120 jobs per user and max. 4 hours run-time per job. Data located on the lustre FS. Mainly used by ALICE group (GSI, Heidelberg, Münster). In average we have 2-5 concurrent users with workers each. 14

23 15

24 16

25 ToDo PBS and SSH plug-ins, out of server UI, native Mac OS X implementation of UI. 17