Anar Manafov, GSI Darmstadt
HEP Data Analysis Implement algorithm Run over data set Make improvements Typical HEP analysis needs a continuous algorithm refinement cycle 2
PROOF Storage File Catalog Query PROOF cluster Scheduler CPUs PROOF Query: data file list, myselector.c Feedback, merged final output Master 3
PROOF PROOF cluster as extension of a local PC, same macro and syntax as in local ROOT session, more dynamic use of resources, real-time feedback, automatic splitting and merging. 4
Dynamic cluster User can entirely control it, can setup and use it on demand, can reserve desired amount of workers, can select a preferable master host, doesn t need admins to take an action, doesn t interact with other users. 5
6
PoD v2.1.x pod-console PoDWorker PoD server management PROOF workers monitor PoD utilities and command line tools Job Manager plug-in system glite LSF PBS SSH Configuration files (PoD, xrootd, PROOF) pod-user-defaults glite-api-wrapper (GAW) pod-agent glite Grid API LSF API PBS (torque) API ssh/scp xrootd/xproof plug-in 7
8
Resource management system User workspace 8
Resource management system User workspace 8
Resource management system User workspace 8
Worker node workspace Resource management system Firewall User workspace User workspace 8
Worker node workspace Resource management system Firewall pod-agent server User workspace xrootd User workspace 8
Resource management system PoDWorker job #1 Worker node workspace xrootd pod-agent worker #1 Firewall pod-agent server User workspace xrootd User workspace 8
Worker node workspace Resource management system PoDWorker job #1 xrootd PoDWorker job #2 pod-agent worker #1 pod-agent worker #2 Firewall pod-agent server User workspace xrootd User workspace 8
Worker node workspace Resource management system PROOF worker #1 PROOF worker #2 PoDWorker job #1 xrootd PoDWorker job #2 pod-agent worker #1 pod-agent worker #2 Firewall pod-agent server User workspace xrootd User workspace PROOF master 8
Worker node workspace Resource management system PROOF worker #1 PROOF worker #2 PoDWorker job #1 xrootd PoDWorker job #2 pod-agent pod-agent xrootd User workspace PROOF master pod-agent server User workspace 9
LIVE DEMO 10
Key features Easy to use GUI & Command-line Different job managers Multiuser/-core environment Native PROOF connections Packet-forwarding User defaults - configuration 11
PoD development PoD is about 85% C++, 13% Bash, 2% (Python and Perl) VCS is git: http://depc218.gsi.de:22222/git/ Continues integration system is buildbot: http://depc218.gsi.de:22000/waterfall Development environment: Xcode and Eclipse 12
PoD Development Build system CMake. Documentation systems DocBook (user manual and web site http://pod.gsi.de), Doxygen (source code doc.). All docs. and web site are generated automatically every night.
PoD at GSI Dedicated LSF queue preemptive, max. 120 jobs per user and max. 4 hours run-time per job. Data located on the lustre FS. Mainly used by ALICE group (GSI, Heidelberg, Münster). In average we have 2-5 concurrent users with 20-120 workers each. 14
15
16
ToDo PBS and SSH plug-ins, out of server UI, native Mac OS X implementation of UI. 17