bwgrid Treff MA/HD Sabine Richling, Heinz Kredel Universitätsrechenzentrum Heidelberg Rechenzentrum Universität Mannheim 20.

Transcription

1 bwgrid Treff MA/HD Sabine Richling, Heinz Kredel Universitätsrechenzentrum Heidelberg Rechenzentrum Universität Mannheim 20. October 2010 Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

2 Course Organization bwgrid Treff Participants Current users of the bwgrid Clusters HD/MA Students and scientists interested in Grid Computing Members of the Universities Heidelberg and Mannheim Scope bwgrid Status and Plans Lectures and/or Workshops User contributions To meet you in person Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

3 Course Organization bwgrid Treff Summer Term 2010 Main Focus: 29. April bwgrid Status and Interconnection HD/MA 20. May Batch-System and Parallel execution of serial jobs 17. June Parallel Programming with Java (Part I) 15. July Parallel Programming with Java (Part II) Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

4 Course Organization bwgrid Treff Fall/Winter 2010/2011 Lecture times: Mannheim: HS Heidelberg: WS Date Location Main Focus MA Matlab & Matlab Clones MA Statistics Software R Nov 2010 HD Dec 2010 HD Jan 2011 HD Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

5 Course Organization bwgrid Treff 20. October 2010 Agenda for today: bwgrid News Working with the statistical software package R on bwgrid Using R for data analysis on cluster computers: distributed computing and statistical modeling with MPI and Rmpi on bwgrid (D. Junge, MZES, University of Mannheim) Discussion of topics for further meetings Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

6 bwgrid News bwgrid News Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

7 bwgrid News What is bwgrid? D-Grid Community project of the Universities in BW Compute Clusters at 8 locations Central storage unit in Karlsruhe Distributed system with local administration Computing centers focus on software in different fields of research Access via at least one middleware supported by D-Grid Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

8 bwgrid News bwgrid Resources Compute cluster: Mannheim/Heidelberg: 280 nodes Direct Interconnection Frankfurt Karlsruhe: 140 nodes Stuttgart: 420 nodes Tübingen: 140 nodes Mannheim (interconnected to a single cluster) Heidelberg Ulm (Konstanz): 280 nodes Hardware in Ulm Karlsruhe Stuttgart Freiburg: 140 nodes Esslingen Esslingen: 180 nodes more recent Hardware Central storage: Freiburg Tübingen Ulm (joint cluster with Konstanz) München Karlsruhe: 128 TB (with Backup) 256 TB (without Backup) Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

9 bwgrid News Access Possibilities Important! Access with local Accounts: Project numbers and User IDs (URZ); User IDs (RUM) Access only to bwgrid cluster MA/HD Access with Grid Certificate: Grid Certificate, VO membership, Grid Middleware Access to all bwgrid resources Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

10 bwgrid User Support bwgrid News General Information: Hardware and software Project descriptions Grid access (server addresses) bwgrid Portals (in development) D-Grid user support: Trouble ticket system News module for maintenances User Support available at all sites: Login messages Local webpages, wikis, mailing lists address for local support Local news man bw-grid Terms and conditions for using bwgrid Home directory, scratch/work space, /tmp Special features Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

11 bwgrid News bwgrid Cluster Mannheim/Heidelberg Belwue VORM Benutzer Benutzer PBS Benutzer Benutzer VORM LDAP Admin AD Cluster Mannheim InfiniBand Obsidian + ADVA passwd InfiniBand Cluster Heidelberg Lustre bwfs MA Lustre bwfs HD Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

12 bwgrid News bwgrid Cluster Mannheim/Heidelberg News Reservation of all compute nodes for tomorrow bwgrid storage in Heidelberg available since end of August Workspaces for temporary data (no backup) $HOME now limited to 100 GB per user (with backup) Project descriptions for New software modules??? Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

13 R on bwgrid Working with R on bwgrid Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

14 What is R? R on bwgrid R is a language and environment for statistical computing and graphics... Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

15 R on bwgrid What is R? R provides a wide variety of statistical and graphical techniques Statistical techniques include linear and nonlinear modeling, classical statistical tests, time-series analysis, classification, clustering,... Well-designed publication-quality plots can be produced, including mathematical symbols and formulae R is highly extensible (via packages; integration in Latex, Open-office; Fortran and C extensions) Free Software (GNU General Public License) Runs on different platforms (Unix, Windows, MacOS) Excellent documentation (Reference manual with about 1500 pages) Excellent support via mailing list Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

16 R on bwgrid More about R Main Project Page: Download, Mailing Lists, Manuals, FAQ s,... Tutorial by G. Sawitzki (Applied Mathematics, Uni Heidelberg) R at LMU Munich: R courses, publications, parallel R Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

17 R on bwgrid R on bwgrid List of available software modules module avail R versions Version set environment with command module load math/r/ module load math/r/ module load math/r/ comes with OS module load math/r/ Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

18 Interactive Usage of R R on bwgrid Request a compute node qsub -I -X -l walltime=3:00:00 Load R module module load math/r/ Start R command line interface R Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

19 R on bwgrid Installation of R package Within R use command install.packages( package-name ) Allow that an installation directory for R packages is created in your home directory Select download mirror Dependent packages are automatically installed Packages must be installed for each version of R separately Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

20 R on bwgrid R Batch Jobs Read Module Help module help math/r/ Starting a single R program example script: bwgrid-r.pbs replace example R-program with your R-program only reasonable if your program needs a lot of memory Starting multiple R programs on one node example script: bwgrid-r-multi.pbs + task.sh determine number of cores in bwgrid-r-multi.pbs edit task.sh to fit your needs Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

21 R on bwgrid Parallel Programming Shared memory: Multithreading methods Parallelization whereby the manager (master) thread forks a number of worker (slave) threads and the tasks are divided among them. OpenMP (shared memory programming via compiler directives in C/C++ and Fortan) Pthreads (C libraries for creating and managing POSIX threads) Distributed memory: Message passing methods Work is divided among processes and the communication is made by sending messages between processes. MPI (Message Passing Interface; standardized and portable message-passing libraries) PVM (Parallel Virtual Machine; for using a network of heterogeneous Unix and/or Windows machines) Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

22 Parallel R packages R on bwgrid M. Schmidberger et al (Tech. Report 47, Department of Statistics, LMU) Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

23 R on bwgrid MPI Basics Number of available processes is given at start of program Each process runs on a single core with its own data Processes communicate via message passing to receive data which is in the memory of other processes MPI implementation provides library with message passing functions Some MPI concepts: Point-to-point communication (different modes) Collective communication (broadcast, gather, all-to-all) Communicators (to group processes) Virtual Topologies (to determine neighbors of processes) Error handling Speed-up depends on time spend for message passing Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

24 R on bwgrid Rmpi Package Package information: Provides interface to MPI functions from R Requires installed MPI implementation Package includes scripts to launch R instances Programmer determines workload of the processes and the communication between the processes Rmpi tutorial: Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

25 R on bwgrid Rmpi Installation Download Rmpi package: wget tar.gz Load R module: module load math/r/ Make installation directory: mkdir -p $HOME/R/2.11.1/Rmpi Install package with: R CMD INSTALL \ --configure-args=--with-mpi=/opt/bwgrid/mpi/openmpi/1.4.2-gnu-4.1 \ -l $HOME/R/2.11.1/Rmpi Rmpi tar.gz Example: bwgrid-rmpi.pbs Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

26 R Presentation (D. Junge) Using R for data analysis on cluster computers: distributed computing and statistical modeling with MPI and Rmpi on bwgrid D. Junge, MZES, University of Mannheim Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27

27 Conclusion Discussion of topics for further meetings Date Location Main Focus User Contribution MA Matlab & Matlab Clones A. Uhlendorff MA Statistics Software R D. Junge Nov 2010 HD Dec 2010 HD Jan 2011 HD Richling/Kredel (URZ/RUM) bwgrid Treff WS 2010/ / 27