Hodor and Bran - Job Scheduling and PBS Scripts
|
|
- Isaac Farmer
- 8 years ago
- Views:
Transcription
1 Hodor and Bran - Job Scheduling and PBS Scripts UND Computational Research Center Now that you have your program compiled and your input file ready for processing, it s time to run your job on the cluster. However, running a parallel program on the cluster isn t as simple as running a serial program. Novice users sometime have the misconception that the cluster is a single large computer. That is not the case. The cluster is a cluster of multiple computers (aka nodes), where there is a head node and multiple compute nodes. Job Scheduler When a user first logs into Hodor using the SSH process described in the Hodor and Bran - Cluster Account Basics tutorial, the user is actually logging into the cluster head node. The head node is where you write and compile your programs. It is also the node that schedules computation runs (aka jobs ) on the compute nodes. The software that manages the schedule of the programs to run is known as a Job Scheduler. Why is a job scheduler necessary? Without a job scheduler, users would randomly assign their programs to run on compute nodes. This type of behavior leads to crashed programs and slow runtimes, as one user s program would compete for compute node RAM and processor time of a compute node with another user s program. For instance, maybe your program needs 64GB of RAM, and another user s program also needs to use 64GB of RAM. Without a job scheduler, your program and the other user s program might end up running on the same compute node that only has 64GB of RAM available for use. If both programs try to use all the RAM, something is going to fail: either your program, his/her program, or the node itself. As such, when running your research code on the cluster it is always important that you use the job scheduler to run your job. Also, you should never run your research code directly from the command line on the head node. Therefore we have a few basic rules that we require users follow, which are listed below. Should a user violate these rules, the CRC reserves the right to ban the user from using cluster resources in cases of repeated or severe rules violations: Cluster Rules: 1. Do not run your research computational program on the head node. The head node is for writing code, editing/transferring files, and compiling programs. 2. Use the job scheduler to run your program on the compute nodes. 3. Do not use programs that you don t have a license to use. 4. Do not flood the cluster with jobs without first notifying the CRC. If you need to submit 1000 s of jobs at a time, please notify the CRC before submitting that many jobs. 5. Follow directions given by the CRC. 6. Be courteous to other users. The cluster a shared resource. Job may need to wait in the queue. 7. Don t give access to your account login/password to others rd party accounts must be approved by the CRC. (aka other universities, private organizations) 1
2 QSUB and PBS Scripts The job scheduler uses the qsub command and PBS scripts to launch a job. A user would type something similar to the following on the command line to submit a job where example.pbs is a PBS script written by the user: qsub example.pbs A PBS script, is simply a text file. Edit it with your text editor of choice and upload it to the cluster, or use VI/VIM to edit it after logging into the head node. Directions on how to use VIM can be found here: OpenMP Programs: Below are the contents of our example.pbs file. It assumes that you are attempting to run an OpenMP program called nbody.x. The line numbers are not part of the file, and have been added for purposes of discussion. 1 #PBS -S /bin/bash 2 #PBS -l nodes=1:ppn=8,walltime=10:00:00,naccesspolicy=singlejob 3 #PBS N mynbody 4 #PBS M john.doe@my.und.edu 5 #PBS m bea 6 #PBS o /${PBS_O_WORKDIR}/out_${PBS_JOBNAME}.txt 7 #PBS e /${PBS_O_WORKDIR}/err_${PBS_JOBNAME}.txt 8 #PBS V 9 10 #Set the environment variable that tells your openmp program 11 #how many threads to use. 12 export OMP_NUM_THREADS=`wc l < ${PBS_NODESFILE}` #Change directory to the current working directory 15 cd ${PBS_O_WORKDIR} /nbody.x inp inputfile.inp Essentially, the PBS script is a shell script, however its file permissions need only be read/write, and it does not need to be executable. For users who are familiar with shell scripts, you might think lines that begin with #PBS are code comments, but this is incorrect. Any line that begins with #PBS, that line is PBS instruction for qsub. A complete list of #PBS can be found here: Let s discuss example.pbs line by line. LINE 1: #PBS S /bin/bash PBS command that sets non-pbs commands to be interpreted as BASH shell commands. Bash and CSH are available on the cluster. 2
3 LINE 2: #PBS l nodes=1:ppn=8,walltime=10:00:00,naccesspolicy=singlejob The pbs command -l that sets the resource request. In other words, it tells the scheduler that it wants: One node (nodes=1) Eight processes per node (:ppn=8) Use of the requested resources for 10 hours (,walltime=10:00:00) where walltime=hh:mm:ss Exclusive use of this node (,naccesspolicy=singlejob). This means that only a single job will have access to this node while this job is running. Each compute node in the cluster has two 3.3GHz quad-core processors. As such, the maximum number that ppn= can be set to is 2x4 in other words, a value of 8. Notice that each variable on line 2 is separated by a comma and no spaces. These aren t the only variables that can be set by the #PBS l command, but most users will only need to the above variables. For a list of more #PBS l variables, see the following web page: LINE 3: #PBS N mynbody The pbs command that sets the name of the job to mynbody. This flag is not required. If not included, the job scheduler will make the job name the same as the job ID number. However, if you want to control the job s name, you should set it yourself. In some cases, it may be more convenient to the user to leave this command out of the PBS script, and instead, set the job name in the following manner: qsub N mynbody example.pbs LINE 4: #PBS M jane.doe@my.und.edu The pbs command that sets the list of addresses that should be sent a message once a mail_option has been triggered. To add more addresses, separate each address by a comma and no spaces. LINE 5: #PBS m bea The pbs command that sets the list of mail_options for this job. Options are: b when the job begins, e when the job ends, a when the job aborts. LINES 6 & 7: #PBS o /${PBS_O_WORKDIR}/out_${PBS_JOBNAME}.txt #PBS e /${PBS_O_WORKDIR}/err_${PBS_JOBNAME}.txt The pbs commands that set the files two which the program s standard out (#PBS o) and standard error (#PBS e) are dumped. The environment variable PBS_O_WORKDIR, is directory path for the current directory when qsub is executed by the user. So if my current directory is: /home/john.doe and my job name is mynbody, then standard out and standard error will be dumped to the following files: 3
4 /home/john.doe/out_mynbody.txt /home/john.doe/err_mynbody.txt Notice in lines 6 & 7 that ${PBS_O_WORKDIR} is proceeded by a forward slash /, but in the BASH shell commands of PBS script it is not. When setting o or e it the forward slash is required due to a bug in the scheduling sosftware. An alternative means of setting the stdout and stderr files is to have them dumped to a single file. This is done using the following PBS commands: #PBS j oe #PBS o /${PBS_O_WORKDIR}/out_${PBS_JOBNAME}.txt LINE 8: #PBS V The pbs command that has all the user s environment variables sent to all the nodes. May or may not be necessary. Ultimately, it s up to the user to determine whether or not this is necessary. ==================== At this point, the rest of the commands in the PBS script are Bash shell commands and a call to run your program. The PBS_NODESFILE environment variable holds the full path to a temporary file that contains a list of the hostnames for every node that your job will use. Since ppn=8, each node s hostname will be listed 8 times, each time on its own line. The BASH commands and your executable will only be executed once on the first node listed in the PBS_NODESFILE temporary file. ==================== LINES 9: blank LINES 10-11: code comments LINE 12 export OMP_NUM_THREADS=`wc l < ${PBS_NODESFILE}` ThIs BASH shell command sets the user s OMP_NUM_THREADS environment variable on the compute node(s) requested by the job with the number of threads required by the user s openmp program. A bit more of an explanation is required to understand what this line is doing. The BASH shell command `wc l < ${PBS_NODESFILE)` simply dumps the list of hostnames to the wc BASH program, which in turn counts the number of lines in that file, and then that count value is stored in OMP_NUM_THREADS. 4
5 LINES 13: blank LINES 14: code comment LINE 15 cd ${PBS_O_WORKDIR} This BASH shell command changes the current directory on the compute node to the path designated by PBS_O_WORKDIR. Notice a forward slash is not needed here. LINES 16: blank LINE 17./nbody.x inp inputfile.inp Here the user is simply executing his program. It assumes that the nbody.x executable is in the directory pointed to by PBS_O_WORKDIR. This ficational program takes two arguments, a flag inp (defined by the fictional program code) and inputfile.inp, which is the filename that this fictional program expects to follow the inp flag. These program arguments are specific to this fictional program. Your program will take completely different arguments as defined by your program MPI Programs: Below are the contents an alternative example.pbs file. It assumes that you are attempting to run an MVAPICH2-x MPI program called mpi_nbody.x. The line numbers are not part of the file, and have been added for purposes of discussion. 1 #PBS -S /bin/bash 2 #PBS -l nodes=2:ppn=8,walltime=10:00:00,naccesspolicy=singlejob 3 #PBS N mynbody 4 #PBS M john.doe@my.und.edu 5 #PBS m bea 6 #PBS o /${PBS_O_WORKDIR}/out_${PBS_JOBNAME}.txt 7 #PBS e /${PBS_O_WORKDIR}/err_${PBS_JOBNAME}.txt 8 #PBS V 9 10 #Set the environment variable that tells your openmp program 11 #how many threads to use. 12 NP=`wc l < ${PBS_NODESFILE}` module load mvapich2-x-runtime #Change directory to the current working directory 17 cd ${PBS_O_WORKDIR} mpirun machinefile ${PBS_NODESFILE} np ${NP}./mpi_nbody.x inp inputfile.inp module unload mvapich2-x-runtime 5
6 Essentially, the PBS script is a shell script, however its file permissions need only be read/write, and it does not need to be executable. For users who are familiar with shell scripts, you might think lines that begin with #PBS are code comments, but this is incorrect. Any line that begins with #PBS, that line is PBS instruction for qsub. A complete list of #PBS can be found here: Let s discuss example.pbs line by line. LINE 1: #PBS S /bin/bash PBS command that sets non-pbs commands to be interpreted as BASH shell commands. Bash and CSH are available on the cluster. LINE 2: #PBS l nodes=2:ppn=8,walltime=10:00:00,naccesspolicy=singlejob The pbs command -l that sets the resource request. In other words, it tells the scheduler that it wants: Two nodes (nodes=2) Eight processes per node (:ppn=8) Use of the requested resources for 10 hours (,walltime=10:00:00) where walltime=hh:mm:ss Exclusive use of this node (,naccesspolicy=singlejob). This means that only a single job will have access to this node while this job is running. Each compute node in the cluster has two 3.3GHz quad-core processors. As such, the maximum number that ppn= can be set to is 2x4 in other words, a value of 8. Notice that each variable on line 2 is separated by a comma and no spaces. These aren t the only variables that can be set by the #PBS l command, but most users will only need to the above variables. For a list of more #PBS l variables, see the following web page: LINE 3: #PBS N mynbody The pbs command that sets the name of the job to mynbody. This flag is not required. If not included, the job scheduler will make the job name the same as the job ID number. However, if you want to control the job s name, you should set it yourself. In some cases, it may be more convenient to the user to leave this command out of the PBS script, and instead, set the job name in the following manner: qsub N mynbody example.pbs LINE 4: #PBS M jane.doe@my.und.edu The pbs command that sets the list of addresses that should be sent a message once a mail_option has been triggered. To add more addresses, separate each address by a comma and no spaces. 6
7 LINE 5: #PBS m bea The pbs command that sets the list of mail_options for this job. Options are: b when the job begins, e when the job ends, a when the job aborts. LINES 6 & 7: #PBS o /${PBS_O_WORKDIR}/out_${PBS_JOBNAME}.txt #PBS e /${PBS_O_WORKDIR}/err_${PBS_JOBNAME}.txt The pbs commands that set the files two which the program s standard out (#PBS o) and standard error (#PBS e) are dumped. The environment variable PBS_O_WORKDIR, is directory path for the current directory when qsub is executed by the user. So if my current directory is: /home/john.doe and my job name is mynbody, then standard out and standard error will be dumped to the following files: /home/john.doe/out_mynbody.txt /home/john.doe/err_mynbody.txt Notice in lines 6 & 7 that ${PBS_O_WORKDIR} is proceeded by a forward slash /, but in the BASH shell commands of PBS script it is not. When setting o or e it the forward slash is required due to a bug in the scheduling sosftware. An alternative means of setting the stdout and stderr files is to have them dumped to a single file. This is done using the following PBS commands: #PBS j oe #PBS o /${PBS_O_WORKDIR}/out_${PBS_JOBNAME}.txt LINE 8: #PBS V The pbs command that has all the user s environment variables sent to all the nodes. May or may not be necessary. Ultimately, it s up to the user to determine whether or not this is necessary. ==================== At this point, the rest of the commands in the PBS script are Bash shell commands and a call to run your program. The PBS_NODESFILE environment variable holds the full path to a temporary file that contains a list of the hostnames for every node that your job will use. Since ppn=8, each node s hostname will be listed 8 times, each time on its own line. The BASH commands and your executable will only be executed once on the first node listed in the PBS_NODESFILE temporary file. ==================== 7
8 LINES 9: blank LINES 10-11: code comments LINE 12 NP=`wc l < ${PBS_NODESFILE}` ThIs BASH shell command sets at temporary environment variable for the user called NP on the first compute node listed in the PBS_NODESFILE temporary file. Unlike the OpenMP version of this PBS script, the NP environment variable is not exported. A bit more of an explanation is required to understand what this line is doing. The BASH shell command `wc l < ${PBS_NODESFILE)` simply dumps the list of hostnames to the wc BASH program, which in turn counts the number of lines in that file, and then that count value is stored in NP. LINES 13: blank LINE 14 module load mvapich2-x-runtime Loads the modulefile mvapich2-x-runtime that sets up the environment variables necessary for running an MPI program compiled using the MVAPICH2-X API. LINES 15: blank LINES 16: code comment LINE 17 cd ${PBS_O_WORKDIR} This BASH shell command changes the current directory on the compute node to the path designated by PBS_O_WORKDIR. Notice a forward slash is not needed here. LINES 18: blank LINE 19 mpirun machinefile ${PBS_NODESFILE} np ${NP}./mpi_nbody.x inp inputfile.inp Here the user is simply executing his program. It assumes that the mpi_nbody.x executable is in the directory pointed to by PBS_O_WORKDIR. There are a few things going on here. 1. It should be noted, that the PBS is actually is executing mpirun not the user s executable. 2. Also mpirun needs to know: a. which compute nodes will are used by this job. i. As such, the PBS_NODESFILE variable is passed to mpirun using the machinefile flag. ii. DO NOT use a custom machinefile here. This will seriously screw up your job run, and most probably other user s as well. iii. ALWAYS use ${PBS_NODESFILE} variable here! b. How many total MPI processes there are in this job i. this is why we pass the value of ${NP} to mpirun using the np flag. 8
9 ii. NP should equal the number of nodes times the number of ppn iii. You could replace np ${NP} with np 16 in this case, just be sure the number is correct. 3. You could also use mpiexec instead of mpirun 4. This ficational program takes two arguments: a. a flag inp (defined by the fictional program code) b. and inputfile.inp, which is the filename that this program expects to follow the flag. c. These program arguments are specific to this fictional program. Your program will take completely different arguments as defined within your code. LINES 20: blank LINE 21 module unload mvapich2-x-runtime Unloads the modulefile mvapich2-x-runtime and removes environment variables that had been previously setup by the load. Not really necessary, because the job is over at this point. It s included in this example to show that it can be done within the PBS script Embarrassingly Parallel Programs: Below are the contents of our example.pbs file. It assumes that you are attempting to run a non-communicating Embarrassingly Parallel program called renderer.x. In other words, you want to run multiple copies of a serial program that don t communicate with each other. The line numbers are not part of the file, and have been added for purposes of discussion. 1 #PBS -S /bin/bash 2 #PBS -l nodes=4:ppn=8,walltime=10:00:00,naccesspolicy=singlejob 3 #PBS N renderjob 4 #PBS M john.doe@my.und.edu 5 #PBS m bea 6 #PBS o /${PBS_O_WORKDIR}/out_${PBS_JOBNAME}.txt 7 #PBS e /${PBS_O_WORKDIR}/err_${PBS_JOBNAME}.txt 8 #PBS V 9 10 #Change directory to the current working directory 11 cd ${PBS_O_WORKDIR} pbsdsh./renderer.x inp inputfile.inp Like the other two PBS script examples, the PBS script shown immediately above is essentially a shell script. Since this script is very much like the OpenMP version, we can skip a discussion of most of this file, down to line 13. LINE 13 pbsdsh./renderer.x inp inputfile.inp 9
10 Here the user is simply wants to execute his/her serial program on all requested processes. It assumes that the renderer.x program is a serial program and is located in the directory pointed to by PBS_O_WORKDIR. In this case, the PBS script is actually executing the pbsdsh program. This PBS script requests the use of four nodes, and eight processes per node, for a total of 32 processes. As such, pbsdsh will execute the renderer.x program on all 32 processes. If the -u flag was added to this line, as so: pbsdsh u./renderer.x inp inputfile.inp then renderer.x would run only one processes per node, for a total of four processes. For more information on the use of pbsdsh see the following web page: PBS_NODENUM and PBS_TASKNUM can be used in combination with the pbsdsh program. For more information see: WARNING - If you wish to use pbsdsh be careful that your program s arguments/flags are different from flags used by pbsdsh. If they are the same, pbsdsh will assume that the flags are meant for pbsdsh, and will not pass them along to your program Other Information qstat and qdel: In addition to the command to submit jobs, you should know the commands to see which jobs are in the job queue, and how to delete a job. To see which jobs are in the queue use the qstat command. Will display all jobs queued Q and running - R qstat a Will display all jobs, plus the nodes on which they are running qstat n Will display all jobs, plus all job information for each job (a lot of information) qstat f If there are a large number of jobs in the queue, you may wish to use grep in addition to qstat such as: qstat a grep jane.doe The figure on the next page demonstrates the output generated from using qstat a 10
11 Notice the Job ID column in the figure above. Use the number in the Job ID column to cancel a running or queued job like demonstrated below: qdel Typing the above text on the command line and pressing return/enter will kill the job with Job ID Users can only kill their own jobs. WALLTIME: When invoking the #PBS l command, be sure to set your walltime variable to a value high enough to ensure your job completes its run before the walltime runs out. In other words, if you feel that your program will take 3 days to run, make sure you set your walltime to at least: 72:00:00. It would probably be wise to set the hours value a bit higher than 72 hours. Estimate too low, and your time may run out before your program completes. Estimate too high, and other jobs with a shorter walltime may jump ahead of your job in the scheduler queue. Be a courteous user and estimate appropriately. NODES: There maybe cases in which a user may wish to select specific compute nodes. Hodor Compute Nodes: node001 thru node032 Bran Compute Nodes: node001-test thru node004-test If you wish to request specific nodes, you can do so by using the following PBS commands Instead of using this: #PBS l nodes=4:ppn=8,wallti. Use this: #PBS l nodes=node002:ppn=8+node003:ppn=8+node004:ppn=8+node004:ppn=8,wallti 11
12 Of course, if you request specific nodes, your job will wait to run until every one of the nodes that you request becomes available. You might wait awhile. Final Thoughts This tutorial only scratches the surfaces of using the Hodor and Bran job scheduler. We haven t covered many things for example: GPU programs, MIC programs, combination MPI/OpenMP just to name a few. If you have any questions at all, please contact the CRC. Aaron Bergstrom HPC Specialist aaron.bergstrom@ .und.edu
Grid 101. Grid 101. Josh Hegie. grid@unr.edu http://hpc.unr.edu
Grid 101 Josh Hegie grid@unr.edu http://hpc.unr.edu Accessing the Grid Outline 1 Accessing the Grid 2 Working on the Grid 3 Submitting Jobs with SGE 4 Compiling 5 MPI 6 Questions? Accessing the Grid Logging
More informationPBS Tutorial. Fangrui Ma Universit of Nebraska-Lincoln. October 26th, 2007
PBS Tutorial Fangrui Ma Universit of Nebraska-Lincoln October 26th, 2007 Abstract In this tutorial we gave a brief introduction to using PBS Pro. We gave examples on how to write control script, and submit
More informationMiami University RedHawk Cluster Working with batch jobs on the Cluster
Miami University RedHawk Cluster Working with batch jobs on the Cluster The RedHawk cluster is a general purpose research computing resource available to support the research community at Miami University.
More informationRa - Batch Scripts. Timothy H. Kaiser, Ph.D. tkaiser@mines.edu
Ra - Batch Scripts Timothy H. Kaiser, Ph.D. tkaiser@mines.edu Jobs on Ra are Run via a Batch System Ra is a shared resource Purpose: Give fair access to all users Have control over where jobs are run Set
More informationNYUAD HPC Center Running Jobs
NYUAD HPC Center Running Jobs 1 Overview... Error! Bookmark not defined. 1.1 General List... Error! Bookmark not defined. 1.2 Compilers... Error! Bookmark not defined. 2 Loading Software... Error! Bookmark
More informationIntroduction to Running Hadoop on the High Performance Clusters at the Center for Computational Research
Introduction to Running Hadoop on the High Performance Clusters at the Center for Computational Research Cynthia Cornelius Center for Computational Research University at Buffalo, SUNY 701 Ellicott St
More informationCluster@WU User s Manual
Cluster@WU User s Manual Stefan Theußl Martin Pacala September 29, 2014 1 Introduction and scope At the WU Wirtschaftsuniversität Wien the Research Institute for Computational Methods (Forschungsinstitut
More informationQuick Tutorial for Portable Batch System (PBS)
Quick Tutorial for Portable Batch System (PBS) The Portable Batch System (PBS) system is designed to manage the distribution of batch jobs and interactive sessions across the available nodes in the cluster.
More informationHPC at IU Overview. Abhinav Thota Research Technologies Indiana University
HPC at IU Overview Abhinav Thota Research Technologies Indiana University What is HPC/cyberinfrastructure? Why should you care? Data sizes are growing Need to get to the solution faster Compute power is
More informationLinux für bwgrid. Sabine Richling, Heinz Kredel. Universitätsrechenzentrum Heidelberg Rechenzentrum Universität Mannheim. 27.
Linux für bwgrid Sabine Richling, Heinz Kredel Universitätsrechenzentrum Heidelberg Rechenzentrum Universität Mannheim 27. June 2011 Richling/Kredel (URZ/RUM) Linux für bwgrid FS 2011 1 / 33 Introduction
More informationBatch Scripts for RA & Mio
Batch Scripts for RA & Mio Timothy H. Kaiser, Ph.D. tkaiser@mines.edu 1 Jobs are Run via a Batch System Ra and Mio are shared resources Purpose: Give fair access to all users Have control over where jobs
More informationRunning applications on the Cray XC30 4/12/2015
Running applications on the Cray XC30 4/12/2015 1 Running on compute nodes By default, users do not log in and run applications on the compute nodes directly. Instead they launch jobs on compute nodes
More informationNEC HPC-Linux-Cluster
NEC HPC-Linux-Cluster Hardware configuration: 4 Front-end servers: each with SandyBridge-EP processors: 16 cores per node 128 GB memory 134 compute nodes: 112 nodes with SandyBridge-EP processors (16 cores
More informationGetting Started with HPC
Getting Started with HPC An Introduction to the Minerva High Performance Computing Resource 17 Sep 2013 Outline of Topics Introduction HPC Accounts Logging onto the HPC Clusters Common Linux Commands Storage
More informationMartinos Center Compute Clusters
Intro What are the compute clusters How to gain access Housekeeping Usage Log In Submitting Jobs Queues Request CPUs/vmem Email Status I/O Interactive Dependencies Daisy Chain Wrapper Script In Progress
More informationThe CNMS Computer Cluster
The CNMS Computer Cluster This page describes the CNMS Computational Cluster, how to access it, and how to use it. Introduction (2014) The latest block of the CNMS Cluster (2010) Previous blocks of the
More informationSLURM: Resource Management and Job Scheduling Software. Advanced Computing Center for Research and Education www.accre.vanderbilt.
SLURM: Resource Management and Job Scheduling Software Advanced Computing Center for Research and Education www.accre.vanderbilt.edu Simple Linux Utility for Resource Management But it s also a job scheduler!
More informationStreamline Computing Linux Cluster User Training. ( Nottingham University)
1 Streamline Computing Linux Cluster User Training ( Nottingham University) 3 User Training Agenda System Overview System Access Description of Cluster Environment Code Development Job Schedulers Running
More informationGrid Engine Basics. Table of Contents. Grid Engine Basics Version 1. (Formerly: Sun Grid Engine)
Grid Engine Basics (Formerly: Sun Grid Engine) Table of Contents Table of Contents Document Text Style Associations Prerequisites Terminology What is the Grid Engine (SGE)? Loading the SGE Module on Turing
More informationSGE Roll: Users Guide. Version @VERSION@ Edition
SGE Roll: Users Guide Version @VERSION@ Edition SGE Roll: Users Guide : Version @VERSION@ Edition Published Aug 2006 Copyright 2006 UC Regents, Scalable Systems Table of Contents Preface...i 1. Requirements...1
More informationIntroduction to HPC Workshop. Center for e-research (eresearch@nesi.org.nz)
Center for e-research (eresearch@nesi.org.nz) Outline 1 About Us About CER and NeSI The CS Team Our Facilities 2 Key Concepts What is a Cluster Parallel Programming Shared Memory Distributed Memory 3 Using
More informationSLURM: Resource Management and Job Scheduling Software. Advanced Computing Center for Research and Education www.accre.vanderbilt.
SLURM: Resource Management and Job Scheduling Software Advanced Computing Center for Research and Education www.accre.vanderbilt.edu Simple Linux Utility for Resource Management But it s also a job scheduler!
More informationParallel Debugging with DDT
Parallel Debugging with DDT Nate Woody 3/10/2009 www.cac.cornell.edu 1 Debugging Debugging is a methodical process of finding and reducing the number of bugs, or defects, in a computer program or a piece
More informationWork Environment. David Tur HPC Expert. HPC Users Training September, 18th 2015
Work Environment David Tur HPC Expert HPC Users Training September, 18th 2015 1. Atlas Cluster: Accessing and using resources 2. Software Overview 3. Job Scheduler 1. Accessing Resources DIPC technicians
More informationIntroduction to Sun Grid Engine (SGE)
Introduction to Sun Grid Engine (SGE) What is SGE? Sun Grid Engine (SGE) is an open source community effort to facilitate the adoption of distributed computing solutions. Sponsored by Sun Microsystems
More informationUsing NeSI HPC Resources. NeSI Computational Science Team (support@nesi.org.nz)
NeSI Computational Science Team (support@nesi.org.nz) Outline 1 About Us About NeSI Our Facilities 2 Using the Cluster Suitable Work What to expect Parallel speedup Data Getting to the Login Node 3 Submitting
More informationJob Scheduling with Moab Cluster Suite
Job Scheduling with Moab Cluster Suite IBM High Performance Computing February 2010 Y. Joanna Wong, Ph.D. yjw@us.ibm.com 2/22/2010 Workload Manager Torque Source: Adaptive Computing 2 Some terminology..
More informationJuropa. Batch Usage Introduction. May 2014 Chrysovalantis Paschoulas c.paschoulas@fz-juelich.de
Juropa Batch Usage Introduction May 2014 Chrysovalantis Paschoulas c.paschoulas@fz-juelich.de Batch System Usage Model A Batch System: monitors and controls the resources on the system manages and schedules
More informationIntroduction to Running Computations on the High Performance Clusters at the Center for Computational Research
! Introduction to Running Computations on the High Performance Clusters at the Center for Computational Research! Cynthia Cornelius! Center for Computational Research University at Buffalo, SUNY! cdc at
More informationHow To Run A Steady Case On A Creeper
Crash Course Introduction to OpenFOAM Artur Lidtke University of Southampton akl1g09@soton.ac.uk November 4, 2014 Artur Lidtke Crash Course Introduction to OpenFOAM 1 / 32 What is OpenFOAM? Using OpenFOAM
More information1.0. User Manual For HPC Cluster at GIKI. Volume. Ghulam Ishaq Khan Institute of Engineering Sciences & Technology
Volume 1.0 FACULTY OF CUMPUTER SCIENCE & ENGINEERING Ghulam Ishaq Khan Institute of Engineering Sciences & Technology User Manual For HPC Cluster at GIKI Designed and prepared by Faculty of Computer Science
More informationGrid Engine Users Guide. 2011.11p1 Edition
Grid Engine Users Guide 2011.11p1 Edition Grid Engine Users Guide : 2011.11p1 Edition Published Nov 01 2012 Copyright 2012 University of California and Scalable Systems This document is subject to the
More informationUsing WestGrid. Patrick Mann, Manager, Technical Operations Jan.15, 2014
Using WestGrid Patrick Mann, Manager, Technical Operations Jan.15, 2014 Winter 2014 Seminar Series Date Speaker Topic 5 February Gino DiLabio Molecular Modelling Using HPC and Gaussian 26 February Jonathan
More informationRA MPI Compilers Debuggers Profiling. March 25, 2009
RA MPI Compilers Debuggers Profiling March 25, 2009 Examples and Slides To download examples on RA 1. mkdir class 2. cd class 3. wget http://geco.mines.edu/workshop/class2/examples/examples.tgz 4. tar
More informationTutorial: Using WestGrid. Drew Leske Compute Canada/WestGrid Site Lead University of Victoria
Tutorial: Using WestGrid Drew Leske Compute Canada/WestGrid Site Lead University of Victoria Fall 2013 Seminar Series Date Speaker Topic 23 September Lindsay Sill Introduction to WestGrid 9 October Drew
More informationParallel Programming for Multi-Core, Distributed Systems, and GPUs Exercises
Parallel Programming for Multi-Core, Distributed Systems, and GPUs Exercises Pierre-Yves Taunay Research Computing and Cyberinfrastructure 224A Computer Building The Pennsylvania State University University
More informationAn Introduction to High Performance Computing in the Department
An Introduction to High Performance Computing in the Department Ashley Ford & Chris Jewell Department of Statistics University of Warwick October 30, 2012 1 Some Background 2 How is Buster used? 3 Software
More informationGRID Computing: CAS Style
CS4CC3 Advanced Operating Systems Architectures Laboratory 7 GRID Computing: CAS Style campus trunk C.I.S. router "birkhoff" server The CAS Grid Computer 100BT ethernet node 1 "gigabyte" Ethernet switch
More informationThe RWTH Compute Cluster Environment
The RWTH Compute Cluster Environment Tim Cramer 11.03.2013 Source: D. Both, Bull GmbH Rechen- und Kommunikationszentrum (RZ) How to login Frontends cluster.rz.rwth-aachen.de cluster-x.rz.rwth-aachen.de
More informationRunning on Blue Gene/Q at Argonne Leadership Computing Facility (ALCF)
Running on Blue Gene/Q at Argonne Leadership Computing Facility (ALCF) ALCF Resources: Machines & Storage Mira (Production) IBM Blue Gene/Q 49,152 nodes / 786,432 cores 768 TB of memory Peak flop rate:
More informationIntroduction to the SGE/OGS batch-queuing system
Grid Computing Competence Center Introduction to the SGE/OGS batch-queuing system Riccardo Murri Grid Computing Competence Center, Organisch-Chemisches Institut, University of Zurich Oct. 6, 2011 The basic
More informationInstalling and running COMSOL on a Linux cluster
Installing and running COMSOL on a Linux cluster Introduction This quick guide explains how to install and operate COMSOL Multiphysics 5.0 on a Linux cluster. It is a complement to the COMSOL Installation
More informationHigh Performance Computing Facility Specifications, Policies and Usage. Supercomputer Project. Bibliotheca Alexandrina
High Performance Computing Facility Specifications, Policies and Usage Supercomputer Project Bibliotheca Alexandrina Bibliotheca Alexandrina 1/16 Topics Specifications Overview Site Policies Intel Compilers
More informationUsing Parallel Computing to Run Multiple Jobs
Beowulf Training Using Parallel Computing to Run Multiple Jobs Jeff Linderoth August 5, 2003 August 5, 2003 Beowulf Training Running Multiple Jobs Slide 1 Outline Introduction to Scheduling Software The
More informationSLURM Workload Manager
SLURM Workload Manager What is SLURM? SLURM (Simple Linux Utility for Resource Management) is the native scheduler software that runs on ASTI's HPC cluster. Free and open-source job scheduler for the Linux
More informationUsing the Yale HPC Clusters
Using the Yale HPC Clusters Stephen Weston Robert Bjornson Yale Center for Research Computing Yale University Oct 2015 To get help Send an email to: hpc@yale.edu Read documentation at: http://research.computing.yale.edu/hpc-support
More informationIntroduction to Linux and Cluster Basics for the CCR General Computing Cluster
Introduction to Linux and Cluster Basics for the CCR General Computing Cluster Cynthia Cornelius Center for Computational Research University at Buffalo, SUNY 701 Ellicott St Buffalo, NY 14203 Phone: 716-881-8959
More informationHigh-Performance Reservoir Risk Assessment (Jacta Cluster)
High-Performance Reservoir Risk Assessment (Jacta Cluster) SKUA-GOCAD 2013.1 Paradigm 2011.3 With Epos 4.1 Data Management Configuration Guide 2008 2013 Paradigm Ltd. or its affiliates and subsidiaries.
More informationNew High-performance computing cluster: PAULI. Sascha Frick Institute for Physical Chemistry
New High-performance computing cluster: PAULI Sascha Frick Institute for Physical Chemistry 02/05/2012 Sascha Frick (PHC) HPC cluster pauli 02/05/2012 1 / 24 Outline 1 About this seminar 2 New Hardware
More informationParallel Computing using MATLAB Distributed Compute Server ZORRO HPC
Parallel Computing using MATLAB Distributed Compute Server ZORRO HPC Goals of the session Overview of parallel MATLAB Why parallel MATLAB? Multiprocessing in MATLAB Parallel MATLAB using the Parallel Computing
More informationManual for using Super Computing Resources
Manual for using Super Computing Resources Super Computing Research and Education Centre at Research Centre for Modeling and Simulation National University of Science and Technology H-12 Campus, Islamabad
More informationJob scheduler details
Job scheduler details Advanced Computing Center for Research & Education (ACCRE) Job scheduler details 1 / 25 Outline 1 Batch queue system overview 2 Torque and Moab 3 Submitting jobs (ACCRE) Job scheduler
More informationUsing Google Compute Engine
Using Google Compute Engine Chris Paciorek January 30, 2014 WARNING: This document is now out-of-date (January 2014) as Google has updated various aspects of Google Compute Engine. But it may still be
More informationHow to Run Parallel Jobs Efficiently
How to Run Parallel Jobs Efficiently Shao-Ching Huang High Performance Computing Group UCLA Institute for Digital Research and Education May 9, 2013 1 The big picture: running parallel jobs on Hoffman2
More informationIntroduction to SDSC systems and data analytics software packages "
Introduction to SDSC systems and data analytics software packages " Mahidhar Tatineni (mahidhar@sdsc.edu) SDSC Summer Institute August 05, 2013 Getting Started" System Access Logging in Linux/Mac Use available
More informationHigh Performance Computing
High Performance Computing at Stellenbosch University Gerhard Venter Outline 1 Background 2 Clusters 3 SU History 4 SU Cluster 5 Using the Cluster 6 Examples What is High Performance Computing? Wikipedia
More informationAdvanced PBS Workflow Example Bill Brouwer 05/01/12 Research Computing and Cyberinfrastructure Unit, PSU wjb19@psu.edu
Advanced PBS Workflow Example Bill Brouwer 050112 Research Computing and Cyberinfrastructure Unit, PSU wjb19@psu.edu 0.0 An elementary workflow All jobs consuming significant cycles need to be submitted
More informationTo connect to the cluster, simply use a SSH or SFTP client to connect to:
RIT Computer Engineering Cluster The RIT Computer Engineering cluster contains 12 computers for parallel programming using MPI. One computer, cluster-head.ce.rit.edu, serves as the master controller or
More informationCaltech Center for Advanced Computing Research System Guide: MRI2 Cluster (zwicky) January 2014
1. How to Get An Account CACR Accounts 2. How to Access the Machine Connect to the front end, zwicky.cacr.caltech.edu: ssh -l username zwicky.cacr.caltech.edu or ssh username@zwicky.cacr.caltech.edu Edits,
More informationRWTH GPU Cluster. Sandra Wienke wienke@rz.rwth-aachen.de November 2012. Rechen- und Kommunikationszentrum (RZ) Fotos: Christian Iwainsky
RWTH GPU Cluster Fotos: Christian Iwainsky Sandra Wienke wienke@rz.rwth-aachen.de November 2012 Rechen- und Kommunikationszentrum (RZ) The RWTH GPU Cluster GPU Cluster: 57 Nvidia Quadro 6000 (Fermi) innovative
More informationThe SUN ONE Grid Engine BATCH SYSTEM
The SUN ONE Grid Engine BATCH SYSTEM Juan Luis Chaves Sanabria Centro Nacional de Cálculo Científico (CeCalCULA) Latin American School in HPC on Linux Cluster October 27 November 07 2003 What is SGE? Is
More informationResource Management and Job Scheduling
Resource Management and Job Scheduling Jenett Tillotson Senior Cluster System Administrator Indiana University May 18 18-22 May 2015 1 Resource Managers Keep track of resources Nodes: CPUs, disk, memory,
More informationGrid Engine 6. Troubleshooting. BioTeam Inc. info@bioteam.net
Grid Engine 6 Troubleshooting BioTeam Inc. info@bioteam.net Grid Engine Troubleshooting There are two core problem types Job Level Cluster seems OK, example scripts work fine Some user jobs/apps fail Cluster
More informationBeyond Windows: Using the Linux Servers and the Grid
Beyond Windows: Using the Linux Servers and the Grid Topics Linux Overview How to Login & Remote Access Passwords Staying Up-To-Date Network Drives Server List The Grid Useful Commands Linux Overview Linux
More informationHow To Run A Tompouce Cluster On An Ipra (Inria) 2.5.5 (Sun) 2 (Sun Geserade) 2-5.4 (Sun-Ge) 2/5.2 (
Running Hadoop and Stratosphere jobs on TomPouce cluster 16 October 2013 TomPouce cluster TomPouce is a cluster of 20 calcula@on nodes = 240 cores Located in the Inria Turing building (École Polytechnique)
More informationParallel Processing using the LOTUS cluster
Parallel Processing using the LOTUS cluster Alison Pamment / Cristina del Cano Novales JASMIN/CEMS Workshop February 2015 Overview Parallelising data analysis LOTUS HPC Cluster Job submission on LOTUS
More informationHOD Scheduler. Table of contents
Table of contents 1 Introduction... 2 2 HOD Users... 2 2.1 Getting Started... 2 2.2 HOD Features...5 2.3 Troubleshooting... 14 3 HOD Administrators... 21 3.1 Getting Started... 22 3.2 Prerequisites...
More informationHPCC USER S GUIDE. Version 1.2 July 2012. IITS (Research Support) Singapore Management University. IITS, Singapore Management University Page 1 of 35
HPCC USER S GUIDE Version 1.2 July 2012 IITS (Research Support) Singapore Management University IITS, Singapore Management University Page 1 of 35 Revision History Version 1.0 (27 June 2012): - Modified
More informationMatlab on a Supercomputer
Matlab on a Supercomputer Shelley L. Knuth Research Computing April 9, 2015 Outline Description of Matlab and supercomputing Interactive Matlab jobs Non-interactive Matlab jobs Parallel Computing Slides
More informationBiowulf2 Training Session
Biowulf2 Training Session 9 July 2015 Slides at: h,p://hpc.nih.gov/docs/b2training.pdf HPC@NIH website: h,p://hpc.nih.gov System hardware overview What s new/different The batch system & subminng jobs
More informationOpenMP & MPI CISC 879. Tristan Vanderbruggen & John Cavazos Dept of Computer & Information Sciences University of Delaware
OpenMP & MPI CISC 879 Tristan Vanderbruggen & John Cavazos Dept of Computer & Information Sciences University of Delaware 1 Lecture Overview Introduction OpenMP MPI Model Language extension: directives-based
More informationUntil now: tl;dr: - submit a job to the scheduler
Until now: - access the cluster copy data to/from the cluster create parallel software compile code and use optimized libraries how to run the software on the full cluster tl;dr: - submit a job to the
More informationUsing the Windows Cluster
Using the Windows Cluster Christian Terboven terboven@rz.rwth aachen.de Center for Computing and Communication RWTH Aachen University Windows HPC 2008 (II) September 17, RWTH Aachen Agenda o Windows Cluster
More informationIntroduction to Grid Engine
Introduction to Grid Engine Workbook Edition 8 January 2011 Document reference: 3609-2011 Introduction to Grid Engine for ECDF Users Workbook Introduction to Grid Engine for ECDF Users Author: Brian Fletcher,
More informationParallelization: Binary Tree Traversal
By Aaron Weeden and Patrick Royal Shodor Education Foundation, Inc. August 2012 Introduction: According to Moore s law, the number of transistors on a computer chip doubles roughly every two years. First
More informationParallel Options for R
Parallel Options for R Glenn K. Lockwood SDSC User Services glock@sdsc.edu Motivation "I just ran an intensive R script [on the supercomputer]. It's not much faster than my own machine." Motivation "I
More informationMATLAB Distributed Computing Server Installation Guide. R2012a
MATLAB Distributed Computing Server Installation Guide R2012a How to Contact MathWorks www.mathworks.com Web comp.soft-sys.matlab Newsgroup www.mathworks.com/contact_ts.html Technical Support suggest@mathworks.com
More informationDebugging and Profiling Lab. Carlos Rosales, Kent Milfeld and Yaakoub Y. El Kharma carlos@tacc.utexas.edu
Debugging and Profiling Lab Carlos Rosales, Kent Milfeld and Yaakoub Y. El Kharma carlos@tacc.utexas.edu Setup Login to Ranger: - ssh -X username@ranger.tacc.utexas.edu Make sure you can export graphics
More informationCompute Cluster Server Lab 3: Debugging the parallel MPI programs in Microsoft Visual Studio 2005
Compute Cluster Server Lab 3: Debugging the parallel MPI programs in Microsoft Visual Studio 2005 Compute Cluster Server Lab 3: Debugging the parallel MPI programs in Microsoft Visual Studio 2005... 1
More informationGeneral Overview. Slurm Training15. Alfred Gil & Jordi Blasco (HPCNow!)
Slurm Training15 Agenda 1 2 3 About Slurm Key Features of Slurm Extending Slurm Resource Management Daemons Job/step allocation 4 5 SMP MPI Parametric Job monitoring Accounting Scheduling Administration
More informationHigh Performance Computing with Sun Grid Engine on the HPSCC cluster. Fernando J. Pineda
High Performance Computing with Sun Grid Engine on the HPSCC cluster Fernando J. Pineda HPSCC High Performance Scientific Computing Center (HPSCC) " The Johns Hopkins Service Center in the Dept. of Biostatistics
More informationMPI / ClusterTools Update and Plans
HPC Technical Training Seminar July 7, 2008 October 26, 2007 2 nd HLRS Parallel Tools Workshop Sun HPC ClusterTools 7+: A Binary Distribution of Open MPI MPI / ClusterTools Update and Plans Len Wisniewski
More informationNotes on the SNOW/Rmpi R packages with OpenMPI and Sun Grid Engine
Notes on the SNOW/Rmpi R packages with OpenMPI and Sun Grid Engine Last updated: 6/2/2008 4:43PM EDT We informally discuss the basic set up of the R Rmpi and SNOW packages with OpenMPI and the Sun Grid
More informationSummary. Load and Open GaussView to start example
Summary This document describes in great detail how to navigate the Linux Red Hat Terminal to bring up GaussView, use GaussView to create a simple atomic or molecular simulation input file, and then use
More informationMaxwell compute cluster
Maxwell compute cluster An introduction to the Maxwell compute cluster Part 1 1.1 Opening PuTTY and getting the course materials on to Maxwell 1.1.1 On the desktop, double click on the shortcut icon for
More informationDocumentation for hanythingondemand
Documentation for hanythingondemand Release 20151120.01 Ghent University Thu, 07 Jan 2016 12:53:15 Contents 1 Introductory topics 3 1.1 What is hanythingondemand?.................................... 3
More informationBatch Systems. provide a mechanism for submitting, launching, and tracking jobs on a shared resource
PBS INTERNALS PBS & TORQUE PBS (Portable Batch System)-software system for managing system resources on workstations, SMP systems, MPPs and vector computers. It was based on Network Queuing System (NQS)
More information24/08/2004. Introductory User Guide
24/08/2004 Introductory User Guide CSAR Introductory User Guide Introduction This material is designed to provide new users with all the information they need to access and use the SGI systems provided
More informationIRS: Implicit Radiation Solver Version 1.0 Benchmark Runs
IRS: Implicit Radiation Solver Version 1.0 Benchmark Runs This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.
More informationSecurEnvoy Security Server. SecurMail Solutions Guide
SecurEnvoy Security Server SecurMail Solutions Guide SecurMail Solutions Guide 2009 SecurEnvoy Printed: 2009 in United Kingdom Publisher SecurEnvoy Publishing Managing Editor SecurEnvoy Training Dept Technical
More informationCNT5106C Project Description
Last Updated: 1/30/2015 12:48 PM CNT5106C Project Description Project Overview In this project, you are asked to write a P2P file sharing software similar to BitTorrent. You can complete the project in
More informationHigh-Performance Computing
High-Performance Computing Windows, Matlab and the HPC Dr. Leigh Brookshaw Dept. of Maths and Computing, USQ 1 The HPC Architecture 30 Sun boxes or nodes Each node has 2 x 2.4GHz AMD CPUs with 4 Cores
More informationINF-110. GPFS Installation
INF-110 GPFS Installation Overview Plan the installation Before installing any software, it is important to plan the GPFS installation by choosing the hardware, deciding which kind of disk connectivity
More informationWINDOWS PROCESSES AND SERVICES
OBJECTIVES: Services o task manager o services.msc Process o task manager o process monitor Task Scheduler Event viewer Regedit Services: A Windows service is a computer program that operates in the background.
More informationCluster Computing With R
Cluster Computing With R Stowers Institute for Medical Research R/Bioconductor Discussion Group Earl F. Glynn Scientific Programmer 18 December 2007 1 Cluster Computing With R Accessing Linux Boxes from
More informationObelisk: Summoning Minions on a HPC Cluster
Obelisk: Summoning Minions on a HPC Cluster Abstract In scientific research, having the ability to perform rigorous calculations in a bearable amount of time is an invaluable asset. Fortunately, the growing
More informationTable of Contents New User Orientation...1
Table of Contents New User Orientation...1 Introduction...1 Helpful Resources...3 HPC Environment Overview...4 Basic Tasks...10 Understanding and Managing Your Allocations...16 New User Orientation Introduction
More informationThe Asterope compute cluster
The Asterope compute cluster ÅA has a small cluster named asterope.abo.fi with 8 compute nodes Each node has 2 Intel Xeon X5650 processors (6-core) with a total of 24 GB RAM 2 NVIDIA Tesla M2050 GPGPU
More informationThe XSEDE Global Federated File System (GFFS) - Breaking Down Barriers to Secure Resource Sharing
December 19, 2013 The XSEDE Global Federated File System (GFFS) - Breaking Down Barriers to Secure Resource Sharing Andrew Grimshaw, University of Virginia Co-architect XSEDE The complexity of software
More information8/15/2014. Best Practices @OLCF (and more) General Information. Staying Informed. Staying Informed. Staying Informed-System Status
Best Practices @OLCF (and more) Bill Renaud OLCF User Support General Information This presentation covers some helpful information for users of OLCF Staying informed Aspects of system usage that may differ
More information