HPC at IU Overview Abhinav Thota Research Technologies Indiana University
What is HPC/cyberinfrastructure?
Why should you care? Data sizes are growing Need to get to the solution faster Compute power is cheap Computing resources are FREE! But only if you parallelize Single core performance has plateaued
What s the hold up?
It s not that hard, but you need a manual You only need to learn a few commands to start with We will go start with a short overview and some practical details Along with a hands-on exercise By the end of the day you will be ready to run you own jobs on Quarry and Big Red II Don t forget to take the cheat sheet that will help you get started on Big Red II
Cyberinfrastructure at IU Supercomputers: Ø Big Red II Ø Quarry Ø Mason Research Database Complex Support & Consul?ng Ø High Performance Systems (Admin) Ø Scien?fic Apps and Perf. Tuning Ø Research Analy?cs Ø Storage Ø Visualiza?on hup://researchtech.iu.edu/ Storage: Ø Data Capacitor 2 high performance file system Ø Data Capacitor Wide Area Network Ø Research File System Ø Scholarly Data Archive SoRware sales - Sta?s?cal and Mathema?cal?tles Ø SPSS Ø SAS Ø Mathema?ca Ø MATLAB Ø Maple
Supercomputing Big Red II 46 th fastest in the world at debut, 62 nd now has both CPUs & GPUs For large parallel apps Quarry a general purpose cluster For serial apps Mason a large memory cluster 512 GB of memory/node For apps that need large memory Shoot us an email if you are unsure about which system to use for your work: sciapt@iu.edu Research Database Complex (RDC) host your research databases and data intensive applications Common terms: CPU, GPU, Core, Node
How do I get access? I am sure you all already did this Create an account: https://itaccounts.iu.edu/ Graduate students with hourly positions may not see BRII listed, please contact us for an account! ssh to: bigred2.uits.iu.edu quarry.uits.indiana.edu mason.indiana.edu IU network ID and pass phrase Read the message of the day (MOTD) hups://kb.iu.edu/d/achr
SSH and SCP Sounds great, where do I login? Not in a browser L SSH is used to login to a remote machine SCP is used to copy files to and from a remote machine Example: ssh username@bigred2.uits.iu.edu!! scp local-file username@bigred2.uits.iu.edu:/path/to/remote/dir! hups://kb.iu.edu/d/aelc hups://kb.iu.edu/d/agye
Environment in your login session Home directory: /N/u/<username>/BigRed2 Scratch space for temporary files: /N/dc2/scratch/<username> Bash shell Changes via hps-admin@iu.edu hups://kb.iu.edu/d/avmj
How to find and use software module command for adding software to your environment module avail module list module load <key> module unload <key> Permanently configure your environment via the.modules file Modules arranged in groups, development, applications, life sciences, etc. hups://kb.iu.edu/d/bcwy
Where else can I look for software For software already on the machine: https://cybergateway.uits.iu.edu/iugateway/searchmodulesinfo Also, on https://cybergateway.uits.iu.edu you can look at your usage stats (both compute and storage) Status of the machines Look at the queues, your jobs in the queue, etc. Statistical Software: http://rt.uits.iu.edu/visualiza?on/analy?cs/software.php HPC Licensed sorware: hup://rt.uits.iu.edu/systems/sciapt/licensed- sorware.php
Queues Hundreds of nodes and thousands of cores are shared among the users This is done with the help of a scheduler and queues Submit jobs: Put together the commands that you want to run Add a few PBS commands to go before the actual commands Submit the job Same system on all three systems Check your cheat sheet for sample scripts hups://kb.iu.edu/d/bdkd
Sample PBS script At the most basic level, on Quarry:! #!/bin/bash! #PBS -l nodes=2:ppn=6,walltime=30:00! cd /path/to/working/dir! mpirun -np 12 -machinefile $PBS_NODEFILE ~/bin/binaryname!! That s it! Just a couple of extra lines to tell PBS what you need! Put these lines in a text file and run qsub script.sh Some other useful commands to see what s going on with the queues: qstat Q #shows available queues qstat u username #shows the status of your queued jobs showq i #shows what s going to run next hups://kb.iu.edu/d/avmy
Interactive jobs For graphical applications you can launch GUIs For testing qsub I Followed by options to specify walltime, queue, etc. qsub I l walltime=00:30:00 q debug! Straightforward on Quarry Will land on an aprun node on Big Red II hups://kb.iu.edu/d/bdsi
How to get support? You can write us if you need help with anything that has to do with these machines Common questions: Help with installing a software How would I best go about doing xyz? I was thinking about xyz, would that work on Quarry? Can you help me do xyz? Most of the answers to beginner level questions are out there on the KB pages already Just need to search and then search again with different terms J
Support via email, phone, in-person We use a ticketing system for questions via email Can call us on the phone, can meet in person We are in the CIB RA is on campus too in the SSRC Personalized presentations and talks for research groups Workshops every semester
How much does this cost? The computing resources and storage are free for normal use (Unless you need huge amounts of dedicated compute time or large data storage) Short term support and consulting is also free Involves help with day to day issues Installing software Troubleshooting problems, etc Long term support and consulting is possible on a paid basis
Contact information Here are a few email addresses, but don t worry, write any one of us and we will get your question to the right people HPS (Admins): hps-admin@iu.edu Scientific Apps: sciapt@iu.edu Research Analytics: researchanalytics@iu.edu HPFS (DC2): hpfs-admin@iu.edu SDA: store-admin@iu.edu Visualization: vishelp@iu.edu