MFCF Grad Session 2015

MFCF Grad Session 2015

Agenda Introduction Help Centre and requests Dept. Grad reps Linux clusters using R with MPI Remote applications Future computing direction Technical question and answer period

MFCF Help Centre MC 3017 Help with Mac minis Laptop connections Printing, accounts... Grad rep - meetings once a term communication between MFCF and grads regarding computing - new developments Request Tracker Lynda.com - online training

Computing Environments Accessing different environments from the Mac mini all share the same file space on files.math - cifs://files.math.uwaterloo.ca simply accessing your files through XQuartz or Terminal - ssh to linux.math, biglinux.math and fastlinux.math which have far more memory and computing power Remote Desktop Connection- windows.math, or another windows machine

Linux environment Brief overview of linux machines: Operating system CentOS6 UNIX shell scripting (default is bash) Access the machines using ssh Uploading your files Background jobs Web home page

Overview - fastlinux What is fastlinux: four computers with fastest cpus, but only 2 cores per machine + less memory What is fastlinux for: jobs where processor speed is most important

Overview- biglinux What is biglinux: two computers (cpu125 and cpu131) What is biglinux used for: very large or multi-threaded jobs Some specifications: 256Gb of RAM in total (with ECC). 4 processors x 12 cores (AMD Opteron 6168) = 48 cores cpu131 is 512 Mb, 64 cores

Access Linux using ssh You will need: uwuserid@uwaterloo.ca password: same as your email password From Mac mini and Linux: open up a terminal window and type: ssh uwuserid@linux.math.uwaterloo.ca you will be prompted for your password

Uploading your files When you first log in, you will likely be in the directory /u/uwuserid (truncated to 8 characters) Your usual MacProfile directory on the Mac mini is /u/uwuserid/macprofile, so you can also place your files in your MacProfile directory to upload as well If you do not have access to your Mac mini, to upload files (e.g. R scripts and data sets), type at your own terminal, not biglinux > scp /path/to/your/file/on/local/machine/file.ext uwuserid@biglinux.math.uwaterloo.ca: /path/to/destination

Background jobs How to free up command line to run more commands How to leave a job running when you log out batch command nohup command

Putting a job in the background - if you do this: $ firefox you can't run another command until firefox exits - if you put an ampersand on the end: $ firefox & it runs in background and you can type more commands - see what jobs you have in the background: $ jobs - e.g. bring the third one to the foreground again: $ fg %3 - e.g. kill the second one: $ kill %2

Leaving a job running after log-out: batch submits job to a queue - runs when conditions allow (load, etc.) - sends you mail about status - restarts if interrupted - syntax: batch queuename -c "command options..." - examples: % batch big -c "./my_big_job.a <infile >outfile" % batch long -c "./my_long_job.a <in >out" % batchstatus % batchcancel -q big 1234 - see "man batch"

Leave a job running after log-out: nohup - normally, log- out sends "hang up" (HUP) to child processes - "nohup" command blocks HUP signal - syntax: just put 'nohup' in front of usual command line - example: % nohup./myprog.a <infile >outfile % nohup matlab - nojvm - r [a,b,c]=myarray - logfile myarray.log % logout

Creating a personal web page 1. Create public_html directory in Unix account on any linux.math machine % cd % mkdir public_html 2. Create an index.html file there % cd public_html % vi index.html... 3. Ensure files and directory are world-readable % chmod a+r. % chmod a+r index.html 4. Now this URL will work: http://www.math.uwaterloo.ca/~uwuserid/ Note: Must have 'www' in there, otherwise it's a different web server.

Remote Apps What is it? allows applications to run on a remote server (windows.math.uwaterloo.ca) Why would I want to use it? faster computers, more memory menu of available applications allows big applications (R, Maple and Matlab) to run with less impact on the general computing environment Only one session (at a time)

Remote Apps Three ways to connect Web page h"ps://remoteapp.math.uwaterloo.ca/ Remote Desktop Client (RDC) Windows, MacOS X, Apple ios, Android, Blackberry OS 10 but not Linux Start menu Microsoft Windows 7, 8, 10, 2008 and 2012 h"ps://remoteapp.math.uwaterloo.ca/rdweb/feed/webfeed.aspx

Remote Apps Caveats Virtual Private Network (VPN) to access Remote apps from off-campus Disconnected Sessions reconnect exactly where you were when you were disconnected Session Timeouts not automatically timed out if apps are running, two hours if no apps are running Scheduled reboots and long running tasks Storage always store all your files in your files.math.uwaterloo.ca space Questions??

Future computing direction Marek Stastna Director of MFCF Associate Dean, Computing

Happy Halloween uwaterloo.ca/mfcf/resources/graduate-students