Introduction to Linux and Cluster Basics for the CCR General Computing Cluster

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Introduction to Linux and Cluster Basics for the CCR General Computing Cluster"

Transcription

1 Introduction to Linux and Cluster Basics for the CCR General Computing Cluster Cynthia Cornelius Center for Computational Research University at Buffalo, SUNY 701 Ellicott St Buffalo, NY Phone: Office: B1-109 cdc at ccr.buffalo.edu Fall 2013

2 CCR Resources CCR provides over 8000 processors in Linux cluster to the UB academic community. RedHat 6 High speed network: Infiniband Extensive store and data backup is available. The cluster environment is a command line interface for the most part. WebMO web portal Cluster machines are accessed through a batch scheduler.

3 Cluster Resources 372 nodes 12 cores with 48GB memory 256 nodes 8 cores with 24GB memory 32 nodes 16 processors with 128GB of memory 18 nodes 32 cores with 256GB memory 1 node 32 cores with 512GB memory 32 GPU nodes 12 cores with 48GB memory 2 GPUs per node NVIDIA Fermi M2050 (CUDA) more information on CCR cluster

4 Data Storage User home directories, group project directories, and application software are accessible to all compute nodes in the CCR cluster. The path to a user s home directory is /user/ubitusername/ The default user quota for a home directory is 2GB. There is a daily backup of home directories. The iquota command will show the usage and quota limit. iquota u UBITusername

5 Data Storage Project directories are available to UB research group. There is a daily backup of data files in the projects directory. A project directory starts at 200GB. The storage can be increased to 500GB without charge, after which there is a fee. The iquota command will show the usage and quota limit. -iquota p group UB Professors can request a projects directory by sending to ccr-help.

6 Scratch Space All scratch spaces are for temporary use. Users can create files and directories in the scratch spaces. The /panasas/scratch file system is available on all nodes in the rush cluster. /panasas/scratch is 180TB. Files older than 2 weeks are automatically removed. There is no backup of data files in /panasas/scratch All compute nodes have the a local /scratch directory. User files are removed when the job exists.

7 Accessing the Cluster The CCR cluster front-end machine is rush.ccr.buffalo.edu Compute nodes are not directly accessible from outside the cluster. The storage spaces is not accessible from outside the cluster This login machine is accessible from computers on the UB network through SSH for login, and sftp or scp for file transfer. ssh, sftp, scp, and X are usually installed on Linux and Mac workstations by default.

8 Accessing the Clusters Windows users must install X-Win32 or PuTTY for the secure login. The X-Win32 program allows for a graphical display. Filezilla is graphical file transfer program, which is available for Windows, Linux and MAC computers. Be sure to specify port 22 to secure login. UBVPN must be used to access the rush login machine from off campus. Users should setup UB-Secure on their laptops for wireless. more on UB software

9 Login to the front-end of the cluster The front-end machine is intended for login to the cluster, compiling code, and submitting jobs. The login machine is a 32-core node with 256GB of memory. This machine is available for short and small scale computations. These computations should be less than 20 minutes. Usage of memory and CPU utilization should be monitored.

10 Accessing the Clusters Use the MyCCR web interface to change or reset the CCR password. more on MyCCR Login from Linux or Mac ssh -X rush.ccr.buffalo.edu ssh -X The -X flag enables a graphical display. This is optional.

11 Accessing the Clusters Details for X-Win32 Select Manual in the configuration window. Select ssh Connection name: rush Host: rush.ccr.buffalo.edu Login: your UBITusername Command: /usr/bin/xterm ls Password: CCR password Be sure to run only one X-Win32.

12 Command Line Environment The rush clusters is command line UNIX environment. The user s login executes a shell. This is a process that accepts and executes the commands typed by the user. By default the shell is set to bash. The.bashrc file is executed on login. This script resides in the user s home directory. Variables and paths can be set in the.bashrc file. It is also executed when a job starts on a compute node.

13 Command Line Environment List the contents of a directory Show details: ls -l Displays ownership and permissions. Show details and hidden files: ls la Show details in reverse time order: ls -latr Show current directory pathname pwd Change directory Change to given pathname: cd /pathname Change to one directory above: cd.. Change to two directories above: cd../.. Change to home directory: cd Create a directory mkdir dirname

14 Command Line Environment Remove Remove a file: rm filename Remove a directory: rm -R dirname If the directory is empty: rmdir dirname Copy Copy a file: cp oldfile newfile Copy a directory: cp -R olddir newdir View a file Display file to the screen: cat filename Display with page breaks: more filename Press space bar to page through the file. Press enter to advance one line. Forward direction only. Control c to quit.

15 Command Line Environment All directories and files have permissions. Users can set the permissions to allow or deny access to their files and directories. Permissions are set for the user (u), the group (g) and everyone else (o). The permissions are read, write and execute. Read (r) permission allows viewing and copying. Write (w) permission allows changing and deleting. Execute (x) permission allows execution of a file and access to a directory.

16 Command Line Environment Show permissions with ls -l. This will also show the user and group associated with the file or directory. drwxrwxrwx -rwxrwxrwx The left most is for a file and d for a directory. The next three are read, write and execute for the user (owner). The middle three are read, write and execute for the group. These permissions set the access for all members of the group. The rightmost three are read, write and execute for other.

17 Command Line Environment Change permissions with the chmod command. chmod ugo+rwx filename chmod ugo-rwx filename ugo is user, group and other rwx is read, write and execute + add a permission - removes a permission Adding write permission for group: chmod g+w filename Removing all permissions for group and other on a directory and its contents. chmod -R go-rwx dirname

18 Command Line Environment The grep command is used to search for a pattern. grep pattern file df will show the disk usage. df -h du will show the estimated disk space usage. du -s -k `ls` sort -rn more This will list the file and directories using the most space first. The pipe symbol is used to indicate that the output of the first command should be used as input to the second.

19 Command Line Environment The asterisk is a wild card for many UNIX commands. List all C files: ls *.c Show the type of file: file filename Manual pages are available for UNIX commands. man ls There are several editors available. emacs, nano and vi emacs can provide a graphical interface.

20 Command Line Environment Files edited on Windows machines can contain hidden characters, which may cause runtime problems. Use the file command to show the type of file. If the file is a DOS file, then use the dos2unix command to remove any hidden characters. See the CCR user guide for more information. more on the User Guide CCR Command Line Reference Card

21 Application Software A large number of software applications are installed on the cluster. more on Software Intel, GNU and PGI compilers CUDA is available for the GPU nodes. more on Compilers Parallel Programming support Message Passing Interface (MPI) OpenMP more MPI and Multi-thread

22 Managing the Environment Software modules are used to manage the user environment. A module is used to set paths, library paths and variables. Most applications installed on the clusters have a module. Loading a module adds the path for the application to path variables and sets variables. Unloading a module removes the path and unsets the variables.

23 Managing the Environment List all available modules module avail module avail intel List modules currently loaded module list Show what a module will do module show module-name Load a module module load module-name Unload a module module unload module-name

24 Managing the Environment Creating your own module. Create the privatemodules directory in your home directory. Copy the template or other module to the privatemodules directory. Edit the module. Using your own module. Load the use.own module. module load use.own This module will be listed by module avail. Load the module. module load your-module-name

25 Managing the Environment Modules can be loaded in the.bashrc file. The module would be loaded automatically when you login. This is helpful if you know that you will always want that module to be loaded. Adding module list is helpful to remind you of the modules that were loaded. Modules are loaded in batch scripts that run jobs on the clusters. Be sure to load the same module that was used in the compilation of the code.

26 Compilers The GNU compilers can be the easiest to use when building some codes. There are no modules for the GNU compilers. These compilers and libraries are part of the Operating System and are in the default path. C compiler: gcc gcc -o hello-gnu hello.c C++ compiler: g++ Fortran compilers: g77, gfortran gfortran is a Fortran 95 compiler.

27 Compilers Typically codes built with the Intel compilers can have the best performance. The Intel compilers can take advantage of the processor and core architecture and features. Load the module for the intel compiler: module load intel C compiler: icc icc -o hello-intel hello.c C++ compiler: icpc Fortran compiler: ifort

28 Compilers The PGI (Portland Group) compilers are also available. Load the module for the PGI compiler: module load pgi C compiler: pgcc pgcc -o hello-pgi hello.c C++ compiler: pgcc Fortran compilers: pgf77, pgf90, pgf95

29 Message Passing Interface There are several implementations of the Message Passing Interface Protocol available on the cluster. Typically, Intel MPI is the best performing and most reliable implementation for running parallel computations on the cluster. Show the Intel MPI modules: module avail intel-mpi Always use the most recent version. Intel MPI can be used with the Intel, PGI and GNU compilers. Use wrappers to compile code: mpicc o cpi-gnu cpi.c

30 Numerical Libraries Intel Math Kernel Libraries (MKL) MKL provides highly tuned BLAS, LAPACK, DFTs, BLACS and ScaLAPACK Intel MKL linking tool is very useful in setting up the linking to the libraries. PETSc (Portable Extensible Toolkit for Scientific Computation) PETSc is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations. more on Numerical Libraries

31 Running Computations The CCR SLURM User Guide has the most complete and recent information for running and monitoring jobs on the CCR Cluster. CCR SLURM User Guide Code, scripts, READMEs and pdfs for download on the right-hand side. Navigation to SLURM topics on the left-hand side. Please do not use any instructions for PBS. We no longer support that scheduler.

32 Getting Help Online resources: User Guide New User Information Tutorials and Presentations more on getting help

Introduction to Running Hadoop on the High Performance Clusters at the Center for Computational Research

Introduction to Running Hadoop on the High Performance Clusters at the Center for Computational Research Introduction to Running Hadoop on the High Performance Clusters at the Center for Computational Research Cynthia Cornelius Center for Computational Research University at Buffalo, SUNY 701 Ellicott St

More information

Introduction to Running Computations on the High Performance Clusters at the Center for Computational Research

Introduction to Running Computations on the High Performance Clusters at the Center for Computational Research ! Introduction to Running Computations on the High Performance Clusters at the Center for Computational Research! Cynthia Cornelius! Center for Computational Research University at Buffalo, SUNY! cdc at

More information

The Asterope compute cluster

The Asterope compute cluster The Asterope compute cluster ÅA has a small cluster named asterope.abo.fi with 8 compute nodes Each node has 2 Intel Xeon X5650 processors (6-core) with a total of 24 GB RAM 2 NVIDIA Tesla M2050 GPGPU

More information

Linux für bwgrid. Sabine Richling, Heinz Kredel. Universitätsrechenzentrum Heidelberg Rechenzentrum Universität Mannheim. 27.

Linux für bwgrid. Sabine Richling, Heinz Kredel. Universitätsrechenzentrum Heidelberg Rechenzentrum Universität Mannheim. 27. Linux für bwgrid Sabine Richling, Heinz Kredel Universitätsrechenzentrum Heidelberg Rechenzentrum Universität Mannheim 27. June 2011 Richling/Kredel (URZ/RUM) Linux für bwgrid FS 2011 1 / 33 Introduction

More information

Using WestGrid. Patrick Mann, Manager, Technical Operations Jan.15, 2014

Using WestGrid. Patrick Mann, Manager, Technical Operations Jan.15, 2014 Using WestGrid Patrick Mann, Manager, Technical Operations Jan.15, 2014 Winter 2014 Seminar Series Date Speaker Topic 5 February Gino DiLabio Molecular Modelling Using HPC and Gaussian 26 February Jonathan

More information

Manual for using Super Computing Resources

Manual for using Super Computing Resources Manual for using Super Computing Resources Super Computing Research and Education Centre at Research Centre for Modeling and Simulation National University of Science and Technology H-12 Campus, Islamabad

More information

NEC HPC-Linux-Cluster

NEC HPC-Linux-Cluster NEC HPC-Linux-Cluster Hardware configuration: 4 Front-end servers: each with SandyBridge-EP processors: 16 cores per node 128 GB memory 134 compute nodes: 112 nodes with SandyBridge-EP processors (16 cores

More information

Cisco Networking Academy Program Curriculum Scope & Sequence. Fundamentals of UNIX version 2.0 (July, 2002)

Cisco Networking Academy Program Curriculum Scope & Sequence. Fundamentals of UNIX version 2.0 (July, 2002) Cisco Networking Academy Program Curriculum Scope & Sequence Fundamentals of UNIX version 2.0 (July, 2002) Course Description: Fundamentals of UNIX teaches you how to use the UNIX operating system and

More information

Agenda. Using HPC Wales 2

Agenda. Using HPC Wales 2 Using HPC Wales Agenda Infrastructure : An Overview of our Infrastructure Logging in : Command Line Interface and File Transfer Linux Basics : Commands and Text Editors Using Modules : Managing Software

More information

UNIX - USEFUL COMMANDS

UNIX - USEFUL COMMANDS UNIX - USEFUL COMMANDS http://www.tutorialspoint.com/unix/unix-useful-commands.htm Copyright tutorialspoint.com This quick guide lists commands, including a syntax and brief description. For more detail,

More information

Cluster@WU User s Manual

Cluster@WU User s Manual Cluster@WU User s Manual Stefan Theußl Martin Pacala September 29, 2014 1 Introduction and scope At the WU Wirtschaftsuniversität Wien the Research Institute for Computational Methods (Forschungsinstitut

More information

Getting Started with HPC

Getting Started with HPC Getting Started with HPC An Introduction to the Minerva High Performance Computing Resource 17 Sep 2013 Outline of Topics Introduction HPC Accounts Logging onto the HPC Clusters Common Linux Commands Storage

More information

Tutorial Guide to the IS Unix Service

Tutorial Guide to the IS Unix Service Tutorial Guide to the IS Unix Service The aim of this guide is to help people to start using the facilities available on the Unix and Linux servers managed by Information Services. It refers in particular

More information

Navigating within the GNU/Linux Filesystem

Navigating within the GNU/Linux Filesystem Navigating within the GNU/Linux Filesystem The purpose of this section is to provide a few basic GNU/Linux commands to aide new users locate areas on the ODU Turing cluster. The commands presented are

More information

Introduction to ACENET Accelerating Discovery with Computational Research May, 2015

Introduction to ACENET Accelerating Discovery with Computational Research May, 2015 Introduction to ACENET Accelerating Discovery with Computational Research May, 2015 What is ACENET? What is ACENET? Shared regional resource for... high-performance computing (HPC) remote collaboration

More information

Tutorial 0A Programming on the command line

Tutorial 0A Programming on the command line Tutorial 0A Programming on the command line Operating systems User Software Program 1 Program 2 Program n Operating System Hardware CPU Memory Disk Screen Keyboard Mouse 2 Operating systems Microsoft Apple

More information

24/08/2004. Introductory User Guide

24/08/2004. Introductory User Guide 24/08/2004 Introductory User Guide CSAR Introductory User Guide Introduction This material is designed to provide new users with all the information they need to access and use the SGI systems provided

More information

MPATE-GE 2618: C Programming for Music Technology. Unit 1.1

MPATE-GE 2618: C Programming for Music Technology. Unit 1.1 MPATE-GE 2618: C Programming for Music Technology Unit 1.1 What is an algorithm? An algorithm is a precise, unambiguous procedure for producing certain results (outputs) from given data (inputs). It is

More information

HPC Wales Skills Academy Course Catalogue 2015

HPC Wales Skills Academy Course Catalogue 2015 HPC Wales Skills Academy Course Catalogue 2015 Overview The HPC Wales Skills Academy provides a variety of courses and workshops aimed at building skills in High Performance Computing (HPC). Our courses

More information

CNAG User s Guide. Barcelona Supercomputing Center Copyright c 2015 BSC-CNS December 18, 2015. 1 Introduction 2

CNAG User s Guide. Barcelona Supercomputing Center Copyright c 2015 BSC-CNS December 18, 2015. 1 Introduction 2 CNAG User s Guide Barcelona Supercomputing Center Copyright c 2015 BSC-CNS December 18, 2015 Contents 1 Introduction 2 2 System Overview 2 3 Connecting to CNAG cluster 2 3.1 Transferring files...................................

More information

Unix Sampler. PEOPLE whoami id who

Unix Sampler. PEOPLE whoami id who Unix Sampler PEOPLE whoami id who finger username hostname grep pattern /etc/passwd Learn about yourself. See who is logged on Find out about the person who has an account called username on this host

More information

A Crash course to (The) Bighouse

A Crash course to (The) Bighouse A Crash course to (The) Bighouse Brock Palen brockp@umich.edu SVTI Users meeting Sep 20th Outline 1 Resources Configuration Hardware 2 Architecture ccnuma Altix 4700 Brick 3 Software Packaged Software

More information

Although Mac OS X is primarily known for its GUI, the under pinnings are all Unix. This

Although Mac OS X is primarily known for its GUI, the under pinnings are all Unix. This BE Computing Web Tutorials: Server Commands Server Commands Indluded: 1. Basic Command Line Tutorial Although Mac OS X is primarily known for its GUI, the underpinnings are all Unix. This tutorial will

More information

University of Toronto

University of Toronto 1 University of Toronto APS 105 Computer Fundamentals A Tutorial about UNIX Basics Fall 2011 I. INTRODUCTION This document serves as your introduction to the computers we will be using in this course.

More information

Tutorial on Linux Basics. KARUNYA LINUX CLUB

Tutorial on Linux Basics. KARUNYA LINUX CLUB Tutorial on Linux Basics KARUNYA LINUX CLUB www.karunya.edu/linuxclub Outline 1. Overview of Linux System 2. Basic Commands 3. Relative & Absolute Path 4. Redirect, Append and Pipe 5. Permission 6. Process

More information

Unix tutorial. Introduction to Unix. VI, March Page 1

Unix tutorial. Introduction to Unix.  VI, March Page 1 Unix tutorial Introduction to Unix http://linux.oreilly.com/ Page 1 Unix tutorial Outline Basic background in Unix structure Directories and files I (listing, navigation, filenames) Directories and files

More information

Introduction to Supercomputing with Janus

Introduction to Supercomputing with Janus Introduction to Supercomputing with Janus Shelley Knuth shelley.knuth@colorado.edu Peter Ruprecht peter.ruprecht@colorado.edu www.rc.colorado.edu Outline Who is CU Research Computing? What is a supercomputer?

More information

Tutorial: Using WestGrid. Drew Leske Compute Canada/WestGrid Site Lead University of Victoria

Tutorial: Using WestGrid. Drew Leske Compute Canada/WestGrid Site Lead University of Victoria Tutorial: Using WestGrid Drew Leske Compute Canada/WestGrid Site Lead University of Victoria Fall 2013 Seminar Series Date Speaker Topic 23 September Lindsay Sill Introduction to WestGrid 9 October Drew

More information

1.0. User Manual For HPC Cluster at GIKI. Volume. Ghulam Ishaq Khan Institute of Engineering Sciences & Technology

1.0. User Manual For HPC Cluster at GIKI. Volume. Ghulam Ishaq Khan Institute of Engineering Sciences & Technology Volume 1.0 FACULTY OF CUMPUTER SCIENCE & ENGINEERING Ghulam Ishaq Khan Institute of Engineering Sciences & Technology User Manual For HPC Cluster at GIKI Designed and prepared by Faculty of Computer Science

More information

An Introduction to High Performance Computing in the Department

An Introduction to High Performance Computing in the Department An Introduction to High Performance Computing in the Department Ashley Ford & Chris Jewell Department of Statistics University of Warwick October 30, 2012 1 Some Background 2 How is Buster used? 3 Software

More information

JUROPA Linux Cluster An Overview. 19 May 2014 Ulrich Detert

JUROPA Linux Cluster An Overview. 19 May 2014 Ulrich Detert Mitglied der Helmholtz-Gemeinschaft JUROPA Linux Cluster An Overview 19 May 2014 Ulrich Detert JuRoPA JuRoPA Jülich Research on Petaflop Architectures Bull, Sun, ParTec, Intel, Mellanox, Novell, FZJ JUROPA

More information

Streamline Computing Linux Cluster User Training. ( Nottingham University)

Streamline Computing Linux Cluster User Training. ( Nottingham University) 1 Streamline Computing Linux Cluster User Training ( Nottingham University) 3 User Training Agenda System Overview System Access Description of Cluster Environment Code Development Job Schedulers Running

More information

New High-performance computing cluster: PAULI. Sascha Frick Institute for Physical Chemistry

New High-performance computing cluster: PAULI. Sascha Frick Institute for Physical Chemistry New High-performance computing cluster: PAULI Sascha Frick Institute for Physical Chemistry 02/05/2012 Sascha Frick (PHC) HPC cluster pauli 02/05/2012 1 / 24 Outline 1 About this seminar 2 New Hardware

More information

C++ Programming on Linux

C++ Programming on Linux C++ Programming on Linux What is Linux?! an operating system! Unix-like CS 2308 Spring 2013 Jill Seaman! Open source! created in 1992 by Linus Torvolds! can be installed on a wide variety of hardware mobile

More information

Frequently Used UNIX Commands

Frequently Used UNIX Commands Frequently Used UNIX Commands Command ls Function Lists directory contents. ls -a List directory contents including hidden files. ls -l List directory contents using a long listing format that indicates

More information

Beyond Windows: Using the Linux Servers and the Grid

Beyond Windows: Using the Linux Servers and the Grid Beyond Windows: Using the Linux Servers and the Grid Topics Linux Overview How to Login & Remote Access Passwords Staying Up-To-Date Network Drives Server List The Grid Useful Commands Linux Overview Linux

More information

High Performance Computing Cluster Quick Reference User Guide

High Performance Computing Cluster Quick Reference User Guide High Performance Computing Cluster Quick Reference User Guide Base Operating System: Redhat(TM) / Scientific Linux 5.5 with Alces HPC Software Stack Copyright 2011 Alces Software Ltd All Rights Reserved

More information

OLCF Best Practices. Bill Renaud OLCF User Assistance Group

OLCF Best Practices. Bill Renaud OLCF User Assistance Group OLCF Best Practices Bill Renaud OLCF User Assistance Group Overview This presentation covers some helpful information for users of OLCF Staying informed Some aspects of system usage that may differ from

More information

Command Line - Part 1

Command Line - Part 1 Command Line - Part 1 STAT 133 Gaston Sanchez Department of Statistics, UC Berkeley gastonsanchez.com github.com/gastonstat Course web: gastonsanchez.com/teaching/stat133 GUIs 2 Graphical User Interfaces

More information

Unix Guide. Logo Reproduction. School of Computing & Information Systems. Colours red and black on white backgroun

Unix Guide. Logo Reproduction. School of Computing & Information Systems. Colours red and black on white backgroun Logo Reproduction Colours red and black on white backgroun School of Computing & Information Systems Unix Guide Mono positive black on white background 2013 Mono negative white only out of any colou 2

More information

Using NeSI HPC Resources. NeSI Computational Science Team (support@nesi.org.nz)

Using NeSI HPC Resources. NeSI Computational Science Team (support@nesi.org.nz) NeSI Computational Science Team (support@nesi.org.nz) Outline 1 About Us About NeSI Our Facilities 2 Using the Cluster Suitable Work What to expect Parallel speedup Data Getting to the Login Node 3 Submitting

More information

The CNMS Computer Cluster

The CNMS Computer Cluster The CNMS Computer Cluster This page describes the CNMS Computational Cluster, how to access it, and how to use it. Introduction (2014) The latest block of the CNMS Cluster (2010) Previous blocks of the

More information

HPCC USER S GUIDE. Version 1.2 July 2012. IITS (Research Support) Singapore Management University. IITS, Singapore Management University Page 1 of 35

HPCC USER S GUIDE. Version 1.2 July 2012. IITS (Research Support) Singapore Management University. IITS, Singapore Management University Page 1 of 35 HPCC USER S GUIDE Version 1.2 July 2012 IITS (Research Support) Singapore Management University IITS, Singapore Management University Page 1 of 35 Revision History Version 1.0 (27 June 2012): - Modified

More information

PuTTY/Cygwin Tutorial. By Ben Meister Written for CS 23, Winter 2007

PuTTY/Cygwin Tutorial. By Ben Meister Written for CS 23, Winter 2007 PuTTY/Cygwin Tutorial By Ben Meister Written for CS 23, Winter 2007 This tutorial will show you how to set up and use PuTTY to connect to CS Department computers using SSH, and how to install and use the

More information

Introduction to MSI* for PubH 8403

Introduction to MSI* for PubH 8403 Introduction to MSI* for PubH 8403 Sep 30, 2015 Nancy Rowe *The Minnesota Supercomputing Institute for Advanced Computational Research Overview MSI at a Glance MSI Resources Access System Access - Physical

More information

Lab 1 Beginning C Program

Lab 1 Beginning C Program Lab 1 Beginning C Program Overview This lab covers the basics of compiling a basic C application program from a command line. Basic functions including printf() and scanf() are used. Simple command line

More information

Introduction to HPC Workshop. Center for e-research (eresearch@nesi.org.nz)

Introduction to HPC Workshop. Center for e-research (eresearch@nesi.org.nz) Center for e-research (eresearch@nesi.org.nz) Outline 1 About Us About CER and NeSI The CS Team Our Facilities 2 Key Concepts What is a Cluster Parallel Programming Shared Memory Distributed Memory 3 Using

More information

Training Day : Linux

Training Day : Linux Training Day : Linux Objectives At the end of the day, you will be able to use Linux command line in order to : Connect to «genotoul» server Use available tools Transfer files between server and desktop

More information

Cluster performance, how to get the most out of Abel. Ole W. Saastad, Dr.Scient USIT / UAV / FI April 18 th 2013

Cluster performance, how to get the most out of Abel. Ole W. Saastad, Dr.Scient USIT / UAV / FI April 18 th 2013 Cluster performance, how to get the most out of Abel Ole W. Saastad, Dr.Scient USIT / UAV / FI April 18 th 2013 Introduction Architecture x86-64 and NVIDIA Compilers MPI Interconnect Storage Batch queue

More information

Linux command line. An introduction to the Linux command line for genomics. Susan Fairley

Linux command line. An introduction to the Linux command line for genomics. Susan Fairley Linux command line An introduction to the Linux command line for genomics Susan Fairley Aims Introduce the command line Provide an awareness of basic functionality Illustrate with some examples Provide

More information

The RWTH Compute Cluster Environment

The RWTH Compute Cluster Environment The RWTH Compute Cluster Environment Tim Cramer 11.03.2013 Source: D. Both, Bull GmbH Rechen- und Kommunikationszentrum (RZ) How to login Frontends cluster.rz.rwth-aachen.de cluster-x.rz.rwth-aachen.de

More information

Parallel Processing using the LOTUS cluster

Parallel Processing using the LOTUS cluster Parallel Processing using the LOTUS cluster Alison Pamment / Cristina del Cano Novales JASMIN/CEMS Workshop February 2015 Overview Parallelising data analysis LOTUS HPC Cluster Job submission on LOTUS

More information

INF-110. GPFS Installation

INF-110. GPFS Installation INF-110 GPFS Installation Overview Plan the installation Before installing any software, it is important to plan the GPFS installation by choosing the hardware, deciding which kind of disk connectivity

More information

Secure Shell. The Protocol

Secure Shell. The Protocol Usually referred to as ssh The name is used for both the program and the protocol ssh is an extremely versatile network program data encryption and compression terminal access to remote host file transfer

More information

SLURM Workload Manager

SLURM Workload Manager SLURM Workload Manager What is SLURM? SLURM (Simple Linux Utility for Resource Management) is the native scheduler software that runs on ASTI's HPC cluster. Free and open-source job scheduler for the Linux

More information

An Introduction to Using the Command Line Interface (CLI) to Work with Files and Directories

An Introduction to Using the Command Line Interface (CLI) to Work with Files and Directories An Introduction to Using the Command Line Interface (CLI) to Work with Files and Directories Mac OS by bertram lyons senior consultant avpreserve AVPreserve Media Archiving & Data Management Consultants

More information

Mathematical Libraries and Application Software on JUROPA and JUQUEEN

Mathematical Libraries and Application Software on JUROPA and JUQUEEN Mitglied der Helmholtz-Gemeinschaft Mathematical Libraries and Application Software on JUROPA and JUQUEEN JSC Training Course May 2014 I.Gutheil Outline General Informations Sequential Libraries Parallel

More information

Grid 101. Grid 101. Josh Hegie. grid@unr.edu http://hpc.unr.edu

Grid 101. Grid 101. Josh Hegie. grid@unr.edu http://hpc.unr.edu Grid 101 Josh Hegie grid@unr.edu http://hpc.unr.edu Accessing the Grid Outline 1 Accessing the Grid 2 Working on the Grid 3 Submitting Jobs with SGE 4 Compiling 5 MPI 6 Questions? Accessing the Grid Logging

More information

RWTH GPU Cluster. Sandra Wienke wienke@rz.rwth-aachen.de November 2012. Rechen- und Kommunikationszentrum (RZ) Fotos: Christian Iwainsky

RWTH GPU Cluster. Sandra Wienke wienke@rz.rwth-aachen.de November 2012. Rechen- und Kommunikationszentrum (RZ) Fotos: Christian Iwainsky RWTH GPU Cluster Fotos: Christian Iwainsky Sandra Wienke wienke@rz.rwth-aachen.de November 2012 Rechen- und Kommunikationszentrum (RZ) The RWTH GPU Cluster GPU Cluster: 57 Nvidia Quadro 6000 (Fermi) innovative

More information

Command Line Crash Course For Unix

Command Line Crash Course For Unix Command Line Crash Course For Unix Controlling Your Computer From The Terminal Zed A. Shaw December 2011 Introduction How To Use This Course You cannot learn to do this from videos alone. You can learn

More information

Thirty Useful Unix Commands

Thirty Useful Unix Commands Leaflet U5 Thirty Useful Unix Commands Last revised April 1997 This leaflet contains basic information on thirty of the most frequently used Unix Commands. It is intended for Unix beginners who need a

More information

Linux Overview. Local facilities. Linux commands. The vi (gvim) editor

Linux Overview. Local facilities. Linux commands. The vi (gvim) editor Linux Overview Local facilities Linux commands The vi (gvim) editor MobiLan This system consists of a number of laptop computers (Windows) connected to a wireless Local Area Network. You need to be careful

More information

Installing a Symantec Backup Exec Agent on a SnapScale Cluster X2 Node or SnapServer DX1 or DX2. Summary

Installing a Symantec Backup Exec Agent on a SnapScale Cluster X2 Node or SnapServer DX1 or DX2. Summary Technical Bulletin Application Note April 2013 Installing a Symantec Backup Exec Agent on a SnapScale Cluster X2 Node or SnapServer DX1 or DX2 Summary This application note describes how to install the

More information

Introduction to Unix. Unix tutorial. Unix tutorial. Outline. Basic background in Unix structure

Introduction to Unix. Unix tutorial. Unix tutorial. Outline.  Basic background in Unix structure Unix tutorial Introduction to Unix http://linux.oreilly.com/ Page 1 Unix tutorial Outline Basic background in Unix structure Directories and files I (listing, navigation, filenames) Directories and files

More information

Research Technologies Data Storage for HPC

Research Technologies Data Storage for HPC Research Technologies Data Storage for HPC Supercomputing for Everyone February 17-18, 2014 Research Technologies High Performance File Systems hpfs-admin@iu.edu Indiana University Intro to HPC on Big

More information

Kernel. What is an Operating System? Systems Software and Application Software. The core of an OS is called kernel, which. Module 9: Operating Systems

Kernel. What is an Operating System? Systems Software and Application Software. The core of an OS is called kernel, which. Module 9: Operating Systems Module 9: Operating Systems Objective What is an operating system (OS)? OS kernel, and basic functions OS Examples: MS-DOS, MS Windows, Mac OS Unix/Linux Features of modern OS Graphical operating system

More information

Introduction to Operating Systems

Introduction to Operating Systems Introduction to Operating Systems It is important that you familiarize yourself with Windows and Linux in preparation for this course. The exercises in this book assume a basic knowledge of both of these

More information

RecoveryVault Express Client User Manual

RecoveryVault Express Client User Manual For Linux distributions Software version 4.1.7 Version 2.0 Disclaimer This document is compiled with the greatest possible care. However, errors might have been introduced caused by human mistakes or by

More information

icer Bioinformatics Support Fall 2011

icer Bioinformatics Support Fall 2011 icer Bioinformatics Support Fall 2011 John B. Johnston HPC Programmer Institute for Cyber Enabled Research 2011 Michigan State University Board of Trustees. Institute for Cyber Enabled Research (icer)

More information

An Introduction to the Linux Command Shell For Beginners

An Introduction to the Linux Command Shell For Beginners An Introduction to the Linux Command Shell For Beginners Presented by: Victor Gedris In Co-Operation With: The Ottawa Canada Linux Users Group and ExitCertified Copyright and Redistribution This manual

More information

WinSCP PuTTY as an alternative to F-Secure July 11, 2006

WinSCP PuTTY as an alternative to F-Secure July 11, 2006 WinSCP PuTTY as an alternative to F-Secure July 11, 2006 Brief Summary of this Document F-Secure SSH Client 5.4 Build 34 is currently the Berkeley Lab s standard SSH client. It consists of three integrated

More information

1 Bull, 2011 Bull Extreme Computing

1 Bull, 2011 Bull Extreme Computing 1 Bull, 2011 Bull Extreme Computing Table of Contents HPC Overview. Cluster Overview. FLOPS. 2 Bull, 2011 Bull Extreme Computing HPC Overview Ares, Gerardo, HPC Team HPC concepts HPC: High Performance

More information

Online Backup Linux Client User Manual

Online Backup Linux Client User Manual Online Backup Linux Client User Manual Software version 4.0.x For Linux distributions August 2011 Version 1.0 Disclaimer This document is compiled with the greatest possible care. However, errors might

More information

Online Backup Client User Manual

Online Backup Client User Manual For Linux distributions Software version 4.1.7 Version 2.0 Disclaimer This document is compiled with the greatest possible care. However, errors might have been introduced caused by human mistakes or by

More information

Intel Do-It-Yourself Challenge Lab 2: Intel Galileo s Linux side Nicolas Vailliet

Intel Do-It-Yourself Challenge Lab 2: Intel Galileo s Linux side Nicolas Vailliet Intel Do-It-Yourself Challenge Lab 2: Intel Galileo s Linux side Nicolas Vailliet www.intel-software-academic-program.com paul.guermonprez@intel.com Intel Software 2014-02-01 Prerequisites and objectives

More information

Overview of presentation

Overview of presentation Overview of presentation What is C3SE & SNIC, a compute cluster, Beda Differences between a cluster and a normal workstation Conceptional overview of a normal work flow Concrete starting points Accessing

More information

How to Use NoMachine 4.4

How to Use NoMachine 4.4 How to Use NoMachine 4.4 Using NoMachine What is NoMachine and how can I use it? NoMachine is a software that runs on multiple platforms (ie: Windows, Mac, and Linux). It is an end user client that connects

More information

Using StarCCM+ at the UB CCR

Using StarCCM+ at the UB CCR Using StarCCM+ at the UB CCR About This Document This document provides a step-by-step tutorial for using the StarCCM + graphical user interface (GUI) on the UB CCR cluster and viz node front-end. It also

More information

Using the Windows Cluster

Using the Windows Cluster Using the Windows Cluster Christian Terboven terboven@rz.rwth aachen.de Center for Computing and Communication RWTH Aachen University Windows HPC 2008 (II) September 17, RWTH Aachen Agenda o Windows Cluster

More information

To connect to the cluster, simply use a SSH or SFTP client to connect to:

To connect to the cluster, simply use a SSH or SFTP client to connect to: RIT Computer Engineering Cluster The RIT Computer Engineering cluster contains 12 computers for parallel programming using MPI. One computer, cluster-head.ce.rit.edu, serves as the master controller or

More information

Parallel Debugging with DDT

Parallel Debugging with DDT Parallel Debugging with DDT Nate Woody 3/10/2009 www.cac.cornell.edu 1 Debugging Debugging is a methodical process of finding and reducing the number of bugs, or defects, in a computer program or a piece

More information

OLCF Best Practices (and More) Bill Renaud OLCF User Assistance Group

OLCF Best Practices (and More) Bill Renaud OLCF User Assistance Group OLCF Best Practices (and More) Bill Renaud OLCF User Assistance Group Overview This presentation covers some helpful information for users of OLCF Staying informed Some aspects of system usage that may

More information

Parallel Programming for Multi-Core, Distributed Systems, and GPUs Exercises

Parallel Programming for Multi-Core, Distributed Systems, and GPUs Exercises Parallel Programming for Multi-Core, Distributed Systems, and GPUs Exercises Pierre-Yves Taunay Research Computing and Cyberinfrastructure 224A Computer Building The Pennsylvania State University University

More information

Fred Hantelmann LINUX. Start-up Guide. A self-contained introduction. With 57 Figures. Springer

Fred Hantelmann LINUX. Start-up Guide. A self-contained introduction. With 57 Figures. Springer Fred Hantelmann LINUX Start-up Guide A self-contained introduction With 57 Figures Springer Contents Contents Introduction 1 1.1 Linux Versus Unix 2 1.2 Kernel Architecture 3 1.3 Guide 5 1.4 Typographical

More information

1. Product Information

1. Product Information ORIXCLOUD BACKUP CLIENT USER MANUAL LINUX 1. Product Information Product: Orixcloud Backup Client for Linux Version: 4.1.7 1.1 System Requirements Linux (RedHat, SuSE, Debian and Debian based systems such

More information

Online Backup Client User Manual Linux

Online Backup Client User Manual Linux Online Backup Client User Manual Linux 1. Product Information Product: Online Backup Client for Linux Version: 4.1.7 1.1 System Requirements Operating System Linux (RedHat, SuSE, Debian and Debian based

More information

Hodor and Bran - Job Scheduling and PBS Scripts

Hodor and Bran - Job Scheduling and PBS Scripts Hodor and Bran - Job Scheduling and PBS Scripts UND Computational Research Center Now that you have your program compiled and your input file ready for processing, it s time to run your job on the cluster.

More information

Linux System Administration on Red Hat

Linux System Administration on Red Hat Linux System Administration on Red Hat Kenneth Ingham September 29, 2009 1 Course overview This class is for people who are familiar with Linux or Unix systems as a user (i.e., they know file manipulation,

More information

Site Configuration SETUP GUIDE. Windows Hosts Single Workstation Installation. May08. May 08

Site Configuration SETUP GUIDE. Windows Hosts Single Workstation Installation. May08. May 08 Site Configuration SETUP GUIDE Windows Hosts Single Workstation Installation May08 May 08 Copyright 2008 Wind River Systems, Inc. All rights reserved. No part of this publication may be reproduced or transmitted

More information

Berkeley Research Computing. Town Hall Meeting Savio Overview

Berkeley Research Computing. Town Hall Meeting Savio Overview Berkeley Research Computing Town Hall Meeting Savio Overview SAVIO - The Need Has Been Stated Inception and design was based on a specific need articulated by Eliot Quataert and nine other faculty: Dear

More information

Distributed convex Belief Propagation Amazon EC2 Tutorial

Distributed convex Belief Propagation Amazon EC2 Tutorial 6/8/2011 Distributed convex Belief Propagation Amazon EC2 Tutorial Alexander G. Schwing, Tamir Hazan, Marc Pollefeys and Raquel Urtasun Distributed convex Belief Propagation Amazon EC2 Tutorial Introduction

More information

Basic Linux and Unix commands, editing and transfering files

Basic Linux and Unix commands, editing and transfering files First version: February 3, 2002 Last update: February 5, 2010 PATRICIA LEDESMA LIÉBANA Basic Linux and Unix commands, editing and transfering files The most recent version of this document can be found

More information

TNM093 Practical Data Visualization and Virtual Reality Laboratory Platform

TNM093 Practical Data Visualization and Virtual Reality Laboratory Platform October 6, 2015 1 Introduction The laboratory exercises in this course are to be conducted in an environment that might not be familiar to many of you. It is based on open source software. We use an open

More information

CS197U: A Hands on Introduction to Unix

CS197U: A Hands on Introduction to Unix CS197U: A Hands on Introduction to Unix Lecture 4: My First Linux System J.D. DeVaughn-Brown University of Massachusetts Amherst Department of Computer Science jddevaughn@cs.umass.edu 1 Reminders After

More information

OPERATING SYSTEM SERVICES

OPERATING SYSTEM SERVICES OPERATING SYSTEM SERVICES USER INTERFACE Command line interface(cli):uses text commands and a method for entering them Batch interface(bi):commands and directives to control those commands are entered

More information

SSH and Basic Commands

SSH and Basic Commands SSH and Basic Commands In this tutorial we'll introduce you to SSH - a tool that allows you to send remote commands to your Web server - and show you some simple UNIX commands to help you manage your website.

More information

Connecting to the School of Computing Servers and Transferring Files

Connecting to the School of Computing Servers and Transferring Files Connecting to the School of Computing Servers and Transferring Files Connecting This document will provide instructions on how to connect to the School of Computing s server. Connect Using a Mac or Linux

More information

Command-Line Operations : The Shell. Don't fear the command line...

Command-Line Operations : The Shell. Don't fear the command line... Command-Line Operations : The Shell Don't fear the command line... Shell Graphical User Interface (GUI) Graphical User Interface : displays to interact with the computer - Open and manipulate files and

More information

ICS 351: Today's plan

ICS 351: Today's plan ICS 351: Today's plan routing protocols linux commands Routing protocols: overview maintaining the routing tables is very labor-intensive if done manually so routing tables are maintained automatically:

More information