On-demand (Pay-per-Use) HPC Service Portal

Transcription

1 On-demand (Pay-per-Use) Portal Wang Junhong INTRODUCTION High Performance Computing, Computer Centre The Service Portal is a key component of the On-demand (pay-per-use) HPC service delivery. The Portal, together with other Cloud computing tools, enables various operations such as online resource reservation, reservation management, automation of HPC cluster provisioning and management of reserved resources. As the Portal undergoes beta testing, I will share with you some details on how it works using a series of screen captures. There will also be some tips on how to make use of the reserved resources from On-demand (pay-per-use) HPC Subscription. The process involves three key components: 1) HPC resources online subscription at the On-demand Service Portal 2) Management of reserved HPC resources at the On-demand Service Portal 3) Access and use of the reserved HPC resources Step - 1 HPC service portal login Valid users can login to the On-demand HPC service portal using their NUSNET accounts (pre-registered at HPC portal) securely. Figure 1 On-demand Portal Login 1

2 Step - 2 Check Project Balance Before you can subscribe resources for your project, you must have positive Funds or Credit in the appropriate project account. The Funds or Credit is predefined between the user and service provider. It can be used by both the service provider and the user to ensure that there are enough funds for subscription. You can check the balance and expense via Bank Your Balance, and the sample output is shown in Figure 2. Figure 2 Home Page of On-demand Portal 2

3 Step - 3 Shop HPC Resources by Making a Reservation To shop on-demand pay-per-use HPC resources, you can make a reservation request, Reservation Make Request as shown: Figure 3 Reservation Menu On the reservation request interface, you can define the Durations with Start Time and End Time; you can choose the Profile Type and server type via Procs Per Node, as in Figure 4. After those are defined, click on Add To Request and continue shopping. Figure 4 Reservation Request Menu Note: the price showed here is for illustration purpose. 3

4 Step - 4 Verify the Requested Reservation The verification of the requested reservation will be displayed at the right column of the portal after you click on Add To Request. The default number of nodes to reserve is 1, you can input the exact number of nodes based on your needs. For example, if two nodes are required, enter 2 at the Qty box and click on Verify Request button to check if the resources are available and the cost for this reservation. Figure 5 Reservation Verification If the resources requested are Available, click on Add to Cart to check out and confirm the shopping. 4

5 Step - 5 the Shopping Cart In your Shopping Cart, click on the button to confirm the reservation request; or click on Remove button to discard the request. Once you check out, the amount of this reservation will be deducted from your project fund (verify it via Bank Your Balance.) With a successful check out, the reservation will be accepted and provisioned based on the requested period, server type and number of nodes. Figure 6 Shopping Cart 5

6 Step - 6 Manage a Reservation Administration Delegation You can delegate the administration work of your reservation to another valid NUS user, who may be your colleague, research fellow or student. Click on Reservation Management, with the specific Reservation highlighted, click on the Manage Users button to open the Manage VPC Users panel. Input the username and enable the Admin access box, then click on OK button to confirm. Figure 7 Administration Delegation 6

7 Step - 7 Manage a Reservation Permission to Use If you make a reservation for your research team or a group of users, you (or the delegated administrator) can set the permission for other team members to access and use this reservation to run their computational jobs. Click on Reservation Authorization, click on the Modify button at the right of the Reservation to open authorization interface. Highlight the user(s) from the Users list at the left column and click on >> button to transfer the account(s) into the Existing Users at right side. Finally, click on Save button to confirm the permission setting. Figure 8 Administration Delegation 7

8 Step - 8 Online at the Portal You may check out the online user guide on the portal to learn how to Use Reserved Resource Job Submission Step-by-step guide lists reference and help. Figure 9 Online At any time, click on Contact Us at the top-right corner of the portal to send us your queries or questions regarding the HPC service portal. Figure 10 Online Feedback 8

9 Step - 9 Login and Use Reserved HPC Resource Users with permission to use the requested HPC reservation can submit and run jobs onto the reserved nodes exclusively via a special Linux-based login server. An example of how to make use the reserved HPC reservation to run a job is given as below, with the following Linux commands to be used. Command myres gcc, icc gfortran, ifort mpicc, mpif77 qsub bjobs, qstat Function To list the authorized reservation that you can use Available C/C++ compiler Available Fortran compiler Available MPI C/Fortran compiler Batch job submission command To list batch jobs and their status List my authorized reservation #> myres rsvid state owner totalnodes users stime etime DRSV.3109 active ccev708 2 ccev708,g Tue Jul 12 14:57: Tue Jul 19 14:57: DRSV.3157 pending ccev708 2 ccev708,t Tue Jul 19 14:39: Tue Jul 26 14:39: Compile a MPI program #> mpicc helloworld.c o hello.exe Submit MPI batch job to the reserved compute nodes #> qsub -l advres=drsv.3109 run_hello stratus.nus.edu.sg List and check batch job status #> bjobs JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME ccev708 PEND NORMAL hostname.s Jul 13 17:32 #> qstat Job id Name User Time Use S Queue stratus hostname.sh ccev708 0 Q NORMAL stratus run_hello ccev708 00:00:00 C NORMAL Note: the status of C listed by qstat indicates that the job is completed. Check and view the result data #> ls -ltr total rw-r--r-- 1 ccev708 svuusers 706 Jul 12 16:29 helloworld.c -rwxr-xr-x 1 ccev708 svuusers Jul 18 14:50 hello.exe 9

10 -rw-r--r-- 1 ccev708 svuusers 222 Jul 18 14:51 run_hello -rw ccev708 svuusers 0 Jul 18 14:52 run_hello.e rw ccev708 svuusers 250 Jul 18 14:52 run_hello.o #> #> cat run_hello.o Hello MPI from the server process! mesg from 1 of 6 on cougar-c05 mesg from 2 of 6 on cougar-c05 mesg from 3 of 6 on cougar-c05 mesg from 4 of 6 on cougar-c05 mesg from 5 of 6 on cougar-c06 mesg from 6 of 6 on cougar-c06 SUMMARY AND CONCLUSION With the On-demand Portal, researchers can shop for on-demand pay-peruse HPC resources when there is a need. The on-demand pay-per-use resources can be requested and reserved dynamically based on the demand and available research fund. The reserved HPC resources are dedicated for authorised users. The owners have the full management right to authorise other users to use the reservation, request to add additional compute nodes and to extend the reservation period. 10