HPC Cluster Decisions and ANSYS Configuration Best Practices. Diana Collier Lead Systems Support Specialist Houston UGM May 2014

Transcription

1 HPC Cluster Decisions and ANSYS Configuration Best Practices Diana Collier Lead Systems Support Specialist Houston UGM May

2 Agenda Introduction Lead Systems Support Specialist Cluster Decisions Job Schedulers - Remote Solve Manager Accessing Support and important resources 2

3 Cluster Decisions 3

4 HPC Best Practice for ANSYS Cluster Decisions Processors Cores and GPUs Memory I/O Storage / File System Network Servers (or Workstations) for ANSYS Solvers (Fluent, CFX, Mechanical) High clock speed Turbo-boost ON Hyperthreading OFF Fastest memory (1866 MHz) Intel E26xx v2 Lower core density, esp. for structures (vs. packaging advantages) Typically cores per job (CFD) Up to cores per job (Fluent) Typically 8-32 cores per job (Mechanical) NVIDIA Tesla K20C or C2075 for solver acceleration (Mechanical, Fluent) 4-8 GB per node Up to 256 GB per workstation Memory channels populated equally (for speed) Nothing special for CFD (SATA) Fast I/O for mechanical (SSD, RAID0) +/- 1TB per year per user Parallel File System best practice Infiniband (or at least 10gigE) for more than 3 nodes Workstations (or Graphics Servers) As above for Pre / Post- 4 processing Typically prep/post on 1 to 8 cores. Graphics intensive! Professional-level NVIDIA Quadro K or ATI FirePro Up to 256 GBytes As above As above Remote display solutions with accelerated OpenGL support (VirtualGL+ RealVNC, Nice DCV, Exceed ondemand)

5 HPC Best Practices Desktop User Workstation and/or Server Users Cluster Users Cloud Solution 5

6 Job Schedulers Job Schedulers 7

7 Job Scheduling Why use a Job Scheduler? Imagine you have two or more engineers wishing to submit jobs to a cluster of computers. Without a job scheduler they would need to rely on each other to determine if the resources were available. There is nothing preventing multiple jobs from running on the same cores/nodes without a job scheduler causing performance bottlenecks. 8

8 Job Schedulers There are several vendors of scheduling products on the market IBM Platform LSF Microsoft HPC 2008 & 2012 R2 Univa Altair PBS Works Adaptive Computing (Moab) (Beta support with Release 15. Full support with release 16 Using Job Schedulers requires IT support and involvement Get an experienced 3 rd party to provide set-up and on-going support. Either through this third party, or directly to the hardware vendor, always purchase the support. 9

9 Job Scheduling In addition to the cores - Job schedulers can schedule many types of resources Cores, Sockets & Nodes Memory GPU s User programmable resources (e.g. Software Licenses) Job Priorities Exclusive Scheduling They also provide ways of providing user or group privileges and limits. Nodes for specific groups/users Run-time limits. 10

10 Job Schedulers ANSYS has our own Scheduler called RSM to help with low level scheduling on workstation groups and to link with schedulers to clusters. It is not meant to compete with traditional Scheduling tools but rather to complement them and link them to the Workbench workflow. 11

11 Remote Solve Manager 12

12 Remote Solve Manager The Remote Solve Manager (RSM) is a job queuing system that distributes tasks that require computing resources. RSM enables tasks to be run in background mode on the local machine, sent to a remote machine for processing, or tasks can be broken into a series of jobs for parallel processing across a variety of computers. 13

13 Remote Solve Manager Computers with RSM installed are configured to manage jobs using three primary services: The RSM Client server The Solve Manager service (typically shortened to Manager ) Compute Server service. Client Submits the job and decides where the results go at the end. Solver Manager schedules the job. Compute Server runs the job or passes the job to a 3 rd party queuing Client Solver Manager Compute Server 14

14 The RSM Configuration Wizard RSM Configuration made easier The recommended method of configuring RSM is using the ANSYS Remote Solve Manager Setup Wizard, a utility that guides you through the process of setting up and configuring RSM. 15

15 The RSM Configuration Wizard Configure the machines to be included in your RSM Layout Perform various cluster configuration tasks Integrate RSM with the following third-party job schedulers (without requiring job script customization): LSF (Windows and Linux) PBS (Linux only) Microsoft HPC SGE (UGE) Create and share RSM directories (Project Directory, Working Directory, and where applicable, Shared Cluster Directory) Define queues Create accounts Test the final RSM configuration 16

16 The RSM Configuration Wizard 17

17 Getting Support Customer Support Portal, or Phone Go to the Support link > Select Service > Select Submit a Service Request Search the Knowledge Base for answers to common problems. 18 Global ANSYS network providing Comprehensive Support

18 Common cluster support calls We bought this cluster for ANSYS and now it doesn t work. Since only ANSYS is used on it ANSYS should fix it for us My IT thought they could look after the Infrastructure but cannot. Now I am very far behind in my work. Please help me! ANSYS is a software provider and therefore we do not provide cluster management or hardware procurement type services. Where large scale infrastructure, such as clusters, is concerned unless you are very experienced in this area do not go it alone. Get an experienced 3 rd party to provide set-up and on-going support. Either through this third party, or directly to the hardware vendor, always purchase the support. This is the sort of area people think they can save money on. Your not saving money when it doesn t work. When something goes wrong having this help is worth its weight in gold. 19

19 Information Available ANSYS Partner Solutions Reference configurations Performance data White papers Sales contact points Performance Data 20

20 Information Available ANSYS Platform Support Platform Support Policies Supported Platforms Supported Hardware Tested systems ANSYS Virtual Demo Room Click on HPC! 21

21 Information Available The Manual Sections on best practices and parallel processing for various solvers Performance Guide for Mechanical Installation walkthroughs for installing the products, parallel processing, licensing and RSM (remote solve manager) ANSYS Advantage Online Magazine 22

22 Questions and Answers 23

23 THANK YOU 24