LS-DYNA: CAE Simulation Software on Linux Clusters

Transcription

1 IBM Deep Computing Group LS-DYNA: CAE Simulation Software on Linux Clusters Guangye Li IBM Deep Computing Team June, 2003 IBM Deep Computing Group

2 Topics Introduction to LS-DYNA LS-DYNA Applications Two versions of LS-DYNA: SMP and MPP An example Performance of LS-DYNA on clusters Performance Improvement with Faster Processors Interconnect Options: Gigabit Ethernet or Myrinet One or two process nodes Comparison of LAM/MPI and MPICH Performance Speedup from Compiler Options Speedup from Faster 533 MHz Front side Bus Chrysler experience 2

3 LS-DYNA: A general purpose transient dynamic finite element program capable of simulating complex real world problems Software Vendor: Livermore Software Technology Corp. (LSTC) Largest application in CAE Large customer base 3

4 LS-DYNA applications include: Occupant safety Metal Forming Metal Cutting Biomedical Blast loading Fluid-structure interaction Earthquake engineering 4

5 Two parallel versions of LS-DYNA SMP (OpenMP) for shared memory multiple processors. Parallelized from a serial code Scalable up to 16 CPUs MPP (Distributed memory version) Using the domain decomposition technique Using MPI for communications between subdomains (processors) Scalable up to more than 100 CPUs. Suitable for both shared memory multiple processors and clusters MPP-DYNA on clusters dramatically reduced the turnaround time and the simulation cost 5

6 Comparison of SMP and MPP Elapsed Time (sec) GHz IBM p690 November 2002 LS-DYNA refined Neon-535k elements 0 1-CPU 2-CPU 4-CPU 8-CPU 16-CPU 32-CPU SMP MPP 6

7 An Example: The Neon Model Frontal crash with initial speed at 31.5 miles/hour Model size number of shell elements: 269,249 number of nodal points: 285,832 Simulation length: 150 ms vehicle bounce back observed at 70 ms Model created by National Crash Analysis Center (NCAC) at George Washington University one of the few publicly available model for vehicle crash analysis based on 1996 Plymouth Neon 7

8 1996 Plymouth Neon 8

9 The model 9

10 The mesh 10

11 Domain decomposition The whole mesh is decomposed into NCPU subdomains. Each domain has about the same number of elements Each link cut corresponding to communications between two nodes. The decomposition should minimize the link cuts Each CPU processes elements in its subdomain CPUs exchange boundary data using message passing (MPI) 11

12 12

13 Simulation results 13

14 Performance Improvement with Faster Processors Elapsed Time (sec) V960 r1488 LS-DYNA Xeon, 2 CPUs per node Gigabit Ethernet Jan-March 2003 LAM/MPI refined Neon-535k elements 0 2-CPU 4-CPU 8-CPU 16-CPU 32-CPU 64-CPU 2.4 GHz 2.8 GHz 14

15 Configuring Each Node with One Processor Elapsed Time (sec) CPU 4-CPU 8-CPU 16-CPU 32-CPU 64-CPU V960 r1488 LS-DYNA 2 CPUs per node 1 CPU per node Gigabit Ethernet x GHz March 2003 LAM/MPI Front crash model 430k elements 15

16 Interconnect Effect on Performance Elapsed Time (sec) GHz IntelliStation Cluster June 2002 MPI LS-DYNA refined Neon-535k elements 0 2-CPU 4-CPU 8-CPU 16-CPU 32-CPU Fast Ethernet Gigabit Ethernet Myrinet 16

17 Interconnect Performance Compared Parallel Speedup CPU 4-CPU 8-CPU 16-CPU 32-CPU x335+fast Ethernet x335+gigabit Ethernet x335+myrinet p655+sp Switch2 V960 LS-DYNA Jan 2003 Refined Neon 535k Elements 17

18 Comparison of LAM/MPI and MPICH Performance Elapsed Time (sec) GHz x335 (Xeon) Cluster Gigabit Ethernet March 2003 LS-DYNA refined Neon-535k elements 0 16-CPU MPICH CPU LAM/MPI

19 Speedup from Compiler Options Intel Compiler Option SSE No_SSE Elapsed time (sec) V960 r1106 MPP-DYNA Feb 2002 LAM/MPI GHz IntelliStation node 12 processor runs 19

20 Speedup from Faster 533 MHz Frontside Bus Model Size (elements) Speedup: 400MHz to 533 MHz Frontside Bus V960 r1488 LS-DYNA March 2003 LAM/MPI 2.8 GHz x335 node 2 processor runs 20

21 Performance Improvement with Version 970 Elapsed Time (sec) GHz x335 (Xeon) Cluster Gigabit Ethernet March 2003 LAM/MPI MPP-DYNA refined Neon-535k elements version 960 r version 970 r CPU 4-CPU 8-CPU 16-CPU 32-CPU 21

22 Chrysler experience Customer requirements Reduced turn around time Price/performance Good accuracy, i.e., The numerical results should match the results on those from the current 64 bit machines A team work Chrysler LSTC IBM Intel Eventually all 22 QA models passed the accuracy requirements and Chrysler bought 108 Xeon based IBM Linux cluster nodes for car crash simulation 22

23 Chrysler is happy with the IBM Linux cluster solution Without parallel processing, we never would have achieved 5* (NCAP) and good (IIHS) on our new Chrysler Sebring and Dodge Stratus within the current product development time. --Subhas Shetty, Chrysler 23

24 Summary MPI based MPP-DYNA has better scalability Linux clusters reduced the turn around time for car crash simulation Linux clusters reduced the simulation cost The accuracy is satisfactory Users today can customize their system in order to pick the features which serve them best Processors Operating system Interconnect 24