Using GPUs in the Cloud for Scalable HPC in Engineering and Manufacturing March 26, 2014

Transcription

1 Using GPUs in the Cloud for Scalable HPC in Engineering and Manufacturing March 26, 2014 David Pellerin, Business Development Principal Amazon Web Services David Hinz, Director Cloud and HPC Solutions HGST, Inc GPU Technology Conference, March 24-27, 2014

2 Motivators for GPU-Enabled HPC in the Cloud Agility: React quickly to changing needs with flexible cloud capacity Scalability: Go wide, go large for faster timeto-results at higher accuracy Global Collaboration: For enhanced IP security, more efficient operations, access to large datasets and models

3 A Simulation-Driven World Computer-Aided Design, Simulation, Analysis, Visualization Across industries, the trend is Simulation-Driven Design and Discovery Aerospace, automotive, civil, construction, energy exploration, consumer products, finance, pharmaceuticals, many others Examples in Design and Manufacturing Computer-Aided Design (CAD) including 3D models Finite Element Analysis (FEA) and Thermal Analysis Electronic Design Automation (EDA) Computational Fluid Dynamics Molecular Modeling High Energy Physics Big Data in Manufacturing

4 Cloud Provides Agility and Scale for HPC Actual demand Resources scaled to demand Predicted Demand Project Delays Actual demand Wasted Resources Rigid On-Premise Resources Elastic Cloud-Based Resources 3 to 5 year architecture commitment Little or no architecture commitment

5 Building and maintaining an HPC cluster is hard Is it worth your time and effort?

6 HPC Job Queues are Evil!? My Job Use scalability to get rid of them

7

8 What Does Scale Mean in the Cloud?

9 Cloud for Design Collaboration Enabling next-generation, GPU enabled cloud applications Virtual Private Cloud Enabling

10 Enabled by GPUs in the Cloud AWS Cluster GPU Instances (CG1, G2) Providing integrated NVIDIA GPUs Ideal for highly parallelized processing - HPC, rendering and media processing CG1 is optimized for GPU computing, HPC G2 for remote desktop, 3D graphics, game streaming

11 GPUs Enable Secure Remote Access Data and computation hosted in a secure, customer-managed virtual private cloud, with controlled access via a wide variety of client devices. Virtual Private Cloud Image courtesy of Calgary Scientific

12 Collaboration is More Secure in the Cloud Bring the users to the data, don t send the data to the users

13

14 Use-Cases in Manufacturing

15 David Hinz, Director Cloud and HPC Solutions HGST, Inc Founded in 2003 through the combination of the hard drive businesses of IBM, the inventor of the hard drive, and Hitachi, Ltd ( Hitachi ) Performance Enterprise PCIe Cloud & Datacenter Enterprise SSD (+3 acquisitions in 2013) SAS 10K & 15K HDDs Acquired by Western Digital in 2012 More than 4,200 active worldwide patents Headquartered in San Jose, California Approximately 41,000 employees worldwide Develops innovative, advanced hard disk drives, enterprise-class solid state drives, external storage solutions and services Delivers intelligent storage devices that tightly integrate hardware and software to maximize solution performance Capacity Enterprise 7200 RPM & CoolSpin HDDs Ultrastar Ultrastar & MegaScale DC

16 Zero to Cloud in less than 12 Month By 31 Dec 2013: Cloud Microsoft Office365 April 2013 Cloud archiving/ediscovery External SingleSignOn (off VPN) Cloud File/Collaboration BOX USe Salesforce.com Integrated to save files in BOX Cloud High Performance Computing (HPC) on AWS Cloud Big Data Platform on AWS Cloud - data mart and provisioning service, with AWS Redshift

17 Evolution of Data Centers and HGST SJC Servo Team Japan Servo Team Old Mail System MN PCB Team US Head Team US HAMR Team Internal Wiki s, etc. Silo ed Site HGST On Site Business, Production and Enterprise Computing HGST Datacenters HPC Clusters Servo Team in SJC Head Team in US PCB Team in MN Servo Team in Jpn HAMR Team in US An Agile Enterprise Datacenter Integrating On-Premise and Cloud Solutions Cloud Off Premise On Premise Evaluate New Storage Technologies and Solutions In House (HDD, SSD, etc.)

18 HPC: Molecular Dynamics Simulations HGST uses Molecular Dynamics Simulation applications for RnD of materials and lubricants needed for HDD s Evaluated to achieve higher memory densities, faster read/write capabilities, small(er) physical form factor, lower power consumption Model Job Size used at HGST Complexity [atoms] Number of Time Steps Job Type Frequency Small 300, per day, 2 days per week once or twice a month Medium 300, Medium jobs during the day, 4 days per month Large 300, large jobs per day, 6 days per month Very Large 300, large job per month

19 Shape Compute To Match Work To Be Done Time 2W Prior: Shared 512 Core Super Computer 512 core waiting 512core 512core 512core 256 core waiting 256 core 128 core waiting 128 core Today: AWS EC2 CC2 (Max Total 512 core) All Jobs Run In Parallel on AWS 1.67x Throughput Improvement

20 HPC: Micro Magnetic Simulation Team models new technologies for future HGST HDD products Finite-difference time-domain (FDTD) numerical analysis solver Accurately simulation of large, complex models of many variable parameters and materials Scale across large clusters AWS C3 Instances provide significant improvement for both scalability and simulation throughput AWS C3 Instances Provided 1.5x or Better Simulation Performance

21 Cloud HPC : What s Next.. AWS C3 Performance Validated Across Many Applications Improve overall performance and reduce monthly AWS compute bill Deploy Graphics User Interfaces HPC Applications Pre and Post Processing in Cloud vs. data migration back to local systems 3D visualization enabled using NVIDIA GRID GPUs in the Cloud

22 Commercial HPC Applications: Cloud Ready? HPC environments desire Cloud Computing with in-house machines Desire Hybrid Mix of on-premise workstations + clusters (some legacy, some new) with burst/over-flow to Cloud EULAs Request: Comprehend cloud use Allow License server placement in cloud/off-prem and accessible on-premise Make it easy to add cloud computing to current licenses Consumption Based Pricing No consistency Not aligned with AWS consumption pricing ISVs are changing, but slowly

23 We ve Only Just Begun. Current Results in less than 12 months Re-aligning Business Group Leadership, Development Teams, Research and Development Teams on New Capabilities Model Demands and Uses To Grow And Accelerate the Business 2013 Heavy Lifting Provides Foundation for 2014 Acceleration at HGST