An HPC Application Deployment Model on Azure Cloud for SMEs

Transcription

1 An HPC Application Deployment Model on Azure Cloud for SMEs Fan Ding CLOSER 2013, Aachen, Germany, May 9th,2013 Rechen- und Kommunikationszentrum (RZ)

2 Agenda Motivation Windows Azure Relevant Technology Our Deployment Model and Framework Performance Analysis Conclusion and Outlook 2

3 The Requirement of SMEs HPC application in SMEs need more and more computing resources The on-premise resource of SMEs cannot satisfy it. The coming of cloud can solve this problem.(big computing power and massive data centers) 3

4 Cloud and On-premise Application Cloud vendors: Various kinds of cloud platforms have different user interfaces. Cloud Users: Difficult to migrate on-premise application into these platform So we need a HPC app deployment Developed a HPC application deployment model on Azure by expending the Azure HPC scheduler. 4

5 Background of Our Cloud Project Project Technical Cloud Computing for SMEs in Manufacturing Application ZaKo3D,WZL(the Institute for Machine Tools and Production Engineering, RWTH Aachen University) ZaKo3D (The figure from WZL) 5

6 Background of Our Cloud Project Problem: Time consuming. Our method: computing variations in parallel on cloud based on MPI. 6

7 Agenda Motivation Windows Azure Relevant Technology Our Deployment Model and Framework Performance Analysis Conclusion and Outlook 7

8 Windows Azure Relevant Technology The Three Roles of Windows Azure Azure HPC Scheduler for High-Performance Computing Azure Storage Blob Service 8

9 The Three Roles of Windows Azure Web Roles(PaaS) Work Roles(PaaS) VM Roles(IaaS) 9

10 HPC Scheduler for High-Performance Computing in Azure Cloud Allows scheduling compute-intensive applications built to use the Message Passing Interface (MPI) Supports Windows Azure roles through offering plug-ins. There are three types of nodes : Head node: work role with HpcHeadNode plug-in, job scheduling and resource management Compute node: work role with HpcComputeNode plug-in, runtime support for MPI and SOA. Front node: web role with HpcFrontEnd plug-in, web portal (based on REST) as the job submission interface for HPC Scheduler. 10

11 Azure Storage Blob Service Azure provide three types of storage service Blobs (Binary Large Objects): storing binary and text data Tables: storing non-relational data Queues: storing messages We employ the Blob storage 11 Three layers concept of Blobs storage (The figure from windows Azure)

12 Agenda Motivation Windows Azure Relevant Technology Our Deployment Model and Framework Performance Analysis Conclusion and Outlook 12

13 Deployment Model 13

14 Deploy Application on Azure based Azure HPC Scheduler 1.Configure head nodes and work nodes 2.Move an application onto the configured Azure instance 14

15 MPI Job Scheduler 15

16 Blob Data Management 16

17 Agenda Motivation Windows Azure Relevant Technology Our Deployment Model and Framework Performance Analysis Conclusion and Outlook 17

18 Experiments on both Cloud Resource and On-premises Deployed ZaKo3D application on both Azure platform and the RWTH Cluster Submit 10 jobs on these two platform(120 variations each) Cluster:12 processes run per node Cloud: only one process per node 18

19 Experiments Results With a portion of 0.87% sequential code, restricted by Amdahl s law and could get a maximum speedup of 116 Runtime of ZaKo3D on Cluster and Azure nodes Scalability of ZaKo3D execution on different number of Cluster and Azure nodes 19

20 Agenda Motivation Windows Azure Relevant Technology Our Deployment Model and Framework Performance Analysis Conclusion and Outlook 20

21 Conclusion and Outlook Conclusion Microsoft Azure Cloud have good scalability and cloud s power can support HPC application s execution This work can give a reference to SMEs (small and medium-sized enterprises) to develop their HPC applications for cloud environments. Current cloud power can only be used to supply to the status the business does not have enough on-premises resources to support its development Outlook Decrease the overhead in the parallel scheduler Investigate the price of cluster and cloud, figure out an available rule for users to make the best decision to choose HPC platforms 21

22 Thank you! 22