Using Windows Azure for High Performance Computing

Size: px

Start display at page:

Download "Using Windows Azure for High Performance Computing"

Kerry Doyle
7 years ago
Views:

1 Using Windows Azure for High Performance Computing Eduardo Rooff, Francis Birck, Matthias Diener, Aexandre Carissimi, Phiippe O. A. Navaux Parae and Distributed Processing Group (GPPD) Federa University of Rio Grande do Su (UFRGS) Porto Aegre, Brazi CLCAR 2012 Panama City, Panama

2 2 Subject Methodoogy to port HPC appications to Coud Microsoft Windows Azure Coud Performance Evauation NAS Benchmarks Evauation of a singe instance of Azure to run HPC

3 3 Motivation The machines are becoming obsoete faster and faster Our ab buys a new one and in few months it doesn t fits Deveopment of Coud Computing has a significant impact on the scientific community Potentiay they can repace custers and grids

4 4 Motivation IaaS are viewed just as virtuaization SaaS are viewed as appication in coud Not a rea new mode of computing! We need a service mode that offers new features Mode of execution Data management (to create new appication)

5 5 Motivation PaaS is reay a new mode of computing Pre-research about: Googe App vs. Microsoft Azure Azure gives you more contro In HPC contro is fundamenta

6 6 Chaenges and Soutions Characteristics of Windows azure - OS and programming mode are based on Windows - Execution is made in a WEB appication Scientific appications has other characteristics - The base OS normay is Unix - The execution is made in a command ine environment Some modifications to the appications are needed

7 7 Chaenges and Soutions First step: Convert to Windows programming mode I - Buid system Natura choice to write Windows appications is Visua Studio The Unix make fies aren t compatibe Was necessary to create a new Visua Studio project Using the VS features Without code modifications

8 8 Chaenges and Soutions First step: Convert to Windows programming mode II - FORTRAN compier Visua Studio doesn t incude a FORTRAN compier Many scientific appications use FORTRAN 8 out 10 NAS benchmarks are written in FORTRAN Inte Visua FORTRAN Composer XE for Windows Integrates with Visua Studio and generates correct code

9 9 Chaenges and Soutions First step: Convert to Windows programming mode III - Differences in system and ibrary cas NAS use some timing functions ony avaiabe on Unix We need to rewrite the entire function for timing cacuation

10 10 Chaenges and Soutions To avoid main code modifications, need to wrote gettimeofday

11 11 Chaenges and Soutions Second step: Convert to Azure programming mode Windows Azure appications need to be WEB based A WEB interface executes functions from an appication (Worker roe) running in background Converting NAS to this mode impies in three issues:

12 12 Chaenges and Soutions Second step: Convert to Azure programming mode I - Native Code Appication FORTRAN and C are not supported nativey by Azure Its necessary to create a native code appication This appication is a Windows ibrary (d)

13 13 Chaenges and Soutions Second step: Convert to Azure programming mode II - Fie System NAS uses fies to get configuration parameters Azure doesn t provide directy access to disk Necessary to create a oca context for fie manipuation We need to rewrite a the fie access routines to use this

14 Chaenges and Soutions 14

15 15 Chaenges and Soutions Second step: Convert to Azure programming mode III - Output instructions NAS uses output with printf functions Azure doesn t have a consoe output We need to use output on the WEB interface We create output variabes to do this Impies in code modifications too in the output routines

16 16 Chaenges and Soutions 1. Remove printf 3. Function header modification 2. Function ca from C# 4. Code adjustment

17 Chaenges and Soutions 17

18 18 Evauation: environments We use this environments for comparison Windows Azure Coud Extra arge instance - 8 cores at 1.6 GHz 14 GBytes of main memory Running Windows 2008 Server

19 19 Evauation: benchmarks We run NAS benchmarks to evauate The input size used was W Used a threads avaiabe on the machines 8 Two benchmarks are not used: DC because its uses a ot of disk and takes too much time FT because it has a probem of stack overfow with IIS

20 20 Resuts: performance Execution time 3 2,5 2 1,5 1 0,5 0,66 2,52 0,19 0,37 Execution time ,81 11,8 9,32 14,85 0 CG EP IS MG Number of threads 0 BT LU SP UA Number of threads Average Time(s) of execution

21 21 Resuts: performance Execution time (seconds) Number of threads Average execution time for a Benchmarks Good scaing ratio in genera Low degradation

22 22 Resuts: speedup 8 7 7,59 Speedup ,24 2,98 3,99 5,82 2,34 4,24 4,06 4, BT CG EP IS LU MG SP UA Avg Benchmark 4 threads 8 threads

23 Resuts: variabiity 23

24 24 Concusions Conversion of HPC appications to Azure is possibe Without modifications on the compute code Need modifications on the input and output code Windows Azure has aceptabe performance A the benefits of Coud Computing A singe instance has same characteristics that a rea machine An interesting research topic is to buid a HPC Coud

25 Using Windows Azure for High Performance Computing Eduardo Rooff, Francis Birck, Matthias Diener, Aexandre Carissimi, Phiippe O. A. Navaux {erooff, fbmoreira, mdiener,

Big Data projects and use cases. Claus Samuelsen IBM Analytics, Europe csa@dk.ibm.com

Big Data projects and use cases. Claus Samuelsen IBM Analytics, Europe csa@dk.ibm.com Big projects and use cases Caus Samuesen IBM Anaytics, Europe csa@dk.ibm.com IBM Sofware Overview of BigInsights IBM BigInsights Scientist Free Quick Start (non production): IBM Open Patform BigInsights