Supercomputing on Windows. Microsoft (Thailand) Limited

Transcription

1

2 Supercomputing on Windows Microsoft (Thailand) Limited

3 W hat D efines S upercom puting A lso called High Performance Computing (HPC) Technical Computing Cutting edge problems in science, engineering and business require capabilities beyond those of standard desktops and servers

4 Business Computing Consumer Computing Technical Computing

5 Computation Transforming The Sciences Life Sciences Social Earth Sciences Sciences Multidisciplinary Research Computer & Information Sciences New Materials, Technologies & Processes Math and Physical Science

6 Technical Computing Computational Modeling Persistent Distributed Storage Real-World Data Workflow Data Mining & Algorithms Interpretation & Insight Computational Modeling

7 High Performance Computing Cutting edge problems in science, engineering and business always require capabilities beyond those of standalone computers Market pressures demand accelerated innovation cycle, overall cost reduction and thorough outcome modeling Aircraft design utilizing composite materials Vehicle fuel efficiency and safety improvements Simulations of enzyme catalysis, protein folding Reservoir Simulation & Seismic Processing Targeted material and drug design Simulation of nanoscale electronic devices Financial portfolio risk modeling Digital content creation and enhancement Supply chain modeling and optimization Volume economics of industry standard hardware and commercial software applications are rapidly bringing HPC capabilities to broader number of users

8 S om e C larification on T erm inology HPC Systems Shared Memory Distributed Memory Dedicated Compute Clusters Non-Dedicated Clusters Collections of computers, networked together and acting as a single large machine E ach n o d e is dedicated to the compute cluster Collections of computers, networked together and acting as a single large machine N o d es h ave d ay-jo b s and can only provide cycles when unused

9 Entry Price Point for HPC Has Dropped

10 Challenges in HPC Today End Users.. w ant to do m ore science & less IT..apps not keeping up w ith ideas App availability & integration Simpler job submission & monitoring IT Pros Developers takes a village to raise a cluster...parallel program m ing is difficult..not enough tools Easier deployment and setup Simplified mgmt environment Integrated & complete development environment Platform standards

11 M icrosoft s V ision for H P C Empower scientists and engineers to interactively employ massive computational resources to solve complex technical problems Make distributed computing resources productive to develop for and costeffective to operate

12 Windows Compute Cluster Server 2003 Mission: Deliver the easiest to deploy and most cost effective solution for solving scaled-out business, engineering and scientific computational problems. Windows Server 2003, Compute Cluster Edition Support for high performance hardware (x64bit architecture) RDMA support for high performance interconnects (Gigabit Ethernet, Infiniband, Myrinet, and others) + Compute Cluster Pack = Support for Industry Standards MPI2 Integrated Job Scheduler Cluster Resource Management Tools Microsoft Windows Compute Cluster Server 2003 Integrated Solution out- of-the-box Leverages investment in Windows administration and tools Makes cluster operation easy and secure as a single system

13 Compute Cluster Server 2003 (CCS) Complete, integrated platform for computational clustering Core Platform Windows Server 2003 Compute Cluster Edition x64 only Cluster Tools Microsoft Compute Cluster Pack Built-in Cluster Mgmt, Job Scheduler & MPI Interconnect support leveraging Winsock Direct Availability Bits at

14

15 Reducing Time to Insight in Cancer Research

16 Demo Architecture FDA-NCI Clinical Proteomics Website Nightly download of new datasets SQL Business Integration Services Signal conditioning and preprocessing Windows CCS Cluster Feature extraction SQL 2005 Interoperability with Linux/Grid 16 node Dell cluster LSF + Linux + Matlab Windows PC Matlab Classification & training Cancer diagnostics 4 node personal cluster Windows CCS + Matlab Remote integration, Windows CCS scalability 128 node cluster in Dupont Windows CCS + Matlab

17 Windows Compute Cluster Server 2003 Head Node Active Directory Job Mgmt Scheduling Cluster Mgmt Resource Mgmt Desktop App User Jobs Policy, reports Job Mgr UI Admin Console Input Cmd line Tasks Management Cmd line Admin Domain\UserA DB/FS Data High speed, low latency interconnect Node Manager Job Execution User App MPI

18 CCS Key Features Easier node deployment and administration Task-based configuration for head and compute nodes UI and command line-based node management Monitoring with Performance Monitor (Perfmon), Microsoft Operations Manager (MOM), Server Performance Advisor (SPA) & 3 rd -party tools Extensible job scheduler 3 rd -party extensibility at job submission and/or job assignment Submit jobs from command line, UI, or directly from applications Simple job management, similar to print queue management Secure MPI User credentials secured in job scheduler and compute nodes Microsoft provided stack reduces application/mpi incompatibility issues Integrated Development Environment OpenMP Support in Visual Studio, Standard Edition Parallel Debugger in Visual Studio, Professional Edition

19 Leveraging Existing Windows Infrastructure Integration with IT infrastructure Kerberos authentication Resource management Group policies Secure job execution Secure MPI Job scheduler Operations manager Windows Update services Systems Management Server Admin console Performance monitor Command line interface Remote Installation services

20 Scaling Excel Excel Services on Windows Compute Cluster Server 2003 Excel Services Microsoft Office Excel 2007 Desktop Servers Clusters

21 Scaling Excel

22 Scaling Excel

23 Simplified Administrator Interface 27

24 Job Scheduler Stack User Admin Third-party Applications User Console Admin Console Jobs/Tasks Client Node Command Line Interface COM API Object Model WS (WSE 3.0) Interface Layer Admission User Interface Handlers Head Node Queueing Scheduling Layer Allocation Job Management Resource Management Execution Layer Activation Compute Node Node Manager Node Manager Node Manager Node Manager Node Manager Node Manager 28

25 2 Popular Methods of Message Passing for HPC OpenMP Works in shared memory situations only- inside one physical chassis Easy to program- simply identify parallelizable loops with pragmas Message Passing Interface (MPI) Works in shared memory (single chassis) and cluster (multiple chassis) situations. Ample MPI API Fine-grained control of messaging between processes Collect and reduce data in a single call Requires moderate-advanced level of programming Harder than OpenMP but MUCH easier than sockets

26 A D eveloper s V iew of C luster C onnectivity Developer Computer on a Corporate Network Developer Computer Head Node SDK Compute Node Compute Node Compute Node Remote Debug Remote Debug Remote Debug

27 Visual Studio 2005 Engineered to provide support for development of HPC applications. Parallel Debugger OpenMP Support

28 Resources Microsoft HPC Web site CCS Community Site Get the Facts Web site Communication from the Open Source Software Microsoft Thai Software Developer Community

29 Q&A

30