Microsoft Compute Clusters in High Performance Technical Computing Björn Tromsdorf, HPC Product Manager, Microsoft Corporation
Flexible and efficient job scheduling via Windows CCS has allowed more of our employees to utilize the HPC cluster resources, which frees up time of professionals to focus on increasing their productivity. Faure Louw Group Leader Structural Analysis PBMR, South Africa An integrated HPC simulation environment is key to supporting product innovation
Entry Price-Points For HPC Has Dropped 1991 1998 2005 Sample System Performance Cray Y-MP C916 Sun HPC10000 Shuttle@ NewEgg.com ~10 GFlops ~10 GFlops ~10 GFlops Price Customers $40,000,000 $1,000,000 (40x drop) $4,000 (250x drop) Government Labs Large Enterprises Every Engineer and Scientist
COTS Components Gaining Ground GigE is gaining (50% of systems) X86 is leading (Pentium 41%, EM64T 16%, Opteron 11%
Distributed Memory Becoming The Norm Clusters over 70%
Challenges In HPC Today want to do get more work done and spend less time on IT takes a village to raise a cluster parallel programming is difficult not enough tools App availability and integration Simpler job submission and monitoring Easier deployment and setup Simplified mgmt environment Integrated and complete development environment Platform standards
Client App Job Console Job Scheduler Management Interface Cluster Application MPI Stack Operating System Networking Hardware Deployment
Simplified Setup Add Head Node to AD Install Compute Cluster Pack and its prerequisites Follow 4 step Todo Wizard 1. Assign networking interfaces 2. Configure Remote Installation Services 3. Add Compute Nodes 4. Manager cluster users
Basic Cluster Architecture
Compute Cluster Server
MD - Compute Cluster Server Enabled Active Directory File Server MOM Server Mail Server Workstation Public (corporate) Network Head Node MS-MPI Interconnect Private Network Computer Node Computer Node Computer Node
Leveraging Existing Infrastructure Active Directory Integration with IT Infrastructure Resource Management Group Policies Windows Security Kerberos Authentication Secure Job Execution Secure MPI Microsoft Enterprise Management Tools Operations Manager Windows Update Services Systems Management Server Built-in Tools Job Scheduler Admin Console Performance Monitor Command Line Interface Remote Installation Services
CCS Key Features Node Deployment and Administration Task-based configuration for head and compute nodes UI and command line-based node management Monitoring with PerfMon, MOM, SPA & 3 rd party tools Integration with existing Windows and management infrastructure Integrates with Active Directory, Windows security technologies, management, and deployment tools Extensible job scheduler 3 rd party extensibility at job submission and/or job assignment Submit jobs from command line, UI, or directly from applications Simple job management, similar to print queue management Secure MPI User credentials secured in job scheduler and compute nodes Standardized MPI stack Integrated Development Environment Parallel Debugger and OpenMP Support in Visual Studio
Benchmark vs. Linux MSC.Nastran - 5 Standard Performance Benchmarks - serial jobs Relative Performance (bigger is better) 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 lg0qdf0 xl0emf0 xl0tdf1 xx0cmda xxafst0 Configuration: HP Cluster Platform 3000 Benchmark courtesy of HP Linux RH4 U2 4GB memory Windows CCS RC1 4GB memory
Partners
CCS Resources Microsoft HPC Web site http://www.microsoft.com/hpc/ MS HPC Community - http://www.windowshpc.net/ Email contact - hpcinfo@microsoft.com Windows Server x64 information http://www.microsoft.com/64bit/ http://www.microsoft.com/x64/ Windows Server System information http://www.microsoft.com/wss/ Get the Facts Web site http://www.microsoft.com/getthefacts
For Microsoft internal use only 2007 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.