REFERENCE Microsoft in HPC Tejas Karmarkar, Solution Sales Professional, Microsoft
Agenda What is HPC? MSC.Software Confidential Microsoft Vision of HPC Microsoft solution & Ecosystem Architecture Proof points Windows HPC = Reduced cost of infrastructure Close
What is HPC? MSC.Software Confidential HPC is computing infrastructure to accelerate innovation Concept / Goal Setting Design & Pre-Processing Testing &/ Simulation Analysis Post processing Design Simulate Analyze Result HPC
Benefits of HPC MSC.Software Confidential
Microsoft s Vision for HPC Provide the platform, tools and broad ecosystem to reduce the complexity of HPC by making parallelism more accessible to address future computational needs. Reduced Complexity Mainstream HPC Developer Ecosystem Ease deployment for larger scale clusters Simplify management for clusters of all scale Integrate with existing infrastructure Address needs of traditional supercomputing Address emerging cross-industry computation trends Enable non-technical users to harness the power of HPC Increase number of parallel applications and codes Offer choice of parallel development tools, languages and libraries Drive larger universe of developers and ISVs
Today s Environment High Speed networking Corporate Infrastructure Clusters/Super Computers Storage Engineers Information workers Specialized languages Compilers Mainstream Technologies Debuggers
High Productivity Computing Combined Infrastructure Integrated Desktop and HPC Environment Unified Development Environment
Application Benefits The most productive distributed application development environment MSC.Software Confidential Microsoft HPC++ Solution Cluster Benefits Complete HPC cluster platform integrated with the enterprise infrastructure System Benefits Cost-effective, reliable and high performance server operating system
Strong Ecosystem with tailored solutions offering
Architecture MSC.Software Confidential Head Node Active Directory Job Mgmt Cluster Mgmt Web Services Scheduling Resource Mgmt User Desktop App Jobs Policy, reports Admin Console Input Job Mgr UI Cmd line Tasks Management Cmd line Admin Domain\UserA DB/FS Data High speed, low latency interconnect Node Manager Job Execution User App MPI
Turnkey Cluster Solution Overall Cluster Setup Head node Instal l x64 OS Install Cluster solution (CCP) Configure Network Configure Node Deployment Compute node Boot WinP E Deplo y Node s Image Head node Run Diagnostic s Network Setup ToDoList Network Configuratio n Wizard Configure for install and update of networking drivers on nodes via Deployment Management Network tests via Diagnostic s Wunderbar
List or Heat Map view cluster at a glance Group compute nodes based on hardware, software and custom attributes; Act on groupings. Receive alerts for failures Track long running operations and access operation history Pivoting enables correlating nodes and jobs together
Skip/Demo Integrated Job Scheduling Services oriented HPC apps Expanded Job Policies Support for Job Templates Improve interoperability with mixed IT infrastructure
Skip/Demo Node/Socket/Core Allocation Windows HPC Server can help your application make the best use of multi-core systems Node 1 Node 2 P0 P1 P0 P1 P0 P1 P0 P1 S0 J1 S1 J1 S0 S1 P2 P3 P2 P3 P2 P3 P2 P3 J2 P0 P1 J3 P0 J3 P1 P0 P1 P0 P1 S2 P2 J1 P3 S3 J3 P2 J3 P3 S2 P2 P3 S3 P2 P3 J1: /numsockets:3 /exclusive: false J3: /numcores:4 /exclusive: false J2: /numnodes:1
Job submission: 3 methods Command line Job submit /headnode:clus1 /Numprocessors:124 /nodegroup:matlab Job submit /corespernode:8 /numnodes:24 Job submit /failontaskfailure:true /requestednodes:n1,n2,n3,n4 Job submit /numprocessors:256 mpiexec \\share\mpiapp.exe [Completel Powershell system mgmt using commands Microsoft.Hpc.Scheduler; are available as well] class Program { static void Main() { IScheduler store = new Scheduler(); store.connect( localhost ); ISchedulerJob job = store.createjob(); job.autocalculatemax = true; job.autocalculatemin = true; ISchedulerTask task = job.createtask(); task.commandline = "ping 127.0.0.1 n *"; task.isparametric = true; task.startvalue = 1; task.endvalue = 10000; task.incrementvalue = 1; task.minimumnumberofcores = 1; task.maximumnumberofcores = 1; job.addtask(task); store.submitjob(job, @"hpc\user, "p@ssw0rd"); } }
Scheduling MPI jobs Job Submit /numprocessors:7800 mpiexec hostname Start time: 1 second, Completion time: 27 seconds
Proof Points Spring 2008, Aachen, #100 2096 cores, 18.8 TF, 76.5% MSC.Software Confidential Spring 2008, NCSA, #23 9472 cores, 68.5 TF, 77.7% Spring 2008, Umea, #40 5376 cores, 46 TF, 85.5% Fall 2007, Microsoft, #116 2048 cores, 11.8 TF, 77.1% 30% efficiency improvement Spring 2007, Microsoft, #106 2048 cores, 9 TF, 58.8% Windows HPC Server 2008 Windows Compute Cluster 2003 Spring 2006, NCSA, #130 896 cores, 4.1 TF
Proof Points Ready for Prime-time Location Champaign, IL Hardware Machines Dell blade system with 1,200 PowerEdge 1955 dual-socket, quad-core Intel Xeon 2.3 GHz processors Hardware Networking InfiniBand and GigE #23 Summer 2008 Number of Compute Nodes 1184 Total Number of Cores 9,472 cores Total Memory 9.6 terabytes Particulars of for current Linpack Runs Best Linpack rating 68.5 TFPs Best cluster efficiency 77.7% For Comparison Linpack rating from November 68.5 TFPs 2007 Top500 run (#14) on the same hardware Cluster efficiency from November 2007 Top500 run (#XX) on the 69.9% same hardware Typical Top500 efficiency for Clovertown motherboards w/ IB 65-77% regardless of Operating System 7.8% improvement in efficiency on the same hardware running Linux About 4 hours to deploy
Proof Points November 2008 Top500
How can we reduce cost? For customers new to HPC Improved hardware/software asset utilization Asset consolidation Improved productivity of engineers More simulation = Less physical testing = Reduced cost Wider adoption HPC in product development process = shorter design cycle Higher fidelity simulation = Improved quality, reduced warranty cost
4/22/2009 28
Closing MSC.Software Confidential Windows HPC can give you best return on your hardware and engineering software assets. Windows HPC can accelerate product development process Microsoft has strong ecosystem in HPC space TCO can be significantly low with Microsoft's HPC solution Visit us on the web @ http://www.microsoft.com/hpc
Contact Details : For further information please contact Tejas Karmarkar Microsoft tejaskar@microsoft.com 22/04/2009 31