Windows Compute Cluster Server Top 5 Most Common Challenges

Transcription

1 Agenda Industry and Market Trends Customer Challenges Windows Compute Cluster Server 2003 Application development Grid computing strategy Summary Sandro Cambruzzi Matej Ciesko Microsoft NCSA Windows Cluster, 906 CPUs, 4.1 TFLOPs Top 500 Supercomputer Trends HPC Market Trends Industry usage rising Clusters over 70% GigE is gaining (50% of systems) x86 is leading (Pentium 41%, EM64T 16%, Opteron 11%) Entry price point for HPC has dropped Customer Challenges

2 Top Challenges Setup is painful Takes a long time to get clusters up and running Keeping systems updated is difficult Lack of integration into IT infrastructure Job management Lack of integration into end-user apps Application availability Limited eco-system of application that can exploit parallel processing capabilities Make high-end computing easier and more productive to use. Emphasis should be placed on time to solution, the major metric of value to high-end computing users A common software environment for scientific computation encompassing desktop to high-end systems will enhance productivity gains by promoting ease of use and manageability of systems. High-End Computing Revitalization Force, 2004 (Office of Science and Technology Policy, Executive Office of the President) Technical Overview CCS Key Features Windows Compute Cluster Server 2003 Node Deployment and Administration - based configuration for head and compute nodes UI and command line-based node management Monitoring with PerfMon, MOM, SPA & 3 rd party tools Integration with existing Windows and management infrastructure Integrates with Active Directory, Windows security technologies, management, and deployment tools Extensible job scheduler 3 rd party extensibility at job submission and/or job assignment Submit jobs from command line, UI, or directly from applications Simple job management, similar to print queue management Secure User credentials secured in job scheduler and compute nodes Standardized stack Microsoft provided stack reduces application/ incompatibility issues Integrated Development Environment Parallel Debugger and OpenMP Support in Visual Studio Active Directory User Input DB/FS Web Services Desktop App Job Mgr UI Cmd line Data Jobs Domain\UserA Head Node Job Mgmt Cluster Mgmt Scheduling Resource Mgmt s Node Manager Policy, reports Admin Console Admin Cmd line High speed, low latency interconnect Job Execution User App

3 Windows Compute Cluster Server 2003 Leveraging Existing Windows Infrastructure Mission: Deliver the easiest to deploy and most cost effective solution for solving scaled-out business, engineering and scientific computational problems. Active Directory Integration with IT Infrastructure Resource Windows Security Kerberos Authentication Secure Job Execution Windows Server 2003, Compute Cluster Edition Support for high performance hardware (x64bit architecture) RDMA support for high performance interconnects (Gigabit Ethernet, Infiniband, Myrinet, and others) + Compute Cluster Pack = Support for Industry Standards 2 Integrated Job Scheduler Cluster Resource Tools Microsoft Windows Compute Cluster Server 2003 Integrated Solution out- of-the the-box Leverages investment in Windows administration and tools Makes cluster operation easy and secure as a single system Group Policies Microsoft Enterprise Tools Operations Manager Windows Update Services Systems Server Secure Compute Cluster Built-in in Tools Job Scheduler Admin Console Performance Monitor Command Line Interface Remote Installation Services Public Network Compute Cluster Topology AD DNS DHCP Corporate IT Infrastructure Windows Update Monitoring MOM 3 rd party Systems SMS 3 rd party Key Features Integration with existing infrastructure Leverage Active Directory and Windows security No special permissions required for installation of cluster Integrate with MS management technologies (MOM, SMS etc) Private Network Compute Node Node Manager Compute Cluster Compute Node Node Manager Head Node Admin / User Cons RIS Job Scheduler NAT Network Simplified administration and node deployment based configuration for head node and adding nodes Multiple ways to add compute nodes Node management through UI and command line Monitoring with Perfmon, MOM, or 3 rd party management tools Job Scheduler Job Scheduling Job ordering Priority- based first- come, first- serve (FCFS) Backfill supported for jobs with time limits Resource allocation License- aware scheduling through plug- ins Parallel application node allocation policies Extensible Third parties can extend by using standard submission and execution Partnering with third parties for deeper integration in the future Job submission Jobs submitted via several methods (e.g. Web, command line) Cleanup Jobs handled uniformly, facilitates easy and complete cleanup

4 Job Life Cycle Objects Descriptions Concepts Cluster The top- level organizational unit of CCP; comprising a set of nodes, a queue and jobs. Queue An organization unit which provides queuing and scheduling of jobs. There is only one queue in a CCP cluster and it contains pending, running and completed jobs. Completed jobs are purged periodically from the queue. Job Node A collection of tasks, using a task to perform a computation, initiated by a user. Jobs are used to reserve resources for subsequent use by one or more tasks. A task represents the execution of a program on some compute nodes. A task can be a serial program (single process) or an program with multiple processes A single compute node, with one or more processors Serial Job Job/ Conceptual Model Parallel Job IPC Parameter Sweep Job Flow Job Queued State Transition Re-queue op Re-queue op Resource allocated Cancel op Cancel Op Failed Run Cancelled Non-zero exit code System failure Run to completion (exit 0) Finished Which Interface to use Interface Best Suited for CLI Any language: Perl, C/C++, Fortran, C# Job Manager COM End user (job submission, monitoring and control) C/C++, VBscript and Perl Web Services Any languages: C#, C++, Java Designed for third party scheduler interop Command Line Interface Job job new [job_terms] create a job container job add jobid [task_terms] add tasks to a job job submit /id:jobid submit job / tasks job submit [job_terms][task_terms] submit a job job cancel jobid cancel a job job modify [options] modify a job job requeue JobID requeue a job job list list jobs in the cluster job listtasks list tasks of a job job view JobID view details of a job task view taskid view details of a task task cancel taskid cancel a task task requeue taskid requeue a task Cluscfg cluscfg view view details of a cluster cluscfg params/setparams view/set config params cluscfg listenvs/setenv list / set cluster wide env cluscfg delcreds/setcreds set /delete user cred

5 Job Submission rem Run a serial job job submit /stdout:%pdcshare%\batchpi10.out %PDCSHARE%\batchpi.exe 10 rem Run a parallel job job submit /stdout:%pdcshare%\batchpi100.out /numprocessors:2 mpiexec -hosts ^%CCP_NODES^% %PDCSHARE%\batchpi.exe 100 rem Run a parametric sweep job for /F "usebackq tokens=4" %j in (`job new`) do set JOBID=%j for /L %i in (100, 100, 1000) do job add %JOBID% /stdout:%pdcshare%\batchpi%i.out %PDCSHARE%\batchpi.exe %i job submit /id:%jobid% setup finish compute rem Run a task flow job for /F "usebackq tokens=4" %j in (`job new`) do set JOBID=%j job add %JOBID% /name:setup /stdout:%pdcshare%\setup.out cmd.exe /c echo Running Setup %TIME% for /L %i in (100, 100, 1000) do job add %JOBID% /name:compute /depend:setup /stdout:%pdcshare%\batchpi%i.out %PDCSHARE%\batchpi.exe %i job add %JOBID% /name:finish /depend:compute /stdout:%pdcshare%\finish.out cmd.exe /c echo Running Finish %TIME% job submit /id:%jobid% Application Development Simplicity through abstraction Compute Cluster Administrator Cluster Job Node ClusterCounter ResourceUsage Compute Cluster Job Manager CCS COM interface Private Sub CommandButton1_Click() Set objcluster = CreateObject("Microsoft.ComputeCluster.Cluster") objcluster.connect (Range("Cluster").Value) Dim objjob, obj Set objjob = objcluster.createjob() Set obj = objcluster.create() obj.commandline = "mpiexec -hosts:mshpch1 -n 2 pi.exe" & Range("Intervals").Value objjob.minimumnumberofessors = Range("s").Value objjob.priority = JobPriority(Range("Priority").Value) obj.stdout = WorkDir & Range("OutputFile").Value objjob.add (obj) ID = objcluster.submitjob((objjob)) Range("JobID").Value = ID End Sub 30

6 Web Service Job Submission Protocol Job Sessions CreateJobSession(schedulerEPR, resourcedescr) {sessionepr, sessiondescr} JobSessionStatus(sessionEPR) sessiondescr JobSessionOperation(sessionEPR, opndescr) sessiondescr Jobs CreateJob(sessionEPR, jobdescr) {jobepr, statusdescr} CreateStandaloneJob(schedulerEPR, resourcedescr, jobdescr) {jobepr, statusdescr} JobStatus(jobEPR) statusdescr JobOperation(jobEPR, opndescr) statusdescr Standard, Integrated Development Cluster scheduler programmable via.net and DCOM Visual Studio 2005 OpenMP, Parallel Debugger Support for 2 based on a reference implementation Standardized stack User credentials secured in job scheduler and compute nodes Performance improvements that take advantage of advanced Windows I/O architecture Support for multiple advanced interconnect types via Winsock Direct architecture Microsoft provided stack reduces application/ incompatibility issues 31 Visual Studio 2005 Engineered to provide support for development of HPC applications. Integration with Office Parallel Debugger OpenMP Support 34 Grid Is A Catch-All Marketing Term Grid computing Grid computing means many different things to many different people/companies: Desktop cycle-stealing Managed HPC clusters Internet access to giant, distributed repositories Virtualization of data center IT resources Out-sourcing to utility data centers Originally this was all called Distributed Systems

7 Challenge: Unified Grid Computational grids have difficulty with modeling and exploiting heterogeneous network environments Data grids are in their infancy: we barely know how to build petabyte-sized data repositories, let alone deal with computations that must join data from multiple distributed, possibly replicated, data repositories of such size IT virtualization is a challenge even within a controlled data center, let alone across an entire organization or a virtual federation of organizations We Know How To Build The Underlying Low-Level Platform Grid computing requires a distributed systems platform that enables broadly speaking the following: Interoperable means of communicating between well-defined system parts, both within and between organizations and dynamic control/configuration of IT resources Microsoft Is Providing This Underlying Functionality Web Services provides the interoperable means of communicating between entities within a distributed system Microsoft s HPC cluster group and its Dynamic Systems Initiative (DSI) are addressing the question of management and dynamic control/configuration of IT resources HPC group is focused on clusters DSI is focused on enterprise IT resources Web Services + HPC Windows Clusters + SQL Server + Dynamic Systems Initiative = Support for Grid Computing Projects TerraService Give programming interface to the images and data contained on TerraServer Launched June 2000 Web Service created in 3 days SkyServer - Access to Federated Astronomy Databases (Fermilab/SDSS, JHU/FIRST, Caltech/2MASS) Live example of Web Services in e-science Virtual Observatory - Build your own - OR Eye Ohio State Medical Center allows authorized users to securely and remotely monitor, record, and replay vital signs data that is generated in operating rooms and intensive care units HPC Resources Microsoft HPC Web site MS HPC Community - contact - hpcinfo@microsoft.com Windows Server x64 information Windows Server System information Get the Facts Web site

8 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.