WCF als Schnittstelle zwischen Industrieanwendung und HPC Cluster

Transcription

1 WCF als Schnittstelle zwischen Industrieanwendung und HPC Cluster Steffen Claus, Horst Schwichtenberg, Karsten Reineck

2 Agenda Industrial application (optimization algorithm) Idea: Exploiting HPC resources for remote computing Using WCF as interface to HPC resources Conclusion

3 Industrial application - AutoNester Solves optimization problem: Placing pieces on fabrics/leather while keeping multiple contraints Optimization contrained by: size of fabrics, shape of pieces, pattern of fabrics (e.g. square pattern), relative position of pieces, quality of fabrics/leather, etc. Benefit of automatic optimization: 1.) minimize waste, 2.) faster results (compared to humans)

4 Visualization of optimization algorithm Demo

5 Automatic placing of patterns on leather hides Map a set of tiles (of car seats, sofas etc.) onto a leather hide quality of leather varies (e.g. holes) additional quality constraints

6 The idea: use remote HPC resources for solver So far: User of application runs the software locally optimization algorithm runs locally on the users machine Idea: Extract optimization algorithm and run it on Windows HPC cluster (transparent for the user) Why bother? MPI-version of the algorithm exists exploitation of cluster resources, faster results with even better efficiency Benefits for software vendor: more sophisticated accounting models (e.g. pay per computation) Benefits for the software developer: straightforward and direct update of the optimization code Benefits for the software user: faster results

7 Technologies for remote access to HPC resources Sockets based (low level based client/server interaction).net Remoting (no real flexibility on client side; integration into existing infrastrucutures might be a barrier) HPC Server SDK by API by Commandline by Powershell our choice: WCF

8 Windows Communication Foundation (WCF) Microsoft API for SOA applications Part of the.net Framework Supports most common SOA technologies and standards: WS-*, WSDL, SOAP etc. Security, Transactions, Bindings (HTTP/HTTPS, NetTCP ) Why did we choose WCF? WCF can easily be integrated into existing code Does not restrict the developer to use a specific language (if WSDL and XSD descriptions of the interface are available) Easiest integration into existing infrastructures (regarding firewall issues) Largest coverage of platforms and languages

9 WCF and HPC cluster Alternative solutions Alternatives for WCF in HPC cluster 1. selfhosted 2. via WCF Broker 3. hosted by IIS our choice: WCF hosted by IIS. Why IIS? 1. Selfhosted is only suitable for small/testing applications (not as stable as IIS) 2. WCF Broker is rather transient and it requires the client to use the HPC SDK; head node must be reachable from the internet 3. IIS is suitable for a stable, always-on service Multiple monitoring threads for running jobs; signal channels for interaction between client and solver

10 Interaction between Application and HPC Cluster Company LAN 2. Authentication and Authorization SCAI Private Network Authentication Client 1. web service DLL computes remote Products Accounting 5. Results and Accounting 6. Results 3. Jobs are submitted to HPC scheduler 4. Jobs Internet Service Provider Head node Compute nodes

11 Features of the remote compute service Interactivity: user is able to see the optimization steps live ; user can decide when to stop the optimization (e.g. once he is satisfied with the result) Job recognition: optimization problems that have already been solved are recognized (don t waste compute resources) Modular Accounting: based on the application, suitable accounting models can be added Role Based Authorization

12 Conclusion Industrial application: placing pieces onto fabrics/leather; optimization problem Idea: extract solver and execute on HPC cluster (esp. MPI version) Solution: WCF services as interface between application and HPC resources Easily accessible, platform-independent, interactive access to HPC resources

13 Thank you!