Wissenschaftliches Rechnen im Paderborner Umfeld

Transcription

1 Wissenschaftliches Rechnen im Paderborner Umfeld ZKI Arbeitskreis Supercomputing 14. März 2013 Prof. Dr. Holger Karl, Dr. Jens Simon paderborn.de, paderborn.de 3/18/2013 1

2 Paderborn University Students total: BA: 7.390, MA: 1.746, Lehramt: 6.767, PhD candidates: 815 Staff total: Professors: 196, Scientific.: 919, Non Scientific: 634 Budget: 180 Mio. Third party funding (2011): 37,8 Mio. 3/18/ Status 2012

3 Paderborn University Faculties Faculty of Arts and Humanities Faculty of Business Administration, Economics and Business Computing Faculty of Science Faculty of Mechanical Engineering Faculty of Computer Science, Electrical Engineering and Mathematics Research 9 central research institutes 3 DFG collaborative research centers (SFB) 1 DFG research group 6 graduate colleges / schools 2 Fraunhofer institutes Status /18/2013 3

4 Computer UPB Informatik (2012) Students total: Starters (only BA): 164 Bachelor /Master graduates: 146 Professors: 21 Scientific Staff: 100 Third party funding: ca. 7 Mio. Rankings of the last years DFG (national science foundation): 7/129 in approval of science funds CHE ranking: leading position among 77 universities regarding 3rd party funds, teaching, infrastructure, research reputation ACM: best German institution for software engineering 3/18/2013 4

5 IT Organization Structure IMT PC² ITD General infrastructure Networks (WAN, LAN, WLAN, VoIP, ) Servers ( , Web, hosts, VMs, ) Storage (Backup, Archive,..) IT security Digital media (recording and live streaming of lectures) Scientific, high performance, high throughput computing Path finding (together with Informatik) IT services for university administration 3/18/2013 5

6 Paderborn Center for Parallel Computing PC² Regional competence center for parallel and distributed data processing in research and application Founded in 1991 Scientific institute of the Paderborn University Goal: Foster and explore efficient use of parallel and distributed systems Funding from regional and federal government, industrial and EUprojects 3/18/

7 PC2 Service & Research An infrastructure provider and service center for highperformance computing Main customers: UPB members and surrounding universities Engineering and natural sciences communities A research institute, working on high performance computing and cloud computing topics In collaboration with several chairs of computer science 3/18/2013 7

8 Compute Center Located in building O, 2 nd floor, main campus Facility available since 2011, shared by IMT and PC² Ca. 400 m 2 floor space (200 m² dedicated to HPC) Electrical Power Up to 550 KW for HPC Up to 200 KW for basic IT services and server hosting 2 x 200 KW UPS (batteries) and 1.6 MW Diesel Generator Cooling Air cooling (1/3) Grad Celsius room temperatures, Cold Aisle Containments Water cooling (2/3) 20/26 Grad Celsius, inrow cooling/backdoors (new HPC system) 3/18/2013 8

9 Compute Center Physical protection Security gate (two doors) with key card access Early fire detection system Nitrogen induction fire suppression system Certification possible (not done yet) Network security Hierarchical Firewall structure for PC² Availability Redundancy of servers for important services, data backup 3/18/2013 9

10 Resources of PC² Cluster OCuLUS [612 compute nodes, >200TFLOPS] (03/2013) Arminius+ [60 compute nodes] PLING2 [57 compute nodes] BIS Grid [8 compute nodes] High Throughput [varying compute nodes] Cloud Green PAD [54 nodes Paderborn, 38 nodes Mainz] Scientific Cloud [38 nodes] Misc 32 nvidia K20, 8 Intel Xeon Phi 2*Maxler FPGA compute nodes, 1* Convey HC FPGA Storage ISILON IQ9000x NAS 50 TB ISILON x400 NAS 800 TB (05/2013) FhGFS parallel filesystem 450TB SMP RX600 with 4 proc. sockets, 24 cores, 128GB 2 x Dell R820 with 4 proc. sockets, 32 cores, 1TB 3/18/

11 OWL Cluster OCuLUS processor cores, 200 TFLOPS, 45 TByte main memory 612 compute nodes, each 2 x Intel E (2.6GHz) 552 nodes with 64 GByte main memory 20 nodes with 256 GByte 32 nodes with 64 GByte and accelerator Nvidia Tesla GK110 8 nodes with 64 GByte and accelerator Intel Xeon Phi 5110P 2 SMP nodes, each 4 x Intel E (2.7GHz) and 1 TByte main memory Parallel Storage System 500 TByte disk capacity Fraunhofer FhGFS 7 Storage nodes and 2 Meta data nodes InfiniBand Interconnect 40 Gbit/s Mellanox HCA and switches Half bisectional bandwidth Redundant Login nodes for Linux and Windows Wikimedia Commons 3/18/

12 Research Topics On going Resource management: Own system, planning based (OpenCCS) Integrating hardware accelerators into HPC (e.g., FPGA) Energy efficient HPC (e.g., project GreenPAD) Near future Support for big data workloads in HPC clusters (e.g., Hadoop, Map/Reduce jobs), supported by resource management Support for software defined networking (SDN) in clusters Mid term SDN and big data workloads jointly integrated in & supported by a cluster resource management system Leverages new Isilon based storage infrastructure 3/18/

13 Research Projects Computer Architecture Custom Computing EPiCS: Engineering Proprioception in Computing Systems (EU) Enhance: Enabling heterogeneous hardware acceleration using novel programming and scheduling models (BMWF) Gomputer: Accelerate Computer Go (Gomputer) (Industry) Middleware & System Software Infrastructure for On the Fly Computing (SFB 901) GreenPAD: Energieoptimierte IKT für regionale Wirtschafts und Wissenscluster (BMWi) Simba: Simulationsplattform für die Automobilentwicklung (BMWi ZIM) Data Center Management: Virtualization technology (Industry) OpenCCS: Resource management system (Long term project) SCALUS: Scalable, Ubiquitous Storage (EU) Data Center 3/18/

14 Collaborative Research Centre On-The-Fly Computing Individualized IT-Services in Dynamic Markets Heinz Nixdorf Institut, Universität Paderborn Friedhelm Meyer auf der Heide GIBU 2012 (Schloss Dagstuhl)

15 The Vision of On-The-Fly Computing Nearly automatic configuration and execution of individual IT services, which are constructed out of base services traded in worldwide available markets Organization of these markets of services Heinz Nixdorf Institut, Universität Paderborn configuration execution SoA Cloud Computing GIBU 2012 (Schloss Dagstuhl) market support Distributed Comp. market organization Mechanism Design

16 The actors in the On-The-Fly Computing markets User/Client OTF Provider Supplier OTF Compute Center OTF Software Provider Heinz Nixdorf Institut, Universität Paderborn Provision of IT services GIBU 2012 (Schloss Dagstuhl) Market organization

17 Challenges for the provision of IT services User/Client s OTF Provider Supplier OTF Compute Center OTF Software Provider How to configure IT services out of base services? Project Area B: Modeling, Composition and Quality Analysis of Services Heinz Nixdorf Institut, Universität Paderborn How to ensure the correct, reliable, and efficient execution of configured services? GIBU 2012 (Schloss Dagstuhl) Project Area C: Reliable Runtime Environments & Application Scenarios

18 Challenges with the market organization How can interactions in large dynamic markets be supported and protected? How can an efficient functionality of the market be guaranteed with strategic behavior of the participants? Project Area A: Algorithmic and Economic Foundations Heinz Nixdorf Institut, Universität Paderborn How can the usefulness of our concept be demonstrated? GIBU 2012 (Schloss Dagstuhl) Project Area C: Reliable Runtime Environments & Application Scenarios

19 SFB 901: Structural Overview Project Area A Algorithmic and Economic Foundations A1: Local Strategies in Dynamic Networks (Meyer auf der Heide, Scheideler) A2: Overlays over Physical Networks (Frey, Karl) A3: Market of Services (Briest, Frick, Haake) Project Area B Modeling, Composition & Quality Analysis of Services B1: Parametric Service Specifications (Engels, Schäfer) B2: Configuration & Rating (Kleine Büning, Kleinjohann) B3: Composition Analysis in Uncertain Contexts (Becker, Wehrheim) B4: Proof-Carrying Services (Platzner, Wehrheim) Heinz Nixdorf Institut, Universität Paderborn Project Area C Reliable Runtime Environments & Application Scenarios C1: Robustness & Security (Blömer, Scheideler, Sorge) C2: On-The-Fly Compute Centers (Meyer auf der Heide, Platzner, Plessl) C3: Modeling of Optimization Problems (Koberstein, Suhl) GIBU 2012 (Schloss Dagstuhl)

20 Entwicklung, Erprobung und Transfer eines energieoptimalen IKT Infrastrukturmodells für regionales Wirtschafts- und Wissenschaftscluster Laufzeit: bis

21 GreenPAD Idee: Regionaler WWC Bündelung der IKT-Infrastruktur an einem Standort (RZ der Uni) (economy of scale -> Energie-Optimierung) Entwicklung attraktiver Service-Modelle für die regionale Wirtschaft (nicht nur, aber auch Cloud-Services -> IaaS, PaaS, SaaS) Zielgruppe in der Projektphase: StartUps, KMU, Pilotkunden vorrangig im Technologiepark und Zukunftsmeile Entwicklung von Übergangsmodellen von regionalen in einen überregionalen Cluster Lastverteilungsverfahren in Abhängigkeit von vorhandener regional erzeugter regenerativer Energien Projektverbund GreenPAD

22 GreenPAD Energiesparpotential Beschaffung Produktion Absatz Energie regionales RZ E.ON IMT Uni/ÖA FUJITSU Technologie und Lösungen PC 2 Unilab HPC Kunden Technologie Park / KMU Smart Grid Konzept Energieeffiziente Komponenten Energieeffizientes RZ (Cloud, Ressourcen Management) Green Office Thin / Zero Clients Energiesparpotential (Maßnahmen kumuliert) Projektverbund GreenPAD

23 Multi Site Cloud Computing Scenario Thousands of Cloud Center Locations Distributed applications (multi tier, P2P,..) Problems Select Cloud Center Matching applications requirements Maximize Service Level Quality Minimize expenses Research Deployment of applications with respect to the geographical distribution 3/18/

24 OpenFlow in Data Centers OpenFlow: centrally computed routing/switching tables Distributed to switches on demand, on per flow basis Possible benefits: Fine grained control, e.g., load balancing Support for data patterns of complex applications Goal: Integrate network configuration with resource management and application knowlegde

25 OpenCCS Planning based Shows planned start times Deadline scheduling Advance reservations usable by multiple groups / users Custom resources Time varying limitations and validities of users / groups Group x gets not more than 4 GPUs on Monday and Friday between 14:00 and 17:30 PBS Pro like resource description 248:ncpus=7:vmem=20g:phi=true+128:ncpus=8:mem=48g, place=scatter:excl Submission rate up to 100Hz Planned Features Management of Virtual Machines Give me 5 Linux and 2 Windows VMs in a separate VLAN using these images Hierarchical Resources User can specify resource (sub )classes: Give me an accelerator whatever available 3/18/

26 OpenCCS on OCuLUS $ccsinfo --allocatable Resource Amount Description Name Used/Avail/Max ====================================== ncpus 0/9840/9856 Cores mem 0/44.8t/45t Phys. Memory vmem 0/54.4t/54.6t Virt. Memory arch - Architecture ib_qdr - Infiniband QDR phi - Intel Xeon Phi tesla - Tesla GK110 $ccsinfo - allocatable - classes Name Class #Hosts Avail/Max ============================ ncpus / /2 mem 64g 592/592 1t 2/2 256g 19/20 vmem 80g 592/592 1t 2/2 264g 19/20 arch SL /614 ib_qdr true 613/614 phi false 605/606 true 8/8 tesla false 581/582 true 32/32 3/18/

27 3/18/

28 And after OCuLUS? Traditional clusters: Performance only relevant metric Today: Performance efficiency E.g., power consumption Tomorrow: Usage efficiency? Sacrifice how much performance to increase usability by how much? To broaden the user base how wide? Number of relevant computations as metric? Convenience Performance

29 3/18/