for High Performance Computing

Transcription

1 Technische Universität München Institut für Informatik Lehrstuhl für Rechnertechnik und Rechnerorganisation Automatic Performance Engineering Workflows for High Performance Computing Ventsislav Petkov Vollständiger Abdruck der von der Fakultät für Informatik der Technischen Universität München zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften (Dr. rer. nat.) genehmigten Dissertation. Vorsitzende(r): Prüfer der Dissertation: Univ.-Prof. Dr. Helmut Krcmar 1. Univ.-Prof. Dr. Hans Michael Gerndt 2. Univ.-Prof. Dr. Felix Gerd Eugen Wolf Rheinisch-Westfälische Technische Hochschule Aachen Die Dissertation wurde am Ü13 bei der Technischen Universität München eingereicht und durch die Fakultät für Informatik am angenommen.

2 Page Acknowledgements iii Abstract v List of Figures xiii ListofTables xvii 1. Introduction Motivation and Problem Statement Performance Analysis and Tuning Methodology Process Automation and Standardization Contributions of This Work Outline of This Work 6 1. Theoretical Background and Technological Overview Software Development Life-Cycle Software Requirements Engineering Software Design Software Construction Software Testing Software Maintenance Process Automation and Design of Workflows Foundations of Process Automation Process Automation Languages and Standards Business Process Management Suites Scientific Workflow Automation Tools 33 ix

3 .' Contents 4. Supportive Software Development Tools Revision Control Systems Client-Server Repository Model Distributed Repository Model Related Work The Need for Parallel Programming Performance Engineering Automation Performance Engineering Tools Performance Tuning Libraries and Frameworks Eclipse and the Parallel Tools Platform 64 II. PAThWay to Performance Analysis and Tuning Workflows Performance Engineering Workflows Performance Engineering Workflows Workflow for Scalability Analysis of Parallel Applications Workflow for Cross-Platform Performance Analysis Workflow for Code Migration and Tuning on GPGPUs Summary and Requirements Overview Architecture for Automation of Performance Engineering Processes Architecture for Automation of Performance Engineering Processes PAThWay Graphical User Interface Logic Implementation Layer Supportive Modules Layer Workflow Modeling Environment Design Goals jbpm Workflow Execution Environment Business Process Model and Notation PAThWay'sCustomDomain-SpecificBPMNNodes Internal Data Storage Repository Motivation and Design Goals Database Entities and Their Application Object-Relational Mapping using Hibernate Project Documentation Module Motivation and Design Goals EclipseWiki and Its Features Integration of EclipseWiki with PAThWay 125 x

4 11. Other Supportive Modules Internal Supportive Modules Revision Control Module Environment Detection Module Runtime Manager and the Parallel Tools Platform Interface Experiments Browser Other Graphical User Interfaces 138 III. Performance Engineering Workflows in Action Scalabiliry Analysis Workflow Scalability Analysis Process Workflow Model LRZ Linux Cluster NAS Parallel Benchmarks Multi-Zone Workflow Execution Results Exploration Cross-Platform Memory Analysis Workflow Cross-Platform Memory Analysis Process Workflow Model STREAM Memory Benchmark Runtime Environment Generic Memory Analysis Strategy for Periscope Workflow Execution Results Exploration Exhaustive Benchmarking Workflow Exhaustive Benchmarking Process Workflow Model SPEC MPI-2007 Benchmarks Online Data Clustering Support for Periscope Workflow Execution Generic Performance Tuning Workflow Performance Tuning and Its Application in HPC Centers Generic Performance Tuning Workflow Conclusion Summary Future Work 180 Appendices 183 xi

5 A. Glossary 185 B. Custom Domain-Specific BPMN Nodes: XML Definitions 191 B.l. Application Configuration 191 B.2. MPI and OpenMP Configuration 192 B.3. Target HPC System Selection 192 B.4. Performance Tool Selection 193 B.5. Source Code Instrumentation 193 B.6. Experiments Creation 194 B.7. Execution Experiment 195 B.8. Runtime Creation Manager 196 B.9. Load Custom Performance Results to the Database 197 B. 10. Store Additional Information to an Experiment 198 B.l 1. Node for Interactive Questions 199 B. 12. Execute a Remote Process 200 C. Internal Database Scheine and Data Persistence Classes 203 C. l. Scheme of the Internal Database 203 C. 2. Data Persistence Classes 217 D. Documentation Module: Modifications to EclipseWiki 219 D. I. PAThWay Extension Classes 219 D.2. Modifications to EclipseWiki's Internal Classes 220 D. 3. Example of a Wiki Markup of a Historical Note 222 E. Other Supportive Modules 223 E. l. Parallel Tools Platform Interface Module 223 E.2. Experiments Browser GUI 225 F. Bibliography 227 xii