Contents PART I: Background

Transcription

1 Contents List of Figures List of Tables xvii xxv PART I: Background 1 1. INTRODUCTION Why Parallel Processing? Parallel Architectures SIMD Systems MIMD Systems Job Scheduling Software Architectures Overview of the Monograph PARALLEL AND CLUSTER SYSTEMS Introduction Parallel Architectures UMA Systems NUMA Systems Distributed-Memory Systems Distributed Shared Memory Example Parallel Systems IBM SP2 System Stanford DASH System ASCI Systems Interconnection Networks Dynamic Interconnection Networks Static Interconnection Networks 29 xi

2 xii HIERARCHICAL SCHEDULING 2.5 Interprocess Communication PVM MPI TreadMarks Cluster Systems Beowulf Summary PARALLEL JOB SCHEDULING Introduction Parallel Program Structures Fork-and-Join Programs Divide-and-Conquer Programs Matrix Factorization Programs Task Queue Organizations Basic Task Queue Organizations Improving Centralized Organization Improving Distributed Organization Scheduling Policies Space-Sharing Policies Static Policies Dynamic Policies An Example Space-Sharing Policy Adaptive Space-Sharing Policy A Modification An Improvement Performance Comparison Performance Comparison Handling Heterogeneity Time-Sharing Policies Hybrid Policies Example Policies IBM SP ASCI Blue-Pacific Portable Batch System Summary 84 PART II: Hierarchical Task Queue Organization 85

3 Contents xiii 4. HIERARCHICAL TASK QUEUE ORGANIZATION Motivation Hierarchical Organization Workload and System Models Performance Analysis Queue Access Overhead Utilization Analysis Centralized Organization Distributed Organization Hierarchical Organization Contention Analysis Centralized Organization Distributed Organization Hierarchical Organization Performance Comparison Impact of Access Contention Effect of Number of Tasks Sensitivity to Service Time Variance Impact of System Size Influence of Branching and Transfer Factors Performance of Dynamic Task Removal Policies Summary PERFORMANCE OF SCHEDULING POLICIES Introduction Performance of Job Scheduling Policies Policies Results Performance Sensitivity to System Load Sensitivity to Task Service Time Variance Sensitivity to Variance in Task Distribution Performance of Task Scheduling Policies Task Scheduling Policies Results and Discussion Principal Comparison Impact of Variance in Task Service Time Impact of Variance in Task Distribution Effect of Window Size 135

4 xiv HIERARCHICAL SCHEDULING Sensitivity to Other Parameters Conclusions PERFORMANCE WITH SYNCHRONIZATION WORKLOADS Introduction Related Work System and Workload Models Spinning and Blocking Policies Spinning Policy Blocking Policies Lock Accessing Workload Results Workload Model Simulation Results Principal Comparison Sensitivity to Service Time Variance Impact of Granularity Impact of Queue Access Time Barrier Synchronization Workload Results Workload Model Simulation Results Impact of System Load Sensitivity to Service Time Variance Impact of Granularity Impact of Queue Access Time Cache Effects Summary 163 PART III: Hierarchical Scheduling Policies SCHEDULING IN SHARED-MEMORY MULTIPROCESSORS Introduction Space-Sharing and Time-Sharing Policies Equipartitioning Modified RRJob Hierarchical Scheduling Policy Performance Evaluation System and Workload Models 174

5 Contents xv System Model Workload Model Performance Analysis Effect of Scheduling Overhead Impact of Variance in Service Demand Effect of Task Granularity Effect of the ERF Factor Effect of Quantum Size Sensitivity to Other Parameters Performance with Lock Accessing Workload Lock Accessing Workload Results Conclusions SCHEDULING IN DISTRIBUTED-MEMORY MULTICOMPUTERS Introduction Hierarchical Scheduling Policy Scheduling Policies for Performance Comparison Space Partitioning Time-Sharing Policy Workload Model Performance Comparison Performance with Ideal Workload Performance with Non-Uniform Workload Performance with distribution Sensitivity to variance in job service demand Performance under distribution Performance under distribution Discussion Conclusions SCHEDULING IN CLUSTER SYSTEMS Introduction Hierarchical Scheduling Policy Job Placement Policy Dynamic Load Balancing Algorithm Space-Sharing and Time-Sharing Policies Space-Sharing Policy 221

6 xvi HIERARCHICAL SCHEDULING Time-Sharing Policy Performance Comparison Workload Model Ideal Workload Results Non-Uniform Workload Results Summary 229 PART IV: Epilog CONCLUSIONS Summary Concluding Remarks 236 References 239 Index 249