Performance Modeling in Industry A Case Study on Storage Virtualization

Transcription

1 Performance Modeling in Industry A Case Study on Storage Virtualization SOFTWARE DESIGN AND QUALITY GROUP - DESCARTES RESEARCH GROUP INSTITUTE FOR PROGRAM STRUCTURES AND DATA ORGANIZATION, FACULTY OF INFORMATICS Nikolaus Huber Steffen Becker Christoph Rathfelder Jochen Schweflinghaus Ralf Reussner nikolaus.huber@kit.edu stbecker@mail.upb.de rathfelder@fzi.de schwefel@de.ibm.com reussner@kit.edu KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association

2 Storage Virtualization on SystemZ Evaluate design alternatives Optimize performance through identifying bottlenecks SystemZ Assess applicability of model-based performance prediction Nikolaus Huber - Performance Modeling in Industry:

3 Storage Virtualization on SystemZ Evaluate design alternatives Optimize performance through identifying bottlenecks Assess applicability of model-based performance prediction Nikolaus Huber - Performance Modeling in Industry:

4 Synchronous Request Handling Nikolaus Huber - Performance Modeling in Industry:

5 Asynchronous Request Handling Nikolaus Huber - Performance Modeling in Industry:

6 Asynchronous Request Handling Performance Questions: Would the asynchronous design alternative perform better? How many I/O threads are required? How many CPUs are sufficient? Nikolaus Huber - Performance Modeling in Industry:

7 Palladio Approach Approach for modelbased performance prediction Simulation & Analysis Tools Structural Model Deployment Model Behavior Model Usage Model Target domain: Business Information Systems Throughput Resource Utilization Response Times Nikolaus Huber - Performance Modeling in Industry:

8 Approach Measurements Measurements Model Parameterization Model Validation Synchronous Model Asynchronous Model Palladio Toolchain Evaluation Nikolaus Huber - Performance Modeling in Industry:

9 Model Implementation Nikolaus Huber - Performance Modeling in Industry:

10 Model Implementation Nikolaus Huber - Performance Modeling in Industry:

11 Model Validation (Request Type Mix) Overall relative prediction error < 22% Nikolaus Huber - Performance Modeling in Industry:

12 Answering Performance Questions Expected Actual Evaluate design alternatives Async. performs better No difference in throughput Async compensates peak load Optimize performance through identifying bottlenecks I/O threads Queue blocking Storage Hardware: Throughput bottleneck at little load I/O Interface: Throughput bottleneck at high load CPU increases throughput Nikolaus Huber - Performance Modeling in Industry:

13 Experiences Gained Assess applicability of model-based performance prediction High initial effort Tradeoff: accuracy modeling effort [6PM] Ease what-if analysis and design alternatives evaluation Valuable to identify performance bottlenecks Improve understanding of the system Cheaper than performance prototype [24PM] Annotated Design 2 ms Feedback Response Time, Utilization, Throughput 10 ms 15 ms Analysis / Simulation Nikolaus Huber - Performance Modeling in Industry:

14 Conclusions Case study results Model-based performance prediction valuable in realistic industrial scenarios (6PM 24PM) PCM mature and applicable beyond its target domain (Overall relative prediction error <22%) Surprising answers to performance questions Nikolaus Huber - Performance Modeling in Industry:

15 Outlook Descartes Research Project Models capturing aspects of dynamic system (e.g. virtualization) Model-based performance prediction at runtime Autonomic and self-aware software systems Nikolaus Huber - Performance Modeling in Industry:

16 Conclusions Case study results Model-based performance prediction valuable in realistic industrial scenarios (6PM 24PM) PCM mature and applicable beyond its target domain (Overall relative prediction error <22%) Surprising answers to performance questions Nikolaus Huber - Performance Modeling in Industry:

17 Any Questions? Thank you! KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association

18 Model Parameterization and Calibration Measurements complex, conducted by IBM Prediction error <10% after model calibration Nikolaus Huber - Performance Modeling in Industry:

19 Model-based Performance Prediction Nikolaus Huber - Performance Modeling in Industry:

20 Virtualization and Performance Modeling I/O Virtualization Performance Modeling I/O Virtualization - Scale-up [WJW07] - Scale-out [WRJ07] Performance Modeling - PCM & CoCoME [KR08] - PCM at CAS Software AG [And08] - Queuing Networks [PGGG06] No intersection of performance modeling and virtualization! Nikolaus Huber - Performance Modeling in Industry:

21 Bibliography [WJW07] J. Wei, J. R. Jackson, and J. A. Wiegert. Towards Scalable and High Performance I/O virtualization - A Case Study. HPCC [WRJ07] J. A. Wiegert, G. Regnier, and J. Jackson. Challenges for scalable networking in a virtualized server. ICCCN [And08] R. Andrej. Evaluation des Vorhersageverfahrens "Palladio" im industriellen Kontext der CAS Software AG, Diploma thesis. [KR08] K. Krogmann and R. H. Reussner. The Common Component Modeling Example, Springer-Verlag Berlin Heidelberg, [PGGG07] U. Praphamontripong, S. Gokhale, A. Gokhale, and J. Gray. Performance analysis of a middleware demultiplexing pattern. In HICSS Nikolaus Huber - Performance Modeling in Industry:

22 Assumptions (Request Queues) RequestGenerator Represents request queues Probability functions for queue locking Nikolaus Huber - Performance Modeling in Industry:

23 Assumptions (Request Queues) II Call of getrequest delayed Blocking probability Delay calculated by Blocked or not Amount of blocked queue accesses Nikolaus Huber - Performance Modeling in Industry:

24 Answering Performance Questions Evaluate design alternatives No difference in throughput Async. can compensate peak loads No overload situation in sync. Optimize performance through identifying bottlenecks Not the I/O threads No queue blocking influences Storage Hardware -- Throughput bottleneck at little load I/O Interface -- Throughput bottleneck at maximum load CPU power increases throughput Nikolaus Huber - Performance Modeling in Industry: