Performance Modeling in Industry A Case Study on Storage Virtualization SOFTWARE DESIGN AND QUALITY GROUP - DESCARTES RESEARCH GROUP INSTITUTE FOR PROGRAM STRUCTURES AND DATA ORGANIZATION, FACULTY OF INFORMATICS Nikolaus Huber Steffen Becker Christoph Rathfelder Jochen Schweflinghaus Ralf Reussner nikolaus.huber@kit.edu stbecker@mail.upb.de rathfelder@fzi.de schwefel@de.ibm.com reussner@kit.edu KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association www.kit.edu
Storage Virtualization on SystemZ Evaluate design alternatives Optimize performance through identifying bottlenecks SystemZ Assess applicability of model-based performance prediction 2 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Storage Virtualization on SystemZ Evaluate design alternatives Optimize performance through identifying bottlenecks Assess applicability of model-based performance prediction 3 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Synchronous Request Handling 4 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Asynchronous Request Handling 5 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Asynchronous Request Handling Performance Questions: Would the asynchronous design alternative perform better? How many I/O threads are required? How many CPUs are sufficient? 6 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Palladio Approach Approach for modelbased performance prediction Simulation & Analysis Tools Structural Model Deployment Model Behavior Model Usage Model Target domain: Business Information Systems Throughput Resource Utilization Response Times 7 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Approach Measurements Measurements Model Parameterization Model Validation Synchronous Model Asynchronous Model Palladio Toolchain Evaluation 8 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Model Implementation 9 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Model Implementation 10 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Model Validation (Request Type Mix) Overall relative prediction error < 22% 11 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Answering Performance Questions Expected Actual Evaluate design alternatives Async. performs better No difference in throughput Async compensates peak load Optimize performance through identifying bottlenecks I/O threads Queue blocking Storage Hardware: Throughput bottleneck at little load I/O Interface: Throughput bottleneck at high load CPU increases throughput 12 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Experiences Gained Assess applicability of model-based performance prediction High initial effort Tradeoff: accuracy modeling effort [6PM] Ease what-if analysis and design alternatives evaluation Valuable to identify performance bottlenecks Improve understanding of the system Cheaper than performance prototype [24PM] Annotated Design 2 ms Feedback Response Time, Utilization, Throughput 10 ms 15 ms Analysis / Simulation 13 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Conclusions Case study results Model-based performance prediction valuable in realistic industrial scenarios (6PM 24PM) PCM mature and applicable beyond its target domain (Overall relative prediction error <22%) Surprising answers to performance questions 14 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Outlook Descartes Research Project Models capturing aspects of dynamic system (e.g. virtualization) Model-based performance prediction at runtime Autonomic and self-aware software systems http://www.descartes-research.net 15 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Conclusions Case study results Model-based performance prediction valuable in realistic industrial scenarios (6PM 24PM) PCM mature and applicable beyond its target domain (Overall relative prediction error <22%) Surprising answers to performance questions http://www.descartes-research.net 16 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Any Questions? Thank you! http://www.descartes-research.net http://www.palladio-approach.net KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association www.kit.edu
Model Parameterization and Calibration Measurements complex, conducted by IBM Prediction error <10% after model calibration 18 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Model-based Performance Prediction 19 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Virtualization and Performance Modeling I/O Virtualization Performance Modeling I/O Virtualization - Scale-up [WJW07] - Scale-out [WRJ07] Performance Modeling - PCM & CoCoME [KR08] - PCM at CAS Software AG [And08] - Queuing Networks [PGGG06] No intersection of performance modeling and virtualization! 20 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Bibliography [WJW07] J. Wei, J. R. Jackson, and J. A. Wiegert. Towards Scalable and High Performance I/O virtualization - A Case Study. HPCC 2007. [WRJ07] J. A. Wiegert, G. Regnier, and J. Jackson. Challenges for scalable networking in a virtualized server. ICCCN 2007. [And08] R. Andrej. Evaluation des Vorhersageverfahrens "Palladio" im industriellen Kontext der CAS Software AG, 2008. Diploma thesis. [KR08] K. Krogmann and R. H. Reussner. The Common Component Modeling Example, Springer-Verlag Berlin Heidelberg, 2008. [PGGG07] U. Praphamontripong, S. Gokhale, A. Gokhale, and J. Gray. Performance analysis of a middleware demultiplexing pattern. In HICSS 2007. 21 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Assumptions (Request Queues) RequestGenerator Represents request queues Probability functions for queue locking 22 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Assumptions (Request Queues) II Call of getrequest delayed Blocking probability Delay calculated by Blocked or not Amount of blocked queue accesses 23 05.05.2010 Nikolaus Huber - Performance Modeling in Industry:
Answering Performance Questions Evaluate design alternatives No difference in throughput Async. can compensate peak loads No overload situation in sync. Optimize performance through identifying bottlenecks Not the I/O threads No queue blocking influences Storage Hardware -- Throughput bottleneck at little load I/O Interface -- Throughput bottleneck at maximum load CPU power increases throughput 24 05.05.2010 Nikolaus Huber - Performance Modeling in Industry: