Austin, TX, USA, 2015-02-04 Towards a Performance Model Management Repository for Component-based Enterprise Applications Work-in-Progress Paper (WiP) International Conference on Performance Engineering (ICPE) 2015 Andreas Brunnert 1, Alexandru Danciu 1, Helmut Krcmar 2 1 fortiss GmbH, 2 Technische Universität München
Agenda Motivation & Vision Basic Technologies Component (Version) Dependencies Handling Resource Demands Related Work Outlook 2
Agenda Motivation & Vision Basic Technologies Component (Version) Dependencies Handling Resource Demands Related Work Outlook 3
Motivation & Vision Performance models are still not in widespread industry use (Koziolek 2010, Mayer et al. 2011): Creation effort often outweighs their benefits (Brunnert et al. 2013, Kounev 2005) Several approaches for automatic generation (Balsamo et al. 2004, Brunnert et al. 2013, Smith 2007) Challenge for applying performance models in industrial practice is the organizational complexity (Brunnert et al. 2014, Schmietendorf et al. 2002): Components of enterprise applications are often under the control of different teams within one or more organizations Teams adhere to different release cycles for their components Challenge to keep a performance model consistent and in sync 4
Motivation & Vision To introduce an integration server for performance models to support the collaboration of distributed teams within an organization. Require - ments Analysis & Design Performance Model Management Repository C 1 C C 2 C 3 C N Team 1 Require - ments Analysis & Design Evaluation Testing Implem - entation Require - ments Analysis & Design Evaluation Require - ments Team Testing Analysis & Design Implem - entation Evaluation Team 2 Testing Implem - entation Evaluation Team 3 Testing Implem - entation Evaluation Require - ments Team N Analysis & Design Implem - entation Testing 5
Agenda Motivation & Vision Basic Technologies Component (Version) Dependencies Handling Resource Demands Related Work Outlook 6
Palladio Component Model (PCM) Basic Technologies PCM as Meta-Model Usage Model Workload Repository Model Components, Interfaces, Relationships, Control Flows, Resource Demands System Model System composed of components within the repository model Allocation Model Mapping of system components to hardware servers Resource Environment Specifies available servers, networks, 7
Basic Technologies Required Enhancements PCM repository models are represented by single files that are hard to maintain by different teams concurrently Multiple PCM repository models with outdated component specifications exist, as multiple component versions need to be maintained at the same time by different teams We propose to use EMFStore 1 as PMMR server: The PCM meta-model is based on the Eclipse Modeling Framework (EMF) EMFStore implements the required versioning features for models based on the Ecore meta-model 1 http://eclipse.org/emfstore/ 8
Agenda Motivation & Vision Basic Technologies Component (Version) Dependencies Handling Resource Demands Related Work Outlook 9
Component (Version) Dependencies Extension of the PCM Meta-Model Callee Version N Caller Interface v1 Interface v2 Interface vn Callee Version 2 Callee Version 1 We propose to extend the PCM meta-model to specify require references across component versions The Palladio-Bench also needs to be extended to support the user while interacting with different component versions 10
Agenda Motivation & Vision Basic Technologies Component (Version) Dependencies Handling Resource Demands Related Work Outlook 11
Handling Resource Demands Hardware-specific Resource Demands PCM repository model components can contain hardware-specific resource demands Resource demands stored in a PMMR are specified relative to a common baseline Measured RD Check-in Hardware Type A Converted RD Check-out Hardware Type B 12
Agenda Motivation & Vision Basic Technologies Component (Version) Dependencies Handling Resource Demands Related Work Outlook 13
Related Work Several approaches for versioning model artifacts exist in literature (Altmanninger et al. 2009) These approaches do not address the specific requirements which arise from the versioning of performance models of individual components Woodside et al. (2007) proposed the Performance Knowledge Base (PKB) as a central performance repository The PKB is intended to store measurement and model prediction results in a PKB instead of the models itself PKB should allow to build performance models on demand PMMR is designed so that performance models can be stored in it directly Koziolek (2010) argues that central performance model repositories (called model libraries) "... could allow rapid performance predictions.... 14
Agenda Motivation & Vision Basic Technologies Component (Version) Dependencies Handling Resource Demands Related Work Outlook 15
Outlook Abstraction Level: Choose abstraction level to reduce the amount of components that need to be represented Level of detail for storing white-box and high-level black-box models Evaluation: Experimental setup to validate the feasibility of the approaches Representative software development project to validate the intended improvements Integration: Danciu et al. (2014) propose an approach to support developers with insights on the response times of the component they are currently developing PMMR Eclipse 16
References K. Altmanninger, M. Seidl, and M. Wimmer. A survey on model versioning approaches. International Journal of Web Information Systems, 5(3):271-304, 2009. S. Balsamo, A. Di Marco, P. Inverardi, and M. Simeoni. Model-based performance prediction in software development: A survey. IEEE Transactions on Software Engineering, 30(5):295-310, 2004. S. Becker, H. Koziolek, and R. Reussner. The palladio component model for model-driven performance prediction. Journal of Systems and Software, 82(1):3-22, 2009. A. Brunnert and H. Krcmar. Detecting performance change in enterprise application versions using resource profiles. In Proceedings of the 8th International Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS '14, New York, NY, USA, 2014. ACM. A. Brunnert, C. Vögele, A. Danciu, M. Pfaff, M. Mayer, and H. Krcmar. Performance management work. Business & Information Systems Engineering, 6(3):177-179, 2014. A. Brunnert, C. Vögele, and H. Krcmar. Automatic performance model generation for java enterprise edition (ee) applications. In M. S. Balsamo, W. J. Knottenbelt, and A. Marin, editors, Computer Performance Engineering, volume 8168 of Lecture Notes in Computer Science, pages 74-88. Springer Berlin Heidelberg, 2013. A. Danciu, A. Brunnert, and H. Krcmar. Towards performance awareness in java ee development environments. In S. Becker, W. Hasselbring, A. van Hoorn, S. Kounev, and R. Reussner, editors, Proceedings of the Symposium on Software Performance: Descartes/Kieker/Palladio Days 2014, pages 152-159, November 2014. S. Kounev. Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction. Shaker Verlag, Ph.D. Thesis, Technische Universit at Darmstadt, Germany, Aachen, Germany, 2005. H. Koziolek. Performance evaluation of component-based software systems: A survey. Performance Evaluation, 67(8):634-658, 2010. M. Mayer, S. Gradl, V. Schreiber, H. Wittges, and H. Krcmar. A survey on performance modelling and simulation of sap enterprise resource planning systems. In The 10th International Conference on Modeling and Applied Simulation, pages 347-352. Diptem Universià di Genoa, 2011. A. Schmietendorf, E. Dimitrov, and R. R. Dumke. Process models for the software development and performance engineering tasks. In Proceedings of the 3rd International Workshop on Software and Performance, WOSP '02, pages 211-218, New York, NY, USA, 2002. ACM. C. Smith. Introduction to software performance engineering: Origins and outstanding problems. In M. Bernardo and J. Hillston, editors, Formal Methods for Performance Evaluation, volume 4486 of Lecture Notes in Computer Science, pages 395-428. Springer Berlin Heidelberg, 2007. M. Woodside, G. Franks, and D. C. Petriu. The future of software performance engineering. In Future of Software Engineering (FOSE), pages 171-187, Minneapolis, MN, USA, 2007. 17
Q&A Alexandru Danciu, Andreas Brunnert performancegroup@fortiss.org pmw.fortiss.org 18
Basic Technologies Performance (Meta-)Models LQN, QNs, QPNs depict workload, hardware environment and performance-relevant aspects of an application in one monolitic model Hard to change a single aspect without modifying the whole model Architecture-level performance models (e.g. the Palladio Component Model (PCM)) allow to specify these aspects independently from each other Used to represent resource profiles Several existing ways to create such models based on static, dynamic or hybrid analysis We propose the use of the Palladio Component Model (PCM) as metamodel for the component performance models managed in a PMMR (Becker et al. 2009) 19
Palladio Component Model (PCM) Implementation of the PMMR PCM as Meta-Model Usage Model Repository models are created by component developers Repository Model System Model Allocation Model Resource Environment 20
Palladio Component Model (PCM) Implementation of the PMMR PCM as Meta-Model Usage Model System models are created by system architects Repository Model System Model Allocation Model Resource Environment 21
Handling Resource Demands Using Benchmark Scores Scores are specified for all relevant hardware resources. r baseline denotes baseline resource demand b baseline denotes baseline hardware resource benchmark score During check-in: r checkinvalue denotes resource demand measured by the user b checkinbenchmarkvalue denotes benchmark score of the hardware resource During check-out: r checkoutvalue denotes resource demand calculated relative to benchmark score b checkoutbenchmarkvalue denotes benchmark score of the target hardware resource 22