Integrating Performance Characterization with Software Development

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol: 11 No: 02 7 Integrating Performance Characterization with Software Development Abstract- The importance of integrating performance considerations into the early stages of the software development process triggers a need to bridge the gap between the fields of software engineering and performance analysis. Failure to detect performance pitfalls in a system at its earliest stages of development could turn up very costly, especially with complex and performance critical systems. Performance engineering strategies and analysis techniques could be used at the design time of the system to avoid such cost. The proposed Integrated Model-based Performance Analysis and Characterization Tool developed attempts to combine already existing standards and proposed frameworks into an automated unified easy-to-use tool. The tool aims to empower the software engineering process and make performance engineering an integral part of the software development cycle. Index Terms- Management, Performance, Design, Experimentation, Standardization, Verification, Software performance engineering, UML, MARTE profile, performance analysis, MDA, model-to-model transformation, QVT relations, LQN I. INTRODUCTION Research in the field of software performance engineering addresses the importance of integrating performance considerations into the early stages of the software development process. Delaying such considerations to later stages of the process with a fix-it-later approach has often proven to be costly and unwisely, especially with complex software systems and performance critical parallel/distributed applications. Several efforts have been made to come up with modeling standards and common frameworks that would concurrently satisfy the requirements of both software engineers and performance experts. Many have proposed systematic transformations of software models to common performance models that already have good tool support and welldeveloped evaluation techniques like Stochastic Process Algebra (SPA), Stochastic Petri-Net (SPN) and Queuing Networks with their different extended forms including Layered Queuing Networks (LQN) [21]. Some also proposed the use of an intermediate performance model like the Core Scenario Model where relevant information is extracted from the software model and represented in a generic performance representation independent of the performance formalism to Reheb A. El-kaedy* and Ahmed Sameh** *The American University in Cairo, **Prince Sultan University AUC Avenue, P.O.Box 74, P.O.Box 66833 New Cairo 11835, Riyadh 11586, Saudi Arabia Sameh.aucegypt.edu@gmail.com be used [17][18]. This intermediate model is then transformed into the target performance model for evaluation and analysis. Considering that UML is gaining more consensus as the de facto modeling framework used by most software engineers, more emphasis has been given to transforming UML system models into their corresponding performance models. The Object Management Group (OMG) [10] leads the process of developing standard UML profiles for annotating UML models with performance information. The profile of Schedulability, Performance and Time (SPT) was the first OMG profile for real-time systems adopted in 2005 [15]. It uses tagged-values to annotate UML 1.4 models with the required performance attributes. The OMG profile for Modeling and Analysis of Real Time and Embedded systems (MARTE) then came to replace the SPT profile [11][14]. The MARTE profile supports UML 2.0 and provides more flexibility in defining performance values using the Value Specification Language (VSL). OMG also advocates the use of the Meta Object Facility 2.0 Query, View, and Transformation (QVT) specification [12] as a standard for model transformation in line with its vision of Model Driven Architecture (MDA) [13]. QVT provides hybrid declarative/imperative model transformation languages used to methodologically describe the transformational mapping between a source and target meta-models. Recently there has been significant progress in the availability of tools supporting the editing, compilation and execution of QVT transformations at its different levels of language abstractions, namely the core, relational and operational mappings languages [9][22]. Our Integrated Model-based Performance Analysis and Characterization Tool (IMPACT) attempts to integrate these standardization efforts into a unified early software performance evaluation platform. The performance information provided through the MARTE annotations of the UML model are transformed using standard QVT transformations into an equivalent performance model for evaluation and analysis. The model is then analyzed using an existing performance model solver and the results are reported back to the designers of the system. In the rest of the paper, we present the structure of the IMPACT system and the tools used for its different components. Section 2 gives a general overview of the system. Section 3 discusses the OMG MARTE profile and the

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol: 11 No: 02 8 Papyrus modeling tool used as the development environment [16]. Section 4 deals with the QVT model transformation language and the mediniqvt tool used for its implementation [9]. Section 5 discusses the performance model generated and the solver used for its analysis [8]. In the end, we discuss the integration platform of our system and summarize possibilities for future enhancements. II. IMPACT SYSTEM OVERVIEW Figure 1 presents the general architecture of IMPACT in accordance with the model analysis framework presented in the MARTE profile specification [14]. The Papyrus modeling tool [16] is used to produce a MARTE annotated UML model of the system being designed. The relational QVT implementation of mediniqvt [9] is then used to execute a relational model-to-model transformation to generate an equivalent LQN model for the system. A relational QVT implementation has been used to allow for the bi-directional transformation between the source and target models. The reverse transformation is necessary for integrating the performance results back into the UML model and implementing the reverse feedback path. required to perform model-based analysis and quantitative predictions of system properties [14]. The profile, as illustrated in Figure 2, consists of foundation modules that provide the basic elements for describing time, non-functional properties, and the use of resources. The design modules refine these foundations to model the features of real-time and embedded systems, and the analysis modules utilize them to define the information needed for model- based analysis of system properties. The Generic Quantitative Analysis Modeling (GQAM) part provides a general analysis framework to support the analysis of generic system properties. Fig. 1: IMPACT Architecture Block Diagram A Layered Queuing Model is used as the performance model as it provides layers to capture contention for both hardware and software resources [1][7]. LQN Model analysis scales well with complex large systems as it is based on mean-value analysis unlike SPAs and SPNs that are based on stochastic discrete-state analysis. A direct transformation to an LQN performance model is used to avoid a double transformation through an intermediate performance model at this stage. This might however be a good option to consider in the future for system generality and tool interoperability. The LQN model generated could then be analyzed using existing LQN solvers. We use the LQNS solver of [8] as an illustration to produce the analysis results. III. MARTE PROFILE AND THE PAPYRUS MODELING TOOL The UML profile for Modeling and Analysis of Real-Time and Embedded systems (MARTE) extends the UML metamodel with the capability to annotate models with information Fig. 2: MARTE Architecture Block Diagram The Schedulability Analysis Modeling (SAM) and Performance Analysis Modeling (PAM) components are specialized analysis sub-profiles that provide particular support for the analysis of schedulability and performance aspects of systems. Performance properties that could be represented in MARTE include five types of quantities: time durations (including forced durations like a user think time), occurrence frequencies, probabilities, repetitions and data sizes. The performance analysis typically yields information like the average response time of components or systems, mean throughput capacity, resource utilization or probabilities of missing delay targets. Some properties may be considered as input properties required for carrying out the analysis of the model, and some properties could be used as output properties that are produced by the analysis. Figure 3 & 4 present a sample UML model of a simple web application annotated with performance information using MARTE (as provided in the MARTE specification document [14]). It illustrate the basic features of the profile including open arrival rates, average processor demands, operation repetitions, multithreaded processes and communication overheads on the processing nodes.

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol: 11 No: 02 9 IMPACT uses the information provided through the MARTE Performance Analysis Modeling framework to build the performance model of the system and analyze it. It transforms the performance properties defined by PAM into their corresponding performance features in the equivalent LQN model. Our system is based on the implementation of the MARTE profile provided by the Papyrus software modeling tool [16]. Papyrus is an open source tool for UML 2 modeling provided by the French Commission of Atomic Energy - Lab of applied research on software-intensive technologies [5]. It is based on the Eclipse environment and follows the Eclipse Modeling Framework for model representations [4]. The Papyrus implementation of the MARTE profile has been added to the latest release (1.12). The main features of the tool are: its full compliance to Eclipse UML2, its full respect of the UML2 as defined by OMG, its full respect of the DI2 (Diagram Interaction) standard, its extendable architecture that allows users to add new diagrams and new code generators, and its profile development support facilities for even nested profiles. Fig 3. MARTE annotated UML diagram for a simple web application Fig. 4 MARTE annotated UML Sequence Diagram for the Example Web Application IV. QVT TRANSFORMATION AND THE MEDINIQVT TOOL OMG s Meta Object Facility 2.0 Query/View/Transformation (QVT) specification provides a standard for expressing model transformations defined precisely in terms of the relationship between a source and a target meta-model [12]. It realizes the Model Driven Architecture (MDA) vision to treat models as the primary artifacts of software development. QVT provides transformation syntax at various language levels. The Core and Relations languages are declarative languages that explicitly define the relationship between the source and target meta-models. The Relations semantics is a higher level language that could be transformed into the simpler, more basic Core transformation semantics. QVT also provides imperative transformation formalisms that explicitly define transformation actions from source to target meta-models. These could either be defined through black-box transformation implementations or through the QVT standard Operational Mapping language. MediniQVT is a tool provided by ikv++ technologies, Germany to implement OMG s QVT Relations specification [9]. It provides an interactive environment to support the development, debugging and execution of QVT Relational transformations. It is also provided as an Eclipse plug-in based on the Eclipse Modeling Framework (EMF). Its trace management option enables incremental model updates and maintains the operation efficiency of model retransformations during the UML model development. The bidirectional nature of the relational transformation allows for the reverse transformation required to integrate the performance results back into the UML model. Figure 5 illustrates sample QVT relational transformations that map the web application model of section 3 into its equivalent LQN model

Fig 5. QVT Operational transformation International Journal of Basic & Applied Sciences IJBAS-IJENS Vol: 11 No: 02 10 V. LQN PERFORMANCE MODEL The Layered Queuing Network is an extended form of Queuing Networks that captures contention for both software and hardware resources in a layered interconnection topology [1][7]. An LQN model contains processors representing actual processors of the system or other logical resources that only accept requests from other servers and clients. Resources including software processes and hardware devices are represented as tasks with entries to accept synchronous or asynchronous requests. Activities are used to represent more sophisticated call sequences in non-sequential execution scenarios. Figure 6 show the Layered Queuing model of the web application model presented in section 3. Once the LQN model of the system is produced by the QVT transformation process, an LQN solver could be used to evaluate the model and generate the required performance indicators. To demonstrate the operation of IMPACT, we use the LQNS solver of [8] to analyze the model generated. The performance results produced should then be parsed and integrated back into the UML design to provide the system designer with the required performance feedback information. LQNS is developed at Carleton University, Canada by the Real-Time and Distributed Systems Group (RADS) [20]. The tool includes both an analytic LQN mean-value solver and an LQN simulator. IMPACT uses the analytic version of the solver, providing it with the generated LQN model as input to generate performance information as illustrated in Figure 7. To the credit of this tool: various previous tools have been proposed or used to implement early stages of the software life cycle performance evaluation, however, none of them have been implemented into a complete environment for specification, performance analysis and feedback to the software designer. An open problem and challenge is to completely automate the process of driving performance models from software specification and to integrate the supporting tools to a unique environment. The feedback to the software designer of the results of the performance analysis is measured by how easily the designer can interpret the quantitative results and figure out what needs to be changed in his design to improve performance. In fact the LQN approach used by IMPACT refers to a higher abstraction level than for example the detailed stochastic model. As such it makes the feedback at the designer level easier to get interpreted. VI. CONCLUSION IMPACT is a performance plug-in that makes use of the modeling capabilities of the Papyrus tool and the relational QVT model transformation implementation of mediniqvt to produce an LQN performance model of a MARTE annotated UML design. The tool is based on the Eclipse environment and Eclipce Modeling Framework. The LQN model produced is in XML format suitable for input to the LQNS analytical model solver, which in turn generates the performance information required. The major challenge is to develop a comprehensive and robust set of transformation rules that makes use of the performance information provided through the stereotypes of the Performance Analysis sub-profile of MARTE. As a future step, the IMPACT tool can be further enhanced to make use of advanced features provided by MARTE like performance analysis contexts with varying workload distributions or resource platforms. The tool has been tested on a reasonable power development station only on small size simple case models. In order to validate its operation, the tool is still to be tested on more complex case studies to evaluate the performance information produced. More work also needs to be done in the area of results feedback. Reverse relational QVT transformations need to be executed to integrate the generated performance information back where they belong on the UML design and provide the user with more detailed feedback about the system s performance profile. REFERENCES [1] C.M. Woodside, Tutorial Introduction to Layered Modeling of Software Performance - Edition 3.0, Department of Systems and Computer Engineering, Carleton University, Ottawa (Canada), May 2002. [2] Cortellessa, V., Di Marco, A., and Inverardi, P. 2006. Software performance model-driven architecture. In Proceedings of the 2006 ACM Symposium on Applied Computing (Dijon, France, April 23-27, 2006). SAC '06. ACM, New York, NY, 1218-1223. DOI= http://doi.acm.org/10.1145/1141277.1141565

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol: 11 No: 02 11 [3] D Ambrogio A.: A Model Transformation Framework for the Automated Building of Performance Models from UML Models. In Proceeding of ACM Workshop on Software and Performance. (2005) [4] Eclipse Modeling Framework (EMF) Project: http://www.eclipse.org/modeling/emf/ [5] The French Commission of Atomic Energy - Lab of applied research on software-intensive technologies. http://www-list.cea.fr/ [6] Gu, G. P. and Petriu, D. C. 2002. XSLT transformation from UML models to LQN performance models. In Proceedings of the 3rd international Workshop on Software and Performance (Rome, Italy, July 24-26, 2002). WOSP '02. ACM, New York, NY, 227-234. DOI= http://doi.acm.org/10.1145/584369.584402 [7] J.A. Rolia, K.C. Sevcik, The Method of Layers, IEEE Transactions on Software Engineering, 21(8):689-700, August 1995. [8] The Layered Queuing Network Solver software package. RADS research group, Carleton University, Ottawa (Canada). http://www.sce.carleton.ca/rads/lqns/ [9] The medini QVT tool official website: http://projects.ikv.de/qvt/ [10] Object Management Group (OMG): http://www.omg.org/ [11] The official OMG MARTE website: www.omgmarte.org [12] OMG. Meta Object Facility (MOF) 2.0 Query/View/Transformation (QVT).Version 1.0, formal/08-04-03, April 2008. [13] OMG. Model Driven Architecture guide. Version 1.0.1, omg/03-06-01, June 2003. [14] OMG. UML Profile for Modeling and Analysis of Real-time and Embedded Systems (MARTE). Version 1.0 Beta2, formal/08-06-08, June 2008. [15] OMG. UML Profile for Schedulability, Performance, and Time Specification. Version 1.0, formal/03-09-01, September 2003. [16] Papyrus UML open source tool for graphical UML2 modeling: www.papyrusuml.org [17] Petriu D.., Woodside M: A Metamodel for Generating Performance Models from UML Designs, Proc. of UML Conference, LNCS 3273, pp 41-53. (2004) [18] Petriu D.., Woodside M: An intermediate metamodel with scenarios and resources for generating performance models from UML designs [19] Ramrao, W.: Transformation of uml design model into performance model: A model driven framework. In ECOOP Student Workshop (2006) [20] Real-Time and Distributed Systems Group, Department of Systems and Computer Engineering, Carleton University, Ottawa (Canada). http://www.sce.carleton.ca/rads/ [21] Simonetta Balsamo, Antinisca Di Marco, Paola Inverardi, Marta Simeoni, "Model-Based Performance Prediction in Software Development: A Survey," IEEE Transactions on Software Engineering, vol. 30, no. 5, pp. 295-310, May, 2004. [22] SmartQVT - An open source model transformation tool implementing the MOF 2.0 QVT-Operational language. http://smartqvt.elibel.tm.fr/

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol: 11 No: 02 12 s AppHost DBHost database [dbthreads] htmlreq 4.5 ms dbreq 12.4 ms Arrivals at 58.8/sec (1.3) Fig 6. Equivalent Layered Queuing model

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol: 11 No: 02 13 Generated by lqns, version 3.10 Copyright the Real-Time and Distributed Systems Group, Department of Systems and Computer Engineering Carleton University, Ottawa, Ontario, Canada. K1S 5B6 Convergence test value: 6.49181e-006 Number of iterations: 5 MVA solver information: Submdl n k srv step() mean stddev wait() mean stddev 1 5 1 1 26 5.2 0.4 272 54.4 8.8 2 9 1 2 46 5.1111 0.31427 708 78.667 10.371 3 5 1 1 26 5.2 0.4 272 54.4 8.8 Total 19 0 0 98 5.1579 0.36464 1252 65.895 15.444 Elapsed: 0:00:00.00.. Type 1 throughput bounds: Task Name Entry Name Throughput Users users 0.000651103 webserver htmlreq 3.06279 database dbreq 0.403226 Mean delay for a rendezvous: Task Name Source Entry Target Entry Phase 1 Users users htmlreq 0 webserver htmlreq dbreq 0 Service times: Task Name Entry Name Phase 1 Users users 1487.86 webserver htmlreq 26.12 database dbreq 12.4 Service time variance (per phase) and squared coefficient of variation (over all phases): Task Name Entry Name Phase 1 coeff of var **2 Users users 2.315e+006 1.04574 webserver htmlreq 1082.03 1.58596 database dbreq 153.76 1 Throughputs and utilizations per phase: Task Name Entry Name Throughput Phase 1 Total Users users 0.000672106 1 1 webserver htmlreq 0.0395198 1.03226 1.03226 database dbreq 0.0497703 0.617152 0.617152 Utilization and waiting per phase for processor: AppHost Task Name Pri n Entry Name Utilization Ph1 wait webserver 0 80 htmlreq 0.395198 0 Utilization and waiting per phase for processor: DBHost Task Name Pri n Entry Name Utilization Ph1 wait database 0 5 dbreq 0.617152 0 Fig 7. The LQN solver output