Metrics for Analyzing the Quality of Model Transformations

Size: px

Start display at page:

Download "Metrics for Analyzing the Quality of Model Transformations"

Cory Grant
10 years ago
Views:

1 Metrics for Analyzing the Quality of Model Transformations Marcel van Amstel 1 Christian Lange 2 Mark van den Brand 1 1 Eindhoven University of Technology, The Netherlands 2 Federal Office for Information Technology, Germany The 7th BElgian-NEtherlands software evolution workshop, December 11 th 2008 Marcel van Amstel, Christian Lange, Mark van den Brand 1/22

Federal Office for Information Technology, Germany The 7th BElgian-NEtherlands software

2 Presentation Outline 1 Background 2 Goal 3 Approach 4 Tool 5 Empirical Study 6 Conclusions 7 Future Work Marcel van Amstel, Christian Lange, Mark van den Brand 2/22

3 Background Model Driven Engineering Domain specific languages for modeling. Model transformations for synthesis. Marcel van Amstel, Christian Lange, Mark van den Brand 3/22

4 Background Model Transformation Formalisms ATL QVT openarchitectureware ASF+SDF TOM Stratego/XT... Marcel van Amstel, Christian Lange, Mark van den Brand 4/22

5 Background ASF+SDF Specification of (domain specific) languages in SDF. Specification of conditional rewrite rules in ASF. Transformations are syntax-safe. Marcel van Amstel, Christian Lange, Mark van den Brand 5/22

6 Background ASF+SDF Example Syntax definition (language) context-free syntax <Name, List[[Attribute]], List[[Operation]]> -> Class String -> Name String -> Attribute String -> Operation Marcel van Amstel, Christian Lange, Mark van den Brand 6/22

7 Background ASF+SDF Example Syntax definition (transformation signatures) context-free syntax addgettersandsetterstoclass(class) -> Class creategettersandsetters(list[[attribute]]) -> List[[Operation]] creategetter(attribute) createsetter(attribute) -> Operation -> Operation variables "$Class"[0-9\ ]* -> Class "$Name"[0-9\ ]* -> Name "$Attribute"[0-9\ ]* -> Attribute "$AttributeList"[0-9\ ]* -> List[[Attribute]] "$OperationList"[0-9\ ]* -> List[[Operation]] Marcel van Amstel, Christian Lange, Mark van den Brand 7/22

$Operation -> Operation variables "$Class"[0-9\ ]* -> Class "$Name"[0-9\ ]* -> Name "$Attribute"[0-9\ ]* -> Attribute$

8 Background ASF+SDF Example Transformation definition (implementation) equations [addgettersandsetterstoclass-1] <$Name, $AttributeList, $OperationList> := $Class, $OperationList1 := creategettersandsetters($attributelist), $OperationList := concat($operationlist1, $OperationList), $Class := <$Name, $AttributeList, $OperationList > ====> addgettersandsetterstoclass($class) = $Class Marcel van Amstel, Christian Lange, Mark van den Brand 8/22

creategettersandsetters($attributelist), $OperationList := concat($operationlist1, $OperationList), $Class :=

9 Background ASF+SDF Example Term definition Marcel van Amstel, Christian Lange, Mark van den Brand 9/22

10 Background ASF+SDF Example Transformation result Marcel van Amstel, Christian Lange, Mark van den Brand 10/22

11 Goal Problem Statement Model driven engineering is becoming increasingly important. Model transformations are similar to traditional software artifacts. Marcel van Amstel, Christian Lange, Mark van den Brand 11/22

12 Goal Research Goal Make the quality of model transformations measurable. Marcel van Amstel, Christian Lange, Mark van den Brand 12/22

13 Goal Research Goal Make the quality of model transformations measurable. Model transformation M t M Marcel van Amstel, Christian Lange, Mark van den Brand 12/22

14 Approach Quality Attributes Quality: Understandability Modifiability Reusability Modularity Completeness Consistency Conciseness Marcel van Amstel, Christian Lange, Mark van den Brand 13/22

15 Approach Metrics Three categories: Function metrics Module metrics (In)consistency metrics Marcel van Amstel, Christian Lange, Mark van den Brand 14/22

16 Metrics Collection Tool Tool architecture ) 5. 5,. 5 F A? E B E? = J E. =? J - N J H =? J H. =? J I A J H E? I, = J = > = I A + =? K = J H ) 5. 5,. ) 2 1. H J * =? A J H E? I 4 A F H J I Marcel van Amstel, Christian Lange, Mark van den Brand 15/22

17 Metrics Collection Tool Metrics report Metric Value Number of functions 47 Number of signatures 80 Number of equations 344 Number of modules 35 Number of signatures without equations 11 Metric Min. Max. Avg. Median StdDev. Number of functions per module ,46 0 3,67 Number of signatures per module ,29 0 6,15 Number of equations per module , ,97 Number of signatures per function 1 6 1,43 1 1,02 Number of equations per function , ,23 Number of equations and conditions per function , ,17 Fan-in (function) ,85 2 9,03 Fan-out (function) , ,12 Fan-in (module) , ,56 Fan-out (module) ,33 7,5 29,42 Number of conditions per equation 0 8 1,53 1 1,84 Number of assignment conditions per equation 0 8 1,37 1 1,79 Number of distinct return values per function 1 3 1,07 1 0,32 Number of import declarations ,77 3 2,92 Number of times a module is imported 0 6 2,09 2 1,36 Number of variables per type ,65 2 2,42 Number of different variables per type 1 4 1,25 1 0,63 Number of types per variable 1 2 1,19 1 0,39 Number of unused variables per module ,51 0 3,25 Marcel van Amstel, Christian Lange, Mark van den Brand 16/22

Number of functions per module 0 19 1,46 0 3,67 Number of signatures per module 0 33 2,29 0 6,15 Number of equations per module 2 92 28,67 18 27,97 Number of signatures per function 1 6 1,43 1 1,02

18 Study Design Introduction Goal: Automated quality analysis Method: Questionnaire Semi-structured interview 6 cases 4 respondents Marcel van Amstel, Christian Lange, Mark van den Brand 17/22

19 Study Design Overview of the cases Transformation # Modules # Functions Purpose ACP2UML Transform process algebra models into UML state machines SL2XMI 8 70 Transform UML activities enriched with surface language constructs into plain activities SLCheck 9 58 Surface language wellformedness checker ASF2C ASF to C compiler UML2DOT 8 28 Transform UML activities into the input language of the visualization engine DOT REPLEO Syntax-safe template engine Marcel van Amstel, Christian Lange, Mark van den Brand 18/22

Surface language wellformedness checker ASF2C 41 396 ASF to C compiler UML2DOT 8 28 Transform UML activities into the input language

20 Quality Analysis Expert evaluation ACP2SM SL2XMI SLCheck ASF2C UML2DOT REPLEO Understandability 2,83 3,17 3,04 1,69 3,38 2,38 Modifiability 3,25 3,17 2,92 1,44 3,25 2,54 Reusability 2,89 3,11 2,89 2,25 3,56 3,11 Modularity 3,89 2,00 2,22 4,00 2,00 3,50 Completeness 3,89 4,11 3,83 4,00 4,06 3,33 Consistency 3,61 3,89 4,00 3,50 4,11 3,06 Conciseness 3,11 3,33 3,22 2,67 3,56 3,11 Marcel van Amstel, Christian Lange, Mark van den Brand 19/22

Modularity 3,89 2,00 2,22 4,00 2,00 3,50 Completeness 3,89 4,11 3,83 4,00 4,06 3,33 Consistency 3,61 3,89

21 Relating Metrics to Quality Attributes Marcel van Amstel, Christian Lange, Mark van den Brand 20/22

22 Conclusions Quality analysis of model transformations is important. 7 quality attributes and 31 metrics. Relation between metrics and quality attributes based on empirical data. Marcel van Amstel, Christian Lange, Mark van den Brand 21/22

23 Future Work Extend the set of metrics and quality attributes. Perform a larger empirical study. Create a quality model. Apply techniques to other transformation formalisms. Propose a methodology for creating high-quality model transformations. Marcel van Amstel, Christian Lange, Mark van den Brand 22/22

DSL Design. Model Transformations. Model Transformations. Language g Implementation Strategies

DSL Design. Model Transformations. Model Transformations. Language g Implementation Strategies DSL Design Generic Language g Technology 2IS15 Model Transformations Language g Implementation Strategies Stand-alone Marcel van Amstel Embedding Translation / Software Engineering and Technology 9-1-2012