RESEARCH ENABLING ACTIONABLE SOFTWARE METRICS IN MODERN COMPANIES ESTABLISHING CUTTING EDGE METRICS RESEARCH AND DEVELOPMENT ENVIRONMENT MIROSLAW STARON WILHELM MEDING KENT NIESEL ANDERS HENRIKSSON CHRISTOFFER HÖGLUND OLA SÖDER VARD ANTINYAN,
Who am I? Background Associate professor, University of Gothenburg Head of Software Engineering for Embedded and Automotive Systems research group (SEAS) at Chalmers University of Gothenburg > 130 papers published in the area of metrics, modelling (UML) and requirements Collaborations with Ericsson, Volvo Cars, AB Volvo, Saab, Sony, Grundfos, Telelogic, Axis, ABB, Wilhelm Meding, Kent Niesel, Ola Söder, Christoffer Höglund, Anders Henriksson, Research interest Metrology in Software Engineering Corporate Performance Measurement Product metrics and indicators Visualizations and dashboards 2
Outline for today Software Metrics theme Global trends in software metrics Metrics theme contributions to the global trends Wrap-up 3
Software Center Metrics Program Goals and wanted position Goal Rapidly empower the company (at all levels) to become excellent in measuring Wanted position Evolving existing best (metrics) practices Enhance release readiness assessments Increase robustness of measurement programs 4
Metrics research organizational context Insight Predictions, simulations, self-evolvning Statistics Org. cataloguing management Elevating organizational performance Decision support Collection Visualization Papers/research contribution SW development and management Metrics research Data analyses 5
Global Trends in Software Metrics Increased importance of software as an element of a global supply chain New metrics Empowered teams embracing diversity New dissemination patterns Status reporting is getting replaced by the need to have deeper insight New visualization patterns High level KPIs are complemented with PIs and low level trends New status reporting Shift from providing information to triggering decisions New intraction patterns Availability of large data sets New analysis methods Based on Staron M, Meding W, Niesel K, Söder O, Evolution of the Role of Measurement Systems in Industrial Decision Support, book chapter in Handbook of Research in Global Supply Chain Management 6
Increased importance of software as an element of a global supply chain SW as an element of a global supply chain Systems of systems Internet of things Suppliers of ECUs with/without SW New metrics Release readiness Innovation IPR 7
Infrastructure and common language set-up: Simplistic measurement system Measurement system is a set of interrelate and interacting elements necessary to achieve metrological confirmation and continual control of measurement processes (JCGM 200:2008) Can be realized using simple tools as MS Excel, Task Scheduler, etc Can be standardized using ISO 15939 8
Pro-active measurement: Self-healing measurement infrastructure Advanced algorithms to automatically recover metrics Increased throughput of metric teams Increased data quality Reduced maintenance cost M. Staron, W. Meding, M. Tichy, J. Bjurhede, H. Giese, Evolving Measurement Systems into Self-Healing Systems for Improved Availability 9
Robustness of measurement programs Motivation Measurement awareness checklist from the workshop in 2013 Goal Quantifying awareness of metrics Impact Cross-company benchmarking Exchange of good practices Development of formal robustness model of measurement programs 10
Robustness of measurement programs Visualization of results (excerpt) Large variability Several categories have leading companies A number of good examples have been identified 11
Automatic Ranking of Textual Requirements Based on Quality Problem About 200 ECUs and 10 000 textual requirements How to automatically identify requirements that are difficult to understand, implement, or test as early as possible Objective Design a method and tool for automatic assessment of requirements Identify Properties Calibrate Measure Evaluate with designers Name LD_Req-1175 v9 LD_Req-30085 v1 LD_Req-20677 v5 LD_Req-28823 v3 LD_Req-13284 v6 LD_Req-11659 v1 LD_Req-12235 v4 LD_Req-11349 v4 LD_Req-9847 v24 LD_Req-9916 v4 1Name Risk index Maintenance 107service Valve Lash 103 & Unit Injector Pre-Load Monitor Evaluation Main-beam 96 activation in Vehicle Mode Running/Prerunning, With AHS Display Customer1 95 Maintenance Display Standard 93 Maintenance VehicleMode_ctrl 81 Standard Oil 79Degradation Monitor Evaluation Standard Tachograph 79 Monitor Evaluation Engine Belt 78 Monitor Evaluation Air Filter Monitor 78 Evaluation 12
Requirements quality model We designed measures for Measuring the Internal quality properties of requirements which affects the external quality properties
Evaluate the ranking accuracy Quality index = Complexity + Dependency + Ambiguity Evaluation Volvo GTT project 1 87% prediction accuracy Volvo GTT project 2 70% Saab project 1-70% Volvo CC project - ongoing xid Conditions Uncertainty Dependency References Structuredness Tool Andreas Mattias Average comments Effectivenes % x0400000002cee415 48 1 31 0 52 78 4 4 4 x0400000002860edf 41 0 13 3 43 67 5 3 4 disagr. of raters x04000000011ae00e 34 1 51 0 60 62 4 4 4 x04000000011ae09e 34 1 51 0 60 62 5 4 4,5 x0400000003062477 34 1 51 0 60 62 4 4 4 x04000000011adff4 33 1 51 0 60 60 4 4 4 Risky 15 by tool x04000000011ae0f1 33 1 87 0 96 60 4 4 4 x04000000031ec161 18 1 11 4 12 58 4 3 3,5 87% x0400000001511db4 39 0 28 0 51 55 4 5 4,5 x0400000002e51331 34 6 3 0 36 53 4 4 4 x0400000002cee406 25 1 21 0 25 49 3 4 3,5 x040000000288b2c1 39 0 17 0 46 49 4 3 3,5 x0400000002fb0bd4 19 1 21 0 15 47 1 1 1 Faulse positive x0400000001bfea10 24 4 18 0 32 46 4 3 3,5 14 x0400000002b66036 25 2 12 0 26 42 3 3 3 Faulse positive
Future of metrics 1. Costs will decrease, speed will increase a. Delivery speed will increase b. Update speed will increase c. Stakeholders will get in control of the entire delivery chain 2. The number of metrics will continue to grow a. New products, New usage scenarios b. New markets, New challenges/problems 3. New analysis methods will appear a. Big data b. Software analytics c. Clouds 4. Complexity of metrics will grow a. Advanced analyses b. Instant access c. More stakeholders 15
VOLVO CARS METRICS SEMINAR SUCCESSFUL MEASUREMENT PROGRAMS HOW TO CREATING NEW PROCESS AND PRODUCT KPI-S SOFTWARE ARCHITECTURE EVOLUTION MEASUREMENT SOFTWARE DEFECT PREDICTION AUTOMATED REQUIREMENTS RISK ASSESSMENT 2015-04-30, PVH 9.00 12.00
17 MIROSLAW STARON miroslaw.staron@gu.se