METRICS RESEARCH ENABLING ACTIONABLE SOFTWARE METRICS IN MODERN COMPANIES

Transcription

1 RESEARCH ENABLING ACTIONABLE SOFTWARE METRICS IN MODERN COMPANIES ESTABLISHING CUTTING EDGE METRICS RESEARCH AND DEVELOPMENT ENVIRONMENT MIROSLAW STARON WILHELM MEDING KENT NIESEL ANDERS HENRIKSSON CHRISTOFFER HÖGLUND OLA SÖDER VARD ANTINYAN,

2 Who am I? Background Associate professor, University of Gothenburg Head of Software Engineering for Embedded and Automotive Systems research group (SEAS) at Chalmers University of Gothenburg > 130 papers published in the area of metrics, modelling (UML) and requirements Collaborations with Ericsson, Volvo Cars, AB Volvo, Saab, Sony, Grundfos, Telelogic, Axis, ABB, Wilhelm Meding, Kent Niesel, Ola Söder, Christoffer Höglund, Anders Henriksson, Research interest Metrology in Software Engineering Corporate Performance Measurement Product metrics and indicators Visualizations and dashboards 2

3 Outline for today Software Metrics theme Global trends in software metrics Metrics theme contributions to the global trends Wrap-up 3

4 Software Center Metrics Program Goals and wanted position Goal Rapidly empower the company (at all levels) to become excellent in measuring Wanted position Evolving existing best (metrics) practices Enhance release readiness assessments Increase robustness of measurement programs 4

5 Metrics research organizational context Insight Predictions, simulations, self-evolvning Statistics Org. cataloguing management Elevating organizational performance Decision support Collection Visualization Papers/research contribution SW development and management Metrics research Data analyses 5

6 Global Trends in Software Metrics Increased importance of software as an element of a global supply chain New metrics Empowered teams embracing diversity New dissemination patterns Status reporting is getting replaced by the need to have deeper insight New visualization patterns High level KPIs are complemented with PIs and low level trends New status reporting Shift from providing information to triggering decisions New intraction patterns Availability of large data sets New analysis methods Based on Staron M, Meding W, Niesel K, Söder O, Evolution of the Role of Measurement Systems in Industrial Decision Support, book chapter in Handbook of Research in Global Supply Chain Management 6

7 Increased importance of software as an element of a global supply chain SW as an element of a global supply chain Systems of systems Internet of things Suppliers of ECUs with/without SW New metrics Release readiness Innovation IPR 7

8 Infrastructure and common language set-up: Simplistic measurement system Measurement system is a set of interrelate and interacting elements necessary to achieve metrological confirmation and continual control of measurement processes (JCGM 200:2008) Can be realized using simple tools as MS Excel, Task Scheduler, etc Can be standardized using ISO

9 Pro-active measurement: Self-healing measurement infrastructure Advanced algorithms to automatically recover metrics Increased throughput of metric teams Increased data quality Reduced maintenance cost M. Staron, W. Meding, M. Tichy, J. Bjurhede, H. Giese, Evolving Measurement Systems into Self-Healing Systems for Improved Availability 9

10 Robustness of measurement programs Motivation Measurement awareness checklist from the workshop in 2013 Goal Quantifying awareness of metrics Impact Cross-company benchmarking Exchange of good practices Development of formal robustness model of measurement programs 10

11 Robustness of measurement programs Visualization of results (excerpt) Large variability Several categories have leading companies A number of good examples have been identified 11

12 Automatic Ranking of Textual Requirements Based on Quality Problem About 200 ECUs and textual requirements How to automatically identify requirements that are difficult to understand, implement, or test as early as possible Objective Design a method and tool for automatic assessment of requirements Identify Properties Calibrate Measure Evaluate with designers Name LD_Req-1175 v9 LD_Req v1 LD_Req v5 LD_Req v3 LD_Req v6 LD_Req v1 LD_Req v4 LD_Req v4 LD_Req-9847 v24 LD_Req-9916 v4 1Name Risk index Maintenance 107service Valve Lash 103 & Unit Injector Pre-Load Monitor Evaluation Main-beam 96 activation in Vehicle Mode Running/Prerunning, With AHS Display Customer1 95 Maintenance Display Standard 93 Maintenance VehicleMode_ctrl 81 Standard Oil 79Degradation Monitor Evaluation Standard Tachograph 79 Monitor Evaluation Engine Belt 78 Monitor Evaluation Air Filter Monitor 78 Evaluation 12

13 Requirements quality model We designed measures for Measuring the Internal quality properties of requirements which affects the external quality properties

14 Evaluate the ranking accuracy Quality index = Complexity + Dependency + Ambiguity Evaluation Volvo GTT project 1 87% prediction accuracy Volvo GTT project 2 70% Saab project 1-70% Volvo CC project - ongoing xid Conditions Uncertainty Dependency References Structuredness Tool Andreas Mattias Average comments Effectivenes % x cee x edf disagr. of raters x ae00e x ae09e ,5 x x adff Risky 15 by tool x ae0f x ec ,5 87% x db ,5 x e x cee ,5 x b2c ,5 x fb0bd Faulse positive x bfea ,5 14 x b Faulse positive

15 Future of metrics 1. Costs will decrease, speed will increase a. Delivery speed will increase b. Update speed will increase c. Stakeholders will get in control of the entire delivery chain 2. The number of metrics will continue to grow a. New products, New usage scenarios b. New markets, New challenges/problems 3. New analysis methods will appear a. Big data b. Software analytics c. Clouds 4. Complexity of metrics will grow a. Advanced analyses b. Instant access c. More stakeholders 15

16 VOLVO CARS METRICS SEMINAR SUCCESSFUL MEASUREMENT PROGRAMS HOW TO CREATING NEW PROCESS AND PRODUCT KPI-S SOFTWARE ARCHITECTURE EVOLUTION MEASUREMENT SOFTWARE DEFECT PREDICTION AUTOMATED REQUIREMENTS RISK ASSESSMENT , PVH

17 17 MIROSLAW STARON