A Framework to Balance Tracing Agility and Formalism

Transcription

1 A Framework to Balance Tracing Agility and Formalism Matthias Heindl, Stefan Biffl Dindin Wahyudin Institute of Software Technology and Interactive Systems Vienna University of Technology

2 Requirements Tracing Context SE projects have the general goal to meet the customer s needs; The customer s needs may change and methods are needed to support change management of requirements and consistency of artefacts Requirements traceability: RT is the ability to follow the life of a requirement in a forward and backward direction. Target Audience Researchers and practitioners in the field of requirements management Project managers Quality managers Software engineering teams 2

3 What is traceability? ability to describe and follow the life of a requirement in both a forward and backward direction (Gotel, Finkelstein, ICRE 1994) The degree to which a relationship can be established between two or more products of the development process (IEEE standard glossary) Numerous standards require traceability: ISO 15504, CMMI, IEEE Std830 Many companies are mandated to implement traceability

4 Example Benefits of Requirements Traceability Requirements Tracebility supports Type of Analysis Impact Analysis Derivation Analysis Description Following links, in answer to the questions: What if this were to change? Following links, in answer to the question: Why is this here? Supported Processes Change Management Cost-Benefit Analysis Coverage Analysis Counting statements that have links, in answer to the question: Have I covered everything? Most often used as a measure of progress. Management Reporting Source: Brian Berenbach, Practical Traceability, Siemens Requirements Engineering Conference, Nürnberg, 2007

5 There are multiple RT approaches available Manual tracing with matrices Tool-supported RT requirements management tools [4], plug-ins [1] Model-driven approaches Automated trace generation approaches (dynamic tracing, data warehouse approach etc.) Event-Based approaches Value-Based approaches

6 Trade Offs and Selection of Requirements Tracing Approaches In a real-world project full tracing of all requirements on the most detailed level can be very expensive and time consuming. The costs and benefits to support the desired fast and complete change impact analysis need to be investigated with empirical data. There is very little systematic discussion on how to measure and compare the tracing effort and effectiveness of tracing strategies

7 A Tracing Activity Model (TAM) The TAM defines common elements of RT approaches: trace specification, generation, deterioration, validation, rework, and application. A framework to systematically describe and help determine the likely efforts and benefits of tracing approaches. Guidance for researchers on research activity in field of requirements tracing. The model can support requirements and test managers in comparing requirements tracing strategies. Validation of the model A feasibility study on retesting compares effort, risk, and delay of 3 tracing strategies (ranging from agile to formal tracing strategies)

8 Requirements Traceability Process Does the TAM model the reality? -> RQs

9 Feability Study To what extent can we balance the agility of a re-testing approach without using traces and the formalism of a systematic tracing approach for retesting with a value-based approach? The main focus of the study was to compare the efforts of each tracing strategy and the expected benefits of trace usage for retesting. Case study object: Re-testing at an Austrian Bank The TAM output variables were (1) the total effort of re-testing, (2) the risk of each strategy, and (3) the delay.

10 3 Requirements Tracing Strategies Ad-Hoc Tracing, standard strategy in the feasibility study context No formal tracing is done the activities of the tracing activity model are not performed and re-testing has to cope without traces Full Formal Tracing (FFT), testers systematically establish traceability by relating requirements and test cases (full tracing) via a tool, e.g. the Mercury Test Director. When a change request occurs, testers check, and adapt existing test cases whenever possible; else testers create new test cases. the upfront effort for FFT is considerably high, because all existing requirements have to be traced to test cases. Value-Based Tracing (VBT). is a hybrid between FFT and ad-hoc tracing VBT tries to reduce tracing effort by establishing traceability on a coarse level (to test case packages instead of particular test cases) and to refine them ad-hoc when necessary.

11 Effort Formulas for RT Strategies No Trace Reuse (NTR) Effort E (NTR) = new test case creation effort + replacing old test case effort E (NTR) = 680 hrs Full Formal Tracing (FFT) Effort E(FFT) = upfront trace effort+ identify test case effort + adapt and reuse test case effort E(FFT) = 173 hrs Value Based Tracing Effort E(VBT) = upfront trace effort (on package level) + change impact analysis (VBT) E(VBT) =395 hrs

12 Results Comparing to the NTR strategy, the FFT alternative takes around 20% less effort. In this case the upfront investment into traceability pays off In many cases, FFT can cause considerably high effort which may prevent tracing in practice. Here, the VBT seems to be a promising approach that can significantly save efforts. Besides effort, the alternatives also differ in delay when traces can be used for the trace application in our case re-testing. VBT has a larger delay, because trace refinement has to be done before re-test. Concerning risk, NTR would be more risky if obsolete test cases were not checked; Inconsistent or redundant test case sets could then result in increased hidden testing effort or lower-quality test sets. Combination of upfront tracing on a coarse level of detail (e.g., package or class level) and just-in-time detailed tracing of really needed traces can help balancing tracing agility (for use in practice) in a formal tracing framework (for research and process improvement).

13 Conclusion TAM provides useful building blocks for reasoning about relevant parameters (efforts, risks, etc.) of a tracing strategy and estimation of outcomes Full tracing provides detailed traces, which are particularly useful for situations when artifacts are not volatile and quick feedback is at a premium, e.g., for comprehensive cross checks at milestone reviews. In volatile parts of the project, agile (just in time) or value-based approaches are favorable as full tracing has a particularly high risk of loosing upfront investments If calculated efforts of tracing strategies do not differ significantly, choose a value-based strategy to provide full (complete) traceability at a coarse level of detail. Future work will be used TAM as (a) framework for a systematic literature review for requirement tracing literature and (b) to apply TAM for studies on tracing strategies in other contexts