SOFTWARE CENTER PROJECT 1 IMPLICATIONS OF CONTINUOUS DEPLOYMENT Agneta Nilsson, Eric Knauss, Miroslaw Staron

Transcription

1 1 SOFTWARE CENTER PROJECT 1 IMPLICATIONS OF CONTINUOUS DEPLOYMENT Agneta Nilsson, Eric Knauss, Miroslaw Staron Sprint 7 Focus: - IntegraFon of RBTS from Project 3 - Explore how to use the CIVIT model to increase speed - Implement RBTS Collabora;ng Companies: - Axis CommunicaFons - Ericsson - Grundfos - Saab - Volvo AB - VCC

2 The CIVIT Model Customer Release Full Product ParFal Product Improvement DirecFons Subsystem Component Once /release Month Week Day Hour Immediate/Minutes

3 Legend F FuncFonal requirements Q Quality requirements Complete coverage Significant tesfng 70% < coverage < 95% ParFal tesfng 30% < coverage < 70% Some tesfng but less than 30% coverage No tesfng of this type at all Coverage for each type of tesfng F Q L E L Legacy funcfonality E Edge cases Level of test automafon Fully automated Significant automafon, between 70 and 95% ParFal automafon, between 30 and 70% Some automafon, less than 30% No automafon at all

4 4 2 Strategies to use Civit model to Increase Speed The CIVIT model Customer Release FullProduct Par=alProduct Subsystem Component F Q L E F Q L E F Q L E F Q L E F Q L E F Q L E F Q L E F Q L E F Q L E Once/release Month Week Day Hour Immediate/ Minutes Software Center, Project 1, Sprint 4

5 5 Stragegy #1: Increase Speed with Default to Green Flag Default: Green flag, move ahead with minimal delay Excep;on: Red flag when more confidence is needed.

6 6 Strategy #2: Increase Speed by Smart Reduc7on of Tes7ng The CIVIT model Customer Release FullProduct Par=alProduct Subsystem Component F Q L E F Q L E Run all F Q L E F Q L E Don't run Run F Q L E F Q L E F Q L E F Q L E F Q L E Don't run Once/release Month Week Day Hour Immediate/ Minutes Software Center, Project 1, Sprint 4 Run

7 Recommending tests (idea) PrioriFze tests based on heatmap Collect which modules/components where changed Sort tests by frequency that the failed in connecfon with collected changes Cut off list by some criteria Module 1 Module 2 Modeul 3 Module n Testsuite 1 Testsuite 2 Testsuite 3 Testsuite m

8 Principle 1: PrioriFze by Test Efficiency Sum (TesHailures) Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7 Test 8 Test 9 Test 10 StarFng from a heatmap we can understand test efficiency

9 Principle 1: PrioriFze by Test Efficieny Sum (TesHailures) Sum (TesHailures) Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7 Test 8 Test 9 Test 10 0 Test 7 Test 8 Test 10 Test 9 Test 4 Test 1 Test 3 Test 2 Test 5 Test 6 We can priorifze these tests (=sorfng) and select tests.

10 Principle 2: Refine Test Efficiency by Context If we know the modules that have recently changed we can characterize test efficiency specifically for these modules

11 Principle 3: Use addifonal knowledge Perhaps a test is not likely to fail, but it would be strategically problemafc. Therefore, we can also take into account business value

12 Longitudinal study on Principle 2 (without selecfon) %- tests recall

13 Recommending tests (prel. Results) Lower recall: Miss tests that would have failed Just take the top n% from priorifzed list Higher precision: Speedup (run less tests that do not fail) Just take the tests that are only m% less likely to fail than the best suggesfon

14 14 IntegraFng RBTS and Civit model to Increase Speed Module 1 Module 2 Modeul 3 Strategy 1: Default to green flag The CIVIT model Testsuite m Testsuite 1 Testsuite 2 Testsuite 3 Enough confidence? Module n Customer Release FullProduct Par=alProduct Subsystem Component F Q L E F Q L E F Q L E F Q L E F Q L E F Q L E F Q L E Define 7me F Q constraints L E F Q L E Once/release Month Week Day Hour Immediate/ Minutes Software Center, Project 1, Sprint 4 Strategy 2: Smart reduc;on of tests

15 15 Suggested Plan for Sprint 8 Increase speed in confnuous integrafon development process through priorifzed test cases InvesFgate further how to priorifze test cases using parerns of test effecfveness through involvement of master students in project InvesFgate further how to connect business value to features to priorifze test cases TransiFon towards confnuous deployment What is the next step in the transifon process and main challenges? IniFaFves taken or planned to advance from confnuous integrafon towards confnuous deployment. What are organizafonal and technical impediments, and what facilitates the transifon towards confnuous deployment?