Lean Kaizen Applied To Software Testing. Thomas Axen

Transcription

1 Lean Kaizen Applied To Software Testing Thomas Axen

2 Agenda Short about ATP - Test Organization Why Test Kaizen - Blitz and Relationship to our test improvement areas Test Kaizen - Blitz seen from the theory to practices The Test Kaizen - Blitz process in ATP Test improvement overall view Summary Questions 4

3 The purpose of this presentation To share my experiences with others We hope that you can use some of this information in your daily work We hope that you come up with some good questions for me, that I can use in my daily work 5

4 Short about ATP - General Employees in ATP FTE at Hillerød Campus FTE in 5 call centers Employees in IT FTE - 80 Consultants Employees in Test - 27 FTE - 23 Consultants Customers ATP livslang Pension (ATP) Arbejdsgivernes Elevrefusion (AER) Arbejdsmarkedets Erhvervssygdomssikring (AES) Barsel.dk DA-Barsel FerieKonto (FK) Finansieringsbidrag (FIB) Lønmodtagernes Dyrtidsfond (LD) Lønmodtagernes Garantifond (LG) Supplerende arbejdsmarkedspension for førtidspensionister (SUPP) Den Særlige Pensionsopsparing (SP) Finansieringsbidrag (FIB) Skattenedslag for seniorer (SFS) Pension Danmark (PD) Objektiv sagsbehandling New customers - Folkepensionen - Udbetaling og indtægtsregulering af fortidspension - Boligstøtte - Barselsdagspenge - Forskudsvis udbetaling af børnebidrag 6

5 Short about ATP - Test Organization The test resources (50 FTE) - 3 Test chiefs - 6 Test Managers (RTM, TI and TM) - 9 Test Coordinators - 3 Test Specialists (automated test and test data) - 6 IT-Testers and students workers (ITU and DTU) - 23 Test consultants Responsibility areas - Test in projects and Sustained Engineering - Test methods and tools (QC, QTP, ATSA, TDM) 7

6 Why Test Kaizen - Blitz We have used Kaizen on other areas with great success Top management see it as a way of tuning the organization A god way of interacting with the stakeholders, to understand their needs and attitude towards test An new positive way of identifying improvements areas on test All the testers can impact/influence on what the improvement areas are and what we shall prioritize to work on It verifies if we are working on the right improvement areas 8

7 Relationship to our test improvements areas Inspiration Process improvement Implementation Management decision Improvement inspiration Feedback TMM assessment Kaizen - Blitz Guru Network Best Practice Test organisation Improvements Governance Prioritizing Execution Project Sustained Engineering Strategy 9

8 Test Kaizen - Blitz seen from the theory to practices Lean thinking: Elimination of waste Flow (production processes are made continuous) Three central concepts: Waste is everything the customer won t pay for Flow implies elimination of batches and queues Production is mass production 10

9 but Isn t All Testing Waste? The answer is yes in the context of software production: Customers will not pay you for making errors. Consequently customer will not pay you for finding and correcting the errors you make. But we are not looking at test in the context of software production. We are looking at test in the context of a customer checking the quality of software received from a vendor. In this context test is not waste. 11

10 How Do We Identify Waste in Testing? ATP chose Kaizen Blitz from the Lean Toolbox. Kaizen means continuous improvements and is by the book : Dealing with a limited and thus manageable part of a production process (comparable to the work of 5 to 6 persons). Being performed quickly but having ambitious goals for removing waste from the area under consideration. Involving the 5 to 6 persons, who operate the process and are familiar with it. Looking for improvements, which the persons involved are themselves in command of. Being based on facts and measurements as far as possible. Being repeated at regular intervals. 12

11 ATP chose a pragmatic approach to Kaizen Dealing with a limited and thus manageable part of a production process. The areas chosen were somewhat big, since it included both Domain, Integration, and User Test. Being performed quickly but having ambitious goals for removing waste from the area under consideration. 3 short workshops over 6 weeks instead of 3 intensive days. Next time Involving exactly the 5 to 6 persons, who operate the process and are familiar with it. Looking for improvements, which the persons involved are themselves in command of. Several useful improvements were owned by other parties. Being based on facts and measurements as far as possible. Neither time nor money for measurement on a large scale. (Common sense) Being repeated at regular intervals. One time must be the first time. Next time 13

12 The Test Kaizen - Blitz process in ATP 1 Step - Define the reason for the investment in kaizen - Business case with focus on cost/benefit and scope. - Communication to sponsor ongoing on status and progress 2 Step - ATP Test Kaizen kick-off - Selecting the tester group that shall work with the kaizen process - Short introduction to kaizen, and the plan (30 min.) 3 Step - Interviewing stakeholder and collecting facts - The VoxPop (Voice of the Customer) input. 4 Step - Workshop, Testers as active players - Present Voxpop and collected facts - Kaizen scope and definition of waste in testing - Value Stream (mapping all test activities and waste pr. Test phase) 5 Step - Workshop, RCA and classification - Root Cause Analysis (Fishbone and selected defects analysis) 6 Step - Workshop, Consultation - Classification and prioritization the improvement areas - Finalize the proposal to management (the presentation) 7 Step - Summary (Presentation for managers and a execution plan) 14

13 3 Step - Interviewing stakeholder and collecting facts Objective: To challenge the Kaizen team members attitude towards their work. Examples: Far to many errors do not surface before full production. The Voice of the Customer: - To little testing. To much test administration. Spend more time testing! - I have never been on a project with so many test phases

14 4 Step - Workshop, Kaizen Scope and Definition of Waste SIPOC - Scope of ATP Test Kaizen (Black outline in Kaizen scope) Programming Component test Domain test Integrations test User test Operation Definition of waste in testing: All activities not directly involved in finding errors or in proving the absence of errors are considered waste. All repetition of test etc. are considered waste. 16

15 Ressourceforbrug/timer 4 Step - Workshop, Value Stream (mapping all test activities and waste pr. Test phase) Objective: To get a clear picture of the time and resources spend on testing and to identify activities where time and resources are wasted. What was done? Detailed mapping of 3 subprocesses: Domain test, Integration test and User test (3 value stream maps). Identification of waste pr. subprocess Fordeling af kundeværdi og spil - Best case Example: Integration test NVA BNVA CVA a 6b 6c 7 8a 8b 9b 9c Aktivitet

16 5 Step - Workshop, Root Cause Analysis Objective: To prepare for elimination of waste by identifying the underlying reasons. What was done? Systematic root-cause-analysis of all activities involving waste. A. Testing is not finished within time and budget I. To few people with proper business and domain knowledge in the individual test phases a. Different (new) people are testing a given piece of software in each of the test phases. 1. To many handovers between phases.

17 Effect on waste elimination High Low 6 Step - Workshop, classification and prioritization the improvement areas Objective: To formulate operational proposals, leading to elimination of waste in testing. What was done: Brainstorm among participants on the basis of the Root Cause Analysis. Identification of 38 proposals for elimination of waste. Consider Don't Prioritisation of proposals: Resources vs. Effect. Do Consider Anchoring: Who is doing what? Low High Resources needed for implementation

18 6 Step - Workshop, Finalize the proposal to management We identified 38 improvement areas We grouped the improvement areas into 4 categories - Go Fix without any additional investments (9 areas) - Investment We need an investment based on a Business case and return of investment analysis (3 areas) - In progress investment already in place project up an running (3 areas) - Relationship to other areas (4 areas) Rewriting of all the improvement areas so management can understand the value of the investments 20

19 7 Step - Summary, Go Fix without any additional investments Priority 1 - Tester should have more focus on progress, instead of roles - Release management plan, must reflect dependencies so it is easy for the tester to plan their test - With critical issues, it should be possible to gather a team across the normal teams Priority 2 - Post-mortem on found defect in production and projects (Root course analysis) - Optimize the defect process - General education plan (Consultants, PM, Business, Testers) Priority 3 - Test cases must follow the BPT standard - Improvement of the code documentation on defect and design changes, so test can focus and minimizing the cost - Quality assurance on deliverables in a structured way (Focus on static testing) 21

20 7 Step - Summary, Investment based on BC and ROI Organization of the test pr. release - All hours on test are used optimal focus on planning, competences and the team work together Quality assurance of all software deliverables collected systematically for reporting to management - Systematic collection of data, for reporting to management so they can use it for negotiations with vendors/teams on what the given quality should be for each of them (Project and SE) Release regressions test coverage - Well documented release regressions test coverage pr. customer/solution, so management can decide if we should go live 22

21 7 Step - Summary, In progress Strategy for how to test letters Test Data Management (Tools and reuse data) More automated regression test (BPT, QTP) 23

22 7 Step - Summary, Relationship to other areas Release management - Overall plan, deliverables, dependencies, risk and issues. - Requirements and change management Deployment to test environments (36 applications using SOA) - System test environments - Integration test environments Tester education with different focus - Consultants, Business unit area, End users, Project management Post-mortem on found defects In production and projects/root cause analysis 24

23 Test improvement overall view Department vision/mission & overall goals - Be proactive (defects prevention, Innovation, think ahead) - Agreements attitude (Clear, on time, expectation, follow-up) - Well-known ABP (Stable, optimized, functional) - Ready for changes (New things, flexible, developing and managed) Test vision & overall goals - From known to respected in ATP - Test in ATP should increase one level on TMM(i) - Effective test with use of Best Practices and front-runners on selected areas - Test process exists; it is always used and improved continually - Competent/ professional test resources in the IT-organization - Competent tests in the business-organization Test leadership - Updating the A4 plans for the whole test area - Updating detailed plans for each sub test areas - Ongoing prioritization and follow-up on tests areas across applications - Focus on improvements to minimize the effort TMM improvements (Level 3) - Continually optimizing/updating/support on test artifacts - Continually optimizing all test processes - Overview of all test tools and usages - Extend the defect root cause analysis - More focus on non-functional requirement and test techniques - More tester education to business, end users and consultants - Focus on test data availability - Use review for testing - Metrics on test Tester feedback loops - The best test case practice in ATP - Optimize the usages of our test environments - Optimizing of the review process on important artifacts Test Kaizen Blitz Improvements area - Quality assurance on deliverables in a structured way - New test organization to handle releases - Release regressions test coverage - Tester should have more focus on progress, instead of roles - Release management plan, must reflect dependencies so it is easy for the tester to plan their test - With critical issues, it should be possible to gather a team across the normal teams, to analyse the issue - Post-mortem on found defects in production and projects - Optimize the defect process - General education plan (Consultants, Business, End users, PM) - Test cases must follow the BPT standard - Improvement of the code documentation on defect and design changes, so test can focus and minimizing the cost - Strategy for how to test letters - Test Data Management (Tools and reuse data) - More automated regression test (BPT, QTP) Guru sanity check Recommendations - All tester roles must be defined and communicated in ATP - Test management meeting each 14 days - PM gets a Tester education day, and understand each others roles Governance projects - Extend the amount of automated regression test - Optimizing of our Test Data Management area - Stabilising of all reg. test cases for automation Project managers improvements areas - Improve the test environment setup - Test statistic (ex. defects in projects / production) - Target groups on test artefacts (Who should read what) - Move the test activities to the start of the project in the risk and QA - Root cause analysis on project defects 25

24 Summary We will definitely have a test Kaizen again, when we are ready It is a good way of making sure that all stakeholders are being heard Great way of getting a buy in from all the testers, where the testers have an influence on the improvements areas to execute on It improves the image and awareness about Test in ATP Remove activities that don t add value All relevant improvements areas are identified and prioritized This is a new angle on ways to identify improvement areas It also spins of improvement areas to other parts of the IT organization Adjust the communication with respect, so the stakeholders can see the value of each improvements area from their point of view. 26

25 Questions 27