Past one. Agenda. Test Maturity Model Integration. Software Testing chapter TI/KVIV. Part 1 TMMi history and model Two TMMi PA s

Transcription

1 Test Maturity Model Integration Software Testing chapter TI/KVIV Agenda Part 1 TMMi history and model Two TMMi PA s Part 2 TMMi assessment approaches Data from 25 TMMi workshops Conclusions 2 Past one 1

2 TMMi essentials Test maturity framework Benchmark model for assessments 5 levels Based on: TMM Evolutionary Testmodel (Gelperin / Hetzel) CMMI for Development In line with ISTQB terminology Developed by the TMMi-foundation Public domain 4 TMMi foundation 5 The TMMi board 6 2

3 TMMi levels 5. Optimization Defect Prevention Test Process Optimization Quality Control 4. Management and Measurement Test Measurement Software Quality Evaluation Advanced Peer Reviews Test Organization Test Training Program 3. Defined Test Life Cycle and Integration Non-Functional Testing Peer Reviews Test Policy and Strategy Test Planning 2. Managed Test Monitoring and Control Test Design and Execution Test Environment 1. Initial 7 The proof of the pudding is in the eating Wanted: one volenteer (preferrably two persons from one organization) They rate two process areas using the SYSQA Quick Scan tool with me as facilitator Test Design and Execution Test Policy and Strategy Other attendents rate the PA s using the forms 8 Part 2 TMMi assessments 3

4 Assessments and TMMi implementations A A A A A A A A A A Start Level 2 Time 11 TMMi assessments types in practice Formal Officially determine extent TMMi requirements are met Informal Reasonably accurate extent TMMi requirements are met 12 4

5 TMMi official assessments types Formal Verified benchmark against TMMi Strengths / weaknesses and full gap analysis Assessment Team Lead: accredited lead assessor Team consists of at least 2 accredited assessors At least two types of evidence Informal No rating against TMMi Reasonable accurate understanding of maturity Assessment team leader: experienced assessor Teamsize: at least 1 person One type of evidence is sufficient 13 TAMAR content - 1 Overview Planning Purpose, scope, contraints Activities Resources (roles / physical resources) Schedule Roles Assessment Sponsor Assessment Organizer Assessment Team Leader Assessment Team Member 14 TAMAR content - 2 Data Management Documentation Interviews Evaluation questionnaires Surveys Process Attribute Rating N: %-15% P: 15%-% M: %-85% F: 85%-% Not Applicable Not Rated 15 5

6 TAMAR content - 3 Reporting Date of assessment Assessment inputs Objective evidence Assessment method Rating (formal assessment) 16 TMMi assessments types in practice Formal Officially determine extent TMMi requirements are met Informal Reasonably accurate extent TMMi requirements are met TAMAR Quick Scans Indication TMMi requirements are met 17 Assessment type: Quick Scan QA manager Tester 1 Project manager Tester 2 Test manager 2 9. Opening 9. TMMi. Rating PA s 12. Lunch 13. Rating PA s Test manager 1 Test environment administrator You 18 6

7 The tool The tool - 2 Assessment type: Quick Scan QA manager Tester 1 Test manager 1 Project manager You Tester 2 Test manager 2 Test environment administrator 9. Opening 9. TMMi. Rating PA s 12. Lunch 13. Rating PA s 14. Conclusions 14. Determine improvement actions 16. Making action plan 16. Evaluation (. On the couch watching TV)

8 Case assessment approaches Group case Summarize the advantaged and disadvantages of the different TMMi assessment approaches 22 Results of 25 quick scans / assessments How I got my data Goals Learn Presales Assignments 25 organizations Telecommunication: 2 Government: Financial services: Industry: 3 Analysis 27 8

9 Focus: level 2 5. Optimization Defect Prevention Test Process Optimization Quality Control 4. Management and Measurement Test Measurement Software Quality Evaluation Advanced Peer Reviews Test Organization Test Training Program 3. Defined Test Life Cycle and Integration Non-Functional Testing Peer Reviews Test Policy and Strategy Test Planning 2. Managed Test Monitoring and Control Test Design and Execution Test Environment 1. Initial 28 Results Fasten your seatbelts Test policy and strategy - goals Establish a test policy Establish a test strategy Establish test performance indicators 9

10 Test policy and strategy: A: 31% V: 15% Test policy and strategy Percentage of sp ecific goals met Organizations 31 Test planning - goals Perform a product risk assessment Establish a test approach Establish test estimates Develop a test plan Obtain commitment to the test plan 32 Test planning: A: 42%, V: 12% Test planning Percentage of spe ecific goals met Organizations 33

11 Test monitoring and control - goals Monitor test progress against plan Monitor product quality against plan and expectations Manage corrective action to closure 34 Test monitoring and control: A: 46%, V: 17% Test monitoring and control Percentage of spe ecific goals met Organizations 35 Test design and execution - goals Perform test analysis and design using test design techniques Perform test implementation Perform test execution Manage test incidents to closure 36 11

12 Test design and execution: A: 61%, V: 9% Test design and execution Percentage of spe ecific goals met Organizations 37 Test environment - goals Develop test environment requirements Perform test environment implementation Manage and control test environments 38 Test environment: A: 6%, V: 7% Test environment Percentage of spe ecific goals met Organizations 39 12

13 Overview Per process area English data Engeland Engeland Test policy and Test planning Test monitoring Test design and Test strategy en control execution environment English data provided by 41 England vs The Netherlands 7 6 Engeland Nederland Test policy and Test planning Test monitoring Test design Test strategy en control and execution environment English data provided by Dutch data provided by 42 13

14 Is the outcome typical for European? New Zealand Australia 43 My conclusion Outside 31%: Test policy and strategy 42%: Test planning 46%: Test monitoring and control Inside 6%: Test environment 61%: Test design and execution 44 We are in a cave! 45 14

15 46 Best practices outside of your cave Perform a product risk assessment and establish a test approach with relevant stakeholders Obtain commitment for your plan from relevant stakeholders Plan stakeholder involvement in your project Monitor stakeholder involvement in your project Make release advice with people outside of testing Establish and maintain a test strategy with relevant management Define, measure and report test performance indicators to relevant stakeholders Establish and maintain a test policy with higher management (business and IT) 47 Stuart Reid s test competence model for testers Test skills Soft skills IT skills Domain knowledge 48 15

16 My test competence model for test consultants Test skills Political skills IT skills Soft skills Domain knowledge 49 To be successful with TMMi, knowledge of TMMi alone is not enough 51 16

17 52 17