Measuring the effectiveness of testing using DDP

Size: px
Start display at page:

Download "Measuring the effectiveness of testing using DDP"

Transcription

1 Measuring the effectiveness of testing using Prepared and presented by Dorothy Graham 1 Contents introduction: some questions for you what is and how to calculate it case studies uses, abuses, common concerns and advice 2

2 Questions you may be asked How good is the testing anyway? Can you prove you are doing a good job? Your testing can still be just as good in less time, can t it? (That deadline pressure really didn t matter, did it?) How many bugs have we missed? Is the testing any better for this release? (Have we learned anything?) (Have we really improved our testing?) Are we better or worse in our testing compared compared to how to other we were groups/organizations? last time / last year? 3 : what you need to have do you keep track of defects? defects found in testing different test stages, e.g. system test, user acceptance test different releases e.g. testing for an incremental release in RAD defects found in live running reported by users / customers can you find these numbers from a previous project and your current project? do you have a reasonable number of defects found? if so, you can use to measure your test effectiveness 4

3 Useful measures a useful measure: supports effective analysis and decision making, and that can be obtained relatively easily. Bill Hetzel, Making Software Measurement Work, QED, easy measures may be more useful even though less accurate (e.g. car fuel economy) useful depends on objectives, i.e. what you want to know 5 Contents introduction: some questions for you what is and how to calculate it case studies uses, abuses, common concerns and advice 6

4 How effective are we at finding defects? defects found in testing or defects found testing defects found in testing defects found wards not found -yet start release benchmark point 7 Defect Detection Percentage () defects found by this testing total defects including those found wards "this" testing could be a test stage, e.g. component, integration, acceptance, regression, etc. testing for a function, subsystem or defect type all testing for a system testing of a sprint or increment Note: Capers Jones Defect Removal Efficiency?? not removal but detection - not efficiency but effectiveness 8

5 Effectiveness at finding defects defects found release time 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% defects found in testing: defects found testing: total defects found: 50 = = % 9 example testing live running live % = = = % 10

6 Exercise 1 Exercise 1 The following data has been recorded for a project. Calculate the of each testing stage based on all the defect information. Fault Information Testing stage Official testing module and integration Tool testing & development Number of faults 299 = = = % 40 = = = % Release testing 19 = = = % User Acceptance test 10 = = = % Pilot 9 = = = % Running ( one month) 20 Hint: you don t really need a calculator just round the numbers to the nearest 10 and you will be close enough! Defects found in this stage of testing = Defects found in this and all subsequent stages of testing Dorothy Graham Page 1 of 2

7 Exercise 1 Exercise 1 Solution: Calculation Testing stage No. faults 1) Official testing module and integration % 300 / 400 2) Tool testing & development 40 40% 40 / 100 3) Release testing 19 33% 20 / 60 4) User Acceptance test 10 25% 10 / 40 5) Pilot 9 33% (or 31%) 10 / 30 ( 9 / 29) 6) Running ( one month) 20 n/a How did we get these figures? Remember = Defects found in testing / all subsequent defects Stage 1 Official testing Test stage 1 found approximately 300 defects this is the numerator (top) The sum of all the subsequent stages is = (rounded up) (rounded up) + 20 = 100 So the denominator (bottom of the equation) is = 400 for Stage 1 is therefore 300/400 or 75% Stage 2 Tool testing Test stage 2 found 40 defects this is the numerator (top) The sum of all the subsequent stages is = 20 (rounded up) (rounded up) + 20 = 60 So the denominator (bottom of the equation) is = 100 for Stage 2 is therefore 40/100 = 40% Stage 3 Release testing Test stage 3 found 19 defects (round up to 20) this is the numerator (top) The sum of all the subsequent stages is = (rounded up) + 20 = 40 So the denominator (bottom of the equation) is = 60 for Stage 3 is therefore 20/60 = 33% Stage 4 User Acceptance test Test stage 4 found 10 defects this is the numerator (top) The sum of all the subsequent stages is = 10 (rounded up) + 20 = 30 (29 to be exact) So the denominator (bottom of the equation) is = 40 for Stage 4 is therefore 10/40 = 25% Stage 5 Pilot Test stage 5 found 9 defects (round up to 10) this is the numerator (top) The sum of all the subsequent stages is = 20 (the only remaining stage is live running) So the denominator (bottom of the equation) is = 30 for Stage 4 is therefore 10/30 = 33% (31% if you calculate 9/29) There is no for live running, since the live running total goes into the calculation of all the previous s. Dorothy Graham Page 2 of 2

8 example: ST UAT ST UAT ST UAT ST UAT All test % = 100 = 100 = % example: ST ST UAT ST UAT ST UAT All test % 40% = 100 = 100 = % 14 14

9 example: UAT ST UAT ST UAT ST UAT All test % 40% 33% = 50 = 50 = % example: all test ST UAT ST UAT ST UAT All test % 40% 33% 60% = = 150 = % 16 16

10 Exercise 2 Exercise 2 The following data has been recorded for a project. Calculate the in the columns on the right. The first one has been done as an example. (A calculator may be useful for some of these your mobile phone has one!) Fault Information Release System Test User Acceptance Test running (1 month) ST UAT ST LR UAT LR All test LR Release % 40% 33% 60% Release Release Release How did we get these figures? Release 1 ST UAT: 100 / ( ) = 100 / 150 = 67% ST LR: 100 / ( ) = 100 / 250 = 40% UAT LR: 50 / ( ) = 50 / 150 = 33% (Remember not to include the ST defects here) All test LR: ( ) / ( ) = 150 / 250 = 60% Dorothy Graham www. DorothyGraham.co.uk Page 1 of 2

11 Exercise 2 Exercise 2 Solution: Calculations Release System Test User Acceptance Test running (1 month) ST UAT ST LR UAT LR All test LR Release % 40% 33% 60% Release % 71% 83% 95% Release % 67% 50% 83% Release % 25% 17% 38% How did we get these figures? Release 2 ST UAT: 150 / ( ) = 150 / 200 = 75% ST LR: 150 / ( ) = 150 / 210 = 71% UAT LR: 50 / ( ) = 50 / 60 = 83% All test LR: ( ) / ( ) = 200 / 210 = 95% Release 3 ST UAT: 200 / ) = 200 / 250 = 80% ST LR: 200 / ( ) = 200 / 300 = 67% UAT LR: 50 / ( ) = 50 / 100 = 50% All test LR: ( ) / ( ) = 250 / 300 = 83% Release 4 ST UAT: 50 / ( ) = 50 / 75 = 67% ST LR: 50 / ( ) = 50 / 200 = 25% UAT LR: 25 / ( ) = 25 / 150 = 17% All test LR: ( ) / ( ) = 75 / 200 = 38% Dorothy Graham www. DorothyGraham.co.uk Page 2 of 2

12 in iterative development 40 new sprint/release of Sprint 1 S of Sprint 2 S = 40 = 80 % 50 of Sprint 1 S3 = = 73 % = = 58 % 17 Contents introduction: some questions for you what is and how to calculate it case studies uses, abuses, common concerns and advice 18

13 Case studies from clients 1 mo year 1 70% year 2 92% 10 mo 50% est Finance (insurance) System Test Group = 38% (before performance testing) Priority 1 & 2 only: = 31% Operating system 23% to 87% by application Defects: 1 / / 200 Scientific software (chemical analysis) Not useful for low numbers of defects 19 Summary for AP Europe 20

14 Information Technology Conclusions UAT more variable than ST mainly personnel Target zone for ST : 75-90% Factors behind the figures size, complexity, tester experience, time, documentation whether UAT started before ST was finished where on the S-curve when stopped Figures don t tell you cost, severity of those you missed cost of finding 30 stopped here? or here? /12/01 08/12/01 09/12/01 10/12/01 11/12/01 12/12/01 13/12/01 14/12/ Summary for AP Europe Project or App. Months Status Comments Before New Testing Process S4 50% ESTIMATED After New Testing Process R1 3 81% FINAL Major re-engineering LBS 4 91% FINAL CP 7 100% FINAL Reporting System DS 3 95% FINAL APC 4 93% FINAL ELCS 4 95% FINAL Eur impl. of US system SMS 3 96% FINAL Enhancement Release C 4 96% FINAL E7 (US) 5 83% FINAL Global Enhancements E7 (Eur) 1 97% Global Enhancements Source: Stuart Compton, Air Products plc 22

15 Rolling Software Testing Defect Detection Percentage Measure (rolling quarterly produced values looking back four quarters) Period under review # Projects Analysed Target Defects in Testing Total Defects Prod'n Bugs Historical Estimate n/a 50 Rolling 1 Qtrs to Q1 Y1 2 n/a Rolling 2 Qtrs to Q2 Y1 1 n/a Rolling 3 Qtrs to Q3 Y1 1 n/a Rolling 4 Qtrs to Q4 Y1 2 n/a Rolling 4 Qtrs to Q1 Y Rolling 4 Qtrs to Q2 Y Rolling 4 Qtrs to Q3 Y Rolling 4 Qtrs to Q4Y Rolling 4 Qtrs to Q1 Y3 90? Rolling 4 Qtrs to Q2 Y3 90? Rolling 4 Qtrs to Q3 Y3 90? Rolling 4 Qtrs to Q4 Y3 90? Source: Stuart Compton, Air Products plc 23 Anonymous client all systems history of this ST UAT ST UAT ST LR UAT LR All LR Extreme % 31% 92% 95% Ultimate % 30% 85% 90% Professional % 40% 89% 94% Realistic % Idiotic % 64% 96% 98% Total % 36% 92% 95% 24

16 Anonymous client (telecoms) % 98.00% not yet stable decline why? Test Rel defects running % 96.00% % 94.00% % % 92.00% % 90.00% test process improvement % 88.00% % % Release % 25 Contents introduction: some questions for you what is and how to calculate it case studies uses, abuses, common concerns and advice 26

17 benefits can highlight test process improvements the effect of severe deadline pressure the impact of overlapping test phases can raise the profile of testing can help predict future defect levels is applicable over different projects reflects testing process in general can give on-going monitoring of testing 27 Uses of when you know test & production defects calculate to monitor the effectiveness of testing in finding defects for different test stages (e.g. ST, UAT) in different releases when you know your prediction: once you know your typical predict the number of production defects (e.g. when software is released) NOTE: not an exact science, but useful to set expectations! 28

18 Prediction of production defects Defects found so far 66% 20 50% 20 80% 20 Predicted defects not found yet Abuses of monitoring individuals or too small a group or timeframe using only other metrics are also important blame the testers if is low prediction only an indication take it with a grain of salt distorted reporting e.g. testers report the same defect 10 times don t report that, it ll make me look bad 30

19 When NOT to use when you don t have many defects in test or in production (i.e. very high quality software) your defect tracking is immature, purely subjective, untrustworthy, or non-existent the software products you produce are never used by anyone (no live running) it doesn t matter how many defects are in them it is impossible to get data on defects found in live running (difficult is OK!) you re not interested in improving 31 What does it mean? is very high ( > 95%) testing is very good? system not been used much yet? next stage of testing was very poor? e.g. ST looks good but UAT was poor, ST UAT is high but live running will find many defects! is low (< 60%) testing is poor? requirements were very poor, affecting tests? poor quality software (too many to find in the time)? deadline pressure testing was squeezed? 32

20 Options for measuring what to measure simplest: all test defects / all defects so far by severity level how "deep" to go? deeper levels give more detailed information deeper levels more complex to measure advice: start simple simple information is much better than none learn from what information you have 33 Technical aspects what time frame for defects found in live? this is arbitrary / whatever makes sense for you many people use 1 month, some use 3 or 6 months can I measure of different test stages? any stage where you have defects that came wards but don t measure individual people!! can I use in agile development? yes: choices accumulate, or measure until next release what if different defect tracking systems? ok to combine if consistently recorded 34

21 Accuracy of defect data most common stumbling block what about duplicates? what about enhancement requests? what if some aren t really defects? the same answer always applies it doesn t matter how you do it as long as you do it the same way each time! most useful aspect of trends, changes over time and why 35 The technical person s trap we re testers we can see all the problems! you may think of lots of problems with this metric yes, (as any measure) can be mis-used but that doesn t mean it can t be useful take the high level view, warts and all, computed simply and consistently, can help you monitor your testing processes and show the effects of both good and bad things 36

22 How to start using suggested first step calculate for a release that is now live what to measure first? most people start with System Test consider looking at highest severity only to start or two s, one for high severity, one for all defects getting data from live running if you don t normally have live defect data, ask for it data collection & calculation should be easy / automatic get your test management tool or defect tracking tool to calculate it for you automatically 37 Summary: key points requires counts of defects but does not need great accuracy is a useful measure easy to calculate based on defect data you probably already have can tell you how effective your testing efforts are and how other things affect it - discussion about 38

Test Automation Objectives

Test Automation Objectives Test Automation Objectives Prepared and presented by Dorothy Graham email: 1 Contents Why test automation objectives are important Commonly-held objectives and why they are not good Good objectives for

More information

Pre-Algebra Lecture 6

Pre-Algebra Lecture 6 Pre-Algebra Lecture 6 Today we will discuss Decimals and Percentages. Outline: 1. Decimals 2. Ordering Decimals 3. Rounding Decimals 4. Adding and subtracting Decimals 5. Multiplying and Dividing Decimals

More information

Selling Agile to the CFO: A Guide for Development Teams

Selling Agile to the CFO: A Guide for Development Teams Selling Agile to the CFO: A Guide for Development Teams You ve learned about agile development, or perhaps you have even worked in an agile organization and have now moved to a traditional one. You re

More information

Automated Testing and Agile

Automated Testing and Agile Automated Testing and Agile Contents Iterate with Confidence... 1 Is Automation Worth the Effort?... 3 The Test Automation Manifesto... 5 Continuous Integration... 7 Regression Testing... 8 Review at the

More information

Unit 1 Number Sense. In this unit, students will study repeating decimals, percents, fractions, decimals, and proportions.

Unit 1 Number Sense. In this unit, students will study repeating decimals, percents, fractions, decimals, and proportions. Unit 1 Number Sense In this unit, students will study repeating decimals, percents, fractions, decimals, and proportions. BLM Three Types of Percent Problems (p L-34) is a summary BLM for the material

More information

A Simple Guide to Churn Analysis

A Simple Guide to Churn Analysis A Simple Guide to Churn Analysis A Publication by Evergage Introduction Thank you for downloading A Simple Guide to Churn Analysis. The goal of this guide is to make analyzing churn easy, meaning you wont

More information

Testing, What is it Good For? Absolutely Everything!

Testing, What is it Good For? Absolutely Everything! Testing, What is it Good For? Absolutely Everything! An overview of software testing and why it s an essential step in building a good product Beth Schechner Elementool The content of this ebook is provided

More information

is your answer, but you are asked to give the answer (correct to 2 decimal places). then the second number is increased by 1.

is your answer, but you are asked to give the answer (correct to 2 decimal places). then the second number is increased by 1. DECIMAL PLACES Sometimes you are required to give a shorter answer than the one which you have worked out. Example 1 3.68472 is your answer, but you are asked to give the answer correct to 2 decimal places

More information

This Unit: Floating Point Arithmetic. CIS 371 Computer Organization and Design. Readings. Floating Point (FP) Numbers

This Unit: Floating Point Arithmetic. CIS 371 Computer Organization and Design. Readings. Floating Point (FP) Numbers This Unit: Floating Point Arithmetic CIS 371 Computer Organization and Design Unit 7: Floating Point App App App System software Mem CPU I/O Formats Precision and range IEEE 754 standard Operations Addition

More information

Testing in Agile methodologies easier or more difficult?

Testing in Agile methodologies easier or more difficult? Testing in Agile methodologies easier or more difficult? Lucjan Stapp Warsaw University of Technology Stowarzyszenie Jakości Systemów Informatycznych L.Stapp@mini.pw.edu.pl L.Stapp@sjsi.org Professor in

More information

Faster, Easier and Cheaper Software Development: Is It Possible?

Faster, Easier and Cheaper Software Development: Is It Possible? Faster, Easier and Cheaper Software Development: Is It Possible? Using Application Lifecycle Management to improve your software development process By Beth Schechner Elementool The content of this ebook

More information

MOBILE SUCCESS STRATEGIES

MOBILE SUCCESS STRATEGIES 7 MOBILE SUCCESS STRATEGIES FOR SALES / SERVICE TEAMS 2014 imason inc. CONTENTS 7 Mobile Success Strategies! The Mobile Explosion in Field Sales & Service 1 Have 2 Make 3 Make 4 Have 5 Allow 6 Make 7 Establish

More information

Section R.2. Fractions

Section R.2. Fractions Section R.2 Fractions Learning objectives Fraction properties of 0 and 1 Writing equivalent fractions Writing fractions in simplest form Multiplying and dividing fractions Adding and subtracting fractions

More information

Unit 7 The Number System: Multiplying and Dividing Integers

Unit 7 The Number System: Multiplying and Dividing Integers Unit 7 The Number System: Multiplying and Dividing Integers Introduction In this unit, students will multiply and divide integers, and multiply positive and negative fractions by integers. Students will

More information

Cut. software development. Improve defect removal efficiency while simultaneously lowering costs and shortening schedules.

Cut. software development. Improve defect removal efficiency while simultaneously lowering costs and shortening schedules. Cut software development Improve defect removal efficiency while simultaneously lowering costs and shortening schedules. Introduction In his paper entitled Software Defect Origins and Removal Methods,

More information

The Top FIVE Metrics. For Revenue Generation Marketers

The Top FIVE Metrics. For Revenue Generation Marketers The Top FIVE Metrics For Revenue Generation Marketers Introduction It s an old cliché but true: you manage what you measure. One implication is that when a business changes, measures should change as well.

More information

Kanban game. Danske Bank version developed by Sune Lomholt based on Software development Kanban 2009-2010 Christina Skaskiw

Kanban game. Danske Bank version developed by Sune Lomholt based on Software development Kanban 2009-2010 Christina Skaskiw Kanban game Danske Bank version developed by Sune Lomholt based on Software development Kanban Kanban Game Backlog Planned Analysis Development Test Deploy Done Doing Done Doing Done Doing Done Redistribute

More information

Agile version control with multiple teams

Agile version control with multiple teams Taking the pain out of branching & merging henrik.kniberg crisp.se Version 1.2, 2008-04-04 Introduction... 2 Purpose of this paper... 2 Who is this paper for?... 2 Disclaimer... 2 Goals... 2 Single page

More information

NF5-12 Flexibility with Equivalent Fractions and Pages 110 112

NF5-12 Flexibility with Equivalent Fractions and Pages 110 112 NF5- Flexibility with Equivalent Fractions and Pages 0 Lowest Terms STANDARDS preparation for 5.NF.A., 5.NF.A. Goals Students will equivalent fractions using division and reduce fractions to lowest terms.

More information

Measurement of Automation

Measurement of Automation 1 Measurement of Automation Tel: +44 1558 685180 Fax: +44 1558 685181 Prepared and presented by Mark Fewster Grove Consultants Llwyncynhwyra, Cwmdu Llandeilo, SA19 7EW, UK Email: mark@grove.co.uk www.grove.co.uk

More information

Applied Software Project Management

Applied Software Project Management Applied Software Project Management Introduction http://www.stellman-greene.com 1 Why do software projects fail? People begin programming before they understand the problem Everyone likes to feel that

More information

THE BUSINESS VALUE OF AGILE DEVELOPMENT

THE BUSINESS VALUE OF AGILE DEVELOPMENT David Chappell March 2012 THE BUSINESS VALUE OF AGILE DEVELOPMENT Sponsored by Microsoft Corporation Copyright 2012 Chappell & Associates When it comes to creating custom applications, too many of us live

More information

Unit 4 Number and Operations in Base Ten: Multiplying and Dividing Decimals

Unit 4 Number and Operations in Base Ten: Multiplying and Dividing Decimals Unit 4 Number and Operations in Base Ten: Multiplying and Dividing Decimals Introduction In this unit, students will learn how to multiply and divide decimals, and learn the algorithm for dividing whole

More information

Partial Fractions. (x 1)(x 2 + 1)

Partial Fractions. (x 1)(x 2 + 1) Partial Fractions Adding rational functions involves finding a common denominator, rewriting each fraction so that it has that denominator, then adding. For example, 3x x 1 3x(x 1) (x + 1)(x 1) + 1(x +

More information

Software development in distributed environments

Software development in distributed environments Software development in distributed environments Index Chapter 1. Software development model Chapter 2. Challenges Chapter 3. Remedy Chapter 4. Conclusions OPENSHPERE - Developing large projects in distributed

More information

Agile QA Process. Anand Bagmar Anand.Bagmar@thoughtworks.com abagmar@gmail.com http://www.essenceoftesting.blogspot.com. Version 1.

Agile QA Process. Anand Bagmar Anand.Bagmar@thoughtworks.com abagmar@gmail.com http://www.essenceoftesting.blogspot.com. Version 1. Agile QA Process Anand Bagmar Anand.Bagmar@thoughtworks.com abagmar@gmail.com http://www.essenceoftesting.blogspot.com Version 1.1 Agile QA Process 1 / 12 1. Objective QA is NOT the gatekeeper of the quality

More information

Developing a Load Testing Strategy

Developing a Load Testing Strategy Developing a Load Testing Strategy Michele Ruel St.George Bank CMGA 2005 Page 1 Overview... 3 What is load testing?... 4 Scalability Test... 4 Sustainability/Soak Test... 4 Comparison Test... 4 Worst Case...

More information

White Paper. Best Practices in Rolling Forecasts

White Paper. Best Practices in Rolling Forecasts White Paper Best Practices in Rolling Forecasts Table of Contents 10 best practices for rolling forecasts 3 1. You need a system and Excel is not a system (it s a personal productivity tool). 3 2. Understand

More information

FORECASTING. Operations Management

FORECASTING. Operations Management 2013 FORECASTING Brad Fink CIT 492 Operations Management Executive Summary Woodlawn hospital needs to forecast type A blood so there is no shortage for the week of 12 October, to correctly forecast, a

More information

Club Accounts. 2011 Question 6.

Club Accounts. 2011 Question 6. Club Accounts. 2011 Question 6. Anyone familiar with Farm Accounts or Service Firms (notes for both topics are back on the webpage you found this on), will have no trouble with Club Accounts. Essentially

More information

Experience, Not Metrics

Experience, Not Metrics Part 7: Consolidating Test Results User Experience, Not Metrics by: R. Scott Barber You ve been running this test for weeks and sending me charts almost every day, but what does it all mean?!? If your

More information

Getting Started With Common Formative Assessment (Slides)

Getting Started With Common Formative Assessment (Slides) Getting Started With Common Formative Assessment (Slides) Kim Bailey Getting Started With Common Formative Assessment PLC Institute Kim Bailey kbailey4learning@me.com kbailey4learning@me.com cjakicic@comcast.net

More information

Chapter 5: Sequential Circuits (LATCHES)

Chapter 5: Sequential Circuits (LATCHES) Chapter 5: Sequential Circuits (LATCHES) Latches We focuses on sequential circuits, where we add memory to the hardware that we ve already seen Our schedule will be very similar to before: We first show

More information

When Good Enough is Not an Option. Whitepaper. Why Test Management. Is Critical. For Small Companies & Startups.

When Good Enough is Not an Option. Whitepaper. Why Test Management. Is Critical. For Small Companies & Startups. Whitepaper When Good Enough is Not an Option Why Test Management Is Critical For Small Companies & Startups www.baytechservices.com What is Test Management? Test management provides an integrated repository

More information

Microsoft. Outlook 2007 Calendar Management Tools For. Jerry Maletsky Dash Designs Consulting Technology Training And Consulting

Microsoft. Outlook 2007 Calendar Management Tools For. Jerry Maletsky Dash Designs Consulting Technology Training And Consulting Microsoft 1 Outlook 2007 Calendar Management Tools For Jerry Maletsky Dash Designs Consulting Technology Training And Consulting Microsoft Outlook 2007 Calendar Management Tools For The Haas School of

More information

Adjust MS Project default settings to meet the needs of your project. Loading Analysis is an iterative process. Where are you overloaded?

Adjust MS Project default settings to meet the needs of your project. Loading Analysis is an iterative process. Where are you overloaded? Resource Loading: who and/or what needed where and when. Adjust MS Project default settings to meet the needs of your project. Loading Analysis is an iterative process. Where are you overloaded? Resource

More information

ATAC. Model-based testing in modern agile software development How to integrate it into the development process? Project number: ITEA

ATAC. Model-based testing in modern agile software development How to integrate it into the development process? Project number: ITEA ATAC Model-based testing in modern agile software development How to integrate it into the development process? Project number: ITEA 2 10037 Edited by: Matti Vuori, TUT Date: 2014/03/18 Document version

More information

Foundation Level PRACTICE EXAM Syllabus Version 2010 Practice Exam Version 2011

Foundation Level PRACTICE EXAM Syllabus Version 2010 Practice Exam Version 2011 ISTQB Certified Tester Foundation Level PRACTICE EXAM International Software Testing Qualifications Board Name: Company address: Phone : Fax : Email: Billing address: Training company: Trainer: Foundation

More information

Morten Mossige, October, 2015 Application lifecycle management with Microsoft s Team Foundation Server at ABB. ABB November 5, 2015

Morten Mossige, October, 2015 Application lifecycle management with Microsoft s Team Foundation Server at ABB. ABB November 5, 2015 Morten Mossige, October, 2015 Application lifecycle management with Microsoft s Team Foundation Server at ABB Slide 1 Agenda Microsoft s ALM Platform, Team Foundation Server Focus Areas Implement process

More information

Investing in Software Testing: The Cost of Software Quality

Investing in Software Testing: The Cost of Software Quality Abstract Investing in Software Testing: The Cost of Software Quality Testing can be considered an investment. A software organization whether an in-house IT shop, market-driven shrink-wrap software vendor,

More information

Relative Risk, Odds, and Fisher s exact test

Relative Risk, Odds, and Fisher s exact test Relative Risk, Odds, and Fisher s exact test I) Relative Risk A) Simply, relative risk is the ratio of p 1 / p 2. For instance, suppose we wanted to take another look at our Seat belt safety data from

More information

Decimals and other fractions

Decimals and other fractions Chapter 2 Decimals and other fractions How to deal with the bits and pieces When drugs come from the manufacturer they are in doses to suit most adult patients. However, many of your patients will be very

More information

Calculus with Analytic Geometry I Exam 5-Take Home Part Due: Monday, October 3, 2011; 12PM

Calculus with Analytic Geometry I Exam 5-Take Home Part Due: Monday, October 3, 2011; 12PM NAME: Calculus with Analytic Geometry I Exam 5-Take Home Part Due: Monday, October 3, 2011; 12PM INSTRUCTIONS. As usual, show work where appropriate. As usual, use equal signs properly, write in full sentences,

More information

Seven Steps to Success Machine Learning in Practice

Seven Steps to Success Machine Learning in Practice Seven Steps to Success Machine Learning in Practice Daoud Clarke Project failures in IT are all too common. The risks are higher if you are adopting a new technology that is unfamiliar to your organisation.

More information

Brillig Systems Making Projects Successful

Brillig Systems Making Projects Successful Metrics for Successful Automation Project Management Most automation engineers spend their days controlling manufacturing processes, but spend little or no time controlling their project schedule and budget.

More information

Years after 2000. US Student to Teacher Ratio 0 16.048 1 15.893 2 15.900 3 15.900 4 15.800 5 15.657 6 15.540

Years after 2000. US Student to Teacher Ratio 0 16.048 1 15.893 2 15.900 3 15.900 4 15.800 5 15.657 6 15.540 To complete this technology assignment, you should already have created a scatter plot for your data on your calculator and/or in Excel. You could do this with any two columns of data, but for demonstration

More information

2.2 Derivative as a Function

2.2 Derivative as a Function 2.2 Derivative as a Function Recall that we defined the derivative as f (a) = lim h 0 f(a + h) f(a) h But since a is really just an arbitrary number that represents an x-value, why don t we just use x

More information

Extreme Programming 1

Extreme Programming 1 Extreme Programming 1 Extreme Programming Waterfall model inspired by civil engineering Civil engineering metaphor is not perfect Software is more organic than concrete You grow the software to meet changing

More information

SharePoint Calendar: Overlay with Color Coded Categories

SharePoint Calendar: Overlay with Color Coded Categories OVERVIEW Are you looking for a way to customize your SharePoint calendar? Color-coding event categories can help users easily identify and prioritize information, and can also help SharePoint administrators

More information

Equations, Lenses and Fractions

Equations, Lenses and Fractions 46 Equations, Lenses and Fractions The study of lenses offers a good real world example of a relation with fractions we just can t avoid! Different uses of a simple lens that you may be familiar with are

More information

Integrating gsix Sigma THINKING into Scrum-Based. Darian Rashid Agile Trainer and Coach darian@agileethos.com

Integrating gsix Sigma THINKING into Scrum-Based. Darian Rashid Agile Trainer and Coach darian@agileethos.com Integrating gsix Sigma THINKING into Scrum-Based Development Environments Darian Rashid Agile Trainer and Coach darian@agileethos.com Lean Six Sigma THINKING in Software Development What is Six Sigma Thinking

More information

Pristine s Day Trading Journal...with Strategy Tester and Curve Generator

Pristine s Day Trading Journal...with Strategy Tester and Curve Generator Pristine s Day Trading Journal...with Strategy Tester and Curve Generator User Guide Important Note: Pristine s Day Trading Journal uses macros in an excel file. Macros are an embedded computer code within

More information

ScrumMasters Considered Harmful

ScrumMasters Considered Harmful ScrumMasters Considered Harmful Where Did It Go Wrong? Paul Hodgetts, Agile Logic, www.agilelogic.com In computer science and related disciplines, considered harmful is a phrase popularly used in the titles

More information

How to Plan an Agile Project in 15 Minutes. Introduction. Simba s Safaris Roars. Create Your First User Story

How to Plan an Agile Project in 15 Minutes. Introduction. Simba s Safaris Roars. Create Your First User Story How to Plan an Agile Project in 15 Minutes Introduction This tutorial will guide you through planning an agile project in fifteen minutes. You will learn how to: Create a User Story Plan a Release Kick

More information

Test Strategies in Agile Projects Q 15

Test Strategies in Agile Projects Q 15 Test Strategies in Agile Projects Q 15 1(19) Contents Agile testing Test methods, tools and planning Definition of Done User Stories Test ideas and guidelines Test charters and exploratory testing Test

More information

Why Test Automation Fails

Why Test Automation Fails Why Test Automation Fails in Theory and in Practice Jim Trentadue Enterprise Account Manager- Ranorex jtrentadue@ranorex.com Thursday, January 15, 2015 Agenda Agenda Test Automation Industry recap Test

More information

Scrum: A disciplined approach to product quality and project success.

Scrum: A disciplined approach to product quality and project success. Scrum: A disciplined approach to product quality and project success. CQAA February 23, 2011 Patricia Rotman Introductions Copyright 2011-2 Alternate Titles Considered Scrum: Just do it! Scrum: It only

More information

Aligning Correct and Realistic Performance Testing with the Agile Development Process

Aligning Correct and Realistic Performance Testing with the Agile Development Process Aligning Correct and Realistic Performance Testing with the Agile Development Process SIGIST Winter 2011 Conference Graham Parsons CEO, Reflective Solutions Overview Introduction A major risk for Agile

More information

Product Design Decisions Help Explanations Information on Rules/Procedures

Product Design Decisions Help Explanations Information on Rules/Procedures Product Design Decisions Help Explanations Information on Rules/Procedures The product design screen involves deciding on the caliber of the components to incorporate in your entry-level and multi-featured

More information

Calculation of Exponential Numbers

Calculation of Exponential Numbers Calculation of Exponential Numbers Written by: Communication Skills Corporation Edited by: The Science Learning Center Staff Calculation of Exponential Numbers is a written learning module which includes

More information

When To Work Scheduling Program

When To Work Scheduling Program When To Work Scheduling Program You should receive your login information by the second week of May. Please login and follow the instructions included in this packet. You will be scheduled based upon the

More information

Induction Problems. Tom Davis November 7, 2005

Induction Problems. Tom Davis  November 7, 2005 Induction Problems Tom Davis tomrdavis@earthlin.net http://www.geometer.org/mathcircles November 7, 2005 All of the following problems should be proved by mathematical induction. The problems are not necessarily

More information

ScrumDesk Quick Start

ScrumDesk Quick Start Quick Start 2008 2 What is ScrumDesk ScrumDesk is project management tool supporting Scrum agile project management method. ScrumDesk demo is provided as hosted application where user has ScrumDesk installed

More information

Benefits of Test Automation for Agile Testing

Benefits of Test Automation for Agile Testing Benefits of Test Automation for Agile Testing Manu GV 1, Namratha M 2, Pradeep 3 1 Technical Lead-Testing Calsoft Labs, Bangalore, India 2 Assistant Professor, BMSCE, Bangalore, India 3 Software Engineer,

More information

Why Top Traders Use The Relative Strength Comparison (RSC) & Desperately Want To Keep It A Secret!

Why Top Traders Use The Relative Strength Comparison (RSC) & Desperately Want To Keep It A Secret! Why Top Traders Use The Relative Strength Comparison (RSC) & Desperately Want To Keep It A Secret! Discover The Best Performing Sectors & The Strongest Stocks In Those Sectors www.meta-formula.com We both

More information

Getting to Done The Secret Sauce of High Performing Teams

Getting to Done The Secret Sauce of High Performing Teams Getting to Done The Secret Sauce of High Performing Teams Hosts: Lowell Lindstrom Jeff Sutherland 2011 Scrum Inc. Is Your Project Agile? 49% of Agile is Bad Agile Source: What is the Primary Reason for

More information

What Is Specific in Load Testing?

What Is Specific in Load Testing? What Is Specific in Load Testing? Testing of multi-user applications under realistic and stress loads is really the only way to ensure appropriate performance and reliability in production. Load testing

More information

Scrum Training Series, Part 3: The Sprint Planning Meeting. Michael James January 20, :13 AM

Scrum Training Series, Part 3: The Sprint Planning Meeting. Michael James January 20, :13 AM Scrum Training Series, Part 3: The Sprint Planning Meeting by Michael James mj4scrum@gmail.com January 20, 2013 12:13 AM Copyright (C) 2011 All Rights Reserved 1 INT. CLASSROOM NARRATOR Welcome to part

More information

Sample from Classroom Stories for SEAL KS1 and KS2 Theme 6 Relationships: Curriculum development resources Year 3

Sample from Classroom Stories for SEAL KS1 and KS2  Theme 6 Relationships: Curriculum development resources Year 3 Theme 6 Relationships: Curriculum development resources Year 3 Max s Bad Week Embarrassment (guilt) Happiness (proud) Story It was Saturday morning and Max was still in bed. He knew that Mum would soon

More information

Math E-301: Homework 4 Notes Due 10/5/09

Math E-301: Homework 4 Notes Due 10/5/09 Math E-301: Homework 4 Notes Due 10/5/09 The notes for homeworks will vary. For some problems, I may give one solution, but there may be other ways to approach the problem. I will try to mention these

More information

The answer is (a) describes the relation between desired consumption expenditures and the factors that determine it, like real disposable income.

The answer is (a) describes the relation between desired consumption expenditures and the factors that determine it, like real disposable income. 1 Solution Set: Chapter 22 Chapter Review: 2. The consumption function The answer is (a) describes the relation between desired consumption expenditures and the factors that determine it, like real disposable

More information

Budgeting and Measuring Your Financial Health. Assignments

Budgeting and Measuring Your Financial Health. Assignments Financial Plan Assignments Assignments While the previous chapter helped you determine where you wanted to be, this chapter helps you see where you are right now. Financial statements help you understand

More information

Managing Agile Projects in TestTrack GUIDE

Managing Agile Projects in TestTrack GUIDE Managing Agile Projects in TestTrack GUIDE Table of Contents Introduction...1 Automatic Traceability...2 Setting Up TestTrack for Agile...6 Plan Your Folder Structure... 10 Building Your Product Backlog...

More information

Lessons Learned from Tool Adoption 1

Lessons Learned from Tool Adoption 1 Lessons Learned from Tool Adoption 1 Karl E. Wiegers Process Impact www.processimpact.com Software engineers talk a lot about using tools to help them perform development, project management, and quality

More information

Quality Meets the CEO

Quality Meets the CEO Quality Meets the CEO Jeffery E. Payne jepayn@rstcorp.com Reliable Software Technologies Corporate management does not care about quality. This is the cold, hard reality of the software world. Management

More information

Alternative approaches Agile processes

Alternative approaches Agile processes Alternative approaches Agile processes Revision PA1 Q Template 6.0 Approved NN Date 1 September 2000 Sesam Stockholm, 021023 Even-André Karlsson Doc. number LD/QLS 00:0001 Agenda Q-Labs and personal background

More information

Our time-tested approach to investing is very straightforward. And we re ready to make it work for you.

Our time-tested approach to investing is very straightforward. And we re ready to make it work for you. What Works Our time-tested approach to investing is very straightforward. And we re ready to make it work for you. Three important steps. Ten effective principles. Three important steps. Ten effective

More information

WHITEPAPER. Delivering More Rigorous Testing of Software Systems to Banks and Financial Institutions. Overview

WHITEPAPER. Delivering More Rigorous Testing of Software Systems to Banks and Financial Institutions. Overview WHITEPAPER Delivering More Rigorous Testing of Software Systems to Banks and Financial Institutions Overview Today, IT is critical to all business segments. Now, most IT departments are feeling the pressure

More information

Fractions. If the top and bottom numbers of a fraction are the same then you have a whole one.

Fractions. If the top and bottom numbers of a fraction are the same then you have a whole one. What do fractions mean? Fractions Academic Skills Advice Look at the bottom of the fraction first this tells you how many pieces the shape (or number) has been cut into. Then look at the top of the fraction

More information

Details of Life Cycle Documentation. From the developers perspective: Requirements doc, functional spec, design doc, implementation doc

Details of Life Cycle Documentation. From the developers perspective: Requirements doc, functional spec, design doc, implementation doc Details of Life Cycle Documentation From the developers perspective: Requirements doc, functional spec, design doc, implementation doc A Note on Different Approaches Documentation The documents I ll describe

More information

Social network downtime in 2008 February 2009

Social network downtime in 2008 February 2009 Social network downtime in 2008 February 2009 This is a survey of the 2008 website availability (uptime) of 15 of the world s largest social networks. It includes analysis of the accumulated downtime of

More information

A Strategy for Transitioning to BGP Security

A Strategy for Transitioning to BGP Security NANOG 52 Denver, CO June 14, 2011 A Strategy for Transitioning to BGP Security Sharon Goldberg Boston University Phillipa Gill University of Toronto Michael Schapira Princeton University Princeton University

More information

Return Path Email Intelligence Report Q3 2012

Return Path Email Intelligence Report Q3 2012 Return Path Email Intelligence Report Q3 2012 HELO and Welcome Welcome to the first edition of the Return Path Email Intelligence Report. Our goal is to provide a quarterly look at email trends and performance

More information

(c) Nine thousand nineteen in standard form is. [1] In 4598, the digit is in the hundreds place, and its value is.

(c) Nine thousand nineteen in standard form is. [1] In 4598, the digit is in the hundreds place, and its value is. Assessment Test for Singapore Primary Mathematics 3A Standards Edition This test covers material taught in Primary Mathematics 3A, Standards Edition (http://www.singaporemath.com/) 1. Fill in the blanks.

More information

Credit cards Personal loans Credit lines Leasing What you should know before entering into an agreement

Credit cards Personal loans Credit lines Leasing What you should know before entering into an agreement Credit cards Personal loans Credit lines Leasing What you should know before entering into an agreement Pre-agreement information on personal financing products for private clients Taking out a loan is

More information

MEASURING SOFTWARE PROJECT HEALTH DURING QUALIFICATION PHASE IN 4 STEPS By Eric Mariacher

MEASURING SOFTWARE PROJECT HEALTH DURING QUALIFICATION PHASE IN 4 STEPS By Eric Mariacher DESIGNING CHANGE REQUEST WORKFLOWS 1 Print suggested options: Only the selected frame MEASURING SOFTWARE PROJECT HEALTH DURING QUALIFICATION PHASE IN 4 STEPS By Eric Mariacher Abstract "Modern" software

More information

The opposite of integrity is manipulation. - Thesaurus. The recent unethical behavior of major corporations like Enron and WorldCom has had a

The opposite of integrity is manipulation. - Thesaurus. The recent unethical behavior of major corporations like Enron and WorldCom has had a How to Maintain Your Integrity at Work Page 1 How to Maintain Your Integrity a t Work The opposite of integrity is manipulation. - Thesaurus The recent unethical behavior of major corporations like Enron

More information

Get Dirty with Diagnostics: Agile Performance Tuning at the Lowest Level

Get Dirty with Diagnostics: Agile Performance Tuning at the Lowest Level Get Dirty with Diagnostics: Agile Performance Tuning at the Lowest Level Adam Reincke Performance Testing Consultant My Background Adam Reincke Performance testing consultant at JDS Australia AIS and ASE

More information

Testing and Scrum. Agenda. Fall 2007 Scrum Gathering

Testing and Scrum. Agenda. Fall 2007 Scrum Gathering Testing and Scrum Fall 2007 Scrum Gathering Ralph van Roosmalen Agenda Introduction The Classical Test Approach Organization Test Documentation Test Activities Recruitment Reporting Test Automation Lessons

More information

Effective Test Management Practices

Effective Test Management Practices Effective Test Management Practices Dr. Magdy Hanna Chairman International Institute for Software Testing mhanna@testinginstitute.com http:// Principles-1 What is most frustrating in your role as a test

More information

How to Plan a Successful Load Testing Programme for today s websites

How to Plan a Successful Load Testing Programme for today s websites How to Plan a Successful Load Testing Programme for today s websites This guide introduces best practise for load testing to overcome the complexities of today s rich, dynamic websites. It includes 10

More information

PLM - Agile. Design Code Test. Sprints 1, 2, 3, 4.. Define requirements, perform system design, develop and test the system. Updated Project Plan

PLM - Agile. Design Code Test. Sprints 1, 2, 3, 4.. Define requirements, perform system design, develop and test the system. Updated Project Plan PLM - Agile Agile Development Evolved in the 1990s as a response to heavyweight methodologies. In 2001 representatives of various new methodologies met to discuss the need for lighter alternatives. The

More information

Atomate Development Process. Quick Guide

Atomate Development Process. Quick Guide Development Process Quick Guide METHODOLOGY Every project is unique You know your business inside out. You have thought and planned your ideas carefully and are keen to see it live as soon as possible.

More information

Test Plan Template (IEEE 829-1998 Format)

Test Plan Template (IEEE 829-1998 Format) Test Plan Template (IEEE 829-1998 Format) Test Plan Identifier Some type of unique company generated number to identify this test plan, its level and the level of software that it is related to. Preferably

More information

Presented by: and. Communicating. Agile. Project Status. Management. Friday, March 16, 12

Presented by: and. Communicating. Agile. Project Status. Management. Friday, March 16, 12 Presented by: and Communicating Agile Project Status to Management 1 Introduction and Agenda Bill Gaiennie, Davisbase Consulting 17 years in software development. 7 years working with software development

More information

A Summary of Error Propagation

A Summary of Error Propagation A Summary of Error Propagation Suppose you measure some quantities a, b, c,... with uncertainties δa, δb, δc,.... Now you want to calculate some other quantity Q which depends on a and b and so forth.

More information

Chapter 9 Software Evolution

Chapter 9 Software Evolution Chapter 9 Software Evolution Summary 1 Topics covered Evolution processes Change processes for software systems Program evolution dynamics Understanding software evolution Software maintenance Making changes

More information

Fuel Pump Replacement on a 2003 Sport Trac By: David Rheman. This is how I changed the fuel pump on my 2003 Ford Explorer Sport Trac 4x2.

Fuel Pump Replacement on a 2003 Sport Trac By: David Rheman. This is how I changed the fuel pump on my 2003 Ford Explorer Sport Trac 4x2. Fuel Pump Replacement on a 2003 Sport Trac By: David Rheman This is how I changed the fuel pump on my 2003 Ford Explorer Sport Trac 4x2. This is for possible information ONLY! Anything I say or do here

More information

Manual Techniques, Rules of Thumb

Manual Techniques, Rules of Thumb Seminar on Software Cost Estimation WS 2002/2003 Manual Techniques, Rules of Thumb Pascal Ziegler 1 Introduction good software measurement and estimation are important simple methods are widely used simple,

More information