Current Defect Density Statistics

Size: px

Start display at page:

Download "Current Defect Density Statistics"

Elijah Simmons
7 years ago
Views:

1 Current Defect Density Statistics Ann Marie Neufelder You may not reprint or copy any part of this presentation without t express and written consent from Ann Marie Neufelder

2 Actual fielded from 90+ projects spanning nearly every industry Defect clusters delivered normalized Percentile group World class Very good Good Fair Poor Ugly This data is in terms of fielded (escaped) s per 1000 lines of effective code normalized to assembler. Seven clusters are visible. A method to predict which cluster your project will fall into was developed from this data.

5 1 Percentile group World class Very good Good Fair Poor Ugly This data is in terms of fielded

3 How to determine normalized effective size Predict/count new and modified lines of code Predict/count deleted lines Multiply existing but unchanged code by 10% Entire functions deleted reduce existing size Effective size = Modified + New + Lines subtracted + (10% of existing code) Multiply effective size by conversion ratio to assembler using industry tables as summarized below Second generation (C, Fortran) 3 Object oriented (Java, C++, Ada 9x) 6 Visual Basic 10

+ New + Lines subtracted + (10% of existing code) Multiply effective size by conversion ratio to assembler using

4 Use the SoftRel Survey to predict the best cluster Appropriate Cluster determined by these things Inherent stability of existing design and code Methods and techniques use to prevent s and develop software Application type Existence of major obstacles (new technology, new environments, etc.) Existence of major opportunities (end user domain experts available to project, etc.) Inherent stability of development process Process alone will not guarantee a world class cluster! An SEI CMM level 1 organization can be world class An SEI CMM level 4 or 5 does not guarantee world class

) Existence of major opportunities (end user domain experts available to project, etc.

5 How the survey score maps to the clusters SoftRel survey score versus clusters SoftRel Survey score World class Very good Good Fair Poor Percentile group Ugly The most variation exists in the world class cluster, however, this cluster is easily predictable because of void of major obstacles and presence of major opportunities as shown next.

5 1 World class Very good Good Fair Poor Percentile group Ugly The most variation exists

6 The World Class Cluster had no major obstacles Major project obstacles versus clusters Number of major project obstacles Percentile group World class Very good Good Fair Poor Ugly Obstacles are defined specifically as new technology, new operating system, new development environment, new compiler, new target hardware

5 1 Percentile group World class Very good Good Fair Poor Ugly Obstacles are defined

7 The Ugly group had no opportunities Major project opportunities versus clusters Number of major project opportunities World class Very good Good Fair Poor Percentile group Ugly Opportunity Explicitly defined as the degree to which end user domain experts are available to the software engineers for this project

5 1 World class Very good Good Fair Poor Percentile group Ugly Opportunity

8 by system type System application type fielded testing Ratio of test to field Command and control Command, control and communications Military ground vehicle n/a Satellite Large stationery capitol equipment Small devices GPS n/a Power systems n/a No special target hardware Total/average

106 n/a Satellite 0.087 0.358 4.1 Large stationery capitol equipment 0.649 2.495 3.8 Small devices 0.

9 by software type Software application type Wireless capabilities Biometrics Domain knowledge can be acquired via public domain in short period of time Client server Real time Multi-tasking DB interfaces Mathematically intensive Web based Target HW is new or evolving Application process evolving fielded testing n/a This is the same set of data sliced a different way Ratio of test to field n/a

Application process evolving fielded 0.165 0.400 0.068 0.108 0.476 0.449 0.456 0.430 0.008 0.642 0.

10 by risk level Risk level fielded testing Ratio of test to field Safety risk (occupational, regional, national or global) Legal risks (banking, etc) Monetary risks (loss of product with monetary value) Recall risk Government regulated This is the same set of data sliced a different way.

9 Monetary risks (loss of product with monetary value) 0.476 1.535 3.2 Recall risk 0.230 3.

11 You can also predict the risk of a late delivery Percentile Ratio of Normalized Fielded Defect Density testing to fielded Stddev Ave Min Max Probability of a late delivery (%) Margin of error when delivery is late (%) World Class Very Good Good Fair Poor Ugly Probability of late delivery If your organization makes 10 releases and the probability of being late is 10% then 1 out of 10 will be late Margin of error Measured as a percentage of the original schedule prediction

180 0.366.0590 36 41 Fair 2.1.618 0.400 0.835.177 85 125 Poor 16.1 1.111 1.0357 1.224.081 100 100 Ugly.5 2.069 1.743 2.674.

12 How to predict your cluster Answer a survey based on Risks Product characteristics Application type Resources Practices, techniques and methods Process stability Determine a baseline cluster for your typical project Each project specific additional obstacle lowers the cluster while adding domain expertise raises the cluster

Process stability Determine a baseline cluster for your typical project Each

13 Lessons Learned Risks cannot be overcome by any of the following New expensive automated tools to theoretically speed up development (this will actually increase the risk level for the first time project) Wishful thinking Risks can be minimized by More granular milestones Addressing high risk items before everything else in the schedule software engineers tend to work on the low risk tasks first Design prototyping when design is a risk Requirements prototyping when end user requirements are volatile Defect prevention techniques such as formal unit testing Increasing the end user domain knowledge of the team This does not mean software experience this means application experience

software engineers tend to work on the low risk tasks first Design prototyping when design is a risk Requirements prototyping when end user requirements are volatile

Reaching CMM Levels 2 and 3 with the Rational Unified Process

Reaching CMM Levels 2 and 3 with the Rational Unified Process Rational Software White Paper TP174 Table of Contents INTRODUCTION... 1 LEVEL-2, REPEATABLE... 3 Requirements Management... 3 Software Project