STATISTICAL PROCESS CONTROL

Transcription

1 SOFTWARE ACQUISITION GOLD PRACTICE TM STATISTICAL PROCESS CONTROL FOCUS AREA: QUALITY - Measurement DESCRIPTION OF THE PRACTICE: SUMMARY DESCRIPTION Statistical Process Control (SPC) can be applied to software development processes. A process has one or more outputs, as depicted in the first figure below. These outputs, in turn, have measurable attributes. SPC is based on the idea that these attributes have two sources of variation: natural (also known as common) and assignable (also known as special) causes. If the observed variability of the attributes of a process is within the range of variability from natural causes, the process is said to be under statistical control. The practitioner of SPC tracks the variability of the process to be controlled. When that variability exceeds the range to be expected from natural causes, one then identifies and corrects assignable causes. Identify and Correct Assignable Cause Attribute Process Measure Output The key steps for implementing Statistical Process Control are: o o o o o o Identify defined processes Identify measurable attributes of the process Characterize natural variation of attributes Track process variation If the process is in control, continue to track If the process is not in control: - Identify assignable cause - Remove assignable cause - Return to Track process variation 1

2 DETAILED DESCRIPTION Statistical Process Control (SPC) can be applied to software development processes. A process has one or more outputs, as depicted in Figure 1. These outputs, in turn, have measurable attributes. SPC is based on the idea that these attributes have two sources of variation: natural (also known as common) and assignable (also known as special) causes. If the observed variability of the attributes of a process is within the range of variability from natural causes, the process is said to be under statistical control. The practitioner of SPC tracks the variability of the process to be controlled. When that variability exceeds the range to be expected from natural causes, one then identifies and corrects assignable causes. Figure 2 depicts the steps in an implementation of SPC. Identify and Correct Assignable Cause Attribute Process Measure Output Figure 1: Statistical Process Control 2

3 Identify Defined Processes Identify Measurable Attributes of Process Characterize Natural Variation of Attributes Track Variation Yes Is Process Controlled? No Identify Assignable Cause Remove Assignable Cause Figure 2: How To Perform SPC In practice, reports of SPC in software development and maintenance tend to concentrate on a few software processes. Specifically, SPC has been used to control software (formal) inspections, testing, maintenance, and personal process improvement. Control charts are the most common tools for determining whether a software process is under statistical control. A variety of types of control charts are used in SPC. Table 1, based on a survey [Radice 2000] of SPC usage in organizations attaining Level 4 or higher on the SEI CMM metric of process maturity, shows what types are most commonly used in applying SPC to software. The combination of an Upper Control Limit (UCL) and a Lower Control Limit (LCL) specify, on control charts, the variability due to natural causes. Table 2 shows the levels commonly used in setting control limits for software SPC. Table 3 shows the most common statistical techniques, other than control charts, used in software SPC. Some of these techniques are used in trial applications of SPC to explore the natural variability of processes. Some are used in techniques for eliminating assignable causes. Analysis of defects is the most common technique for eliminating assignable causes. Causal Analysis-related techniques, such as Pareto analysis, Ishikawa diagrams, the Nominal Group Technique (NGT), and brainstorming, are also frequently used for eliminating assignable causes. 3

4 Table 1: Usage of Control Charts Type of Control/Attribute Chart Percentage Xbar-mR 33.3% u-chart 23.3% Xbar 13.3% c-chart 6.7% z-chart 6.7% Not clearly stated 16.7% From Ron Radice s survey of 25 CMM Level 4 and Level 5 organizations [Radice 2000] Table 2: Location of UCL-LCL in Control Charts Location Percentage Three-sigma 16% Two-sigma 4% One-Sigma 8% Combination 16% None/Not Clear 24% From Ron Radice s survey of 25 CMM level 4 and level 5 organizations [Radice 2000] Table 3: Usage of Other Statistical Techniques Statistical Technique Percentage Run Charts 22.8% Histograms 21.1% Pareto Analysis 21.1% Scatter Diagrams 10.5% Regression Analysis 7.0% Pie Charts 3.5% Radar/Kiviat Charts 3.5% Other 10.5% From Ron Radice s survey of 25 CMM level 4 and level 5 organizations [Radice 2000] Control charts are a central technology for SPC. Figure 3 shows a sample control chart constructed from simulated data. This is an X-chart, where the value of the attribute is graphed. Control limits are graphed. In this case, the control limits are based on a priori knowledge of the distribution of the attribute when the process is under control. The control limits are at three sigma. For a normal distribution, 0.2% of samples would fall outside the limits by chance. This 4

5 control chart indicates the process is out of control. If this control chart were for real data, the next step would be to investigate the process to identify assignable causes and to correct them, thereby bringing the process under control. Upper Control Limit Attribute Lower Control Limit Sample Figure 3: A Control Chart Some have extended the focus of SPC in applying it to software processes. In manufacturing, the primary focus of control charts is to bring the process back into control. In software, the product is also a focus. When a software process exceeds the control limits, rework is typically performed on the product. In manufacturing, the cost of stopping a process is high. In software, the cost of stopping is lower, and few shutdown and startup activities are needed [Jalote and Saxena 2002]. SPC is one way of applying statistics to software engineering. Other opportunities for applying statistics exist in software engineering. Table 4 shows, by lifecycle phase, some of these uses of statistics. The National Research Council recently sponsored the Panel on Statistical Methods in Software Engineering [NRC 1996]. The panel recommended a wide range of areas for applying statistics, from visualizing test and metric data to conducting controlled experiments to demonstrate new methodologies. Phase Table 4: Some Applications of Statistics in Software Engineering Use of Statistics Requirements Design Coding Testing Specify performance goals that can be measured statistically, e.g., no more than 50 total field faults and zero critical faults with 90% confidence. Pareto analysis to identify fault-prone modules. Use of design of experiments in making design decisions empirically. Statistical control charts applied to inspections. Coverage metrics provides attributes. Design of experiments useful in creating test suites. Statistical usage testing is based on specified operational profile. Reliability models can be applied. Based on [Dalal, et. al. 1993] 5

6 Those applying SPC to industrial organizations, in general, have built process improvements on top of SPC. The focus of SPC is on removing variation caused by assignable causes. As defined here, SPC is not intended to lower process variation resulting from natural causes. Many corporations, however, have extended their SPC efforts with Six Sigma programs. Six Sigma provides continuous process improvement and attempts to reduce the natural variation in processes. Typically, Six Sigma programs use the Seven Tools of Quality (Table 5). The Shewhart Cycle (Figure 4) is a fundamental idea for continuous process improvement. Tool Check Sheet Histogram Pareto Chart Cause and Effect Diagram Scatter Diagram Control Chart Graph Table 5: The Seven Tools of Quality Example of Use To count occurrences of problems. To identify central tendencies and any skewing to one side or the other. To identify the 20% of the modules which yield 80% of the issues. For identifying assignable causes. For identifying correlation and suggesting causation. For identifying processes that are out of control. For visually displaying data, e.g., in a pie chart. Plan to improve a process, collect data, and create a schedule. Act; apply the improvement to the process throughout or abandon it if improvements not found. PDSA Cycle for Continuous Improvement Study/Check; Do the collected data show an improvement or increase your understanding of the process? Do; Implement the improvement on a small scale. Figure 4: Shewhart Cycle 6

7 CHARACTERISTICS OF IMPLEMENTATION: SUMMARY CHARACTERISTICS NO DATA CURRENTLY AVAILABLE ANTICIPATED BENEFITS OF IMPLEMENTATION: SPC is a powerful tool to optimize the amount of information needed for use in making management decisions [Eickelmann and Anant 2003]. Statistical techniques provide an understanding of the business baselines, insights for process improvements, communication of value and results of processes, and active and visible involvement. SPC provides real time analysis to establish controllable process baselines; learn, set, and dynamically improve process capabilities; and focus business on areas needing improvement. SPC moves away from opinionbased decision making [Radice 2000]. These benefits of SPC cannot be obtained immediately by all organizations. SPC requires defined processes and a discipline of following them. It requires a climate in which personnel are not punished when problems are detected. It requires management commitment [Demmy 1989]. 7

8 DETAILED CHARACTERISTICS The processes controllable by SPC are unlimited in application domain, lifecycle phase, and design methodology. Processes need to exhibit certain characteristics to be suitable for SPC (as shown in the table below). In addition, a process to which SPC is applied should be homogeneous. For example, applications of SPC to software inspections have found that inspections must often be decomposed to apply SPC effectively. Florence [1999] found, for instance, that the inspection of human machine interface specifications should be treated as a process separate from the inspection of application specifications in the project he examined. Weller [2000] found that inspections of new and revised code should be treated as separate processes in the project he examined. A trial application of SPC is useful in identifying homogeneous sub processes. Issues other than identifying defined homogeneous processes arise in implementing SPC. A second table presents such implementation issues. Criteria Of Processes Suitable for SPC Well-defined Have attributes with observable measures Repetitive Sufficiently critical to justify monitoring effort (Based on [Demmy 1989]) Define Process Choose Appropriate Measures Focus on Process Trends Calculate Control Limits Correctly Investigate and Act Provide Training SPC Implementation Issues Consistent measurements cannot be expected from software processes that are not documented and generally followed. Measures need not be exhaustive. One or two measures that provide insight into the performance of a process or activity are adequate, especially if the measures are related to the process or activity goal. Measures that can be tracked inexpensively are preferable. Control charts should be constructed so as to detect process trends, not individual nonconforming events. Straightforward formulas exist for calculating control limits and analyzing distributions. Introductory college courses in statistics usually do not address process-control techniques in detail. SPC only signals the possible existence of a problem. Without detailed investigations, as in an audit, and instituting corrective action, SPC will not provide any benefit. Problems in following the above recommendations for implementing SPC can be decreased with effective training. SPC training based on examples of software processes is to be preferred. (Based on [Card 1994]) 8

9 RELATIONSHIPS TO OTHER PRACTICES: The Figure below represents a high-level process architecture for the subject practice, depicting relationships among this practice and the nature of the influences on the practice (describing how other practices might relate to this practice). These relationship statements are based on definitions of specific best practices found in the literature and the notion that the successful implementation of practices may influence (or are influenced by) the ability to successfully implement other practices. A brief description of these influences is included in the table below. Determine which attributes, at what levels, should be controlled Assess progress towards process control Define whether environment is appropriate for process control Statistical Process Control (SPC) Communicate progress towards controlling processes Provide data upon Improve testing which decisions efficiency and can be based effectiveness Process Architecture for the "Statistical Process Control" Gold Practice TM Summary of Relationship Factors INPUTS TO THE PRACTICE Determine which attributes, at Statistical Process Control can only be effective if the most critical what levels, should be processes are identified and addressed using the technique. controlled Practices that help establish clear goals and decision points, and are based on meaningful metrics and attributes based on specific program or technical goals, stand to gain the most payback from using SPC. SPC techniques need not be restricted to the present, i.e., planning for the insertion of new technology later in the life cycle should also plan for the use of SPC to ensure that processes are controlled and reliability of the resulting software artifacts is optimized. Define whether environment is Practices such as Performance-Based Specifications and appropriate for process control Commercial Specifications/Open System can imply the generation and collection of data. Such data may serve as appropriate input to SPC techniques such as control charts. Therefore, an environment that is data-rich provides an excellent opportunity to leverage the benefit statistical process control techniques. Provide data upon which decisions can be based An initial step in applying SPC is often to discover controllable and homogeneous processes. Past performance data can be used for this purpose. Formal inspection processes and processes for leveraging COTS/NDI explicitly call for metrics to be collected. These metrics can be used as the basis for SPC. 9

10 Assess progress towards process control Communicate progress towards controlling processes Improve Testing Efficiency and Effectiveness Summary of Relationship Factors (continued) OUTPUTS FROM THE PRACTICE SPC can be used not only to control processes, but also to determine if quantitative requirements on software processes are being met. The results of SPC, then, provide valuable data and information that can be used to manage progress towards achievement of software requirements. Part of this ability to manage progress is supported by the quantitative progress measurements that are inherent in the SPC process, primarily in the form of defect tracking and correction against specific, quantitative quality targets. Management go/no-go decisions can be based on whether development processes are under control. SPC presents graphical displays supporting such decisions. The number and types of available graphical formats that can be used as part of the SPC process provide accessible visibility into progress being made to all program stakeholders. Demonstration-based reviews provide an excellent vehicle for communicating the progress being made in controlling processes through the use of SPC. By providing control over software development processes, SPC will result in more predictable and more reliable software. Rigorous testing that is guided by specifications and supported by well-documented and accurate operational and usage-based models will be much more effective under the controlled processes resulting from SPC. 10

11 RESOURCES: Websites 0 American Society for Quality 0 Software Division of the American Society for Quality 0 Six Sigma Forum established by the American Society for Quality (ASQ) 0 A Web portal devoted to Six Sigma programs. 0 Quality Digest. A magazine. 0 Software Process Resource Collection 0 Practical Software and Systems Measurement. Sponsored by the U.S. Department of Defense and the U.S. Army. 0 A page at Clemson University for Continuous Quality Improvement, including software and tutorials. 0 An online textbook in statistics 0 Virtual Laboratory in Probability and Statistics (Kyle Siegrist s tutorial) Tools and Methods Tools for plotting control charts, plotting other graphs, and calculating various statistics support SPC. Common spreadsheets, such as Excel, provide much of the needed capabilities, but a number of vendors have produced tools customized for SPC. Some are listed below: Measurement Control Chart Toolkit (MCCT). A toolkit for SPC available from the United Nations Industrial Development Organization. A page at Clemson University for Continuous Quality Improvement, including software and tutorials. A company providing SPC software, training, and consulting A windows-based SPC and Process Improvement software tool sold by Qualitrain Professional Services Incorporated. A software package provided by the company To The Point. Pankaj Jalote s Software Process Design and Analysis page. Includes an applet for computing control limits in control charts. Experts/Contact Points 0 Donald Reifer, a consultant. 0 R. S. Pressman and Associates 0 Software Quality Engineering. A private company providing training and consulting. 0 A company providing SPC software, training, and consulting. 11

12 Training Opportunities: Software Engineering Institute (SEI) course on SPC. A company providing SPC software, training, and consulting Software Quality Engineering. A private company providing training and consulting. Bibliography: [Bertolino, et. al. 2002] A. Bertolino, E. Marchetti, R. Mirandola, G. Lombardi, and E. Peciola, Experience of Applying Statistical Control Techniques to the Function Test Phase of a Large Telecommunications System, IEE Proceedings Software, V. 149, N. 4, August 2002, pp [Card 1994] D. Card, Statistical Process Control for Software? IEEE Software, V. 11, No. 3, May 1994, pp [Cobb and Mills 1990] [Dalal, et. al. 1993] [De Lucia, et. al. 2002] [Demmy 1989] [Florac and Carleton 1999] [Florac, et. al. 2000] [Florence 1999] [French 1995] [Gardiner and Montgomery 1987] [Hayes 1998] R. H. Cobb and H. D. Mills, Engineering Software under Statistical Quality Control, IEEE Software, V. 7, No. 6, November 1990, pp S. R. Dalal, J. R. Horgan, J. R. Kettenring, Reliable Software and Communication: Software Quality, Reliability, and Safety, Proceedings of the 15th International Conference on Software Engineering, May 1993 A. De Lucia, A. Pannella, E. Pompella, S. Stefanucci, Empirical Analysis of Massive Maintenance Processes, Proceedings of the Sixth European Conference on Software Maintenance and Reengineering (CSMR 02), 2002 W. S. Demmy, Statistical Process Control in Software Quality Assurance, Proceedings of the IEEE National Aerospace and Electronics Conference (NAECON 1989), May 1989, pp W. A. Florac and A. D. Carleton, Measuring the Software Process: Statistical Process Control for Software Process Improvement. Addison-Wesley, 1999 W. Florac, A. D. Carleton, and J. R. Barnard, Statistical Process Control: Analyzing a Space Shuttle Onboard Software Process, IEEE Software, V. 17, no. 4, July/August 2000, pp A. Florence, SEI CMM Level 4 Quantitative Analysis, Twenty-Fourth Annual Software Engineering Workshop V. A. French, Applying Software Engineering and Process Improvement to Legacy Defence System Maintenance: An Experience Report, Proceedings of the International Conference on Software Maintenance, October 1995 J. S. Gardiner and D. C. Montgomery, Using Statistical Control Charts for Software Quality Control, Quality and Reliability Engineering International, V. 3, 1987, pp W. Hayes, Using a Personal Software Process to Improve Performance, Proceedings of the Fifth International Software Metrics Symposium, November 1998 [Ishikawa 1982] K. Ishikawa, Guide to Quality Control, Unipub/Quality Resources, 1982 [Jacob and Pillai 2003] A. L. Jacob and S. K. Pillai, Statistical Process Control to Improve Coding and Code Review, IEEE Software, V. 20, No. 3, May/June 2003, pp

13 [Jalote and Saxena 2002] [NRC 1996] [Paulk, et. al. 1995] [Prowell, et. al. 1999] [Radice 2000] [Turner 2002] [Weller 2000] P. Jalote and A. Saxena, Optimum Control Limits for Employing Statistical Process Control in Software Process, IEEE Transactions on Software Engineering, V. 28, N. 12, December 2002, pp Panel on Statistical Methods in Software Engineering of the National Research Council, Statistical Software Engineering, National Academy Press, 1996 M. C. Paulk et. al., The Capability Maturity Model: Guidelines for Improving the Software Process, Addison-Wesley, S. J. Prowell, C. J. Trammell, R. C. Linger, and J. H. Poore, Cleanroom Software Engineering: Technology and Process. Addison-Wesley, 1999 R. Radice, Statistical Process Control in Level 4 and 5 Organizations Worldwide, Proceedings of the 12th Annual Software Technology Conference, (Also at R. Turner, Implementation of Best Practices in US Department of Defense Software-Intensive Systems, Dissertation, George Washington University, January E. F. Weller, Practical Applications of Statistical Process Control, IEEE Software, V. 17, no. 3, May/June 2000, pp

14 APPENDICES DEFINITIONS: An application of statistics to controlling industrial processes, including processes in software development and maintenance. Statistical Process Control (SPC) is used to identify and remove variations in processes that exceed the variation to be expected from natural causes. The purpose of process control is to detect any abnormality in the process. - [Ishikawa 1982] 14

15 SOURCES (Origins of the Practice): Walter Shewhart developed Statistical Process Control (SPC) in the 1920s. Shewhart sought methods applying statistics to industrial practice. Acceptance testing and SPC grew out of this work. Shewhart proposed the use of control charts, a core technique for SPC, in a historic internal memorandum of 16 May 1924 at Bell Telephone Laboratories. For a long time, SPC was most widely adopted in Japan, not the United States. Shewhart mentored W. Edwards Deming, and Deming went on to introduce quality technologies into Japanese industry. The Guide to Quality Control Ishikawa [1982], first published in 1968 in Japanese, is a guide to quality control techniques that became prevalent in Japan after World War II. Corporations in the United States began adopting quality technology, including SPC, more widely in the 1980s. Recently, many United States corporations have instituted Six Sigma programs. These programs, through continual process improvement, attempt to reduce the natural variation in industrial processes. Some began applying SPC techniques to software in the 1980s. Gardiner and Montgomery [1987] report an example. Software inspections seem to provide the processes that are most commonly monitored with SPC in software. Some recently proposed software process models include opportunities for SPC. The spiral lifecycle provides a natural time for tuning software processes, namely before the start of the development of each increment. SPC yields analyzed data that managers can use in selecting processes to tune. Cleanroom software engineering combines incremental development and software inspections with other technologies, such as reliability modeling. The Software Engineering Institute (SEI) Capability Maturity Model (CMM) mandates that SPC be used in Level 4 organizations. 15

16 RECOMMENDING SOURCES: Capability Maturity Model, Version 1.1 SPC is described as part of the Key Process Area, Quantitative Process Management, in the CMM. This Key Process Area must be implemented to achieve Level 4, Managed, in the CMM. The purpose of Quantitative Process Management is to control the process performance of the software process quantitatively. Software process performance represents the actual results achieved from following a software process. Quantitative Process Management involves establishing goals for the performance of the project s defined software process, which is described in the Integrated Software Management key process area, taking measurements of the process performance, analyzing these measurements, and making adjustments to maintain process performance within acceptable limits. When the process is stabilized within acceptable limits, the project s defined software process, the associated measurements, and the acceptable limits for the measurements are established as a baseline and used to control process performance quantitatively. [Paulk, et. al. 1995] This process area is renamed Organizational Process Performance at Maturity Level 4 in the CMMI, Version 1.1. Turner, R., Implementation of Best Practices in US Department of Defense Software- Intensive Systems, Dissertation, George Washington University, January, 2002 Turner surveyed a sample of 35 software experts from academia, industry, and government participating in various software-intensive working groups. The survey partitioned 32 best practices into four focus areas. Statistical Process Control, a best practice identified by Donald Reifer, was grouped into the Measurement focus area. The best practices were ranked based on the 23 survey respondents individual rankings; rating of effectiveness; and groupings into the eight best practices, eight worst practices, or otherwise. SPC was ranked thirtieth out of the 32 best practices. Cleanroom Software Engineering Cleanroom alludes to a technique used in semiconductor fabrication to prevent defects, and is a methodology for software development that integrates several software engineering technologies: 0 Incremental development 0 Formal specifications 0 Stepwise refinement 0 Structured programming 0 Formal verifications 0 Formal reviews 0 Statistical testing 0 Certification (reliability) measurement Incremental development supports SPC: Incremental development is based on the engineering principle of controlled iteration in product development. Rather than a single pass through the development process, incremental development involves a series of smaller, cumulative development passes. Each pass (increment) is cumulative, involving all work in previous increments plus some new work 16

17 In addition to the benefits of intellectual control, customer feedback, and risk management, incremental development enables the project team to employ statistical process control. Product quality is measured at the end of each increment and is compared with the team s quality goals. The deviation between actual results and goals is used to determine whether the project is under control. A minor deviation confirms that the project is on track, whereas an unacceptable deviation occasions a careful performance review. If problems are identified, the team can make process changes to improve performance in the next increment. [Prowell, et. al., 1999] 17

18 GLOSSARY c-chart CE Diagram CMM Control Chart Fishbone Diagram Histogram Ishikawa Diagram Kiviat Chart LCL NGT Pareto Analysis p Chart Pie Chart Radar Chart Regression Analysis Run-Chart Scatter Diagram SEI SPC A control chart displaying counts per item where the range of the counts is fixed. Contrast with a u-chart. Cause and Effect diagram. A diagram, developed by Kaoru Ishikawa, that resembles a fish skeleton. The diagram shows the causes and sub causes leading to an effect. Capability Maturity Model A graph with limit lines that is used to detect changes in the process from the graphed data is collected. [Ishikawa 1982] See CE Diagram. A graph showing for defining intervals, the number of values in a sample or the percentage in that interval. A visual display of a probability distribution. See CE Diagram. A chart in the shape of a circle with evenly spaced radii, where each radius represents the value of a (non-negative attribute). Attributes are often plotted with respect to user-defined thresholds. Lower Control Limit. Used in constructing a control chart. Nominal Group Technique The use of a bar chart that displays by frequency, in descending order, the most important defects. Proper use of this chart will have the cumulative percentage on a second y-axis (to the right of the chart). If the Pareto principle is evident, about 20% of the categories on the far left will have about 80% of the impact on the problem. A control chart in which a fraction formed from the ratio of discrete variables (e.g., fraction of items defective) is plotted against time. A graph in which the percentage in one of a small number of categories is displayed as an area in a circle. See Kiviat Chart A statistical technique for determining the best mathematical expression describing the functional relationship between one response and one or more independent variables. (From A performance measure of a process over a specified period of time used to identify trends or patterns. A plot in which two attributes of the data are plotted, one on the abscissa and the other on the ordinate. Useful for detecting a relationship between the two attributes. Software Engineering Institute Statistical Process Control 18

19 u-chart UCL X-Chart Xbar Chart Xbar-mR Chart XmR Chart z-chart A control chart displaying counts per item (e.g., total defects per thousand source lines of code) where the range of the counts is not fixed. Contrast with the c-chart. Upper Control Limit. Used in constructing a control chart. Individual control chart A control chart in which the average for subgroups of data is plotted by subgroup. A control chart in which both the average and the range for subgroups of data are plotted by subgroup. Individual and Moving Range control chart. A control chart in which the plotted data has been transformed to be from a normal distribution if the process is under control. 19

20 CASE STUDIES FROM THE LITERATURE: The table below summarizes recent experiences of a number of organizations with SPC. SPC Technique Examples of the Use of SPC Application Process/Attributes Discussion/Results Reference Control chart Military software unit maintaining almost 380,000 source lines of code Defects per Person- Day and Defects per Thousand Lines of Code. SPC techniques introduced along with software engineering project management techniques, inspections, and metrics. Process improvement resulted in 20% more code being produced than on previous version, rework in Integration and Test dropping from 4.7 to 0.09 person-years. [French 1995] X Control Chart Process automation and consumer electronics projects in C++, ranging from 150 to 400 Function Points Defects per Thousand Lines of Code for code review and testing. Lines of Code per Hour for inspection preparation. Illustrates how using control charts identified out-ofcontrol processes. Corrective actions were implemented to remove assignable causes. [Jacob and Pillai 2003] XmR Control Chart GCOS 8 (an operating system) Inspection review rate and defect-detection effectiveness of software inspections. Identified sub processes (inspections of new code and revised code). Demonstrated inspections were under control. Estimated defects remaining in the code. [Weller 2000] XmR Control Chart and u-chart Space Shuttle Onboard Software Inspection review rate, defect-detected in inspections and test. Trial study of SPC. Identified inspection sub processes and natural variation of these processes. [Florac, et. al. 2000] Z-Chart A sample of 298 engineers training in the Software Engineering Institute s Personal Software Process (PSP) Defects discovered in design reviews. Process control charts presented in the context of metrics collected by engineers going through PSP training. Demonstrations of the effectiveness of the PSP based on size estimation accuracy, effort estimation accuracy, defects per source line remove early in [Hayes 1998] 20

21 the lifecycle, and productivity. Confidence intervals based on statistical models of failures Ericsson Lab, Italy Effectiveness in failure detection of function testing. Pilot study. Concluded process control for function test can be based on examined models. [Bertolino, et. al. 2002] Regression model, Outlier analysis, Box plot Large Y2K maintenance project. Part of a major international software enterprise in Caserta, Italy. Size and productivity metrics during maintenance. Study analyzed correlation, characterized patterns of variations, developed regression model to predict impact of maintenance. Concluded that maintenance process is predictable, repeatable, and suitable for SPC. [De Lucia, et. al. 2002] 21