Use and interpretation of statistical quality control charts

Transcription

1 International Journal for Quality in Health Care 1998; Volume 10, Number I: pp Methodology matters VIII 'Methodology Matters' is a series of intermittently appearing articles on methodology. Suggestions from readers of additional topics they would like to see covered in the series are welcome. Use and interpretation of statistical quality control charts JAMES C BENNEYAN Department of Mechanical, Industrial and Manufacturing Engineering, 334 Snell Engineering Center, Northeastern University, Boston, MA 02115, USA Originally developed at Bell Laboratories by Dr Walter Shewhart [1] in 1924 specifically to help detect statistical changes in process quality, control charts have since become one of several primary tools of quality control and process improvement. Quality control charts are chronological graphs of process data that, although based in statistical theory, are easy for practitioners to use and interpret. These charts also can help users to develop an understanding of the performance of a process and to evaluate any benefits or consequences of process interventions, complementing traditional methods by providing additional longitudinal information that otherwise might not be detected [2]. This article provides a brief introduction to the use of statistical quality control charts for analyzing, monitoring, and improving health care processes. After an overview of key concepts, several examples illustrate control chart use and interpretation. The article concludes with some common pitfalls to avoid and references for further exploration. Readers unfamiliar with the topic should also read introductory materials on the principles of quality management and the philosophies of the late quality pioneer, Dr W. E. Deming [3]. Key concepts: natural variability and statistical control Most processes exhibit some variability, all of which can be classified into one of two categories: 'natural' or 'unnatural'. The natural variability of a process is the systemic variation inherent as a regular part of the process. Some examples of 'common causes' of natural variability might include the time of day, hospital census and case-mix, weight and physical condition of patients, other patient-to-patient differences, and varying behaviors and demographics. Because they are caused by regular sources within the process or its environment, data exhibiting natural variation occur in predictable and relatively common frequencies. Conversely, outcomes or in-process observations that have very small probabilities of occurrence, assuming only natural variability, usually represent deviations from the regular process. Such events suggest that the process fundamentally has changed, for better or worse, due to atypical unnatural variability that should be traced to root assignable causes for management intervention. Examples of 'special causes' of unnatural variability might include changes in clinical procedures, skill degradation, equipment failure, new staff, and changes in population demographics, the rate of disease, or a patient's physiology. Finally, the term 'statistical control' refers to the stability and predictability of a process over time and to the type of variability that exists. A process that is completely stable over time exhibits only natural variability, with its regular random behavior remaining unchanged, and is referred to as being in a state of statistical control. Conversely, a process that changes from its norm will exhibit unnatural variability and is referred to as being out of statistical control. Note that it is usually quite difficult to determine intuitively, without the aid of control charts, which type of variation exists and therefore whether direct intervention ultimately would be beneficial or harmful to process outcomes. Control chart format, use and interpretation Figure 1 shows the general format of a statistical control chart. Process data are collected at certain intervals over time and formed into rational time-ordered subgroups (such as Address correspondence to James C Benneyan. Tel: (+1) ; Fax: (+1) ; benneyan@coe.neu.edu 69

2 Methodology Matters: J. C. Benneyan Upper Control Limit (UCL) Subgroup Statistic (e.g., average, standard deviation, rate, number, proportion) Center Line (CL) Lower Control Limit (LCL) Figure I General format of a quality control chart I I I I 1 I I I 1 I I I I I Subgroup Number by day or week). Some value of interest is calculated from each subgroup soon after being collected, plotted in chronological sequence on the chart, and then evaluated for typical versus statistically irregular behavior. At least 25 subgroups are recommended to start a control chart. Along with these subgroup values, three horizontal lines are also calculated and plotted on the chart. These are called the centerline, the upper control limit, and the lower control limit. The centerline and control limits help define the incontrol central tendency and natural variability of the process, respectively. The centerline is almost always set equal to the arithmetic mean or expected value of the plotted statistic, so that approximately half of the subgroup values will fall on each side. The control limits are then usually set equal to the centerline plus and minus three theoretical standard deviations of the plotted values. A process is considered to be in statistical control if all of the plotted subgroup values are between the control limits over a sufficient span of time. There also should be no evidence of unusual behavior between the limits, such as trends, cycles, visibly evident changes, and other forms of non-random variation such as those defined by the additional rules listed in Table 1. Note that unequal subgroup sizes (such as a differing number of medications filled each week) result in varying control limits, although chart interpretation is basically the same. In this fashion, control charts are valuable for several purposes throughout the process improvement cycle. These uses are described and illustrated in greater detail elsewhere [1,3-5] and include: (1) testing for and establishing a state of statistical control; (2) monitoring an in-control process for changes in process and outcome quality; (3) identifying, testing, and verifying process improvement opportunities. The long-term objectives are to tighten the control limits by reducing process variation and to move the centerline so that Table I Criteria for not being in a state of statistical control Control chart out-of-control signals Any single subgroup value outside either control limit Eight consecutive subgroups on one particular side of the centerline (CL) Twelve of fourteen consecutive subgroups on one particular side of the centerline Three consecutive subgroups beyond 2 standard deviations on a particular side of the CL Five consecutive subgroups beyond 1 standard deviation on a particular side of the CL Thirteen consecutive subgroups within +1 standard deviation (on both sides) of the CL Six consecutive subgroups with either an increasing or decreasing trend Cyclical or periodic behavior the process is operating closer to some target value. Several health care examples of these uses can be found in reference [5]. Common types of statistical control charts Several common types of control charts exist, each constructed using slightly different formulas and each being appropriate for different types of data. The appropriate equations for a given chart can be found in any good quality control text [4,6,7]. Determining which specific type of control chart should be used is based on identifying the type of continuous or discrete data to be plotted. For example, three of the most common statistical distributions are the normal, binomial, and Poisson. While numerous other types are 70

3 Use of quality control charts? 50 I < 35 -r c Q 15-: < H I h I H I H -I Subgroup Number Figure 2 A'and S control charts of laboratory beta-sub turnaround-time possible, usually one of the following can be reasonably appropriate for describing many processes. For continuous values that are normally distributed (i.e. 'bell-shaped'), both an X (pronounced 'x bar 3 ) and S control chart should be used together. These two charts monitor the process mean and standard deviation, respectively. Examples of this type of data might include recovery duration, times to complete various activities, patient weights, lung capacity, and intake and output. For example, Figure 2 illustrates X and S control charts for the laboratory time to complete beta-sub pregnancy tests. The process is considered to be out of control if either chart exhibits out-of-control signals. In this case, note that both charts have out-of-control values above the upper control limits. The root causes of these outcomes should be identified and removed in order to achieve a consistent and in-control process. Also note that the chart later has a run of 10 consecutive values beneath the centerline, signaling (via the second criterion in Table 1) a now improved process. For discrete data, the two most common pairs of control charts are called np and p charts (for binomial distributions) and c and u charts (for Poisson distributions). Unlike the example given above, once the appropriate pair is identified only one of the two charts should be used alone, rather than both together (due to statistical reasons beyond the current scope of this article). The choice within a pair is largely preferential when all subgroups are the same size, whereas p and u charts are preferred over np and c charts, respectively, when dealing with unequal subgroup sizes. Each of these four charts is appropriate in the following situations. Given a certain number of dichotomous events, np and p charts are used for monitoring the number and fraction of these, respectively, that have a particular characteristic of interest (e.g. a billing error, a late patient arrival, or a Cesarean section birth), where the probability of occurrence is reasonably the same for each event. For example, Figure 3 illustrates a p chart of the fraction of medication errors per week (with non-constant control limits due to a varying number of medications per week). Note that these data reveal a gradually increasing trend in the error rate (criterion seven), to the point where two subgroup values fall above the upper control limit. Again, identifying and removing the root causes of this unnatural variation is the first step in stabilizing and improving this process. Alternately, Poisson-based c and u charts often are appropriate for count data for which no theoretical maximum exists, such as the number of some particular type of occurrence per examined area, volume, or time period. Examples can include the number of arrivals to an emergency room per shift, telephone calls per hour, leukocytes per CBC, or skin melanomas per patient. For example, Figure 4 illustrates a u type of control chart for the number of patient falls per month (with non-constant control limits now due to a varying monthly census). Again, note the clear signals of atypical outof-control events (corresponding to the months 10/91, 1/93, and 8/93) in an otherwise stable and consistent process. Further examples of each of the above types of control charts can be found in the references cited. While these types are the most commonly used, they are based on certain assumptions that are usually but not always true. For example, g and h control charts [8] are appropriate in situations where the underlying count distribution is geometric, rather than Poisson. These charts also are useful in situations involving low rates by monitoring the number of dichotomous events between outcomes, such as the number of surgeries between postoperative infections [9]. As an example, Figure 5 illustrates a g chart of die number of days between Clostridium difficile nosocomial infections. As can be seen via multiple violations of the second and third criteria, for statistical control an increase in the infection rate appears to have occurred in the vicinity of subgroup 34, with 18 of the next 22 subgroup values including a run of nine consecutive values beneath the centerline. Some final cautions While ease of use certainly is valuable, the effects of some common oversimplifications and misunderstandings about the use of control charts can be to decrease significandy either specificity or the ability to detect true process changes. For example, periodic statistical errors include basing control limits on the overall standard deviation of the data instead of on the recommended formulas, not using three standard deviation limits in almost all cases, and not accounting for significantly autocorrelated or multivariate processes. Other pitfalls include not using at least rational subgroups, not taking sufficiendy large subgroups if process data are not reasonably bell-shaped, and failing to design charts with 71

4 Methodology Matters: J. C. Benneyan o c &2 1 ' t. Q. F*^» / \ / \ w-\ j \ i \ / \ I H^ J it "T" i V i V i i i V i 'i i i '» i i V i i i i i V i i V i i \ i i i ' 1 i I i i \\ Z0 25 Week Number 30 Figure 3 p control chart of fraction of medication errors per week M > M I I I I t I I I I t I I I I I M I I I I I I I 1 I > I t I I I I I 1/91 4/91 7/91 10/91 1/92 4/92 7/92 10/92 1/93 4/93 7/93 10/93 1/94 4/94 Month Figure 4» control chart of patient falls per month 6- Figure 5 control chart of nosocomial infection rate i u i i l Occurrence of Infection desirable statistical properties (i.e. sensitivity and specificity). Common errors in control chart selection include failing to identify an appropriate chart, using an X chart alone without an J" chart, overuse of the so-called 'individuals' chart (especially in cases for which any of the above charts are more appropriate), and using standard control charts when combining data from non-homogenous processes. Related implementation concerns include reacting to natural variability 72 J

5 Use of quality control charts ('process tampering'), using control charts primarily to 'hold the gains', and over-reliance on software. Although not the current focus, additional information on these topics, related quality control methods, and more advanced concepts can be found in some of the listed references. References 1. Shewhart W. A. The Economic Control of Quality of Manufactured Product. New York: D. Van Nostand and Co., BenneyanJ. C, Kaminsky F. C. Another view on how to measure health care quality. Qual. Progress 1995; 28: Montgomery D. C. Introduction to Statistical Quality Control, 3rd edn. New York: Wiley, Benneyan J. C. Statistical quality control methods in infection control. Infect. Contr. Hosp. Epidemiol, (in press). 6. Gitlow H., Gitlow S., Oppenheim A., Oppenheim R. Tools and Methods for the Improvement ofquality. Homewood, IL: Irwin, Banks J. Principles of Quality Control. New York: Wiley, Benneyan J. C, Kaminsky F. C. Modeling discrete data in SPC: the g and h control charts. Am. Soc. Qual. Contr. (Ann. Qual. Congr. Trans.) 1994; BenneyanJ. C. Design of statistical^ control charts for nosocomial infection and other alternatives. In International Applied Statistics in Medicine Conference Proceedings (in press). 3. Deming W. E. Quality, Productivity, and Competitive Position. Cambridge, MA: Massachusetts Institute of Technology Center for Advanced Engineering Studies, Received in revised form 25 July