Using Run Charts. Complex Care Management Toolkit Resource. Reasons to plot data over time (not pre-post!)

Transcription

1 Using Run Charts Complex Care Management Toolkit Resource Reasons to plot data over time (not pre-post!) Make performance of the process visible Determine if change is an improvement by comparing data before and after test Aggregate measures alone do not lead to predictions about future performance or insights to explain past variations Displaying data over time allows us to make informed predictions, and thus manage effectively Determine if holding the gain 2 1

2 Wait Time (Minutes) Time series graphs Line graph series of data over time connected by lines Run chart line graph with a median allows application of rules to detect signals Statistical process control chart (SPC) series of data over time with a mean center line, upper and lower control limits allows identification of common or special cause variation Jan Feb Mar Apr Change Implemented May Jun Jul Aug Sept Oct Nov Dec 3 Run Charts Display ordered sequence of data and provide running record over time Can be used for any data that are sequenced over time (trending) Require no statistics Visually illustrate progress toward goal Allow us to detect signals of improvement or degradation in a process over time Adapted from, NHS Scotland Tutorial Guide on Statistical Process Control

3 Run Chart Anatomy Individual measures Center line = median Time-Ordered Data Source: Richard Scoville, PhD 5 Run Chart Anatomy Annotations tell the story 2/5 Project Start = baseline data collected 2/6 New data system, no data available 6/6 New triage procedure Title names the measure Appropriate Scale: Data fills most of scale Source: Richard Scoville, PhD 6 3

4 % Patients Screened during Week of Visit Jan10-14 Jan17-21 Jan24-28 Jan31- Feb4 Feb7-11 Feb14-18 Feb21-25 Feb28- Mar4 Mar7-11 Mar14-18 Mar21-25 Mar28- Apr1 Apr4-8 Apr11-15 Apr18-22 Apr25-29 May2-6 May9-13 May16-20 May23-27 May30- Jun3 Jun6-10 Jun13-17 Sample Run Chart % Patients with Chlamydia Screen Go-live; process changes 100% 90% 80% New team mtg MD/staff educ 60% Target 70% 60% 50% 40% 30% 20% 10% 0% Week Slide used with permission from Medical Group 7 Understanding Variation All data demonstrate variation Sources of variation People, methods, environment, materials, measurements Methods: measuring, collecting, analyzing, interpreting Two types of variation Random / Common cause Non-random / Special cause Adapted from, NHS Scotland Tutorial Guide on Statistical Process Control

5 Random Variation Aka unassigned variation Is present in all processes reflects business as usual Does not judge whether the process is good or bad Is predictable Example: Arrival time to work 9 Non-random Variation Aka assignable variation Is assignable to a specific cause Is a special circumstance that is not part of the process not business as usual Helps you determine if your change is an improvement 10 5

6 Rules of Probability Run Charts Statistically significant evidence that there is a < 5% probability that would occur by chance Signals = Runs, trends, shifts Control Charts Statistically significant evidence that < 1% probability of occurring by chance Common cause Special cause 11 Constructing a Run Chart 1. Identify your measure for which want to display data 2. Horizontal x-axis: time units (days, weeks, months) 3. Vertical y-axis: outcome units Wait time of appointment: minutes Wait time to get appointment: days Blood pressure: mmhg LDL: mg/dl Adapted from, NHS Scotland Tutorial Guide on Statistical Process Control

7 Constructing a Run Chart (cont) 4. Scale both axes for current and future data points 5. Plot and join data points over time (ideal of 15 pts) Plot baseline data 6. Calculate median as the center line Median is the data point where half the data points are above and half are below aka the 50th percentile Sort all of your data points and take the middle value if you have an even number of data points, average the middle two Adapted from, NHS Scotland Tutorial Guide on Statistical Process Control. and Richard Scoville, PhD 13 A sidenote about the Median Why not the mean? The median is used because: It is the point at which half of the data points are expected to be above and below (this allows us to apply the rules we will discuss in a bit) Not influenced by extreme points in the data set Which points do I use to calculate the median? Ideally, calculate the median using baseline data and extend into the future When do I recalculate the median? 14 7

8 Rules for Detecting Signals of Non-Random Variation (aka, a change!) Four decision rules: 1. Astronomical point One point wildly different from all the others 2. Number of Runs 3. Shift 4. Trend 15 Rule 2: Number of Runs Run = sequence of one or more consecutive useful observations on same side of centerline Useful observation = data point that does NOT fall on centerline Total # observations number on centerline = # useful observations Too many or too few runs suggest non-random variation If number of runs < lower limit or > upper limit, indicates non-random variation 16 8

9 Total # of data points on the run chart that do not fall on the median Number of Runs: Lower and Upper Limits Lower limit for the number of runs (< than this number runs is too few ) Upper limit for the number of runs (> than this number runs is too many ) Total # of data points on the run chart that do not fall on the median Lower limit for the number of runs (< than this number runs is too few ) Upper limit for the number of runs (> than this number runs is too many ) Source: The run chart: a simple analytical tool for learning from variation in healthcare processes, Rocco J Perla, Lloyd P Provost, Sandy K Murray, BMJ Qual Saf 2011;20: There is a shift if: Rule 3: Shift 6 or more consecutive points either all above or all below the median Values that fall on the median neither add to nor break a shift. Skip values that fall on the median and continue counting. Source: The Improvement Guide by Langley et al., 2 nd edition, page

10 Rule 4: Trends There is a trend if: 5 or more consecutive points all going up or all going down. If the value of 2 or more successive points is the same, ignore one of the points when counting; like values do not make or break a trend Source: The Improvement Guide by Langley et al., 2 nd edition, page