Measuring return on investment of improvement activities. User guide

Transcription

1 Measuring return on investment of improvement activities User guide

2 If you would like to receive this publication in an accessible format, please phone using the National Relay Service if required, or This document is available as a PDF on the internet at: Copyright, State of Victoria, Department of Health, 2013 This publication is copyright, no part may be reproduced by any process except in accordance with the provisions of the Copyright Act Authorised and published by Victorian Government, 50 Lonsdale Street, Melbourne. March 2013 ( )

3 Measuring return on investment of improvement activities User guide The Commission for Hospital Improvement: inspiring and enabling leadership to deliver real and measurable healthcare improvements

4

5 Contents Section 1: Introduction 1 Background 1 Role of measurement in redesign and improvement 1 Purpose of the document 1 Section 2: Measuring the benefit of redesign and improvement 3 Key terms and defi nitions 4 Measuring fi nancial benefi t 5 Before starting 6 Section 3: Data requirements 7 Section 4: Calculating return on investment 10 Step 1. Select project type 10 Step 2: Populate the tool 11 Step 3: Interpret the results 12 Section 5: Limitations of the model 14 Section 6: Using the ROI prospectively 15 Prospective ROI template 15 Appendix: Checklist for using the ROI 17

6

7 Section 1: Introduction Background The Redesigning Hospital Care Program (RHCP) is an initiative focusing on improving the delivery of health services and, in turn, patient outcomes. There are currently 32 Victorian health services participating in the program. The RHCP, as of August 2012, is part of the Commission for Hospital Improvement. The two main objectives of the program are to: achieve measurable improvement in health service capability for redesign measurably improve health delivery processes and outcomes across the system. The program provides a systematic and integrated approach to improvement, assisting health services to tackle local access, effi ciency and service-quality challenges. Role of measurement in redesign and improvement Measurement is an essential component of redesign and improvement. Measurement is used in multiple ways including: identifying improvement opportunities (for example, assessing against performance targets) understanding a process or a set of processes (for example, using the Tool for organisations to reveal constraints in health (TORCH) to gain a high-level understanding of constraints to fl ow) monitoring implementation to determine if an intervention is an improvement (for example, it measures a process and also a check measure to ensure the intervention is not having a negative impact on another part of the system) monitoring changes to ensure an intervention is sustained as part of routine practice and the improvement is built upon (for example, using visual boards such as Knowing How We are Doing boards) measuring the spread of improvement assessing take-up of change ideas for improvement (for example, the number of wards using patient rounding as a standard) prioritising potential improvement activities for example, assessing the potential impact of a project such as on the service s fi nancial position, patient safety and access to determine where to invest. Purpose of the document This document has been developed as a guide to assist health services to understand the ROI approach and its application as part of the redesign approach. In addition it provides a step-by-step guide on using the ROI tool and how to interpret the results. This document will: provide an overview of ROI as one measurement component provide an overview of the ROI tool structure provide a defi nition of key terms and data requirements document the four stages involved in completing the ROI tool provide details on completing each stage provide information on how to interpret the ROI output. 1

8 Return on investment package The ROI package includes: a user guide ROI tool. The ROI tool The ROI tool is an Excel spreadsheet that requires users to input data into predetermined fi elds to then generate the ROI. The spreadsheet will be made available to you alongside this document. A graphing template is included as part of the Excel tool. 2

9 Section 2: Measuring the benefit of redesign and improvement Measuring the benefi ts or potential benefi ts of improvement activities in health services is multifaceted. The benefi ts are both non-fi nancial and fi nancial. Both are important in understanding and assessing the success of a redesign project. The RHCP has a well-developed approach to measuring a range of non-fi nancial benefi ts. These benefi ts typically include: increased patient throughput reduced wait time capacity and demand improved patient experience and staff satisfaction reduced rates of harm (falls, pressure ulcers and medication errors as examples). An often challenging measure to determine is the fi nancial benefi t or return on investment (ROI). To this end the RHCP has partnered with Bevington Group to develop an ROI model specifi cally designed for health. The model has successfully been applied to 45 completed redesign projects and has shown that the redesign methodology is an effective means to implement improvement, through demonstrating a positive ROI. The ROI tool is currently designed to examine projects focusing on emergency department, ward or theatre improvements. These three areas were selected because they have been a major focus of improvement using the redesign methodology. The tool is designed to be used in the following way: Prospectively (prior to the project selection or commencement) to identify the potential fi nancial benefi ts of a project, and assist in selecting or prioritising projects. This may form part of the business case development. Retrospectively (at the completion of a project) as part of evaluating the success of the project to determine the fi nancial return. This may form part of the evaluation phase of the redesign approach. The model defi nes the ROI as released value. Released value is the additional capacity created to (potentially) enable more patients to be treated for the same (or similar) cost described in dollar terms. It does not include savings achieved through incremental cost reduction or a bottom-line saving. Furthermore, the ROI can be used throughout the project cycle to assist in decision making about whether to continue, modify, scale up or terminate the project. It is envisaged that over time the ROI tool will be further developed to address other focus areas for improvement. 3

10 Key terms and definitions Term Definition General Gross released value Net released value Released value Return on investment The total released value as a result of the project. The released value as a result of the project, less the cost of the project. The value released by the project, measured in additional bed days/hours or theatre capacity. The value is released, allowing more patients to be treated with the same amount of resource. The value cannot be easily mapped to budget or cash fl ow, particularly if funding caps have been reached. A measurement used to evaluate the (fi nancial) value or benefi t of an investment (project) or evaluate the difference, in terms of fi nancial value or benefi t, between a number of investments. Emergency department specific Additional patients per year Cubicle hours released per year Number of cubicles The potential additional number of patients who could be seen per year. Calculated by dividing the additional cubicle hours released by the new length of stay. The potential additional cubicle hours released as a result of the project. Calculated by multiplying the reduction in length of stay by the total number of presentations annually. The number of cubicles affected by the project. Theatre specific Theatre hours released per day Theatre hours released per year Theatre utilisation The potential additional theatre hours released per day as a result of the project. Calculated by multiplying the increase in theatre utilisation by the total available theatre hours per day. The potential additional theatre hours released per year as a result of the project. Calculated by multiplying the daily increase in the released theatre working hours by the number of working days in the year (250 days). The total working days can be adjusted. The average utilisation rate of the theatre(s) affected. This is calculated by dividing the total available hours by the actual operating time, multiplied by

11 Ward specific Length of stay at the start of the project (days) Length of stay at the end of the project Number of beds Additional patients admitted per year Bed days released per year The average length of stay before the project starts. This is the baseline against which the change is assessed. The average length of stay once solutions have been implemented. The number of beds currently used to treat patients in the wards or diagnostic-related groups involved in the project. If this number is variable use an average of the number of beds over the preceding three months. The potential additional number of patients who could be seen per year. Calculated by dividing the additional bed days by the length of stay at the end of the project. The number of beds that are released as a result of the project. This is calculated by multiplying the reduction in length of stay by the total patients admitted annually. Measuring financial benefit Measuring the fi nancial or potential fi nancial benefi t is one element of a comprehensive approach to articulating the outcomes from redesign or improvement activities. Financial benefi t alone does not tell the whole story and it is recommended that ROI is used in conjunction with other measures such as: patient experience staff satisfaction patient access quality and safety indications wait times. The ROI model developed will provide health services with a way to capture and articulate fi nancial benefi ts. The model outlines the benefi ts anticipated over a three-year period. The formula for calculating the ROI is benefi t cost/costs. For the purpose of the ROI model the fi nancial benefi ts are expressed as follows: Ward projects Potential bed days released per year Potential additional patients admitted per year Net and gross value released Gross value released per dollar invested 5

12 Emergency department projects Potential cubicle hours released per year Potential additional patients per year Net and gross value released Gross value released per dollar invested Theatre projects Potential theatre hours released per day Potential theatre hours released per year Net and gross value released Gross value released per dollar invested In determining the above benefi ts, data relating to the following is required: implementation costs ongoing maintenance costs project benefi ts (anticipated or achieved). Further information about the data requirements is provided in section 3 Data requirements. Before starting It is recommended that health services involve a representative from the fi nance department and/or clinical costing unit to assist with collecting data relating to fi nancial costs and provide expert input when interpreting and using the fi ndings. Familiarise yourself with the data required to complete the tool, relevant to the project type (emergency department, ward or theatre). This is outlined in detail in Section 3 Data requirements. Including the data requirements up-front as part of the data management plan for the project prior to commencing the project will ensure data accuracy is optimised. Best practice for collecting the data includes: Decide on the measures at the start of the project (during the planning phase). Determine how much data will be collected and assign responsibility. As part of good practice project management ensure data collection progress is monitored throughout the project tollgates to ensure it is completed. Collate the data in a central location for use at the completion of a project. Where this has not been done the ROI can still be assessed if the relevant costs and benefi ts are available or can be derived. However, the data must be interpreted acknowledging the limitations of how the data was captured. 6

13 Section 3: Data requirements The data requirements to complete the ROI assessment are provided in the table below. The data elements referred to appear as yellow cells in the Excel spreadsheet under the headings of implementation costs, ongoing maintenance costs and benefi ts, and need to be collected to complete the tool. Data element Definition 1. Implementation costs This includes all costs related to implementing the project that the health service would not have incurred had there not been a project. The implementation costs should refl ect and align with the project budget. In calculating the implementation costs consider the following cost categories. Project staff Clinical staff Materials Equipment Other The cost of all staff assigned to the project. This includes salary and contract costs along with any associated on-costs. If staff were assigned to the project part time, consider the costs as a pro-rata charge. The cost of clinical staff should be considered in two ways: the cost of backfi ll required to release other clinical staff the cost of any new positions created as a result of the project. This includes salary and contract costs along with associated recruitment, on-boarding and other on-costs. The cost of any materials used during the project including: equipment for 5S 1 or visual management workshop materials for example, value stream mapping materials for data collection such as tracking for example, stop watches and clipboards. The cost of any additional equipment required to support the implementation of solutions. Examples include electronic whiteboards/journey boards, new surgical equipment and trolleys for storage. The total of any other costs incurred as part of the project. This may include venue hire, staff training on the new standard introduced by the project, and external consultancy fees. 2. Ongoing maintenance costs This is likely to be an estimate only as the costs will not have been incurred. It includes costs associated with keeping the solutions in place. In calculating the ongoing maintenance costs consider the following cost categories. Generally maintenance costs will be lower than the implementation costs. Note: When estimating costs consider if: patient numbers will stay constant or grow (this will impact on the number of staff and materials required) equipment or materials will need replacing. Costs will need to be adjusted accordingly. Project staff Clinical staff The ongoing costs of all staff assigned to the project. Only complete this section if project staff are going to continue working on the project. Again, this includes salary and contract costs along with any associated on-costs. If staff were assigned to the project part time, consider the costs as a pro-rata charge. The cost of clinical staff created as a result of the project. This includes salary and contract costs and any on-costs. 1 5S is a workplace organisation method that employs fi ve phases: sorting, set in order, systematic cleaning, standardising and sustaining. 7

14 Materials Equipment Other The cost of any materials used on an ongoing basis. The cost of any additional equipment required to support the implementation of solutions following the project. It also includes maintenance or replacement of existing equipment. The total of any other costs incurred to sustain the solutions implemented during the project. This may include training or external consultant fees. 3. Benefits Specifi c measures are required to enable the fi nancial benefi ts to be calculated. In order to fully assess the benefi ts and determine a fi nancial value, health services will need to calculate local costs. The benefi ts have been separated into emergency department, ward and theatre type projects. Emergency department projects Length of stay at the start of the project (average hours) Length of stay at the end of the project (average hours) Total number of patients per year Number of cubicles Emergency department cubicle cost per hour The average length of stay in the emergency department prior to beginning the project. This is the baseline against which change is assessed. The average length of stay at the end of the project, after the implementation of solutions has been completed. The total number of patients seen by the emergency department in a year. The number of cubicles impacted by the project. Average cost per hour of an emergency department cubicle. It is recommended that this cost includes: fi xed average cost of heating, lighting, laundry and food average variable cost of medications and staff. Costs recommended to exclude are: capital costs such as buildings. Theatre projects Utilisation rate at the start of the project Utilisation rate at the end of the project Maximum daily available hours Number of theatres Cost per theatre hour The average utilisation rate of the theatre(s) affected prior to the commencement of the project. This is calculated by dividing the total available hours by the actual operating time, multiplied by 100. The average utilisation rate of the theatre(s) affected after the implementation of solutions has been completed. This is calculated by dividing the total available hours by the actual operating time, multiplied by 100. The total available hours that the theatre could operate for if there was no waste in the system. The number of theatres in the health service affected by the project. Average cost per hour of a theatre. It is recommended that this cost includes: fi xed average cost of heating, lighting, laundry and food average variable cost of medications and staff. Costs recommended to exclude are: capital costs such as buildings. 8

15 Ward projects Length of stay at the start of the project (days) Length of stay at the end of the project Number of beds Total number of patients per year Bed cost per day The average length of stay before the project starts. This is the baseline against which the change is assessed. The average length of stay once solutions have been implemented. The number of beds currently used to treat patients in the wards involved in the project. If this is variable use an average of the number of beds over the preceding three months. The total number of patients seen by the ward(s) in a year prior to beginning the project. Average cost per bed per day. It is recommended that this cost includes: fi xed average cost of heating, lighting, laundry and food average variable cost of medications and staff. Costs recommended to exclude are: capital costs such as buildings. A note on diagnostic-related group projects The ROI tool can be used for projects that are focused on a diagnostic-related group (DRG) rather than a functional area more broadly (emergency department or wards, for example). For such an assessment modify the data in the benefi ts section to refl ect the specifi c DRG. For example, the average length of stay at the beginning of a project would be the average for the relevant DRG only and the total patients per year similarly would refl ect the patients for that particular DRG. 9

16 Section 4: Calculating return on investment This section will guide users through using the Excel tool provided as part of this pack to calculate the fi nancial or potential fi nancial benefi t of an improvement or redesign activity. At the conclusion of the section additional information is provided specifi cally for using the tool prospectively (provided as a separate Excel worksheet). The steps to follow to complete the tool prospectively are the same; however, benefi t-related data will be related to the anticipated outcome of the project. Refer to Section 6 Using the ROI tool prospectively for more details. A checklist for completing the ROI has been provided as an appendix to this document. Step 1. Select project type The selection of project type is based on the purpose of the project and the intention of the improvement rather than the location of the project in the health service. It may be useful to think about this in terms of the project s main measure. The project may operate directly within or outside of the area where the project benefi t is being sought. This version of the tool relates to three project types: emergency department measure: emergency department length of stay ward measure: ward length of stay theatre measure: theatre utilisation. The following table provides examples of characteristics for the three project types within or outside the project location to assist with determining the appropriate template and therefore data requirements. Project type characteristics examples Project type Emergency department The outcome sought by the project is to reduce emergency department length of stay. Operates within the location of the project Operates directly on the emergency department to reduce length of stay. Example projects include: improving planning and treatment processes. Operates outside the location of the project Operates outside of the emergency department to reduce length of stay by improving the interface with other areas or improving fl ow to create capacity. Example projects include: improving patient transfer processes improving access to shared services such as pathology or imaging redesigning the patient journey to support the patient being pulled from ED to the ward. 10

17 Ward The outcome sought by the project is to reduce ward length of stay. Theatre The outcome sought by the project is to improve the utilisation of theatres (optimise the time theatres are used). Operates directly on the ward to reduce length of stay. Example projects include: improving treatment schedules reducing variation of care reducing ward-based harm (falls, medication errors or pressure injuries). Operates directly on theatres to improve utilisation. Example projects include: reducing the time between theatre cases. Operates outside of the ward to reduce length of stay by improving the interface with other areas or improving fl ow to create capacity. Example projects include: improving pharmacy turnaround time for discharge scripts to enable patients to be discharged in a more timely manner. Operates outside the theatre to improve utilisation by improving the interface with other areas or improving patient transfer times. Example projects include: improving patient scheduling improving ward transfer processes. An Excel worksheet is provided for each of the three project areas. Step 2: Populate the tool There are three sections of the tool to populate to produce the results. These align with the three sections detailed in section 3 Data requirements: implementation costs ongoing maintenance costs benefi ts. The tool has been colour-coded to assist with data entry. All yellow cells require data to be entered. The cells that are shaded blue will automatically populate based on the data entered in the yellow cells. Populate the implementation and ongoing maintenance costs fi rst. An example is provided below. 11

18 Then enter the relevant project measures in the benefi ts section. See example below. Step 3: Interpret the results The key term used to describe the ROI of a project or to assist in prioritising potential projects is released value. The released value is used to describe the additional capacity created to (potentially) allow more patients to be treated for the same (or similar) cost. Released value is not shown as a benefi t on the budget or cash fl ow. It also does not describe incrementally reduced costs. Examples of released value include: improving theatre utilisation to enable more cases per theatre list reducing the length of stay on a ward to enable additional patients to be treated within the same or similar resources reducing the length of stay in the emergency department to enable additional patients to be treated within the same or similar resources. Additional benefi ts that the ROI model demonstrates are the potential: bed days saved per year emergency department cubicle hours saved theatre hours saved per year additional patients seen. This data is located in the row titled Summary of benefi ts (see ward project example below). The model outlines the benefi ts anticipated over a three-year period. The methodology assumes a decrease in the maintenance of the benefi t to provide a more conservative expectation of the benefi t. In year 1 gross released value is calculated as 75 per cent of the total released value. In years 2 and 3 this is further reduced by 10 per cent each year. The following example for a ward project has determined the total released value as $656,250. The gross released value of $492,188 is calculated for year 1 as 75 per cent of that fi gure. For years 2 and 3 the amount reduces by a further 10 per cent year on year to $442,969 and $398,672 respectively. Health services may elect to revise the percentage of total benefi t achieved each year. 12

19 The net released value is calculated by subtracting the total costs from the gross released value. For this example it equates to $392,188, $392,969 and $348,672 for years 1, 2, and 3 respectively. The fi nal row is the gross released value per dollar invested (gross released value / total costs). For the example below it shows that for each dollar invested a positive ROI was yielded across each of the three years described. Where this value is less that $1.00 a negative ROI has occurred. However, consideration of the value in years 2 and 3 (or even beyond) may be required, particularly where investment in year 1 is large. It is also important to consider this in using the tool to prioritise projects because the benefi t may be seen in later years. Further to this, the benefi ts need to be interpreted in context as projects may have the same (or similar) gross released value per dollar invested but signifi cantly different net released value. This is again particularly important when using the tool to prioritise projects. Example graphs to help summarise the fi ndings are included as part of the Excel worksheets provided. 13

20 Section 5: Limitations of the model A number of limitations to the ROI model should be considered when using the tool and interpreting the data including: The ROI model only assesses the fi nancial benefi ts of a project. However, improvement projects commonly create non-fi nancial benefi ts such as improving redesign or improvement capability within a health service quality improvements including improved patient safety improved patient experience greater staff satisfaction. Although excluded from the ROI model, many of these non-fi nancial benefi ts will have a fi nancial impact. For example, the fi nancial benefi ts of improved staff satisfaction cannot be calculated, but it can lead to improved productivity and decreased absenteeism. This in turn creates a fi nancial benefi t. As such the ROI calculation may understate the fi nancial benefi ts arising from improvement projects. The ROI assessment may be completed at the end of the project, shortly after the solutions are implemented. This may mean that the full impact of the solutions are not refl ected in the results, potentially underestimating the benefi ts of the project. Health services are constantly undergoing changes, with change impacting across the organisation. As such it may be diffi cult to attribute the benefi ts of change to a single project. The cost for treatment using emergency departments, wards and theatres can vary depending on a number of factors including when and where the patient is treated. However, a single average cost is used in the calculation. The current ROI model does not include potential revenue impact where additional revenue may be generated because a funding cap has not been reached, nor does the model measure incremental cost reduction. The current ROI model has a limited project scope. The following are examples of projects that are currently out-of-scope: improvements in turnaround times to access shared services such as radiology, pathology or pharmacy (where emergency department or ward length of stay reduction is not the intention of the project) cost reduction in terms of reduced stock and inventory cost reduction related to patient transport waitlist reduction to improve access to services reduction in staff overtime or agency costs reduction in quality and safety incidence. 14

21 Section 6: Using the ROI prospectively The ROI tool can be used before a project begins to estimate the potential fi nancial benefi t associated with the project. The ROI tool can be used to: assist in prioritising or selecting particular projects provide an indication of the break-even period (where implementation and ongoing costs are offset) assess if a project will bring about a positive ROI, and thereby justify the investment. However, when doing this it must be acknowledged that although ROI can assess the potential fi nancial return on investment it does not assess the range of benefi ts offered by a potential project. Hence it is recommended that the ROI is not used as the sole measure to select improvement projects. Project selection should consider the non-fi nancial benefi ts of the project as outlined previously before making a fi nal decision. Prospective ROI template A separate Excel spreadsheet has been developed for health services looking to use the ROI prior to beginning a project. The structure of the tool is the same as for the retrospective tool; however, the template requires a target measure. Yellow cells are cells that require the user to enter data. The data requirements are the same as outlined in Section 3 Data requirements. The template has been created to enable analysis based on one or two target values. This allows potential projects to be based on best and worst (or what-if) case scenarios. The benefi t of doing this is that a range of ROI values are determined that consider various project risks. Setting appropriate targets When using the tool prospectively it is important to set realistic targets in order to predict the likely ROI outcome. In setting a target consider: current performance compared with statewide (or other relevant) average, such as Health Roundtable data best practice benchmarks project scope (including scale of the project) project timeframe changes intended to be made as part of the project. If using what-if scenarios consider the risk of the project. A risk assessment may also have been completed as part of the initial project brief or business case. Risks and issues may arise from: change fatigue or change readiness other projects impacting on the same area of the health service timing issues What else is occurring at the same time? Is it a busy or quiet period of the year/month? resources available technology issues organisational issues restructure, changes to key personnel scope changes. 15

22 Interpreting the ROI In using the ROI prospectively the data must be considered in the context of the broader project outcomes sought and risks associated with each scenario impacting on the likelihood of project success. It is recommenced to consider the net released value along with the gross released value per dollar invested and to do this across the three-year period. The example provided below shows a vast difference between the scenarios. Scenario 2, which yields a reduced bed day saving, has a negative net released value and gross released value per dollar invested for the three-year period. The decision on which approach to take with the project would need to be considered given that both alternatives potentially yield a positive ROI. Similarly in using the model to prioritise or select projects from a range of projects the predicted ROI should not be used in isolation. It is designed to be one part of an informed decision-making process. 16

23 Appendix: Checklist for using the ROI Below is a checklist designed to assist with using the ROI both retrospectively or prospectively. Retrospective ROI tool completion Before the project starts Task Completed Select the appropriate model based on the intent of the project Understand the budget and total costs for the project Determine the methodology and frequency for collecting data Assign responsibility for collecting and recording data Measure and record baseline data During the project Task Completed Continue collecting data as the project progresses Monitor the budget and update costs as required At the conclusion of the project Task Completed Populate the ROI template with the data Report the results adding context and information on the limitations as required Prospective ROI tool completion Before the project starts Task Completed Select the appropriate model based on the intent of the project Set targets (include differing scenarios) Understand the budget and estimated costs for the project Measure and record baseline data Populate the ROI template Report the results adding context and information on the limitations as required 17

24