Patient Reported Outcome Measures

Transcription

1 Patient Reported Outcome Measures September 2011 Health Intelligence Specialists, YHQO Tel:

2 Role of Quality Observatories High Quality Care for all Requirement for a Quality Observatory to be set up in every region. Set up nationally in December 2009, 10 across England YHQO is a virtual between the PHO and the SHA PHO QO SHA There have been 2 full time employees since Aug 2010, plus contributions from PHO and SHA staff Overarching purpose of Quality Observatories: Enable local benchmarking, development of metrics and identification of opportunities to help frontline staff innovate and improve health services Work programmes for the Yorkshire and Humber Quality Observatory: QIPP, Clinical Thresholds, Quality Accounts, Quality Dashboards, Patient Experience, PROMs, End of Life and more recently cancer

3 Why develop PROMs? Just as important is the effectiveness of care from the patient s own perspective which will be measured through patient-reported outcome measures (PROMs) Darzi - High Quality Care for all (2008) Aims of PROMs: Evaluate clinical quality Support benchmarking Evaluate clinical and cost effectiveness of procedures Assess appropriateness of referrals Support reductions in inequalities PROMs considered the nearest measure to output and productivity of the NHS. Better measure than prevalence of adverse outcomes: death, infection, readmission From patients perspective measure of outcome rather than process (patient experience). Patient s own assessment of health related quality of life To be in used conjunction with other clinical data

4 PROMs procedures Currently covers 4 procedures: Groin hernia repair Unilateral hip replacements Unilateral knee replacements Varicose vein surgery OPCS codes define operations to be included (includes primary and revisions for hip and knee) Covers NHS providers and independent sector providers for NHS patients Expansion of PROMs: Piloting PROMs for coronary re-vascularisation Piloting cancer survivorship Ongoing pilot for long term conditions Developing generic PROM to cover most acute elective interventions

5 Collecting PROMs Patient completes pre op questionnaire given by provider (ideally on the day of admission) Patient completes post-operative questionnaire several months after procedure (3 months for groin hernia and varicose vein, 6 months for hip and knee surgery) Questionnaire includes generic questions (EQ5D), condition specific questions (Oxford hip and knee replacement and Aberdeen varicose vein score) and visual analogue scale EQ5D generic questionnaire 5 dimensions: mobility, self care, usual activities, pain/discomfort and anxiety/depression Answer questions as: no problems, some problems, severe problems Converted to score from 1 to (where 1 is the best outcome possible) Oxford hip/knee score 12 questions score 1-4, converted to score out of 48 (48 being the best outcome) Aberdeen varicose vein score scores out of 100 where 0 indicates no evidence of varicose veins and100 the most severe problems

6 PROMs data Data is produced as aggregated scores by Trust/PCT Data includes participation, linkage, issue and response rates Participation rate - number of valid pre-op questionnaires received divided by the number of FCEs Linkage rate - number of linked pre-op questionnaires divided by the number of valid questionnaires received Issue rate - number of post-operative questionnaires sent divided by the number of valid pre-ops received Response - number of valid post op questionnaires received divided by the number sent to patients Average pre and post operative scores for generic and condition specific measure Number of post operative questionnaires that have increased/ decreased/ stayed the same compared to pre-op Health gain calculated as the difference between pre and post-operative scores Case mix adjusted post operative scores and health gains to enable comparison of organisations based on assuming a standard case mix. Factors include: Age Sex Pre-existing conditions Pre-operative score Adjusted data includes confidence intervals

7 Example from May 2011 YHQO report Knee Replacement Participation and Linkage Rates Knee Replacement participation and linkage rates Participation Rate Linkage Rate Harrogate NHS FT 98.2% 73.4% Barnsley Hospital NHS FT 94.4% 79.0% The Rotherham NHS FT 93.0% 64.8% N Linc & Goole NHS FT 91.7% 84.5% Hull and EY NHS Trust 90.6% 90.2% Airedale NHS FT 87.1% 71.2% Scarborough NHS Trust 84.3% 80.2% Cald & Hudd NHS FT 80.7% 90.4% Leeds Teaching NHS Trust 75.2% 63.7% Sheffield Teaching NHS FT 66.4% 78.6% Bradford Teaching NHS FT 65.1% 73.2% York Hospitals NHS FT 63.0% 101.1% Mid Yorks NHS Trust 60.9% 97.9% Donc & Bass NHS FT 60.7% 103.1% England Avg 81.9% 67.4% 0% 50% 100% EQ5D Groin Hernia Repair Unadjusted health gain Adjusted health gain adjusted health gain confidence intervals England Avg Leeds Teaching NHS Trust N Linc & Goole NHS FT Barnsley Hospital NHS FT Donc & Bass NHS FT Sheffield Teaching NHS FT The Rotherham NHS FT York Hospitals NHS FT Mid Yorks NHS Trust Harrogate NHS FT Cald & Hudd NHS FT Airedale NHS FT Scarborough NHS Trust Hull and EY NHS Trust Bradford Teaching NHS FT

8 Publication of data Up to July 2011 the data was cumulative since April 2009 Published monthly as spreadsheets on the Information Centre website: Refresh changes to publications. Provider/commissioner spreadsheets available quarterly Freezing older data ( ) in August 2011 (this data is classed as final) Cumulative data from April 2010 to March 2011 classed as provisional until next freeze point (August 2012) can be updated with new data until it is finalised Cumulative data from April National highlights published monthly, spreadsheets published quarterly data for Apr11-Jun11 published Nov 2011 Deep dives into specific issues

9 Using the data PROMs data should be used to support the aims listed in slide 3 Work on inequalities was undertaken nationally, early PROMs reports on pre-operative data showed patients from deprived areas waited longer for treatment Potential work on referrals to look at optimal point for referral to ensure the best outcome for the patient and value for money Ongoing work to look at the possibility of linking PROMs data to best practice tariff Potential work on looking at clinical significance of changes in scores. It is difficult to pin point a MID (minimally important difference) as clinical differences are hard to use and present, the next option is to use statistically significant results (Getting the most out of PROMs) Assessing the statistical significance of results. Confidence intervals are used on adjusted data only where there is no overlap in the confidence interval can a result be said to be statistically significant at 99.8% It is possible that going forward data can be made available at consultant level and could potentially form part of appraisals etc.

10 Caveats in PROMs data Caveats within the published data: National sample size is only a proportion of operations (approx 33% of operations have returned post-operative questionnaires Voluntary for patients to complete Time lags in data (e.g. Apr 10 to Mar 11 data will not yet include all postoperative questionnaires) Variation in timeliness of questionnaire completion (pre and post-op) Retrospective refreshing/updating of data Aggregate scores only by Trust and PCT

11 Changes to PROMs data Changes to consent model - Patient gives explicit permission for data to be used but can tick an opt out box if they do not want personal details sharing data with the provider and consultant team 3 levels Full acceptance of personal data to be used Partial acceptance (at aggregate level only) by ticking opt out box Refusal to complete questionnaire Enables more detailed data to be produced as patients have consented to personal details being used Need deeper analysis to understand potential outliers Going forward there are plans to produce record level extracts for providers and commissioners which are potentially linked to some HES fields

12 PROMs outlier policy The outlier policy is currently under consultation it is essentially a way of classifying PROMs data as positive or negative outliers Data presented as funnel plots with control limits set to 2 standard deviations (95% control limit) and 3 standard deviation (99.8% control limit) Organisations outside of the inner control limits defined as alert and outside outer limits as alarm. Alert and alarms to be triggered for negative outliers To be part of CQC Quality Risk Profiles Future supporting documentation expected to be released Link to outlier policy: Guidance/DH_ Example funnel plots on following slides taken from Finalised score comparison tool spreadsheet at : 82

13 adjusted health gain Interpreting funnel plots Procedure Scoring Mechanism Organisation Level Organisation Name Knee Replacement Oxford Knee Score PCT of responsibility East Riding of Yorkshire PCT Lower 95% control limit Upper 95% control limit Lower 99.8% control limit Upper 99.8% control limit national rate All Selected volume of modelled records

14 Record level data Accessing record level data Record for each patient in HES (PROMs centric universe) Includes all responses to questionnaires Includes patient data such as age, admission date, gender Request analysis through QO or pay for extracts through IC What analysis have we done/planned? Comparison of health gains for different hip procedures Breakdown of EQ5D dimensions Analysis of effects of age on health gains Assessment of post-operative complications Caveats Can not include identifiable/sensitive fields if used in analysis these must be anonymised Data must be aggregated to a level which does not breach HES protocol around small numbers (i.e. can not include numbers less than 6) Data is raw data (i.e. it has not been case-mix adjusted)

15 YH QO work on PROMs Linking with the Department of Health and Information Centre Produced PROMs newsletter to update the region on latest developments Looking a developing a specific PROMs section of the QO website Working on setting up a regional PROMs network Analysis using record level data for specific organisations and potential plans to undertake ad hoc regional analysis of data at this level Produce quarterly PROMs report latest report produced May 2011, next report currently in development, altered to reflect refresh publication. Expected Sep 11 Link to YHQO website:

16 PROMPT PROMT Patient reported outcomes measures tool for Commissioners. DH tool developed for commissioners to allow commissioners to benchmark and compare information on PROMs procedures with other PCTs to improve investment decisions Select PCT and use drop down boxes to select measures of interest: pre-op health status, health gain (raw and adjusted), post-op health status (raw and adjusted), volume rate and expenditure rate Types of charts distribution and quadrant Guidance for using tool - Link to PROMT :