Supporting GPs to interpret and learn from Patient and Colleague

Transcription

1 Supporting GPs to interpret and learn from Patient and Colleague Feedback Dr Di Jelley Northern Deanery Associate Advisor for Appraisal and Revalidation APCE Appraisal Conference 14/11/12

2

3 What is Multi-source feedback [MSF]? MSF provides the opportunity for patients, nonmedical co-workers (including other health professionals, managers and administrators) and medical colleagues (including trainees and juniors) to reflect on the professional skills and behaviour of a doctor. GMC 2011

4

5 What is multi-source feedback for? Seen as a developmental tool to examine behaviours key to a job such as teamwork, communication and interpersonal skills Aim is to provide a set of colleagues and patients opinions based on GMC criteria, and to compare these with Dr s self assessment Looks less at what people do, and more at how they do it Aim is to enhance individual and team performance

6 Experience of MSF in the health service Used extensively in North America as part of reaccreditation Can achieve high levels of reliability if raters are used Allowing Dr to choose raters does NOT lead to more +ve feedback than random selection Already widely used in junior Dr training Several feedback tools have now been evaluated in UK general practice

7 GMC feedback questionnaires-research findings Campbell et al report via GMC guidance on pilot work with > 17,000 colleague and patient responses - uk.org/executive_summary_of_research.pdf_ pdf GMC work and other studies have shown that accurate feedback on Dr performance can be provided not only by other Drs but also by nurses, other health professionals and administrators

8 Potential Benefits of 360 feedback You don t know how others see you until you ask Praise is a very powerful motivator Can be a very powerful a learning experience Self assessment of communication skills, team relationships, leadership skills etc is very inaccurate

9 Possible Pitfalls with 360 degree feedback Destructive feedback can be very damaging to the individual and the team Anonymity may be used to express grudges or personal animosity Feedback without facilitated discussion unlikely to change behaviour Time consuming and respondents may get feedback fatigue

10 What tools are to be used for Patient and Colleague Feedback The RCGP Guide [version 7 June 2012] only recommends CFEP and Edgecumbe, as well as GMC questionnaires, as tools for both Colleague and Patient feedback other tools for either/or The surveys must be independently collected and collated They must be developed and piloted in line with GMC Guidance The GMC has worked with CFEP to develop their own feedback questionnaires these are free of charge but must be collated and compared against national norms using external provider or by Trust/PCT etc

11 GMC Questionnaires These have been developed and piloted by CFEP The questionnaires themselves are free to download from the GMC website, but they must be collated independently and provide the Dr with comparisons to national norms CFEP are one of the organisations who collate them for doctors [at a cost] Clarity and RCGP include them in their e-portfolio Collation can also be done by the PCT or Trust

12 CFEP -CFEP uses both the GMC questionnaires and their own version of feedback tools-they are very similar but only the latter has locum specific data at present. CFEPhttp:// eral-practice/360.aspx

13 Edgecumbe Edgecumbe uses their own surveys which are similar to the GMC ones RCGP approves both of these Edgecumbe Edgecumbe

14 Clarity and RCGP Clarity anr RCGP both offer GMC questionnaires as part of their annual fee Neither yet have much of a comparative data base, but this will improve No clear indication yet as to how either will present their comparative data

15 Interpreting feedback some general points Patient Responses Bias towards the positive majority of all responses good to excellent only 1% less than satisfactory or poor 98% happy to see that doctor again Colleague responses Again tend to be very positive majority ranking their colleague as good to excellent Only 1% ranked their colleague as less than satisfactory or poor 97% agree their colleague was fit to practise medicine

16 Bench-marking [1] The data base of responses from the pilot studies is from volunteers who are unlikely to be a truly representative sample of all UK doctors The differences between the lowest and highest quartiles is often very small this may increase as data base expands to include all doctors Appraisers should also be aware of rating biases when interpreting the doctor s data

17 Bench-marking [2]- Rating biases Patient factors- the following all tend to lead to higher ratings: Perceived importance of the consultation Well-established doctor/patient relationship Ethnic group responses higher from white than ethnic group patients Age -patients over 40 Colleague factors--higher ratings with High frequency of contact between Dr and rater Non medical peers rate higher highly than medical

18 Bench-marking [3]- Rating biases Doctor factors the following score less highly overall from patients and colleagues Locums Doctors whose primary medical degree is from outside the UK Ideally the doctor should be compared with a cohort of patients/colleagues with similar characteristics, but this data is not yet widely available. CFEP does have some locum specific data with their own not the GMC questionnaire

19 Interpreting the feedback CFEP and Edgecumbe do it differently Edgecumbe give a % value for every question if the doctor s responses are marked as 10% this means only 10 % of doctors had responses to this question which scored lower than this doctor s response. If the doctor has a score of 85% for a particular question, this means that only 15% of doctors would have a higher score than this doctor on this question

20 Interpreting CFEP CFEP uses quartiles the doctor s responses are graded as lowest quartile -25% middle quartiles 25-75%- and top quartile above 75% The concept is the same as Edgecumbe but the figures look a little different. If responses to a particular question are in the bottom quartile, this means they are the lowest 25% of scores-if the responses are in the top quartile then for that question the doctor s responses are in the highest 25%.

21 Interpreting the figures Given the research data findings that suggest most doctors are highly rated most of the time, it is worth identifying for discussion any areas where The responses are in the lowest quartile [CFEP] or below 25% [Edgecumbe] Any area where the range of responses is quite wide The responses are higher or lower than the doctor s self assessment

22 Structuring the discussion about feedback Focus first on the overall results most are likely to be good/very good overall and this should set the scene for the discussion Consider discussion about the process How many questionnaires were returned How and where was the patient survey done Did the sample have any special characteristics How many valid responses were received

23 Considering strengths and weaknesses Celebrate all the positives -high ratings and positive comments Do all the items get a similar rating or are there marked differences between items Does the doctor score significantly lower than other doctors [ie lowest quartile or <25%] on any items Does the self-assessment vary significantly from the patient/colleague scores Are there any comments that need specific review

24 General advice advice on how to discuss feedback Most doctors find eliciting feedback stressful and fear they will be found wanting -be sensitive to this and supportive in your comments celebrating the positives The process is formative- NOT pass or fail your aim is to make sense of feedback and use it to inform to shape the doctor s CPD as needed Use open questions, identify strengths, development areas, and any unhelpful behaviours Should lead to agreement on what to keep doing and what to do differently Finish by summarising strengths

25 Consider the PINs model for supporting change-solution focused [Dr Steve Blades] Platform for change (P) Describing the ideal (I) Where are you now? (N) Next steps (S)

26 Platform for change (P) What is it important for you to change / improve? What will be the benefits of the change e.g. to you, to colleagues, to patients What would be the positive knock on effects of the change? How important is it to change on a scale of 1-10

27 Describing the ideal (I) Imagine you were doing this really well. What would be happening? What would you be doing? What would others see and hear? (Concentrate on the observable not on feelings)

28 Where are you now? (N) On a scale of 1-10 where 1 is never doing any of this and 10 is doing it all of the time where are you now? What makes it n? (rather than less than n) When are you able to do better than this? Take the opportunity to be affirming about the ability to accomplish the change at least partly or some of the time.

29 Next steps (S) What would be happening if you were doing n+1? What would you be doing? What would others see and hear? What small steps can you take to get you to n+1? How can you experiment with changes? What and when will you actually do? Who can support you in making these changes? What will be the first signs of change?

30 Summarising the discussion [1] When considering areas for change it may be helpful to think about some of these points What might be the benefits of change? These What might be the benefits of change? These might be benefits for the doctor, patients or colleagues.

31 Summarising the discussion [2] Be as specific as possible about the change. For example if the doctor needs to communicate better with staff be clear about what is meant by this and how it might be achieved and reviewed? Making changes in this area can be difficult so be realistic about how many things can be changed at once

32 PDP aims make them specific For some changes further training might be indicated. Actions might include attendance at a communication skills update, doing some joint or videoed consultations, specific training on shared decision making skills, leadership skills, mentoring, coaching etc Try and include a specific action on the PDP not just a vague aspiration

33 Self-assessment of your feedback review skills The University of Exeter Medical School are currently developing an on-line self-assessment tool for appraisers [using CFEP questionnaires] Collated feedback for a range of doctors will be available to view and assess Appraisers will be able to see how their interpretation of the feedback compares with that of other raters This will be a valuable education tool for appraisers and may help to identify appraisers who assess at the hawk or dove ends of the spectrum, and move them towards the middle ground

34 Northern Deanery Website Northern deanery/primary care/appraisal and revalidation latest news FAQs on patient and colleague feedback

35 !"##$%$&'($)*(+$