MSC Assessment Activities

Transcription

1 MSC Assessment Activities The Medical Schools Council wishes to bring to the attention of the GMC Undergraduate Board a number of developments upon which it has recently embarked. The MSC trusts that the GMC will be supportive of these activities and would wish to support the MSC in further work to raise the quality of assessments conducted by Medical Schools both in the UK and internationally. MSC Assessment Alliance and the development of a common bank of questions In 2006, as a result of its consultation on Strategic Options for Assessment, the GMC suggested that Medical Schools should consider sharing examination questions. The Deans debated this suggestion and agreed that a constructive way forward would be to build on the work done by the 16 Medical Schools who already collaborate in the Universities Medical Assessment Partnership. The benefits of this are that by working together on the development of shared items, Schools will address the original GMC suggestion, and also, through the sharing of expertise, raise the standards of assessment across the UK. All 31 Medical Schools have agreed to participate in the development and use of common assessment items through the MSC Assessment Alliance. The Alliance will be based in the specialist Academy focusing on assessment at the University Hospital of South Manchester. The Director is Professor Ged Byrne. The Alliance will be managed by a Board chaired by the MSC Chair, Professor Tony Weetman. Development and Operational activities will be managed by subgroups with members from a range of partner Schools. The sub-groups are chaired by Dr Diana Wood of Cambridge and Professor Ged Byrne of Manchester, respectively. A survey has already taken place to determine the numbers and types of assessments (MCQs, EMQs, Short answers, Essays etc) used in Finals, the subject areas covered and the timing of the assessments.

2 A full day meeting of all the UK s assessment leads took place on 7 July to agree a way forward that adds maximum value for all partners and raises the quality both of the assessments and the assessors. The full report of the meeting will be available at the next meeting of the Undergraduate Board but one of the issues raised was a request for clarification by the GMC of paragraphs 112 and 117 of Tomorrow s Doctors woth regard to no compensatory mechanisms Meanwhile work will continue train item writers and to develop and quality assure test items mapped against Tomorrow s Doctors. Software is being developed to support the writing of the questions, their quality assurance, banking and performance analysis. Rigorous security arrangements are being put in place. Prescribing Skills Assessment Background Prescribing is a fundamental part of the work of Foundation year doctors, who write and review many prescriptions each day. It is a complex task requiring knowledge of medicines and the diseases they are used to treat, careful judgement of risks and benefits of treatment, and attention to detail. As well as offering the potential for improving health, it is an activity associated with potential hazards: a recent GMC sponsored study found that 9% of hospital prescriptions contain errors 1. It is also apparent that this is the area of work that new graduates find the most challenging 2. As a result In Tomorrow s Doctors 2009 the GMC has issued guidance about the competencies expected of new medical school graduates in relation to the safe and effective use of medicines. The Prescribing Skills Assessment The Medical Schools Council and the British Pharmacological Society are now working together to develop a Prescribing Skills Assessment that will allow all students to demonstrate these competencies. The aim is to provide a reliable and validated assessment that will serve to ensure that satisfactory standards of practice are achieved by all graduates. The assessment will be based on core competencies such as prescribing, reviewing medication charts, planning treatment, monitoring the effects of medicines, communicating important information and calculating drug doses. Although a reasonable knowledge of common medicines will be required, students will also have to demonstrate the ability to interpret clinical information and make appropriate prescribing decisions based upon it. The ultimate goal is to create an online assessment, lasting around one hour, to be undertaken by final year medical students at a time to be decided by their medical school. This flexible format will allow students who fail the assessment to undertake further training and retake the assessment prior to graduation. Current medical school curricula will provide the preparation that is necessary to pass the assessment and this will be supplemented by Prescribe, a national elearning resource for clinical pharmacology and prescribing ( It is expected that all UK students will be required to pass the assessment. Students who fail on the first attempt will be permitted a second attempt on a date to be agreed. A second failure would result in a period of remediation with support from the Medical School before a third attempt at the test. 1 An in depth investigation into causes of prescribing errors by foundation trainees in relation to their medical education - EQUIP study (2009): 2 How prepared are medical graduates to begin practice? (2008):

3 NHS Employers has suggested that the Person Specification for application to the Foundation Programme should be amended, to include a mandatory requirement to have passed an approved Prescribing Skills Assessment prior to employment in the NHS. We are working with the Rules Group to confirm that such an approach is legal. How is the Assessment Being Developed? The Prescribing Skills Assessment is being driven by a cross sector Steering Group, which is keen to engage with all stakeholders, including medical students. The assessment will be developed over the course of 2010 to 2011, initially using paper-based pilots in 11 medical schools and then using an online hub to disseminate pilot assessments and collect user feedback. Over 1200 students have already taken the pilot test and feedback has been extremely positive and encouraging. It is hoped that the online assessment will be up and running in all medical schools in the 2011/2012 academic year although the challenge of electronic delivery is not being under-estimated. Common prescription form In addition to the common prescribing assessment, the MSC, along with the GMC and others, has lobbied for the introduction of a common prescription form across the UK. It is pleased that a group convened by the NHS Medical Director and the Chief Pharmaceutical Officer has agreed to take forward work to pilot this in the northwest building on the GMC EQUIP study as baseline. Work on the principles defining a good prescription chart will begin shortly. Improving Selection into the Foundation Programme The MSC is leading a consortium to devise an improved method of selection into the Foundation Programme. A detailed and extensive Option Appraisal with input from a literature review and Expert Panel, as well as wide consultation, suggested that quartiles should be replaced by a standardised scheme for Medical Schools to grade knowledge and skills based on a student s performance to the end of the penultimate year an educational performance measure (EPM). White space questions would be replaced by an invigilated, machine-markable test of professional attributes the Situational Judgement Test. Situational Judgment Tests Situational Judgment Tests (SJTs) are a measurement method designed to assess judgment in work-relevant situations: - They present challenging situations likely to be encountered at work - Make judgments about possible responses and are - Scored against a pre-determined answer key

4 They are increasingly popular in large scale selection with well-established validity (both predictive and incremental). They have been used to assist selection into GP training for some time and recently there has been a trial for postgraduate specialty selection too. Job shadowing and interviews have taken place across the UK in order to research the role of the FY1 doctor. The findings have informed what might be taken into account when selecting applicants for the Foundation Programme. FY1s have been interviewed as well as clinical tutors, programme directors, patients, and others. The academic literature has also been surveyed, and the requirements in Tomorrow s Doctors have been taken into account. Through this job analysis a list of characteristics that make good trainee doctors, over and above the clinical skills and knowledge they gain at medical school, has been developed. The following questionnaire has been widely circulated in order to obtain feedback about the relative importance these professional attributes, and to check that nothing has been missed in the earlier research. Please rate the importance of each professional attribute to the role of the FY1 doctor. PROFESSIONAL ATTRIBUTE Commitment to Professionalism Coping with Pressure Effective Communication DESCRIPTION Displays honesty, integrity and awareness of confidentiality & ethical issues. Is trustworthy and reliable. Demonstrates commitment and enthusiasm for role. Willing to challenge unacceptable or unsafe behaviour when appropriate. Takes responsibility for own actions. Capability to work under pressure and remain resilient. Demonstrates ability to adapt to changing circumstances and manage uncertainty. Remains calm when faced with confrontation. Develops and uses appropriate coping strategies. Demonstrates judgment under pressure. Actively and clearly engages patients and colleagues in equal/open dialogue. Demonstrates active listening. Communicates verbal and written information concisely and with clarity. Adapts style of communication according to individual needs and context. Able to negotiate with colleagues & patients effectively. How important is this professional attribute for an FY1 doctor? little extremely importance important

5 Learning & Professional Development Organisation and Planning Patient Focus Problem Solving and Decision Making Self Awareness and Insight Working Effectively as Part of a Team Demonstrates desire and enthusiasm for continued learning, takes responsibility for own development. Willing to learn from others and from experience. Is open and accepting of feedback. Demonstrates a desire and willingness to teach others. Manages and plans workload effectively, displaying efficient time management and delivering tasks on time. Able to prioritise effectively and re-prioritise where appropriate. Is conscientious and maintains accurate records. Ensures patient is the focus of care. Demonstrates understanding and appreciation of the needs of all patients, showing respect at all times. Takes time to build relationships with patients, demonstrating courtesy, empathy and compassion. Works in partnership with patients about their care. Demonstrates an ability to assimilate a range of information and identify key issues. Engages with the wider issues and thinks creatively to solve problems and reach appropriate decisions. Is proactive and demonstrates initiative. Is able to attend to detail. Demonstrates awareness of the boundaries of their own competence. Willing to seek help when required, recognising that this is not a weakness. Exhibits appropriate level of confidence and accepts challenges to own knowledge. Shows capability & willingness to work effectively in partnership with others and in multi-disciplinary teams. Demonstrates a facilitative, collaborative approach, respecting others views. Offers support and advice, sharing tasks appropriately. Demonstrates an understanding of own and others roles within the team and consults with others where appropriate. The SJTs will be piloted in 3 Medical Schools this autumn and in a further 10 Schools in Spring If, following psychometric analysis the test is found to offer a fair and valid means of distinguishing between candidates, a further large scale pilot will take place in autumn 2011, prior to live implementation by the UKFPO on 3 dates to be agreed -in late 2012/early 2013 for students starting their F1 post in August Overseas applicants will be invited to the UK to sit the SJT, as well as the Prescribing Skills Assessment.

6 Educational performance measures (EPM) Stakeholders were very clear that performance at Medical School should inform selection into the Foundation programme. At present there is no common framework for the calculation of the quartiles required by the current system. A draft framework has been put together which separates knowledge based and clinical assessments and provides greater transparency and granularity than the current system. It is currently being piloted. Request The MSC hopes that the GMC will commend the way all UK Medical Schools are working together to share best practice and raise the standards of their assessments. It is hoped that this will prove a significant contribution to patient safety. The innovative and constructive approach taken and the commitment to analysing the results of the initiatives was recognised at the recent Ottawa Conference as at the forefront of best practice. The Medical Schools Council looks to the GMC to support the continuation of these developments and would be happy to discuss ways of enhancing them further.