The objectives of the FEP were to develop an assessment and coaching tool for use with AAs

Transcription

1 Anaesthesia Assistant Follow- up and Enhancement programme, Nepal Dr Oliver Ross, Dr Laura Flutter, Lisa Dali, Shovana Rai, Indra Rai, Dr Resham Rana, Dr Chattra Shreshta, Dr Shristi Malla, Dr Manik Lal, Dr Steve Pickering, Laxmi Deo, Rabina Shakya, Dr Mark Zimmerman. Nick Simons Institute, Kathmandu, Nepal Presented (in part) on April 24 th 2012 at the Ministry of Health and Population, Kathmandu, Nepal. Overview Executive summary AA Background FEP methods Overall Findings Discussion Recommendations Executive summary Why How Chief findings Recommendations Anaesthesia assistants (AAs) are essential, often isolated health- workers. Few receive ongoing support, advice or professional development after completing initial training. Providing a robust system of support, education, ongoing learning and connection with senior staff outside their hospital is vital to maintain their skill, confidence and even presence. The Nick Simons Institute (NSI) has pioneered the Follow- up and Enhancement Programme (FEP), a system of follow- up and onsite coaching for health workers. FEP has been successfully developed for key mid- level practitioners and is now being implemented across Nepal with support of the National Health Training Centre (NHTC). FEP is the first part of establishing a comprehensive system of continuous professional development (CPD). A FEP was undertaken for AAs across Nepal between November 2011 and April practising AAs from 21 hospitals in 18 districts were visited with a core pool of 38 6 month trained AAs forming the majority of the participants. The objectives of the FEP were to develop an assessment and coaching tool for use with AAs

2 2 to assess levels of knowledge and skill using reproducible objective tests to provide onsite coaching to bring all AAs up to an acceptable standard in important clinical skills. to assess the enabling environment- equipment, drugs, supplies and hospital environment for AAs and surgeon/aasupervisors to assess the AAs own confidence and perceived barriers to safe practice and further learning to investigate factors that influence competence explore a potential framework for continuous professional development (CPD) for AAs The AA FEP was an effective and useful tool for the evaluation and coaching of AAs. The study confirmed AAs form the backbone of anaesthetic provision in government and mission hospitals across Nepal. There are very few anaesthesiologists working in government hospitals and one only at district level. The caseload managed by the AA s can be high and the cases difficult. In general AAs were found to be competent in anaesthesia required for caesarean section and significant surgery at their hospital. Skills related to general anaesthesia and the management of anaesthetic emergencies were performed less well. Minimal coaching was required to bring AAs to the necessary standard in these skills. The enabling environment in most government hospitals was not adequate for general anaesthesia, with deficits in equipment and drug supply, Supportive environments were rarely seen but some good examples exist. Almost all AAs working in a CEOC site were regularly engaged in newborn resuscitation yet few have received any training in this. Our recommendations from this FEP include:- acknowledgement of the vital role AAs play in hospitals with support to training programmes and professionalisation of the cadre urgent correction of the environmental factors that inhibit delivery of safe anaesthesia a possible model for ongoing educational support for AAs support for the OT team and a greater focus on surgery and anaesthesia as a key non- communicable health problem. AA BACKGROUND Anaesthesia in district hospitals- a vital service The majority of surgery conducted in district and Nepal s eleven zonal hospitals is for emergencies and in support of Comprehensive Emergency Obstetric Care (CEOC); however in 2011, fewer than 50% of Nepal s 75 districts have functioning government surgical services. Up- scaling of CEOC and providing functional operative

3 3 services has been recognised as essential to reduce stillbirths as well as maternal and neonatal mortality and morbidity. The importance of AAs in delivery of safe anaesthesia for CEOC services is widely recognised, not just in the OT but for sick mothers and newborns. This was recently highlighted in the recent 2011 Nepal CEOC Readiness Survey. This survey, analysing the use of specific funds for CEOC services at district hospital level, indicated large deficits in provision across Nepal with CS readiness generally low and CS rates of 0.4% compared with a target of 5%. Deficits were in large part due to personnel shortages, including AAs who were either absent or few in number. General anaesthesia back- up was available in only four of the 18 hospitals analysed. The report placed an emphasis on the preservation of a functional OT team, with additional capacity to deal with sick mothers (40% of maternal deaths occurred in hospitals with haemorhage and eclampsia the main causes). These deaths were caused by three main factors: inability to treat complications at a previous referring facility; inability to treat complications at the facility where the woman died and inadequate clinical expertise. Professional protection, and ongoing training and support are required for all essential CEOC workers. Two continuously mentored AAs were considered the minimum for each CEOC site with an implementation plan to increase the number of AAs as a key recommendation. In addition to CS need, the unrecognised and largely unmet need for surgery is a significant public health concern. A busy hospital such as Tansen Mission Hospital operates on more than 7000 total cases per year, including over 2800 major and intermediate surgeries (Nepal s Annual Report, 2009/10). Detailed data for surgical need across Nepal are lacking, but trauma, burns and acute abdominal emergencies account for a large proportion of the acute surgical burden. Hysterectomies, cholecystectomies and hernias account for common elective conditions. Shortfalls in service delivery are made up in the private sector in some districts. Camps have been a long- standing solution, particularly in remote areas, but increasing provision at government hospitals is an essential aim. Task- shifting to non- doctor anaesthetists in Nepal In Nepal, despite rapid increases in the number of anaesthesiogists being trained, there are currently very few anaesthesiologists working in government district, and zonal hospitals. As a result, AAs have for many years been delivering anaesthesia across the country. Initiatives to train and support these providers have been supported by the Government of Nepal and recognised as essential to maintain anaesthesia for emergency surgery for the majority of the population. These AAs are supplementary to the existing anaesthesiologist labour force and work only under the supervision of a medical doctor. Task- shifting to non- doctors is recognized globally as necessary to improve service delivery, especially in rural areas. In both the developing and developed world,

4 4 anaesthesia service provision by non- doctors is well established. The World Health Organisation define task- shifting as the rational redistribution of tasks among health workforce teams to make more efficient use of existing human resources and ease bottlenecks in service delivery [WHO 2008]. There is increasing evidence that task- shifting is effective, including a recent study from Nepal, demonstrate adequate clinical provision by mid- level practitioners. Few of these workers receive ongoing support, advice or professional development; the consequence of which is that daily decision making is vulnerable, ongoing confidence and ability diminishes and problems with delivery of care are not highlighted. Providing a robust system of support, education, ongoing learning and communication with senior staff outside their hospital is vital to maintain their skill, confidence and even presence. Evaluation of the training of key workers is also essential to ensure such training is effective and targeted to clinical need. Published data exists for anaesthesia- related mortality, coverage and caseloads for non- doctor anaesthetists but not from Nepal. There is little global data on evaluations of knowledge, skills, confidence and barriers to practise for non- doctor anaesthetists. AA training and support to date For many years, training of AAs in Nepal was delivered on an informal basis by individual doctors at individual hospitals. In 1996, the Ministry of Health initiated a three month AA training programme at Bir hospital in Kathmandu. However, it was recognised that a more comprehensive course was required and in 2002, a six- month government (NHTC) approved competency based training course was designed and implemented by Dr Paul Foster and colleagues. Between 2003 and 2010, 94 AAs were trained on this scheme at several hospitals across the country. Despite the success of this programme, there remained a need to increase AA numbers, improve and prolong AA training, and move towards greater recognition of AAs as a professional cadre. Over a period of five years, a group of experts and advocates developed a new twelve month government approved academic training course. This commenced in 2011 at the National Academy of Medical Sciences (NAMS) at Bir Hospital and is supported by the Nick Simons Institute (NSI) and the Nepal Health Training Council (NHTC). This course supersedes the six- month course and provides a modern curriculum based on spiral learning, mentoring, the use of multiple learning resources, theory and a thorough practical apprenticeship. It is consistent with global recommendations on health worker training and task shifting (WHO 2008). The qualification will be registered by the Nepal Health Training Council or Nepal Nursing Council. Key to the ongoing success of this programme is the implementation of a system of follow- up evaluation and support including onsite coaching and CPD for AAs as they

5 5 qualify and enter clinical practice. Currently there is no formal system of follow- up and support for AAs in the field. Five AA NSI refresher training courses for a small number of AAs have taken place but few government AAs have been able to access this training to date. Previous anaesthesia assistant assessments In 2004, Nepal s Safe Motherhood Project commissioned a follow- up of AAs trained on the six- month programme. Seventeen hospitals (and three primary health- care centres) with twenty- two working AAs were visited; of these, eleven AAs were assessed for competence. Anaesthetic practice was predominantly spinal and ketamine anaesthesia. GA with intubation was very infrequent used. Knowledge was found to be generally good; skills in spinal, basic airway and ketamine anaesthesia were found to be adequate to good (80% were defined as competent); GA less good (40% defined as competent)(none were performing intubations); whilst other aspects of spinal care (hygiene, post spinal care) and pre- anaesthesia check (PAC) could be improved. The report highlighted the need for facilities to be improved, and it was noted that documentation of complications and surgical data was lacking. Most AAs were sufficiently confident but there were significant hindering factors to their practice: surgeons trust and communication, availability of equipment, numbers of cases, lack of supervision and monitoring and financial allowances. Key recommendations included a need for institutionalised training and a system of follow- up. In 2008/9, NSI undertook an extensive informal review across Nepal. The aim was to inform development of the new twelve- month curriculum by looking at place of work, caseloads, and AA needs. AA graduates were found in a wide variety of hospitals and clinical environments. Ketamine IVA and spinal anaesthesia were practiced at all sites and the case load of the AAs was broad including a substantial number of paediatric cases. AAs expressed a desire for refresher training and continuing education. An NSI telephone survey in 2011 found that 49 AAs were currently working of the original 91 trained; the majority were government workers but 11 of the 38 in government hospitals were not working in anaesthesia as there was no surgeon in their hospital. Supporting effective sustained health- worker service in rural areas Key issues for long- term health delivery in Nepal include health- worker training and retention in rural areas, as highlighted in the CEOC review. In the literature, key areas for effective health care provision and health worker retention in rural areas include adequate pre and in- service training, maintenance of adequate facilities to enable delivery of clinical care, attention to reduced health- worker numbers, professional development, mentorship, career development and long term support.

6 6 Additionally, poor health- worker performance or competence is linked to inadequate training, lack of guidelines and inadequate supervision. Vital components for effective health- worker education and training include integration of pre and in- service training, providing learning in the community, increasing information and communication technologies and the use of quality assurance programmes. A model of health- worker follow- up for Nepal: the follow- up and enhancement (FEP) programme NSI and the NHTC have developed FEP to address these issues. FEP provides onsite follow- up and coaching for rural health care workers (HCW). It was developed to answer the question of whether training has made a difference. In addition it was designed to further enhance the skills and build the capacity and confidence of the rural HCW in their place of work through one- to- one coaching and mentorship. As such it links pre- service training to continued in- service training and support. It provides feedback on the relevance and effectiveness for a particular training programme a framework for assessment onsite coaching to provide immediate enhancement of knowledge, skills and confidence education on the use of Quality Improvement (QI) tools feedback to participants, trainers, NSI, NHTC, GoN FEP is now recognised by Nepal s Government NHTC as an effective tool for onsite coaching and assessment of a variety of HCW training. A target follow- up of 30% of all health- worker trainings has been set. Establishing FEP for mid- level practicum, skilled birth attendant and biomedical technician training is ongoing and to date eight FEPs have taken place. THE AA FEP OBJECTIVES AND METHODS Overall Objective To improve delivery of safe anaesthesia at district hospitals in Nepal by assessing and enhancing the skills and confidence of AAs. Specific AA FEP outputs 1. Individual onsite encouragement and coaching in core clinical skills to enhance AA confidence, knowledge and skills

7 7 2. A documented evaluation of the practice, knowledge & clinical skills, confidence and work environment of the AAs: what AAs know what they need to know whether their environment helps them deliver safe anaesthesia. 3. An insight into anaesthesia at differing hospitals to give indicators of success and inadequacy in anaesthetic/ot provision. 4. Development of a comprehensive, sustainable assessment and support tool for AAs in Nepal with establishment of an effective feedback system: AA FEP to AA training sites and NHTC/Government of Nepal 5. Establishment of a CPD programme for working AAs AA stakeholders AAs, local colleagues and supervisors, hospital administration, DHO AAC training sites, NSI, NHTC, Government of Nepal Ministry of Health and Population (MoHP), CEOC planners and funders Anaesthesiologists (including Society of Anethesiologists of Nepal), surgeons and MDGP trainers Participants The main participants were graduates of the six- month AA course. 38 of these were visited in addition to 6 other AAs trained on different schemes but working at the same FEP sites. (Green figures below represent AAs visited) This represents approximately 70% of AAs thought to be currently working in Nepal. Hospitals were selected on the basis of where the AAs were working; the distribution was wide and representative: all regions, rural/urban; district, government, non- government, zonal and central hospitals.

8 8 FEP timetable/logistics Development of the AA FEP tool in early November 2011 was followed by a series of two- week regional tours from Nov 2011 to April The AA FEP team was led by a UK consultant anaesthesiologist with experience in Nepal and the NSI FEP team and variously one of four Nepali consultant anaesthesiologist AA trainers, a UK registrar anaesthesiologist, NSI Head of Training, and the NHTC deputy director. Permission was sought in writing prior to each visit but also at each hospital by meeting with the hospital director/medical superintendent at the start of each visit. AA FEP tool structure A. Knowledge test (40 question true/false; with coaching and re- test) B. Skills assessment (with coaching to competent standard) (four in total) C. Case based discussion anaesthesia emergencies (with coaching to competent standard) (four in total) D. Enabling environment (OT facility and staff, drugs and equipment, OT record of cases) E. Participant interview: experience, confidence, barriers and self- learning F. Supervisor interview G. Review of provisional CPD- QI materials H. Participant Evaluation The aim was to assess and coach core knowledge and techniques relevant to anaesthesia practice in district hospitals. The tool was based on existing NSI- government FEP tools for other health- workers. The goal of coaching and enhancement was integral to the design and included a spiral repetition of core topics and principles throughout the assessments and coaching. A variety of educational methods were incorporated to enhance the strength of assessment and the educational value of coaching. Principles of adult learning were incorporated. The use of a knowledge test and skill checklists was deliberately similar to the current core AA training curriculum and similar to the original 6- month training course. Assessments included a combination of quantitative and qualitative methodology. Knowledge, skills and case- based discussions were all assessed quantitatively using objective tools. After each assessment, discussion and coaching was delivered with subsequent repeat of the assessment to demonstrate knowledge and skill acquisition. Coaching was continued until competency was demonstrated. The intention was to assess and coach each AA with the same tool, on identical methods and manikins and by the same evaluator. The lead FEP consultant anaesthesiologist delivered or attended all, but two of the assessments and coaching sessions. This led to consistency in assessment and coaching.

9 9 Specific topics for assessment and coaching were: knowledge test : basic anaesthesia knowledge (ketamine, spinals, drugs, GA etc) core skills: demonstration on a manikin with discussion using checklists: pre- anaesthesia check, basic airway and laryngeal mask, intubation with rapid sequence induction and spinal anaesthesia case based discussions on anaesthesia emergencies; high spinal, post- partum haemorrhage, hypoxia under GA and regurgitation under ketamine anaesthesia. Directly observed clinical practice could not be incorporated due to inconsistent OT activity levels across sites. Qualitative assessments included AA confidence, challenges and barriers to performance and supervisor s assessment of the AA. Themes were selected for reporting and further analysis. Specific sources for the FEP tool were: Other NSI health- worker FEP tools Expert panel of AA trainers, overseas anaesthetists, NSI trainers and stakeholders (CEOC) by forum meeting and e- discussion current 12 month AA curriculum, AA refresher course, Basic Anaesthesia Manual ( six month textbook - the core training text for the majority of participants) previous AA surveys as detailed above International Anaesthesia standards: WFSA and WHO guidelines for anaesthetic services at district hospitals [refs] UK Workplace based assessments for core anaesthetic training The tool was developed, discussed and refined at a stakeholder meeting and piloted in two hospitals (with three AAs); a senior AA trainer Dr Resham Rana took part in the pilot. The pilot informed adjustments to the FEP tool including reducing the number of questions in the knowledge test, refining questions for clarity, reducing the number of skill assessments to four and replacing anaphylaxis with regurgitation as the fourth case- based discussion. The enabling environment questionnaire was based on the AAC curriculum, World Federation of Societies of Anaesthesia and WHO standards for facility, equipment, drugs and supplies for level one and level two district hospitals (CS capable). An assessment of pulse oximeter availability, knowledge of hypoxia and awareness of the WHO Safe Surgery Checklist was incorporated on behalf of Lifebox ( FEP Equipment was supplied by NSI. The proposed CPD tools were developed at NSI and the video library compiled by the lead FEP anaesthesiologist and colleagues at University Hospital, Southampton, UK these were in part filmed at Tansen Mission Hospital.

10 10 FEP visit structure Each visit commenced with introductions to theatre staff and hospital management and permission to complete the FEP was sought from the hospital director. This was followed by completion of the FEP tool with individual AA s. Each visit was spread over a minimum of two days to allow the development of a relationship with the AA and to enable a more comprehensive assessment at each facility. Individual AA assessment with coaching work was completed in approximately six hours. All assessments and coaching were delivered in either Nepali or English with Nepali translation. Throughout the creation and delivery of the assessment and coaching tools, adequate comprehension was confirmed. This ensured the assessments were a test of knowledge and skill, not English comprehension. Data analysis: Data were collected in written form in each FEP book, and then entered into Excel and SPSS for analysis. OVERALL FINDINGS Demographics: 44 AAs working in 21 hospitals across 18 districts completed the FEP. Chart 1 - Type of hospitals visited Chart 2 AA cadre levels (HA health assistant (highest non- nursing cadre), SN staff nurse, AHW auxiliary health worker, CMA community medical assistant, ANM Auxiliary Nurse Midwife)

11 11 Chart 3 AA original training site The AAs were a diverse group of health- workers with a broad variety of anaesthetic experience ( years). They ranged from an isolated AA with little or no available anaesthesia work, to isolated AAs in a busy zonal hospital with a high case load to experienced and well supported AAs in busy AA training hospitals. Irrespective of their level of anaesthesia experience, the AAs were key health- workers at all the hospitals visited, not only in the OT. This was most evident in district hospitals where their primary work commitment was as their main cadre (HA, CMA, SN) on the ward or outpatients, though this rarely limited their availability for anaesthesia. In zonal hospitals, in addition to providing essential anaesthesia care for large numbers of emergency and elective cases, the AAs were frequently relied upon for non- surgical emergencies : for example newborn resuscitation, trauma and prolonged respiratory support for snake bites. Most AAs had received no training for this work.

12 12 Annual anaesthesia caseloads Table 1 shows the total number of theatre cases per year at each facility. Data was retrieved from the OT record book for a three month period then extrapolated to provide data for one year (possible seasonal variations not included). Large central Kathmandu valley hospitals are not included here, as AAs rarely performed anaesthesia. Table 1 Theatre cases per facility (Caesarean section=cs) No. of non- % CS of doctor Annual number of total cases C/ Name of Facility (East to anaesthesia Cases Annual Annual Ca West) providers (major/intermediate) number CS Case per AA pe Koshi Zonal Hospital Okhaldhunga Sagarmatha Zonal Hospital Bhaktapur Hospital Bharatpur Hospital Janakpur Zonal Hospital Dhaulagiri Zonal Hospital Tamghas Hospital Lamjung Community 58 District Hospital Gorkha Hospital Tansen Mission Hospital Bheri Zonal Hospital Gulariya District Hospital Dadeldhura District Hospital HDCS Hospital Dadeldhura Seti Zonal Hospital Mahakali Zonal Hospital No personal logbooks were kept so individual data was extrapolated from facility case numbers. There were no anaesthesia critical events or mortality records, and perioperative mortality from the Annual Report reflects a multitude of factors in addition to anaesthesia so is not reported here. In the above table, the hospitals highlighted as examples are government district hospital (Gorkha, Tamghas and Gulairya) and government zonal hospitals (Janakpur, Seti and Sagarmatha) - none of these six hospitals have anaesthesiologists present.

13 13 In the busiest hospitals (Tansen Mission Hospital, Bharatpur District Hospital, Janakpur Zonal Hospital), AAs were doing approximately 500 cases per year. In quieter places with intermittent GP surgeon availability, few to no cases were done. The most common significant operations were CS, appendectomy and closed fracture reductions. Spinals and ketamine intravenous anesthesia (KIVA) formed the bulk of anaesthesia, even in centres with GA facility. Regional anaesthetic blocks were commonly practiced. These techniques are cheap, quick and avoid more complex anaesthesia. Most AAs were self- taught or taught by more experienced AA colleagues. 61% of AAs anaesthetise children under 5 years but only 9% have received any training in paediatric anaesthesia. Regular pre- anaesthesia assessment was infrequently performed. Predominantly this was due to habit, but poor communication with the AA, unavailability of anaesthesia charts and surgeons denying time to perform full checks were also contributory factors. Pre- anaesthesia assessment was assessed as a core skill during FEP (see below). Cases described by AAs Many cases described were life- saving events of very sick patients, mostly obstetric (uterine rupture, PPH, very prolonged labour) or the successful management of anaesthetic emergencies. Inevitably most of these related to spinal complications, a reflection of their workload. Obstetric complications and deaths were described; one busy zonal in the Eastern region saw a high volume of very sick obstetric cases with significant mortality on the wards reported. Anaesthesia complications described included difficult life- threatening airway events both in the OT and presenting as emergencies at the hospital. Emergency airway management is a core anaesthesia skill yet many AAs felt under prepared for this clinical scenario. Almost all AAs working in a CEOC site were regularly engaged in newborn resuscitation yet few of them have received any training in this. This was due to shortages of OT staff and skilled neonatal resuscitation providers. Though not formally assessed, modern protocols were generally not followed.

14 14 Knowledge, skills and case- based discussions All Scores shown are for 43 AAs; the AA at the pilot site (Gorkha District Hospital) was excluded as the knowledge and skill assessments were adjusted after the pilot. For the skills and case discussions, all AAs were coached to the same standard of competence; hence, all post- coach skill scores were 100%. Some AAs took longer to coach to this standard- this was not quantified for this FEP. Scores for all AAs Graph 1 Mean scores of all AAs for the nine core knowledge and skills tested pre- coaching are shown. In general, theoretical knowledge was good with a few consistent knowledge gaps across all AAs (e.g. modern CPR ratios). The mean pre- coaching knowledge test score was 33/40 (83%) (range 28-39). The mean post- coaching knowledge score was 39/40 (98%) (range 33-40). Skills scores were also good with adequate skills in basic airway techniques and intubation with rapid sequence induction. Spinal technical skill and knowledge was very good, demonstrating that regular use of a skill leads to practical competence.

15 15 Graph 2 Box plot for key knowledge and skill assessments (showing medians, interquartile ranges, minimum and maximum and outliers) There were exceptions with some AAs in quiet facilities with good skills and other AAs in busy places with less good skills. Overall, technical skills were better than emergency management skills (eg knowing how to deal with complications and unexpected problems such as hypoxia, total spinal) with consistently lower mean scores seen for the emergency case discussions and a greater range in knowledge.(see graphs) Of note all AAs work under the supervision of a doctor, providing a potential source of advice and help. In addition many work with other AA colleagues who also provide support. How these scenarios would be managed in real situations would be almost impossible to assess in this environment. However the reduced ability of many AAs to manage the scenarios presented to them indicates that a structured approach to emergencies needs to be taught and embedded in practice (see below) Defining a level of overall competence for individual AAs Graph 3 shows a scatter plot for each AA s mean score for all nine AA scores; an arbitrary competency line is set at 70%. Most AAs were above this level. Graph 3 summated skill scores for individual AAs

16 16 Using 70% as the cut- off, Graph 4 shows the percentage of AAs scoring above this value for each of the nine skills. It demonstrates clear gaps in clinical knowledge, such as hypoxia and anaesthetic preparation. This highlights areas for intervention, support and further training, and for feedback to core AA training sites. Graph 4 Percentage of AAs achieving 70% or greater in the core skills Analysis of the individual skills highlights areas of excellence and weakness for all but this information can be used to tailor additional training and intervention. (see below) eg. for rapid sequence induction and intubation, difficulty with intubation can be addressed with algorithms and further training.

17 17 Graph 5 - Key steps for RSI During the rapid sequence induction skills station, preparation of equipment and pre- oxygenation were done well. Management of a difficult view on laryngoscopy was performed less well. Overall, steps for spinal anesthesia were performed well by the majority of AAs assessed. Areas for improvement for some participants included post spinal care and the use of a circular skin cleaning technique. Graph 6 steps for spinal anesthesia (gaps in knowledge)

18 18 Analysis of factors affecting skills A cross- tab analysis was performed to assess variables which may affect AA skill scores. Original cadre, number of cases and current GA availability made no difference to the skills score. Years of experience (on the x axis in the two graphs below) showed a positive trend across skills, perhaps outweighing other factors. Attendance at AA Refresher Training appeared to positively affect certain skills (knowledge test, airway, intubation, spinal complication). Graph 7 Graph 8 In summary, Most AAs were above a 70% competence cut- off. Of the skills assessed, spinal anaesthesia was performed the best, airway management was adequate but decision making and emergency management were poor. Experience seems to influence skills.

19 19 Coaching A key output of the FEP visit was the onsite coaching delivered by senior experienced anaesthesiologists. All AAs were coached until competence in knowledge and skills was demonstrated. This core part of the FEP ensures the visit leaves the AA more confident and competent, and elevates FEP above a simple assessment visit. Few needed much coaching to achieve the required standard. The coaching was highly rated by all AAs and co- workers during their feedback of the FEP process. Enabling environment - equipment, drugs, facility Each hospital was assessed against a WHO basic standard for anesthesia for small rural and district hospitals with CS capability as described above. (see appendix) None of the hospitals visited achieved all of the standards and half the hospitals achieved less than 80% of the standard. The deficit was greatest in government district and zonal hospitals. The missionary hospitals generally had the best facilities. Graph 9 Percentage of hospitals reaching WHO facility score Four government district hospitals and three zonal hospitals had no functional GA service. A further zonal hospital had very outdated GA equipment (ether EMO). Some facilities, including some government hospitals, had enough equipment to provide a good general anaesthesia service; however, many others were both poor and unsafe. Zonal hospitals were found with a high turnover of cases but inadequate equipment and drugs, and poor hospital support. In one such hospital, any anesthesia was provided only for CS. Other emergency or elective surgeries rely on

20 20 the patient s family to buy the equipment and drugs required. On the days of the FEP visit, these were not available in any pharmacy close to the hospital. At this zonal hospital, essential drugs such as ketamine for simple procedures for children were unavailable. Occasionally a district hospital had appropriate facilities for the surgery undertaken. This was due to good leadership and planning on behalf of the doctors, AA and hospital management. Drug availability was mostly adequate for basic anaesthesia. GA drug provision was lacking even in hospitals with an anaesthesia machine.anaesthesia machines were inconsistent; examples of poor facility included out dated equipment (ether anaesthesia via EMO), a new machine never used as staff felt it was unsafe despite regular servicing by a government appointed contractor and a CEOC site/zonal hospital with no GA equipment at all. Graph 10 GA readiness : a sample of some of the equipment and drugs required for CS under GA. (% availability at all hospitals shown) Post- operative recovery was virtually non- existent at district and zonal hospitals; space and staff were the primary limitations to this. This is a significant safety concern, particularly for sick mothers and busy hospitals. Several hospitals across the country reported spinal bupivacaine failure despite good spinal technique; there is no case documentation available but this is likely to be a drug quality problem.

21 21 Pulse oximetry Of the hospitals visited, basic provision was good with only one hospital (a Government district hospital) having none available. This reflects government provision in response to recommendations made. Graph 11 shows percentage of hospitals with a functional oximeter in each area. Graph 11 Pulse oximeter availability by clinical area Many hospitals had two or more oximeters: Graph 12 - Availability of pulse oximeters

22 22 Knowledge associated with oximeter use was generally good. Case discussion scores on hypoxia were lower, consistent with the general pattern for emergency scenarios. Graph 13 Mean scores for hypoxia related assessments Operating theatre protocols and staff Knowledge of the WHO Safe Surgery Checklist was very low amongst AAs: 95% were unaware of it; only 39% of supervisors were unaware of it but none were using it. Only one central hospital was using a similar checklist. Trained OT staff numbers are low (not formally assessed in FEP). This potentially leads to reduced patient safety and throughput in the OT. Workloads, leadership and enabling environments varied enormously between hospitals. These subjectively affected AA confidence and actual or perceived barriers to safe anaesthesia. There was marked variety across zonal and district hospitals. The latter were frequently affected by reduced patient numbers and inconsistent staffing eg. MDGPs on government rotations. Both motivation and leadership varied across institutions. Certain hospitals functioned well; often this was due to leadership from an individual or the maintenance of a cohesive OT team for a period of time.

23 23 Enabling environment: Motivation: Education/support/confidence Graph 14 - perceived barriers (% of AAs reporting each barrier) AAs are providing anaesthesia for complex emergency cases without appropriate senior support (MDGP/surgeon notwithstanding). Over half of AAs (52%) felt a lack of supervision was a barrier to their anaesthesia practice AAs often work in isolation, both educationally and or physically. Some feel vulnerable though others, particularly in well- run hospitals, feel well supported. Supervisor anesthesia skills were not tested but their training or experience was not high. There were very few anaesthesiologists in the hospitals visited and they were themselves isolated. Graph 15 AA confidence

24 24 Confidence levels reflected AA practice but were generally good. Some remained motivated despite minimal support from supervisors or hospital management. Many AA s have felt neglected since completing their training and were keen to engage in follow- up coaching. They are an accessible workforce- relatively few in number and located in urban sites. Modern educational facilities are limited, 30% have no access to a computer and 48% no access to the internet. 61% have no textbook other than their basic training manual. There were examples of considerable motivation and self- learning, both in AA teams and individually, often when working with a supportive GP doctor leading the hospital. Box 1 - Some personal responses: AA challenges and frustrations AAs are not recognized as important members of the OT team AAs are not valued in this hospital ; surgeons are always in a hurry and this is Nepal (on the poor conditions in one zonal hospital). What is the best thing about anesthesia here? Saving the life of two patients (mother and child) in a good safe environment (zonal hospital AA), lots of involvement in anaesthesia (zonal hospital AA); it s unpleasant to be a doctor here without AAs (medical superintendent, mission hospital) Why is surgery/anaesthesia good here? consistent team of OT, nurse in charge, coo- operative doctors, management and AAs over years (Nepalgunj) team approach and full involvement in anaesthesia (Nepalgunj) more cases means more practice (Maternity) teaching others helps my own learning (Bharatpur) FEP process evaluation These are busy health- workers with limited time but there was almost universally positive feedback regarding the FEP from participants and most supervisors. The time scale for FEP was realistic, though short. The major advantage of FEP, apart from data acquisition, was immediate onsite coaching. Deficits in knowledge or skills were addressed immediately, improving confidence and hopefully clinical delivery. The major limitation in assessment was the use of classroom- based scenarios, not clinical evaluations and coaching. This was due to inconsistent numbers and types of

25 25 clinical cases across the hospitals. The addition of clinical case exposure, assessment of supervisor anaesthetic skills and observation of the running of OT would be a great advantage. The FEP tool needs some revision to aid comprehension, ease of data entry and subsequent data analysis. Discussion and recommendations AA presence and effectiveness The study confirmed that AAs remain the backbone of anaesthetic provision in government and mission hospitals across Nepal. CEOC services would not exist at district and most zonal hospitals without them. AAs additionally provide vital assistance across the hospital with emergencies, including newborn resuscitation. Their surgical caseload can be high and the cases difficult. There are very few anaesthesiologists working in government hospitals and none at district level. AAs feel unsupported and vulnerable. Barriers to safe delivery and confidence have been documented. Key amongst these are the lack of a supportive enabled environment and AA recognition. Recommendation One: Acknowledge the vital service provided by AAs across Nepal by ensuring: Continuous AAC courses and sustained AA follow- up AA incentives to encourage AA recruitment, retention and respect AA professional registration and government posts as AAs AA competence AA FEP was an effective and useful tool for the evaluation and coaching of AAs. AAs were found to be competent in anaesthesia required for Caesarean section and significant surgery at their hospital. GA skills were less good, though for the majority adequate; few required much coaching to bring them to the required standard. Analysis for each of the nine skills shows clear gaps in knowledge, such as emergency management and anaesthetic preparation. This would not be unique to an AA but rather an anaesthetic provider with a small caseload who is isolated from professional support and training updates. The data from this FEP identifies key areas for intervention, support and further training. This is also essential feedback to core AA training sites and course developers.

26 26 AAs with greater experience performed better in the skills; there was no link between numbers of cases and skills, indicating that it takes time to acquire a high level of competence. Additionally, the more experienced AAs in the FEP generally worked in mission hospitals and as such they work in a highly functional, well supported and often very busy OT and hospital. They perform a wide variety of cases with regular GAs and the hospital is a long term training site providing a steady, though not continuous, input of anaesthesiologists These factors are incorporated into the model for maintenance of competence shown below. From this FEP, we have defined key components required for the support of AA competence. Figure 1 Key components for AA competence Core training There is now an established 12- month AA training programme at six AA training sites. This is run by NAMS and is NHTC approved. Enough cases and experience

27 27 An AA in a busy hospital will do approximately 500 major or intermediate cases, plus minors (under ketamine). AAs additionally provide vital assistance across the hospital with emergencies. Logbooks are essential to document personal case load. For the less busy facilities, caseloads can be increased by occasional attendance at busy hospitals and camps. A few AAs have arranged such additional experience but this requires support from their base hospital. The number of GAs should not be taken as an acid test of an AA s experience but there is no doubt that performing GAs enhances confidence and skill. In addition a significant planned surgery (elective) case load enhances confidence and knowledge and improves OT team performance. Enabled environment This is equipment, drugs, adequate numbers of trained co- workers, a functional OT team and good leadership- (see below). Supervision This needs to be both local and distant. Local support in the form of a supportive surgeon, supervisor, AA colleagues and hospital management provides an invaluable network. Informal mechanisms for individual case review exist but these should be formalised to ensure quality and sustainability. The proposed CPD model below provides a model for continued support. Supervisors can themselves feel frustrated and isolated. They need to be encouraged and provided with the tools to support safe anaesthesia in their hospital. This may include short refresher training in anaesthesia. Continuous learning and mentoring (Continuous professional development (CPD)) AAs have no structured continuous learning or support, highlighted in the literature as essential to maintain skills and confidence. Many AAs have felt neglected since completing their training and were keen to engage in follow- up coaching. They are an accessible workforce- relatively few in number and located in urban sites Recommendation two a. acknowledge and support the key elements to maintain AA competence b. support the development and delivery of the above CPD model We propose a model of AA CPD support that acknowledges the findings of this FEP. The model incorporates self- learning and constant interaction with mentors, uses tested educational programmes linked with core AA training, and includes

28 28 algorithms and guidelines specifically designed for their environments and learning needs. In combination these could provide a workable comprehensive structure of support for AAs. Figure 2 A model for CPD (two year model) Entry to the programme (red arrow) is a current working AA or graduation from the 12 month AA course. Two- way communication is an essential element to this model. A faculty of mentors, based on the AAC trainers with support from outside anaesthesiologists, must first be established To begin the process, the AA will be sent a personal logbook, anaesthesia emergency algorithms, WHO Safe surgery checklist and other checklists to be used at their facility. To demonstrate engagement with the CPD process, AAs will first be asked to complete the logbooks followed by sequential donations of educational material.

29 29 Over time, with regular engagement and communication with mentors, a library of educational materials and regular educational practice, with clinical case logbook recording will be built up at each hospital and for each AA. Regular onsite FEP visits (each AA visited every 2-3 years) with assessment and coaching at their own place of work will take place. Each AA can additionally attend a three day educational refresher course (each AA attends every 2-3 years) to improve knowledge, clinical skills and emergency scenario management. Regular attendance at busier hospitals may be incorporated into refresher training Cell phone communication (SMS reminders and structured follow- up calls) can also be included. Fourteen new Critical event algorithms have been created specifically for the AA programme following the FEP: these are all based on an ABC approach and include high spinal, failed intubation and hypotension (see appendix). The WHO hypoxia and UK Resuscitation Council 2010 newborn life support algorithms will be used in addition. The key resources for the CPD model hence include:- Logbooks: personal and central clinical record and assessment tool (pilot logbook use underway) Standardised anaesthesia chart content Critical event algorithms: these have now been specifically created for the AA programme (see appendix) Checklists: WHO Safe Surgery Checklist plus AA curriculum checklists Case Scenario worksheets (to be created) Textbooks (handbooks, accessible via AAGBI, WFSA) DVDs including video library (forthcoming ela DVD, own Southampton Anaesthetic video library) Lifebox pulse oximeters : educational materials incorporated into CPD matrix and oximeters themselves as donations through this matrix Options for refresher courses include short 3 day courses for 20 AAs similar to ALSO or PTC courses (e.g. such a course exists and can be used in Nepal - the AAGBI SAFE obstetric anaesthesia course) or the previously run two week AART course at an AAC site. This is an excellent resource but capacity to run this regularly would need to be significantly increased to ensure all AAs can attend. The enabling environment When assessed against a basic standard, this was not adequate in most hospitals. Capital outlay has addressed some absolute shortages, eg pulse oximeters, monitoring and GA equipment, but much of that GA equipment is not serviced

30 30 adequately, may be out of date or simply absent. Drug provision in many hospitals is unreliable and limited, only sufficient to support simple spinals for straightforward Caesarean section. The AA shortage is significant in many districts. In addition, an AA working alone cannot be expected to sustain a service or their own competence and confidence. The recent CEOC review recommends at least two AAs in each hospital, as well as ongoing support. Building a functional OT team is vital and can help balance workload, encourage retention, maintain motivation, and provide support and advice. Good leadership is recognised as integral to a functional hospital: the NSI RSSP model is one such model for the development of sustainable district hospital service. Hospital support for OT services has an impact on AAs. It is possible to maintain a safe professional environment in the government sector and there were a number of examples of good practice in such hospitals (Nepalgunj, Bharatpur, Baglung, Gorkha). Core elements to success appear to be sustained teamwork, adequate facility and drugs, sufficient numbers of trained and motivated staff, enough cases and good leadership in place for a significant length of time. Learning from excellent centres and working with new WHO global initiatives (e.g. WHO Safe Surgery checklist) may provide a way forward to developing safe, effective OT services. Some AAs were seen to take personal responsibility for their work environment with occasional excellent examples (Gorkha). However, this is understandably difficult in the face of disinterested or obstructive management. The lack of AA recognition as a professional cadre places them in a weak and vulnerable position, unable to change things for the better or even advocate for change.

31 31 Recommendation three Urgent correction of the factors that inhibit delivery of safe anaesthesia ( e.g. drug and equipment supply) at district and zonal hospitals. Recommendation four Build a sustained functional surgical and anaesthetic team at each hospital a. support all staff required for effective, safe surgical and anaesthetic provision at district hospitals support learning from examples of best OT practice around Nepal and incorporate global safe surgery initiatives eg the WHO Safe Surgery Checklist c. recognition of AAs as a professional cadre Recommendation five There is a need for a greater focus on safe surgery and anaesthesia in national guidelines in national policy, guidelines and data collection. Summary The AA FEP was an effective and useful tool for the evaluation and coaching of AAs. The study confirmed that AAs form the backbone of anaesthetic provision in government and mission hospitals across Nepal. There are very few anaesthesiologists working in government hospitals and none at district level. The caseload managed by the AAs can be high and the cases difficult. In general, AAs were found to be competent in anaesthesia required for Caesarean section and significant surgery at their hospital. Skills related to general anaesthesia and the management of anaesthetic emergencies were performed less well. Minimal coaching was required to bring AAs to the necessary standard in these skills. The enabling environment in most government hospitals was not adequate for general anaesthesia, with deficits in equipment and drug supply. A facility list has been created. Models of safe, effective surgical and anaesthetic practice exist but have not been developed sufficiently in most sites. AAs need further support and professional development to maintain competence. A model of the components of competence and a suggested model for continuous professional support are proposed.