Robert Englander, MD Laura Edgar, EdD Lois Margaret Nora, MD, JD, MBA

Transcription

1 Robert Englander, MD Laura Edgar, EdD Lois Margaret Nora, MD, JD, MBA

2 Competency-based Medical Education Robert Englander, MD, MPH March 27 th, 2015

3 Definitions CBME: An outcomes-based approach to the design, implementation, assessment, and evaluation of medical education programs, using an organizing framework of competencies. Domain of Competence: Broad distinuguishable areas of competence that in the aggregate constitute a general descriptive framework for a profession (e.g. the ACGME six domains of competence)

4 Definitions Competency: An observable ability of a health professional that integrates knowledge, skills, values, and attitudes. Milestone: A defined, observable marker of an individual s ability along a developmental continuum. (In the ACGME framework-a level of performance along a continuum for a given competency)

5 Competency-based Medical Education Acad. Med. 2002;77: ic Medicine Moving from: Fixed Time/Variable Outcome to Fixed Outcome/Variable Time Medical Education and Training

6 Step 1: Define the Competencies The Vision: Physicians will spend their careers, from premed to exit from practice, on a developmental trajectory building mastery in

7 Eight Domains of Competence Patient Care Interpersonal and Communication Skills Medical Knowledge Professionalism Interprofessional Collaboration The Physician Practice-based Personal & Learning & Professional Improvement Development Systems-based Practices The Complete Physician

8 Eight Domains of Competence Patient Care Interpersonal and Communication Skills Medical Knowledge Professionalism Interprofessional Collaboration* The Physician Practice-based Personal & Learning & Professional Improvement Development Systems-based Practices *From the Core Competencies for Interprofessional Collaboration The Complete Physician

9 Eight Domains of Competence Patient Care Interpersonal and Communication Skills Medical Knowledge Professionalism Interprofessional Collaboration Practice-based Learning & Improvement The Physician Systems-based Practices Personal & Professional Development* *From the Pediatrics Milestone Project The Complete Physician

10 Step 2: Define Performance Levels We must agree on the Milestones of competency development in each discipline. We must agree on and implement common evaluation tools in each discipline to document resident achievement of these milestones.

11 Step 3: Assessment of Competence

12 Step 3: Assessment of Competence O l ti t th d ti i t One solution to the reductionist versus Holistic dilemma:

13 EPAs in UME Core Entrustable Professional Activities for Entering Residency EPAs in GME OB/GYN (Netherlands) Psychiatry (New Zealand/Australia) Pediatrics (USA) Internal Medicine (USA)

14 DOC C 2 M 1 M 2 M 1 M C 3 M 2 EPA DOC C 1 M 1 M 2 M 1 M 2 C M 1 C 4 EPA: Entrustable Professional Activity DOC: Domain of Competence C: Competency M: Milestone DOC C 2 C 5 M 1 M 2 M 1 M 2

15 M 1 M 2 M 1 M 2 Narrative description of a pre-entrustable learner M 1 M 2 M 1 M 2 M 1 M 2 Narrative description of the entrustable learner M 1 M 2

16 Step 4: Program Evaluation H ill k th t th di hift t How will we know that the paradigm shift to CBME was a success?

17 Accreditation Council for Graduate Medical Education Milestones In Practice Laura Edgar, EdD, CAE Executive Director, Milestone Development ACGME

18 Disclosures Dr. Edgar is a full-time employee of ACGME 2012 Accreditation Council for Graduate Medical Education (ACGME)

19 Presentation Topics Milestone Background and Purpose Evaluation and Reporting What s Next? 19

20 Milestones: What Milestones describe performance levels residents are expected to demonstrate for skills, knowledge, and behaviors in the six competency domains. Milestones will lay out a framework of observable behaviors and other attributes associated with residents development as physicians. In the next accreditation system, aggregate resident performance on the milestone level l will be used as one indicator of a program s educational effectiveness. Nasca, TJ et. Al. The Next Accreditation System :

21 Milestones: Why Fulfill the promise of the Outcome Project Increased use of educational outcome data in accreditation ACGME accountability to public Support the educational process

22 Milestones: Who More than 600 volunteers worked on this project over the last three and a half years Representatives from ACGME, Certification i Boards, Colleges and Academies, Program Director Groups, Residents and Fellows

23 Uses and Implications ACGME Accreditation i continuous monitoring i of programs; lengthening of site visit cycles Public Accountability report at a national level on competency outcomes Community of practice for evaluation and research, with focus on continuous improvement Residency Programs Guide curriculum development More explicit expectations of residents Support better assessment Enhanced opportunities for early identification of under-performers Milestones Certification Boards Potential use ascertain whether individuals have demonstrated qualifications needed to sit for Board exams Residents Increased transparency of performance requirements Encourage resident self-assessment and self-directed learning Better feedback to residents 23

24 Milestone Template 24

25 Milestones are progressive over time. There is no prescribed speed at which residents must complete a milestone set. Selecting a response box in the middle of a level implies that milestones in that level and in lower levels have been substantially demonstrated. Selecting a response box on the line in between levels indicates that milestones in lower levels have been substantially demonstrated as well as some milestones in the higher level(s). Optionto to select Not yet achieved Level 1

26 By the numbers The number of reportable milestone sets ranges from Several specialties have other milestone sets that are not reportable to the ACGME but can be used for remediation or other curricular purposes

27 What the Milestones Are An attempt to define explicit, practical, relevant, and manageable set of domains of clinical competence An attempt to describe levels of competency development during training, leading to unsupervised practice of medicine A way to better inform trainees of some of the expectations of training i

28 What the Milestones are Not Not everything a graduating resident needs to know or be able to do Not a mandate for specific rotations or durations of experience N l f l Not a replacement for regular assessment and evaluation

29 Milestones: Evaluation and Reporting Evaluation completed by the Clinical Competency Committee All programs within a specialty use the specialty s milestones Programs will report semi-annually Milestone data will be reported to ACGME through direct entry into ADS

30 Milestones: What s Next? Milestones v2.0 General Competencies: Best of the Best Currently working on PBLI More than 650 individual rows of text Organizing by key words and concepts 30

31 Accreditation Council for Graduate Medical Education Questions? 31

32 Accreditation Council for Graduate Medical Education Thank you! 32

33 More Information Frequently Asked Questions: pdf Comments and Questions can be sent to: 33

34 Competency Across A Professional Career: The Contributions of Continuing Board Certification Alliance of Independent Academic Medical Centers 27 March 2015 Lois Margaret Nora, MD, JD, MBA

35 Disclosures» Salaried employee, American Board of Medical Specialties ABMS Board Certification and Maintenance of Certification, are programs of the ABMS and its 24 Member Boards.» Member, Council on Graduate Medial Education (COGME)» Advisory Board - Sam s Club Healthy Living i Made Simple Magazine» Advisory Board, National Center for Interprofessional Practices and Education 35

36 Outline» Professional Self-regulation of physicians» ABMS Board Certification Initial certification and impact on GME Continuing certification and impact on career-long assessment, learning, and improvement» Sample Innovations and a Request for Engagement 36

37 Characteristics of a Profession - Society s Trust is Fundamental» Special knowledge and skills acquired by certain members of society and not others; often with the substantial investment of society in the educational process» The commitment of the profession s members to, and the trust by the other members of society that, the special knowledge and skills will be used not in the professional s self-interest for the good of society and its members» Trusting in that commitment, Society grants the profession substantial bt til autonomy to determine dt educational standards, d self-assess, and to self-regulate 37

38 American Board of Medical Specialties» ABMS Board Certification originated early 1900s» Umbrella organization of 24 Member Boards; high standards for inclusion» ABMS Member Boards are independent organizations» ABMS Board of Directors includes at-large members and members of the public» The mission of the American Board of Medical Specialties (ABMS) is to serve the public and the medical profession by improving the quality of health care through setting professional standards for lifelong certification in partnership with Member Boards. 38

39 ABMS Member Boards 24 ABMS Member Boards 37 specialties and 123 Sub-specialties Allergy and Immunology Orthopaedic Surgery Anesthesiology Otolaryngology Colon and Rectal Surgery Pathology Dermatology Pediatrics Emergency Medicine Physical Medicine and Family Medicine Internal Medicine Medical Genetics and Genomics Neurological Surgery Nuclear Medicine Rehabilitation Plastic Surgery Preventive Medicine Psychiatry and Neurology Radiology Surgery Obstetrics and Gynecology Ophthalmology Thoracic Surgery Urology 39

40 ABMS Board Certification» ABMS Board Certification Initial Continuing (Maintenance of Certification)» Initial Board Certification Meet all requirements for licensure Requires completion of an extended high-quality period of training and assessment in knowledge, skills, and professionalism (usually ACGMEapproved residency/fellowship) Board standards impact the curriculum and conduct of residency education Complete additional assessments of knowledge, clinical skills, and professionalism specific to the discipline Board standards impact what is assessed and how it is assessed 40

41 ABMS Member Board Engagement in the UME and GME Components of the Continuum (examples)» ACGME/ABMS Core Competencies development involved all ABMS Member Boards» American Board of Pediatrics Entrustable Professional Activities Education in Pediatrics Across the Continuum Project R. Englander, AAMC (Lead) Josiah Macy Foundation grant GME Program ram reports to ABPeds on resident performance annually 41

42 ABMS Member Board Engagement in the UME and GME Components of the Continuum (examples)» American Board of Family Medicine MOC begins during residency: completion of one Selfassessment/Cognitive Simulation and completion of one Improvement in Medical Practice activity during residency; certification examination given during residency» American Board of Surgery Work with the broader surgical community for Surgical Resident Preparation courses in UME Requirements for certain activities prior to eligibility for the exam have impacted GME Principles of Valid, Affordable, and Easily Accessible ACLS, ATLS, Laparoscopic Skills 42

43 Why Transition from Diploma-style Board Certification to ABMS Continuing i Certification?» Public expectations; fulfilling the public trust» Substantial change in knowledge, procedures, society over time» People don t self-assess particularly well» Skills can diminish over time» Safety concerns in healthcare settings» Practice in other high-consequence industries 43

44 International Movement Toward Continuing i Professional Self-regulation lti» Renewal of licensure FSMB and state medical boards considering greater expectations for maintenance of licensure (MOL)» Continuing Certification ABMS Programs for MOC AOA Programs for COC» International UK Revalidation Program Canada RCPSC MOC Program Australia Revalidation i 44

45 Goals of Continuing Certification» Improve the quality of care provided to patients and communities; Improve health outcomes» Meaningful professional self-regulation» Provide a systematic, rigorous, relevant method for on-going g professional learning and assessment in knowledge, skills, judgment, and professionalism Trusted by the public Believed in by the profession 45

46 ABMS - Movement to Continuing Certification» 1970s: Periodic recertification» 2006: Entire ABMS Boards Community moved toward program for continuing certification (Maintenance of Certification; MOC)» : In-depth review of MOC; development of new program standards» 2015: Program Standards in place as of January 1 46

47 ABMS Maintenance of Certification» Must have achieved Initial ABMS Board Certification» Career-long process» Based in the six ABMS/ACGME Competencies Practice-based Learning and Improvement; Patient Care and Procedural Skills; Systems-based Practice; Medical Knowledge; Interpersonal and Communication Skills; and Professionalism» General Program Standards include ABMS Member Boards will work to increase the Program s value, relevance, and meaning and be sensitive to the time, administrative burden, and cost associated with participation. ABMS Member Boards will engage in CQI of their Program for MOC and engage in ABMS-wide review process. Diplomate and Public engagement in CQI and review processes 47

48 ABMS Program for MOC Standards» The Program is based in educational and quality science research» Four Integrated Elements Professionalism and Professional Standing Medical licensure; Actions against license; Route to recertification; Assessment and learning in professionalism Lifelong Learning and Self Assessment Expectation of relevance to specialty and area of physician practice; Emphasis on Patient Safety; Emphasis on Accredited CME Assessment of Knowledge, Judgment and Skills Expectation of board-derived assessment; Encourage innovation Improvement in Medical Practice At individual and/or system level 48

49 MOC - Select Examples of High Value-Added Activities iti from the Physician i Perspective» ABOG, ABEM, ABMGG Push key articles, practice guidelines with self-assessment tools to diplomates» ABMS Portfolio Program a program that aligns MOC with high quality, relevant, institutional based QI activities 36 active health system/hospital/specialty society sponsors Currently, more than 50 sponsor-applicants 21 ABMS Member Board participants 49

50 MOC - Select Examples of Emerging Innovation Pilots and Planning» Registries for Improvement in Medical Practice activities» Modifications in the Board-developed developed Examination Distributed secure testing Expanded access to materials during testing» Greater use of simulation in the MOC Process» Fundamental changes to examination approach 50

51 The MOCA Redesign Project Web-based based learning platform that supports personalized knowledge acquisition, assessment, and demonstration of proficiencies. Identifies knowledge strengths and gaps among diplomates as a group and individually using a longitudinal assessment approach. Optimizes diplomate learning experiences by providing customized feedback and learning recommendations. Connects diplomate to a universe of relevant continuing education content and learning opportunities. The Big Data created better informs CME providers about diplomate needs and future product development. The American Board of Anesthesiology 51

52 The American Board of Anesthesiology 52

53 My MOCA: MOCA Minute The American Board of Anesthesiology 53

54 MOC - Making a Difference» ABMS Evidence Library Evidence about the value of Board Certification Evidence underlying the methods used in MOC Developing evidence base related to MOC» MOC improving evidence-based testing Urology Urine creatinine» MOC improving physician learning, patient care and teaching» MOC impacting population health Pediatric screening and referral patterns Adult immunization ACP» MOC improving quality of care across health systems Mayo experience Harvard Business Review blog 54

55 Educators Wanted - Becoming More Involved» Expertise To inform debate To champion the philosophy p To contribute to, and improve, the methods» Participation Portfolio Program sponsors Product and Program Offerings» Research and scholarship 55