CONTINUING PROFESSIONAL DEVELOPMENT

What we heard Survey Continuum Physician Specialty LEADER Manager EVOLVING CanMEDS ENGAGE Competency MILESTONES Education Update Scholar Relevant Patient Safety Framework Feedback Collaborator Adapted Professional Medical Expert Faculty Development Communicator HEALTH ADVOCATE Revision Focus Groups Provide CARE STAKEHOLDERS CONTINUING PROFESSIONAL DEVELOPMENT Sharing the results of the CanMEDS 2015 Series I and II consultations

Contents Executive summary...3 1. About this report...6 2. Asking: How we heard from you...7 3. Listening: What you told us...11 4. Consolidating: What we ll do with what you said...15 Appendix: Engagement methodologies...18 Navigation tools: Click the section in the Contents to jump to that chapter. To return to the Contents click on the CanMEDS 2015 logo. Royal College of Physicians and Surgeons of Canada 774 Echo Drive Ottawa, ON K1S 5N8 Canada Toll free 1 800-668-3740 TEL 613-730-8177 FAX 613-730-8262 WEB royalcollege.ca EMAIL canmeds@royalcollege.ca Copyright 2014 by the Royal College of Physicians and Surgeons of Canada All rights reserved. This material may be reproduced in full for educational, personal, non-commercial purposes only, with attribution to the source as noted below. Written permission from the Royal College is required for all other uses, including What commercial we heard: use Sharing of the the CanMEDS results of illustrations the CanMEDS or 2015 its framework. Series I and II consultations 2

Executive summary The CanMEDS Physician Competency Framework Fundamentally, CanMEDS is an initiative to improve patient care by enhancing physician training. From its beginning in the 1990s, its main purpose has been to articulate a comprehensive definition of the competencies needed for all domains of medical practice and thus provide a comprehensive foundation for medical education. <We> are pleased to note many changes that we believe will benefit medical education and physicians in practice and their patients. Written submission The most difficult hurdle, given the scope of this competency framework, relates to its application. The framework will have to be made accessible and flexible for the various faculties and programs. The diversity of learning environments and governing authorities in medical education presents a major challenge. Written submission 3

CanMEDS 2015 Project When something is worth doing, it s worth doing it right. In 2012, a three-year plan was developed to update the CanMEDS Physician Competency Framework to ensure that it incorporates recent developments in medicine and remains responsive to the practical needs of medical educators. This, the third update to CanMEDS in the 20 years since it was first released, occurs in a special context. It is part of the Competence by Design initiative of the Royal College, a major multi-year project to implement an enhanced model for competency-based medical education in residency training and specialty practice in Canada. As part of the CanMEDS 2015 project, we engaged as many experts and partners as possible to ensure that the 2015 version of the framework is a valid and practical foundation for excellence in patient care now and in the future. In 2014, we released a series of drafts of the framework and milestones to give you, the stakeholders, the opportunity to influence the competencies as they developed. With each release, we conducted in-depth consultations with you to hear whether we are on the right track. Our members commend the RC for its leadership in advancing and developing milestones and think it could be a useful guide when designing curricula. Written submission This report Transparency is important to us. Over 1,000 people participated in our consultations in 2014. We feel that it s essential to report back on how you responded to the draft documents and how the feedback is shaping subsequent releases. This report summarizes the results of the comprehensive consultations that took place in the first six months of 2014 in response to the Series I and II drafts of the framework and milestones. It is being released at the same time as the Series III draft so that we can demonstrate that we have heard and considered numerous points of feedback, and enable participants to see the collective feedback of the CanMEDS community. Our process Our approach was to ask, listen, consolidate and ask again. We undertook three major engagement activities to generate feedback between February and July 2014: online surveys; web-based focus groups, including one with members of the public; and written submissions. Through the spring and summer of 2014, teams of educators and researchers reviewed and analyzed the data received from these consultations. It was then organized into a series of detailed internal reports for the CanMEDS 2015 Integration Committee, which used it to help create the Series III drafts of the framework and the milestones. 4

What we heard: the top five themes After analyzing the consultation data, we saw five clear recurring themes and identified the primary actions we need to take in response to each one: We are on the right track, but we are not done yet Continue to engage stakeholders through consultation; make changes that reflect the feedback; proceed with the new additions such as patient safety. CanMEDS competencies are good, but there is still room for improvement Clarify and streamline the competencies and milestones; eliminate jargon; make the resources practical. Manager vs. Leader: no clear consensus Build a better understanding of the options; be open to different perspectives. Milestones need to be realistic and relevant Clarify how competencies are organized within and between roles; consider reducing the number of milestones. CanMEDS 2015 requires a careful roll-out Plan for how the new milestones will be rolled out locally. Be practical. Help ensure that people on the ground have the knowledge, experience and support to implement CanMEDS 2015. Overall, I think the revised Framework is much clearer and easier to follow. I struggle with some of the terminology you use though. For example, why do you say inter and intraprofessional colleagues, try using just colleagues. Written submission Next steps After the Series III draft is released in September 2014, stakeholders will be encouraged to review it and to provide feedback by way of an online survey. This feedback will help shape the Series IV draft, which we will present for approval to our Royal College Council in February 2015. The final CanMEDS 2015 Framework will be launched simultaneously online and at the International Conference on Residency Education in October 2015 in Vancouver. CanMEDS 2015 project timeline 5

1. About this report The third iteration of the CanMEDS Physician Competency Framework, CanMEDS 2015, has been under development since late 2012. We launched the CanMEDS 2015 project with two key goals: 1. Update the CanMEDS Framework to further align with competency-based medical education. This update will ensure that the framework supports recent developments in medicine, anticipates and supports the practical needs of medical education programs, contributes to the strategic directions of medical education, and considers the practical implementation needs of our partners. 2. Identify competency milestones as a foundational initiative under the Royal College s Competence by Design multi-year transformational program. Milestones for each role will mark a physician s progression of competence from the start of residency training into practice through to retirement. Our process for developing the framework and the accompanying CanMEDS Milestones Guide involves our releasing a draft for stakeholder input and consultation, receiving and analyzing that input, and then using it to revise the subsequent draft, which is then in turn released for further input. In other words, feedback is integral to the process; the drafts created are your drafts. This report summarizes the feedback we received on the framework and milestones up to the end of June 2014. We ve already put that feedback to work, using it to shape the Series III drafts of the framework and milestones that will be released concurrently with this report and that will, we hope, generate the same powerful feedback that has informed this report. We gratefully acknowledge all of those who responded. Your thoughtful suggestions and incisive recommendations shared during the consultation process are greatly valued; your engagement is crucial to the CanMEDS 2015 iteration. Thank you! 6

2. Asking: how we heard from you Hearing all perspectives: the people and places Given how many people and organizations use and are affected by CanMEDS, a comprehensive and inclusive consultation process was essential to generate the feedback needed for the next revisions of the framework and milestones. We need to know that CanMEDS 2015 truly expresses and reflects the diversity of experiences and perspectives alive in the different geographic regions, practice settings and disciplines across the country. Our consultations connected us with the perceptions, thoughts and ideas of medical learners and medical educators of 17 Canadian medical schools, as well as with Royal College Fellows, family physicians, other health professionals and members of the public. Many participants have told us that a key strength of the CanMEDS Framework is the fact that its original design in the 1990s was informed in part by public input. Knowing this, we wanted to be sure that the 2015 update includes, again, the perspectives of the people who depend on Canadian physicians for their health care. Finally, because CanMEDS has been adapted in a number of international jurisdictions, we also looked to our international partners for their perspectives on the framework. What we did and why: the process Our goal was to design a consultation process that would engage all of our stakeholder groups and generate comprehensive feedback. We chose three channels for engagement: online surveys, web-based focus groups and written submissions. These different methods gave participants several options for sharing their feedback over many months. For participants to provide the practical, informed input we sought, we understood that they first needed to be briefed on the specific scope, goals and intended outcomes of the CanMEDS project. We provided this background information through a variety of communication channels, such as the Royal College website, regular online open-access webinars and in-person conversations with key stakeholder groups. 7

Engagement channel Quick description Number of participants National online surveys Two surveys for the Series I and Series II 901 releases respectively, eliciting specific recommendations for improvement Web-based focus groups Eight focus groups composed of participants 58 recruited from six different stakeholder groups to ensure feedback from direct users Written submissions Written commentaries solicited from 131 representatives of national institutions and partner organizations, epanel members and designated experts Total 1,090 I think milestones are a reframing of already existing guideposts, but it is helpful to explicitly state these things at a national level. Survey respondent This is an extremely well-put-together document that is easy to follow and understandable. I find extremely gratifying to see the extent to which patient safety and the domains of the Safety Competencies have been interwoven throughout this document. Written submission 8

National online surveys We conducted an online survey twice in 2014, the first in February, responding to Series I, and the second in May, responding to Series II. Both times the survey was conducted in English and French and was open for an average of six weeks. We promoted the survey to all stakeholders by direct email, the Royal College s corporate publications and social media. The survey questions were designed so that participants could give us specific direction on what should change in the next round of revisions. Web-based focus groups In March and April, we held eight web-based focus groups in English and French. We recruited participants strategically from five key stakeholder groups (see Figure 2) and from each of the 17 medical schools to ensure that we would be able to hear from the people who are most affected, either directly or indirectly, by the framework. The focus groups were designed to encourage informed exploration in a safe environment. We wanted participants to interact, ask questions, share their comments and generate new ideas. Figure 1 2% Figure 2 9% 12% 10% Survey participants Series I and II surveys combined 12% 14% Focus group participants 27% 79% 20% 14% Practising physician Resident/Medical student Other Retired physician Medical learners Program directors Members of the public National Specialty Society reps and Fellows Health professionals Deans (Undergraduate medical education, postgraduate medical education, continuing professional development) 9

Written submissions In February and again in May, we invited representatives of key national institutions and partner organizations to submit written commentaries on the Series I and II draft revisions respectively. We asked them to identify the strengths, weaknesses and practicality of the revised framework, as well as new challenges and new opportunities that require consideration. Written commentaries were also solicited from our CanMEDS 2015 epanel members. The epanels were created so that interested Fellows, residents and stakeholders could review and comment on the 2015 framework in a practical and meaningful way. Comments from this group were, by design, rolespecific, and contained very detailed suggestions for modifying the framework. Analyzing what we heard Over the course of six months, over 1,000 people responded to our requests for feedback. Together, they generated more than two and half thousand comments on the revisions to the draft framework and milestones. In the spring and summer of 2014, teams of educators and researchers carefully reviewed and analyzed the information received from the surveys, focus groups and written submissions. The data were then organized into a series of reports for the CanMEDS 2015 Integration Committee, which is responsible for generating the Series III drafts of the framework and the milestones. Please read on for a high-level overview of the comments and feedback we collected, as well as details of what we will do as a result of the feedback. In May and June, we also contacted designated reviewers, people who are considered experts in their respective fields. We asked them to use the lens of their expertise to identify gaps or potential flaws in the framework design or content. Examples of the perspectives these experts brought to bear on the framework include those of undergraduate and continuing professional development, credentialing, and international medical graduates. For more detailed information on the three engagement channels, please see Appendix. 10

3. Listening: what you told us This section describes what we learned from participants about the strengths, weaknesses and opportunities of the Series I and II drafts. Your feedback to us, which is drawn from the quantitative and qualitative data we collected in the first six months of 2014, has been organized under five recurring key themes: Theme 1: We are on the right track, but we are not done yet Theme 2: CanMEDS competencies are good, but there is still room for improvement Theme 3: Manager vs. Leader: no clear consensus Theme 4: Milestones need to be realistic and relevant Theme 5: CanMEDS 2015 requires a careful roll-out I think this looks very good. As an incoming program director, my biggest concern is one of implementation (i.e., how will this work in the real world). There are a number of challenges including how to rationalize this with rotations, service needs, etc. Written submission The Manager vs. Leader is going to be contentious. It is my general experience that when physicians become leaders, if they have limited leader skills, they tend to default back to being a manager. Written submission 11

We are on the right track, but we are not done yet Most of the participants in one or more of the three consultation activities in 2014 responded positively to the drafts they reviewed. Respondents generally felt that the revisions improved the current 2005 framework. The new content areas that have been incorporated into the framework were positively received. These include new competencies around patient safety and quality improvement, intraprofessionalism, and ehealth. Respondents also expressed satisfaction that they were involved in the process, and they told us they were reassured that their own comments from earlier consultations had been incorporated into subsequent drafts. They encouraged us to continue to engage stakeholders in the revision process. CanMEDS competencies are good, but there is still room for improvement Although respondents were generally pleased with the content of the draft framework and milestones, we still received many suggestions on ways to clarify, streamline and improve the documents. One message was loud and clear: please finetune words and concepts to make the content accessible and practical for those people who use the standards daily. Many respondents asked us to reduce or eliminate the medical education jargon that still appears in various places throughout the documents. Respondents also pointed out confusing terminology, asking that it be replaced with plainer language, as well as inconsistencies in terminology, advising us to address them. Some respondents told us that they continue to struggle with how competencies are organized within and between roles. Others felt that competencies are in the wrong place or that there is too much or too little overlap between roles. There were a few suggestions to add new content, and questions about where content can be found. For example, there were some recommendations to make cultural competence, history of medicine, technology, and Indigenous Health more prominent within the framework. There were a number of comments from participants who looked at the Competence Continuum Diagram, asking us to develop the Transition Out of Professional Practice Milestones. Thoughtful comments were made about the need to consider how the competencies and milestones will be translated and interpreted in French. Manager vs. Leader: no clear consensus The Series I release of the framework proposed renaming the Manager Role to the Leader Role to better reflect physicians scope of practice in this domain. Not surprisingly, respondents felt strongly about the proposed title change. However, no clear consensus emerged. Those who supported the title change were happy to see that physician leadership skills were being given more prominence within the framework. They felt that leadership is paramount to effecting positive change to an individual practice, within a collaborative environment, and to health care administration. They expressed the belief that physicians need to be leaders and that leadership competencies include being collaborative and knowing when to follow and when to take the lead. Respondents opposed to the proposal felt that the Leader title gives the wrong message and represents a step backward for medicine. They felt it reinforces a long-standing hierarchical culture in medicine, one that many are working hard to change. Among these respondents, very few voiced concern about the decision to reinforce leadership competencies within the Manager Role. Instead, it appeared the concern was about the label itself. 12

Many respondents who were uncomfortable with the proposed title suggested alternative options, including Leadership, Leader-Manager and Manager-Leader. Another suggestion was to split the role into two, one called Leader and the other called Manager. Milestones need to be realistic and relevant The Royal College defines a milestone as the description of an ability expected of a trainee or physician at a defined stage of professional development. With the release of new milestones in the draft CanMEDS Milestones Guide, trainees, educators and practitioners took the first of many opportunities to provide us with their feedback and direction. There are too many milestones! Most respondents understood and supported the rationale for creating milestones and agreed that the milestones will provide a practical structure for medical educators when they integrate the new 2015 competencies into their respective programs. However, many told us that the sheer number of milestones was impractical, and that it will be too difficult to teach and assess trainees against them. The milestones are very clinically oriented; many are not applicable to lab medicine specialties or radiology where direct patient interactions may not occur. This is particularly true for Communicator. Survey respondent We haven t got it quite right yet the milestones are too easy or too hard. The milestones are not quite right, according to many comments. In some cases respondents felt the competencies could be attained earlier or later in training and that we should take the time to review and adjust the milestones. Respondents saw a need for the milestones to evolve over time, and they encouraged us to be open to that evolution and to support a process of continual improvement. Not all milestones apply to all specialties or at all levels of medical education. A number of respondents felt that the competencies do not represent the practice experience of all specialties equally. Some specialties have infrequent direct contact with patients, for example, and therefore have limited opportunity to develop the competencies defined within some roles of the framework. Other physicians described how they fulfill a role as an expert witness regularly in their work and questioned the decision to de-emphasize this competency within the framework. Overall, respondents want us to consider and adjust the milestones for each specialty. There were mixed reactions to the decision to draft milestones specifically undergraduate milestones across the continuum of medical education. Many, both within Canada and abroad, found the approach to be extremely helpful. They saw the structure as a logical, coherent approach for looking at medical education across the continuum. Others questioned whether the Royal College, its volunteers and its partners have the required authority to define milestones for undergraduate education. The decision to draft milestones for the transition periods in a physician s career received broad support. These are typically difficult periods for physicians, and respondents noted that an increased emphasis on the transitions is a welcome addition. 13

CanMEDS 2015 requires a careful roll-out Despite the general support expressed for CanMEDS 2015 and the introduction of the milestones, most respondents indicated that they are anxious about the roll-out. Many told us that they perceive a lot of change coming, and they worry that they will not have the knowledge, experience or support to implement CanMEDS 2015 on the ground. They had questions about Royal College expectations with respect to milestones, and whether the system changes will account for the practical realities of a medical education system that has service requirements. Commonly asked questions included: Success requirements for CanMEDS implementation, according to survey respondents: 1. Positive attitude and involvement of teachers and learners 2. Flexibility 3. Strong evaluation tools 4. Specialty-specific 5. Relevant to all specialties How will the revised framework and new milestones be customized for each specialty? How will the milestones be assessed? How will CanMEDS 2015 be supported (i.e., tools, expertise, resources, transition period) and by whom? When will our program need to comply with the new competencies? If the idea is to move away from time-based training to competency-based training, how will this affect service scheduling? Challenges raised by survey respondents include: 1. Being able to assess the milestones 2. Distinction between resident skill levels 3. Risk of minimizing other skills 4. Requirements of attaining a milestone (i.e. one-time demonstration vs. multiple) 5. Support for the transition period 14

4. Consolidating: what we ll do with what you said This section of the report fulfills our commitment to the many people who participated in our consultations. It tells you how we intend to act on your feedback, and what you can expect from us over the coming months. Incorporating the valuable feedback We received an impressive amount of thoughtful feedback from many individuals and groups over the first six months of this year. Although it would be impractical to explain how all of these suggestions will be reflected in the next drafts of the framework and the CanMEDS Milestones Guide, we want to assure you that all of the feedback has been carefully reviewed by those responsible for revising those documents. Note that at times respondents presented conflicting perspectives. In these cases, the Integration Committee will decide how to proceed, based on best practice and evidence. A few examples of the changes you can expect as a result of the feedback, and when you can expect to see them: Addressing concerns around the clarity of the framework: - We will use less jargon (e.g., substitute plainer language for terms like inter- and intraprofessional in the Collaborator Role) in Series III - We will remove all unnecessary adverbs such as appropriately and consistently in Series III - We will introduce greater consistency in our choice of terminology between roles (e.g., adverse event ) in Series III - We will develop a glossary for the final version of the framework Accounting for differences between disciplines and practice settings: - As the roll-out of CanMEDS 2015 begins, we will engage individual specialty committees to revise the milestones in ways that are appropriate to the disciplines (e.g., direct patient contact, legal testimony) - We will revise key competency 5 in the Communicator Role to better account for the fact that not all physicians use electronic medical records in their practices in Series III 15

Increasing clarity about the scope of certain roles: - We will add explanations in Series III to the Communicator and Collaborator Roles to help clarify the rationale for separating patient-physician communication from communication between physicians and between physician and health care professionals Manager vs Leader: - We will continue the discussion by including a specific question on the Series III survey that explores this issue The Competence Continuum: - We will respond to requests to remove the undergraduate milestones from the CanMEDS Milestones Guide and continue our efforts to work with key partners on these milestones - We will revise the Competence Continuum Diagram, including removing the Medical School Fundamentals and the Early Clinical Activity Stages The CanMEDS Milestones Guide: - We will reduce the total number of milestones in September 2014 - We will adjust the milestones in September 2014 to ensure they are written as observable behaviours that can be taught and assessed appropriately - We will focus on strengthening the milestones for Continuing Professional Development and as future versions of the CanMEDS Milestones Guide evolve - We will develop draft milestones for the Transition Out of Practice Milestones in 2014 and 2015 Implementation and faculty development: - We will develop tools and resources to support those on the front line who will be responsible for implementing CanMEDS 2015 - We will help support faculty development activities that are aimed at building local capacity to implement CanMEDS 2015 - We will support specialties as they choose from the palette of milestones and begin to customize the standards for their discipline 16

Following up on the findings At the time of writing, we are busy considering your feedback and integrating it into the next draft of the framework and the milestones. When you read this, the Series III draft will have been released, and we will be embarking on the next round of engagement with you to review the draft and to generate feedback by way of an online survey. This feedback will in turn help shape the Series IV draft of the framework, which we will then present for approval to our Royal College Council in February 2015. The CanMEDS Milestones Guide is a living document and will receive continual revisions. The Royal College, its partners and its volunteers are in the process of developing a robust implementation plan to ensure a smooth transition to CanMEDS 2015 on the front lines. This plan includes timely and clear communications, new faculty development initiatives, and practical online resources. The official launch of CanMEDS 2015 will take place simultaneously online and at the International Conference on Residency Education in Vancouver in October 2015. We look forward to hearing from you again soon! As I review it, I truly don t have anything to add. I referred back to my earlier feedback from November 2013 and note that this version has addressed my comments and, I assume, those of many other contributors. epanelist Acknowledgments Many people contributed to the design, implementation and analysis of the 2014 consultation activities associated with CanMEDS 2015, including the preparation of this report. We wish to acknowledge and thank the following people: Cynthia Abbott, Melanie Agnew, Andrée Boucher, Ginette Bourgeois, Ming-Ka Chan, Lana Dixon, Jane Fulford, Jennifer Gordon, Lisa Gorman, Jolanta Karpinski, Cynthia MacLachlan, Megan McComb, Sarah Matthews, Anna Oswald, Ashley Ronson, Marvel Sampson, Jon Sherbino, Linda Snell, Elaine Van Melle, and Elizabeth Wooster. How to reference this document: Abbott C, Bourgeois G, Frank JR, MacLachlan C, Ronson A, and Van Melle E. What we heard: Sharing the results of the CanMEDS 2015 Series I and II consultations. Ottawa: The Royal College of Physicians and Surgeons of Canada; 2014 September. 17

Appendix: Engagement Methodologies National online surveys Methodology: Two national online surveys were conducted to coincide with the Series I and Series II releases respectively. Focusing on the strengths and weaknesses of the draft CanMEDS 2015 framework, both of the surveys took approximately 15 minutes to complete and contained sections with demographic questions, general questions and role-specific questions. The second survey contained an additional section on milestones, and also allowed participants to enter an anonymous identifier that would link their responses to those of the first survey, if both were completed. Participation: The surveys were promoted to all stakeholders by direct email, the Royal College s corporate publications and social media. National Advisory Committee members also encouraged their constituents to respond to the survey. The first survey was open for approximately eight weeks and the second for four weeks. Overall, there was a total of 901 respondents from the two surveys combined. Approach: The questions were designed to validate that the framework revisions were on track and practical, and to give respondents the opportunity to identify and recommend improvements where needed. Data analysis: The quantitative results of each survey were analyzed to obtain descriptive statistics using SPSS. The qualitative results arising from the open-text responses were analyzed using a simplified thematic analysis: each survey question was coded individually and the codes were organized into themes. In other words, no overarching themes were developed to describe the data set as a whole; rather, preliminary themes were developed on a question-by-question basis only. This was an appropriate approach considering the nature of the questions and the goals of the survey consultation. Focus groups Methodology: Based on six distinct target audiences (medical learners, program directors, members of the public, National Specialty Society representatives and Fellows, health professionals, deans), eight online focus groups were held, each of 90 minutes duration. An external consultant firm, Hill & Knowlton Strategies (H&K), was chosen to facilitate the discussions and analyze the results. Recruitment and participation: The public focus group participants were recruited through H&K s national online screening process to reach members of the public who had recent experiences in the health care system and interest in the topic. The remaining focus group participants were recruited to provide a reflective sample of medical educators and learners from geographic regions across the country (with the exception of one French session where all individuals resided in Quebec). A total of 58 participants were involved in the discussions. Approach: The process was customized to each audience s areas of interest, experience and expertise. The process used with the public and with health professional participants focused on exploring each role in-depth. The process developed for the learners, program directors, deans and National Specialty Society/front-line Fellows, who will be directly affected by the changes to CanMEDS, focused on the overall framework, the samples of milestones, and the implementation of CanMEDS 2015. Data analysis: Each of the eight focus group sessions was recorded and transcribed to produce textual data. These records were imported into NVivo qualitative data analysis software and coded by discussion topic into related nodes or themes. Using NVivo software allowed the analyst to process the large volume of data collected in the focus group sessions efficiently and effectively. From there the analyst was able to discern both common themes supported by many participant references as well as hidden gems that may not have been mentioned often but are still pertinent. 18

Written submissions Methodology: We sent invitations for written commentaries to key health care partners, epanel members, and experts in areas such as continuing professional development. Participation: A total of 131 submissions were received from representatives of national institutions and partner organizations, epanel members and expert reviewers. Approach: Participants were invited to respond to the strengths, weaknesses, challenges and opportunities of the revised framework. Data analysis: The commentaries received were reviewed. Anonymous quotes were extracted and overall themes summarized; these will be presented to the members of the CanMEDS 2015 National Advisory Committee and to each of the CanMEDS 2015 Expert Working Groups. 19

Survey Continuum Physician Specialty LEADER Manager EVOLVING CanMEDS ENGAGE Competency MILESTONES Education Update Scholar Relevant Patient Safety Framework Feedback Collaborator Adapted Professional Medical Expert Faculty Development Communicator HEALTH ADVOCATE Revision Focus Groups Provide CARE STAKEHOLDERS CONTINUING PROFESSIONAL DEVELOPMENT