PRECEPTOR S GUIDE TO THE RLS THIRD EDITION

PRECEPTOR S GUIDE TO THE RLS THIRD EDITION American Society of Health-System Pharmacists 7272 Wisconsin Avenue Bethesda, MD 20814 2008 American Society of Health-System Pharmacists ASHP is a service of the American Society of Health-System Pharmacists, Inc. Registered in the U.S. Patent and Trademark Office

THE PRECEPTOR S GUIDE TO THE RLS THIRD EDITION 2008 American Society of Health-System Pharmacists, Inc., Bethesda, MD ASHP logo and name are registered trademarks of the American Society of Health-System Pharmacists, Inc. All rights reserved.

The following materials were developed for use in ASHP- accredited residency training programs. Permission to use these materials outside this application must be obtained from ASHP.

Table of Contents Chapter 1 Introduction... 1 Chapter 2 The Residency Learning System (RLS)... 3 An Introduction to Systems Theory... 3 The Environment of Residency Training... 4 The Residency Learning System Concept... 5 Figure 2-1: The Environment of the Residency Learning System (RLS).....5 Figure 2-2: Components of the Residency Learning System (RLS).....6 Chapter 3 Component 1 Educational Outcomes, Goals, and Objectives... 9 What Are Goal Statements and Educational Objectives and Where Do They Come From?...9 Classification of Objectives... 10 Chapter 4 Component 2 Instruction by Preceptors....11 The Four Roles of the Preceptor in Teaching Problem-Solving Skills....15 How to Use the Instructional Objectives....17 How to Use the Criteria Lists to Provide Effective Feedback... 17 Figure 4-1: The Learning Pyramid... 12 Table 4-1: Guide to Selecting Techniques for Instruction for Cognitive Learning... 13 Table 4-2: Practice-Based Teaching Techniques Associated with Preceptor Roles and Theories of Cognitive Problem Solving... 14 Chapter 5 Component 3 Assessment... 19 Preceptor Evaluation of Residents' Attainment of Educational Goals and Objectives...19 Residents' Self-Evaluation of Their Attainment of Educational Goals and Objectives...24 Residents' Evaluation of the Preceptor and Learning Experience... 24

Table of Contents Chapter 6 Component 4 The RLS Decision Process... 25 Step 1: Identify the Residency Program s Purpose and Desired Outcomes... 26 Step 2: Establish Program Structure... 27 Step 3: Assign Educational Goals and Objectives to Specific Learning Experiences... 35 Step 4: Designate Learning Activities for Each Learning Experience and Write Learning Experience Descriptors... 45 Step 5: Design Program Assessment Strategy, Design Assessment Strategy for Each Learning Experiences, and Design Evaluation Tools... 47 Step 6: Design Customized Training Plan for Each Resident... 50 Step 7: Precept the Learning Experiences....51 Step 8: Monitor Resident Progress... 53 Step 9: Conduct Quality Improvement Activities on the Program... 54 Figure 6-1: Flow Diagram of RLS Decision Process... 26 Figure 6-1: Professional Competence Equation... 28 Table 6-1: Comparison of Elements of Structure... 29 Table 6-2: Generic Template for Structure of PGY1 Learning Experiences... 33 Table 6-3: RLS Form for Recording PGY1 Step 3 Goal and Assessment Assignment Decisions... 36 Appendices Appendix A: Appendix B: Appendix C: Appendix D: Appendix E: Appendix F: Principle 4 of the ASHP Accreditation Standard for Postgraduate Year One (PGY1) Pharmacy Residency Programs... 55 Principle 4 of the ASHP Accreditation Standard for Postgraduate Year Two (PGY2) Pharmacy Residency Programs... 59 PGY1 Required and Elective Educational Outcomes, Goals, and Objectives... 63 Numbered and Classified by Taxonomy and Level of Learning Required and Elective Educational Outcomes, Goals, and Objective Plus Instructional Objectives to Assist with Teaching for Use with RLS... 75 PGY1 Required and Elective Educational Objectives Criteria For Measuring Resident Performance... 91 PGY1 Example Outcomes of Application of the RLS Decision Process... 104

Chapter 1: Introduction This guide will introduce you to the Residency Learning System, (RLS), a set of processes and tools that will assist you in meeting the Principle 4 requirements of the ASHP Accreditation Standard for Postgraduate Year One (PGY1) Pharmacy Residency Programs and the ASHP Accreditation Standard for Postgraduate Year Two (PGY2) Pharmacy Residency Programs. These standards were approved in 2005 and replace the 2001 pharmacy practice and 1994 specialized residency standards. In both of the 2005 standards Principle 4 contains requirements for the design, conduct, and evaluation of the training program. Appendix A is Principle 4 of the PGY1 standard, and Appendix B is Principle 4 of the PGY2 standard. In both the PGY1 and PGY2 standards, Principle 4 requires that, the resident s program be designed, conducted, and evaluated using a systems-based approach. Such an approach requires that there be a direct correlation among the expectations of resident performance, the type of instruction provided, and the evaluation of resident performance. Both standards go on to say that programs are free to develop their own systems-based approach, but that in acknowledgement of the time, skill, and effort required to do so, ASHP offers the RLS for the use of programs who chose to use it. Principle 4 describes the products and outcomes of a systems-based approach for which programs will be surveyed, but does not describe the processes by which these products and outcomes might be achieved. The RLS offers a full description of processes by which each of the products can be developed or outcomes achieved along with various tools to assist in carrying out the processes. RLS processes and tools are introduced in this manual as they are needed to sequentially design and implement the program using the system-based approach. All tools are available on the ASHP website in the RLS section of the Residency Programs area. The RLS has been utilized by ASHP-accredited residencies since 1993, the year in which the ASHP Model for Pharmacy Practice Residency Learning Demonstration Project was initiated. Completed in 1996, programs were offered the first edition of the RLS Model and related tools. The model and tools were revised in 2001 to reflect what had been learned from five years of experience with the model by many programs. With the introduction of the significant changes in Principle 4 and other areas of the 2005 standards plus five more years of RLS experience, ASHP determined it time to do a another and even greater revamp of the RLS processes and tools. Training to use the new and improved RLS nicknamed The Next Generation when choosing a new title for the RLS workshops that have been conducted since 1996 continues to consist of manuals (this preceptor guide and one for residents), workshops (at ASHP Midyear Clinical Meetings, National Residency Preceptors Conferences, and by special arrangements at program sites), and access to tools on the ASHP website. Here is a current list of the RLS tools available on the ASHP website. As experience grows with the RLS and as educational outcomes, goals, and objectives are updated for specialized residencies, expect this list to grow. Check the ASHP website for the most current listing..

General Use RLS Tools Blank Snapshot Form Blank Summative Evaluation Form Comparison of Elements of Structure Guide for Instruction by Preceptors Preceptor and Learning Experience Evaluation Form Quarterly Evaluation Form Example of Quarterly Evaluation and Tracking RLS Decision Process Flow Diagram Who Does What in the RLS PGY1-specific RLS Tools PGY1 Condensed Criteria Evaluation Tool Example of Care Plan Snapshot PGY1 Criteria for Objectives PGY1 Form for Step 3 Goal and Assessment Assignment Decisions PGY1 Match of RLS Products with Principle 4 Requirements PGY1 Progressive Detail Lists of Outcomes, Goals, Objectives, Instructional Objectives PGY1 Template for Structure PGY1 All Required Goals Summative Evaluation Form PGY1 Top 10 Snapshots PGY1 Summative Evaluation Template for Direct Patient Care PGY1 Summative Evaluation Template for Drug Policy/Drug Information Elective PGY1 Summative Evaluation Template for Practice Management PGY1 Summative Evaluation Template for Staffing PGY1 Progressive Detail Lists of Outcomes, Goals, Objectives, Instructional Objectives And more..

Chapter 2: The Residency Learning System (RLS) An Introduction to Systems Theory The RLS is a system. By definition, a system is a group of interacting, interrelated, or interdependent elements forming a whole. Systems theory makes clear to us the four principles upon which systems operate. All are reflections of the interrelatedness of the parts. Principle One, Containment, is that systems are likely to be subsystems of other systems. For example, the pharmacy department is really a subsystem of the hospital which, in turn, may be a subsystem of a health system. Whatever happens in the pharmacy department will be affected by what is happening in the hospital, and what happens in the hospital will be affected by what is happening in the health system. Principle Two, the Ripple Effect of Change, is that a change in one place in a system affects all other components of the system. For example, if we tamper with one part of our ecosystem and discharge large amounts of carbon dioxide, other gasses, and pollutants into the atmosphere, the seas warm, and we have hurricanes of increased ferocity. Likewise, if your residency program changes the objectives it wishes to teach, your instruction and what you assess must change accordingly. Principle Three, Synergy, is that the whole is greater than the sum of its parts. The circulatory, pulmonary, nervous, skeletal, and other systems that compose the body are complex and fascinating in themselves, but put together they produce a conscious, thinking being. A residency program that singles out for use one or more specific components of the RLS, but not all, will have an exemplary approach to design, or well-worded objectives, or a legitimate assessment process, or a sound approach to instruction; but only when all are adopted does the program gain the exponential effect of synergy among the parts.

Principle Four, the Rule of the Weakest Link, says that a system is only as good as its weakest link. For instance, even though you link a high-performance tuner/amplifier to a high-performance MP3 player, attaching low-quality speakers results in low-quality sound. Do a lousy job of connecting your methods of instruction to the objectives, and no matter how faithful your program is to the other components, your program will function at the lower level of your instruction. The Environment of Residency Training Your PGY1 or PGY2 residency is a subsystem of total pharmacy learning and education. Figure 2-1 shows how residency training fits into the scheme of things -- what influences it, and, in turn, what it influences. Note that where the spheres of current pharmacy practice and the vision of pharmacy practice coincide is the level of practice reflected in the PGY1 and PGY2 standards. The standards, in turn, specify requirements for level of practice, site characteristics, preceptor qualifications and what residents should learn, which, in turn, affects residency training. Notice that the learning requirements for residents tie back directly to the practice responsibilities they are expected to fulfill. As practice changes, training must also change. The directive to make those changes comes through the reformulation of the accreditation standards and its impact on requirements for accreditation. The differences contained in the 2005 PGY1 and PGY2 standards demonstrate this principle. Among the major changes in the 2005 standards are a response to the Institute of Medicine s mandate that all health professionals attain competence in five areas interdisciplinary practice, patient-centered care, use of informatics, evidence-based practice, and quality improvement.

Figure 2-1: The Environment of the Residency Learning System (RLS) The Residency Learning System Concept Now take a second look at Figure 2-1. Notice that the requirements of the accreditation standard all influence decisions made by training sites as they use the RLS process and tools. For instance, the decision of how to structure the program s learning experiences will be heavily impacted by the standard s specification of practice requirements. However, the contribution made by the RLS is strongest in helping programs satisfy the standard s preceptor and learning requirements. The Residency Learning System is diagrammed in the lower half of Figure 2-1. The RLS is a way of thinking about training in pharmacy practice residencies based on systems thinking. This systematic approach can be applied to any area of residency training in pharmacy. The RLS was created by customizing the traditional instructional systems design (ISD) model so it fits the pharmacy residency environment. The driving principles of ISD are that learning should be efficient (take as little time as possible) and be effective (achieve the desired ends). It attempts to be comprehensive for all desirable outcomes of PGY1 or PGY2 training. The RLS has four subsystems (Figure 2-2): Component 1: Statements of educational outcomes, goals, and objectives derived from task analysis of the job responsibilities of pharmacists who have completed a residency

Component 2: Instruction by preceptors designed to facilitate resident attainment of the educational outcomes through mastery of the educational goals and objectives Component 3: Assessment strategies for three areas: Preceptor evaluation of resident attainment of the educational goals and objectives of training Resident self-evaluation of attainment of the educational goals and objectives of training Resident evaluation of the quality of the preceptor and of the learning experience Component 4: A decision process to guide and balance the selection of different elements in the other three subsystems Figure 2-2: Components of the Residency Learning System (RLS) Component 1 Goals and Objectives Decision Process Instruction by Preceptors Component 2 Component 4 Assessment Component 3

The arrows show the interrelations of the subsystems, indicating that a change in one subsystem affects the others. As an illustration of how this works, let us say that we have the following in the residency program educational objectives, Explain the organization s medication-use system and its vulnerabilities to adverse drug events (ADEs). To teach so that the resident can adequately perform this educational objective, the preceptor will need to provide reading materials and/or talk with the resident about central concepts of systems theory, the concept of system error, the meaning of terms associated with ADEs, the meaning of the term culture of safety, the role of automation and information technology in both preventing and contributing to medication errors, etc. Evaluation, a term to be used interchangeably with assessment, might be a dialogue between the resident and preceptor about these topics, or the resident could write a page or two summarizing his or her understanding of the organization s medication-use system and its vulnerabilities. But what if the goal of the resident s training is to go beyond understanding the medication-use system and its vulnerabilities to actually doing something about it? Then we must have a different the educational objective. The new objective might read, Design and implement pilot interventions to change problematic or potentially problematic aspects of the medication-use system with the objective of improving quality. The instructional setup must also be different. The resident must go beyond acquiring an understanding of the medication-use system and its problems to engage in problem-solving the design and the implementation of an intervention. The preceptor must add modeling of how to do design and implementation and then coaching of the resident as he or she engages in the design and implementation of the intervention. Evaluation must also be different. The preceptor will evaluate the quality of the intervention and its implementation when it was independently designed and implemented by the resident. One of the criteria for determining quality will be whether the design was practical to implement. The principle of the Ripple Effect of Change implies that if you change an educational objective in your residency program, you also need to change your instruction and your assessment. Likewise, if you change your instruction, it will affect your educational objectives and assessment strategy. Or, if you choose to evaluate differently, you must look again at your educational objectives and your instruction. The RLS is a tool for empowering preceptors and residents because it exponentially increases the power of the preceptor to teach and the resident to learn. But what if you choose to concentrate on just one or, perhaps, two of the subsystems of the RLS? Let's say you have the best educational objectives in the business but you don't use them to drive the instruction. You lose the effect of synergy. All four subsystems need to be in place and their interrelationships attended to if you want the maximum effect. Finally, when you choose to use the RLS, the residency training you provide will be as strong as your weakest subsystem. Therefore, the quality of your instruction will be pulled down by poorly stated educational objectives or lack of effective evaluation. Just like in a stereo system, each subsystem is inextricably linked to, and has a profound effect on, the others. There is an important message here that we will return to time and time again. The RLS is a system. You should not expect to get the benefits of using a system unless you do the whole thing. That means starting with step one of the decision process and moving forward a step at a time, involving all preceptors in the process as you go.

In this manual we will introduce you to each of the components of the RLS, one component at a time, beginning with the educational outcomes, goals and objectives and ending with the RLS decision process.

Chapter 3: Component 1. Educational Outcomes, Goals, and Objectives Your ability to interpret the required and elective residency educational outcomes, goals, and objectives is the foundation of everything you do in designing and delivering residency training in accordance with Principle 4, and consequently with the RLS. The educational outcomes, goals, and objectives will guide what you teach, how you teach it, and when you judge learning has been achieved. If the learning you expect to take place is not clear in your mind, then the path you take through instruction and assessment will meander (inefficient) and may not get your residents where you want them to go (ineffective). What Are Educational Outcomes, Goals, and Objectives, and Where Do They Come From? The residency educational outcomes and goals come from a task analysis -- an examination of the health system pharmacy practice environment to identify areas of job responsibility that residents assume when they enter their first post residency position. At the time of the writing of the third edition of this guide, ASHP has initiated the updating of existing sets of educational goals and objectives for the various areas of specialized residency training so that they are not only current but also employ the patterns and definitions described below for the 2005 PGY1 educational outcomes, goals, and objectives. As you look over Appendix C, which contains all educational outcomes, goals, and objectives designated for use in ASHP-accredited PGY1 residencies, note that those which are required by the standard are prefixed with the letter R and those that are not required but may be of interest to programs are prefixed with the letter E. A definition of terms is at the top of the document and repeated here: Educational Outcomes (Outcome): Educational outcomes are statements of broad categories of the residency graduates capabilities. Educational Goals (Goal): Educational goals listed under each educational outcome are broad sweeping statements of abilities. Educational Objectives (OBJ): Resident achievement of educational goals is determined by assessment of the resident s ability to perform the associated educational objectives below each educational goal.

In the Appendix D RLS document, one of many RLS tools to which you will be introduced in this manual, the listing of all the available educational outcomes, goals, and objectives has an additional category of objectives. These are instructional objectives. Instructional Objectives (IO): Instructional objectives are the result of a learning analysis of each of the educational objectives. They are offered as a resource for preceptors encountering difficulty in helping residents achieve a particular educational objective. The instructional objectives falling below the educational objectives suggest knowledge and skills required for successful performance of the educational objective that the resident may not possess upon entering the residency year. Instructional objectives are teaching tools only. They are not required in any way nor are they meant to be evaluated. Classification of Objectives Note that each educational objective and instructional objective in Appendix D starts with a word enclosed in parentheses. These words designate the type and level of learning specified in the objective according to one of three well-accepted learning taxonomies Bloom s Taxonomy for cognitive learning, Krathwol for affective learning, and Simpson for psychomotor learning. Please stop now and read Chapter 9, Developing Training Materials and Programs: Creating Educational Objectives and Assessing Their Attainment, in Staff Development for Pharmacy Practice. This chapter is available on the ASHP website, www.ashp.org.

Chapter 4: Component 2. Instruction by Preceptors The second component of the RLS is instruction by preceptors. As this component was developed, the educational outcomes, goals, and objectives were directly linked to how instruction by preceptors should be designed and delivered. This reflects the key characteristic of the systems approach that each component directly affects all other components. The content and level of learning of the educational objectives serves as the preceptor's guide for selecting teaching strategies, methods, and techniques. On the whole, preceptors for residency training are chosen because they are excellent practitioners. Excellence in providing patient care and managing a pharmacy do not necessarily mean excellence in teaching others to provide patient care or to manage the pharmacy. The latter requires teaching skills. This component helps with their acquisition and refinement. There are two sets of RLS tools for assisting with component 2. First, added to the PGY1 residency list of educational outcomes, goals, and objectives, there is the set of instructional objectives (IOs) associated with each of the educational objectives, a category of objectives briefly defined in chapter 3. The instructional objectives listed under an educational objective pinpoint what the resident needs to learn in order to successfully perform the educational objective. The instructional objectives can be used by preceptors as they identify learning activities that will help the resident achieve each educational objective assigned to the learning experience. They may serve as both a source of ideas of what to do when initially setting up a learning experience and as a quality control checkpoint for an established learning experience. Reference to the list of instructional objectives for a specific educational objective may also help to diagnose what additional instruction is needed when a resident has difficulty performing that educational objective. Take a moment to peruse some of the IOs in Appendix D. Part two of the RLS tools for the instruction component consists of an overall strategy for practicebased teaching and accompanying set of practice-based teaching techniques for use by preceptors. The instruction component, in keeping with the systems approach, prescribes that the manner in which instruction is delivered be matched with the educational goals and objectives of training. The Learning Pyramid, Figure 4-1, summarizes preceptor roles as residents progress through the three stages of learning to solve patient-care problems a carefully scheduled transition from the role of direct instruction to modeling to coaching to facilitation.

Figure 4-1. The Learning Pyramid (Source: Nimmo CM. Developing training materials and programs: facilitating learning in staff development. In: Nimmo CM, Guerrero R, Greene SA, Taylor JT, eds. Staff development for pharmacy practice. Bethesda, MD: ASHP; 2000.) Stage of Learning Preceptor s Role Culminating Integration Practical Application Foundation Skills and Knowledge Facilitating Coaching Modeling Direct Instruction Table 4-1 is a guide to help preceptors select an appropriate instructional method that matches with both the Learning Pyramid's framework and the level of learning specified in the objective to be taught. The table associates the preceptor's role as specified in the Learning Pyramid with specific practice-based teaching techniques for clinical problem solving.

Table 4-1. Guide to Selecting Techniques for Instruction for Cognitive Learning (Source: Nimmo CM. Developing training materials and programs: facilitating learning in staff development. In: Nimmo CM, Guerrero R, Greene SA, Taylor JT, eds. Staff development for pharmacy practice. Bethesda, MD: ASHP; 2000.) STAGES OF LEARNING IN THE FRAMEWORK OF INSTRUCTIONAL STRATEGIES Foundation Knowledge and Skills Practical Application Culminating Integration BLOOM'S LEVELS OF COGNITIVE LEARNING Knowledge Comprehension Application Analysis Synthesis Evaluation APPROPRIATE INSTRUCTIONAL METHODS Reading Lecture Guided discussion Interactive lecture Case presentation Case-based teaching Simulation/Role play Practice-based teaching Table 4-2 gives more detailed guidance for practice-based teaching, listing specific techniques for use in each of the four preceptor roles identified in the Learning Pyramid.

Table 4-2: Practice-Based Teaching Techniques Associated with Preceptor Roles and Theories of Cognitive Problem Solving (Source: Nimmo CM. Developing training and programs: facilitating learning in staff development. In: Nimmo CM, Guerrero R, Greene SA, Taylor JT, eds. Staff development for pharmacy practice. Bethesda, MD: ASHP; 2000. Text modified with permission of the author. ) PRECEPTOR ROLE Direct Instruction Modeling TECHNIQUE Direct learners to content specific to their practice problems. Teach how a new piece of content relates to other pieces. Introduce new content in the context of solving a direct patient care practice problem. Teach strategies to help clarify problems. Teach the patterns that characterize different categories of direct patient care practice problems. Explain out loud what you are thinking as you solve a problem. ASSOCIATED COGNITIVE APPROACH Organize content in the mind for quick recall. Organize content in the mind for quick recall. Organize content in the mind for quick recall. Define and classify problems. Define and classify problems. Master problem-solving strategies. Coaching Give learners opportunities to practice solving direct patient care practice problems coupled with feedback on their use of strategies. Provide sufficient problem-solving practice to build speed. Master problem-solving strategies. Master problem-solving strategies. Ask learners to explain out loud what they are thinking as they solve a problem. Self-monitor quality of problem solving. Facilitating Teach learners to evaluate their own work. Self-monitor quality of problem solving. The uses of the Learning Pyramid, the Guide to Selecting Techniques, and the specific practicebased teaching techniques for clinical problem solving are fully explained in two chapters of a book published by ASHP, Staff Development for Pharmacy Practice edited by Nimmo, Guerrero, Greene, and Taylor. The first is Chapter 10, Developing Training Materials and Programs: Facilitating Learning in Staff Development. The second is Chapter 12, Developing Training

Materials and Programs: Practice-Based Teaching. These chapters can be found on the ASHP website, www.ashp.org. The following section on the preceptor s four roles will help you to apply what you have learned to the pharmacy residency situation. This section is a direct lift from the guidance we have provided to residents in The Resident s Guide to the RLS. The section reproduced below discusses what residents should expect their preceptors to do. **As you read this next section of text, keep in mind that you are seeing the preceptor roles from the perspective of the resident. The Four Roles of the Preceptor in Teaching Problem-Solving Skills From your study of Bloom s Taxonomy in this guide you have already learned about the hierarchical nature of learning. Now we will take this one step further and apply it to the learning situation in which you mostly find yourself during this year learning to solve practice problems. Figure 3 shows both how your learning will progress on a particular problem solving skill and how the preceptor s role will change as you advance. Let s say that you are entering a learning experience in oncology. You will not be much good at designing, recommending, monitoring, and evaluating patient-specific therapy for oncology patients until you have a firm understanding of the various cancers and medications used to treat them. So, your preceptor may begin by having you read chapters in an oncology text or current journal articles on therapies for the kinds of cancers you are about to work with. In this stage of your learning your preceptor is taking on the role of direct instruction.

Figure 3. The Learning Pyramid. (Source: Nimmo CM. Developing training materials and programs: creating educational objectives and assessing their attainment. In: Nimmo CM, Greene SA, Guerrero R, Taylor JT, eds. Staff development for pharmacy practice. Bethesda, MD: ASHP; 2000. Modified with permission from the editor.) Stage of Learning Preceptor s Role Culminating Integration Practical Application Foundation Skills and Knowledge Facilitating Coaching Modeling Direct Instruction Your foundational learning accomplished, you are ready to acquire the thinking strategies necessary to solve a patient care problem in this area of practice. Knowing about a particular type of cancer and the medications that can be used to treat it does not tell you how to put that information together with the characteristics of a particular patient, what to pull into the consideration, how to weight each piece of information, and how to come up with a solution to a case that doesn t fit the textbook. What you need at this point in your learning is for your preceptor to talk out loud as he or she solves a series of cases so you can see how he or she thinks. That role is called modeling. What is being modeled is thinking strategy. When you have a good idea of how to approach a problem, it s time to try solving one yourself. At this stage of the game you would expect yourself to be uncertain about the fine points and to get stuck once in awhile. The preceptor now switches into the role of coach you talk out loud about what you are thinking as you work on the patient s case and the preceptor provides steady feedback about when you re thinking is on the mark and when you are veering off. This is called feedback. The feedback you receive will be

focused on an evaluation of how well what you are doing matches up with the criteria for competent performance. As you get better, the preceptor will fade back the comments. One day you will find that you are doing cases together and the preceptor is silent because you don t need any further guidance. Now your skills will be in place, your preceptor will be confident of your ability to solve similar cases entirely on your own, and it s time to build your confidence in your ability to work independently. You are at culminating integration, and your preceptor will switch to the role of facilitator. This is the day when you come to work and the preceptor says, There s a patient with a melanoma in 5C. She s your patient. I ll be in the ambulatory clinic today if you need me. You will find that in those early independent cases your preceptor has picked patients whose problems are ones the preceptor thinks you re ready to solve. As you prove yourself to yourself, the scope of who you care for on your own will widen until you are the independent practitioner in this area that you need to be for that job you are going to walk into next year. This is the end of the excerpt from the residents manual. How to Use the Instructional Objectives Look in the list of educational outcomes, goals, objectives, and instructional objectives in Appendix D for educational objective R2.4.1 which refers to creating the patient-specific information base. Two of the IOs for this educational objective specify that the resident must have comprehensionlevel learning about disease states and medications. These IOs can be of help to the preceptor when deciding the learning activities in which the resident should engage to be able to work on therapeutic plans for a particular patient population. IOs help guide the preceptor's thinking about what content knowledge the resident must master to do the required problem solving. The content knowledge will vary from practice area to practice area; e.g., if the resident is learning to care for patients with diabetes, the disease state or medication knowledge will not be the same as it would be if the resident is learning to care for patients with asthma. We believe that having these prerequisites clearly spelled out for preceptor use can be helpful. Instructional objectives, then, are educational objectives that identify for preceptors what the resident must know or learn for success on the objectives. The instructional objectives, then, are tools for the preceptor to use in designing instruction. They are rarely, if ever, seen by residents. They are not intended to be formally evaluated. How to Use the Criteria Lists to Provide Effective Feedback Providing feedback on performance is an enormously important teaching task for preceptors. The resident must receive feedback, for without it the resident is destined to simply repeat the same approach to performing a task, resulting in repetition and not improvement. It is the presence of the preceptor s feedback that makes residency training an incubator for professional growth and differentiates it from the slower growth that can occur in regular practice as the pharmacist integrates feedback received from patients, peers, or other health care providers.

Feedback that supports change resulting in improvement must focus on the doing of a task and how it matches criteria for competent performance. Timing of feedback to support improvement is important. It must be regular. Daily is best even as the tasks are being done. At a minimum, there must be detailed feedback at least once a week. Batching, withholding feedback until the end of a learning experience, is not effective since acquisition of complex skills requires repeated shaping to get them right. That means the resident may perform the same task 20 times with feedback each time, making a modification to the approach with each performance, before getting it right. The RLS provides a tool for facilitating criteria-based feedback and formal assessment. This is Appendix E. Each list of criteria for competent performance provided in the RLS tool focuses on whatever is specified to be done in the educational objective. The list delineates the qualities of competent performance of the task specified. The criteria lists were developed by an instructional designer in collaboration with health system pharmacists with teaching background. The team was tasked with moving educational objective by educational objective and answering the question, What are the characteristics of a competent performance of this task? Each list is accompanied by a suggested context in which to assess the resident s performance of the educational objective. This guidance is intended to help assure that assessment of the right performance occurs and that the assessment is focused on a single performance and not a look back at cumulative performance of that task. If the performance specified in the educational objective asks that the resident perform a task done in practice, then the suggested assessment activity will involve the resident doing that task in practice. In those cases the assessment method will be direct observation of either the performance of the task, as with the delivery of patient education, or of the product of the task, as with the development of a monitoring plan. The type of feedback provided by the preceptor for supporting improvement is critical. That which does not focus on the quality of performance does not provide the right information for growth. Feedback that discusses multiple performances of the same task simultaneously makes it harder for the resident to relate the information provided to a specific performance that requires change. Feedback that does not occur with some immediacy to the performance of the task brings in the problem of recall of specifics for both resident and preceptor. Therefore, the preceptor should talk with the resident as soon as practical after a specific task performance, focus comments on that single performance, and discuss the resident s work relative to the criteria provided in the model. In some circumstances, when the resident is having particular difficulty in perfecting performance and verbal feedback doesn t seem to be doing an effective job, it can be useful to try written feedback this is the function of the snapshots which are criteria-based checklists. These are discussed in component 4 of the model. Written feedback can often have more impact than verbal because writing the feedback information down lends an aura of importance, and it also provides a chance for the resident to review the feedback over and over again.

Chapter 5: Component 3. Assessment The assessment component allows you to let residents know up front that you are going to be looking very objectively at certain aspects of their practice and evaluating them. It's almost like physicians' peer review; someone looks over how you handle a case and comments on it. It has an impact on your overall level of practice. Preceptor, Demonstration Project The third component of the RLS is assessment. The RLS tools support assessment in three areas: Preceptor evaluation of residents' attainment of educational goals and objectives Residents' self-evaluation of their attainment of educational goals and objectives Residents' evaluation of the preceptor and learning experience As previously noted at the beginning of Chapter 3, at the time of the writing of the third edition of this guide, ASHP has initiated the updating of existing sets of educational outcomes, goals, and objectives for the various areas of specialized residency training. As each is completed, sets of criteria for each of the areas will be developed to support accurate assessment in the specialized residencies. The principles of assessment are the same whether assessing PGY1 or PGY2 performance. Preceptor Evaluation of Residents' Attainment of Educational Goals and Objectives In a systems-based approach to training, the preceptors' assessment of resident performance must directly link to the residents' educational goals and objectives. A program is not using a systematic approach to the design and delivery of training if this direct link to the goals and objectives is not there. Preceptors determine resident success or failure by examining residents' ability to satisfactorily perform the assigned educational goals. Since goals are not measurable, preceptors base judgments of success on the goal on resident achievement of the educational objectives associated with each goal. When using the RLS, preceptors record their evaluations at the end of each learning experience (or quarterly for longitudinal experiences), using a form for summative evaluation. A blank summative evaluation form is Appendix F. Read the instructions at the top of this form. Notice that they direct preceptors to focus on judging goal attainment, but to do so by looking at performance on the associated educational objectives. On the form, these educational objectives are listed directly below the goal so the preceptor can keep them clearly in mind during the evaluation process. Note that there are blocks in which to check an overall rating for the goal and for rating performance on each of the educational objectives listed below it. The interpretation of the ratings follows:

Achieved. Satisfactory Progress. Needs Improvement. The resident has fully accomplished the ability to perform the educational goal or the objective. No further instruction or evaluation is required. This applies to an educational goal or objective whose achievement requires skill development during more than one learning experience. In the current learning experience the resident has progressed at the required rate to attain full ability to perform the goal by the end of the program. The resident's level of skill on the goal or objective does not meet the preceptor's standards of either "Achieved" or "Satisfactory Progress," whichever applies. The scale selected for use with the RLS summative form is one of many that might be chosen for use by the program. One such scale familiar to many is the three part: Does not meet expectations Meets expectations Exceeds expectations. It is up to the program to determine the scale most suited to its needs. Spaces for writing specific comments accompany the educational goals and objectives. Providing narrative commentary is even more important than the check rating. The narrative should provide specific information on meeting criteria that apply to the objectives under each goal as a way to help the resident to improve his or her future performance. As discussed in Chapter 4, Appendix E lists a sample of PGY1 assessment criteria for each educational objective. In addition, the lists describe the appropriate assessment situation for the measurement of the educational objective. Primarily these activities involve direct observation of the resident s carrying out of the task itself or review of the products resulting from doing the task. Each of these situations is an exact match with the performance the objective specifies. Use of these criteria and assessment situations when evaluating resident performance assures that assessment is criteria-based and a measure of the objective as it is written. You may once again want to take some time to read through these lists. A complete listing of PGY1 assessment criteria can be found on the ASHP website, www.ashp.org. How do preceptors make use of these criteria in their evaluations? First, they use them on a daily basis to judge what the resident is doing and to provide feedback to the resident. Chapter 4 of this guide described in detail the process of providing criteria-based feedback. Some preceptors remember the details of what the resident does and do not need to keep a record to refer to when they complete the summative evaluation. Others find using written criteria-based checklists (snapshots) to rate specific instances of an objective performance of help for providing feedback to the resident during the learning experience, and also for recalling details when rating performance at the end of the learning experience. We call completed checklist evaluations "snapshots" because they provide a picture of what the resident does during one performance of a specified task. Among the RLS tools are several of the most used snapshots. See the ASHP website, www.ashp.org for a sample list of snapshots for PGY1 educational objectives.

As described above, the RLS summative evaluation form records the preceptor s assessment of goal achievement based on the resident's achievement of the associated educational objectives. To help make the evaluation more objective, the preceptor may choose to use snapshots of resident behavior collected during that learning experience, attaching them to a completed summative evaluation. The following are three examples of how the narrative commentary might be filled out in a summative evaluation. Note that the sample commentaries are each a cumulative look at the resident s performance and not a snapshot of one performance. Instead the narrative is a set of conclusions about the resident s progress and current skill level drawn from the succession of observations occurring during the learning experience. Satisfactory narrative will always relate back to the criteria set for successful performance and describe how closely the resident has come to meeting those criteria. Since in the RLS approach evaluations are shared not only with the resident but also with the resident s other preceptors, the summative evaluation provides a wealth of information to the resident and to those who will next teach the resident helping the new preceptor pick up where the previous preceptor left off. Example 1: First, imagine that you are looking at an evaluation form for a patient care learning experience that lists all of goals of the R2 outcome including goal R2.4. Goal R2.4: Collect and analyze patient information. Imagine that a resident is in the last of her direct patient care learning experiences. She has been rated as making "Satisfactory Progress" in three previous learning experiences. In this last experience the preceptor judges that she is fully proficient in building a pharmacist's patient database. The preceptor checks "Achieved" and enters the following narrative alongside the two objectives that fall under the goal:

OBJ R2.4.1 (Analysis) Collect and organize all patient-specific information needed by the pharmacist to prevent, detect, and resolve medication-related problems and to make appropriate evidence-based, patient-centered medication therapy recommendations as part of the interdisciplinary team. OBJ R2.4.2 (Analysis) Determine the presence of any of the following medication therapy problems in a patient's current medication therapy: medication used with no medical indication: 1. Medication used with no medical indication 16 Patient not adhering to medication regimen Consistently records all information needed to make therapeutic decisions. Worked out method to efficiently record information & it is working well for her. Consistently identifies pertinent medication and disease-related problems. Corrected tendency to base decisions on objective data without referring to patients expressions of needs and preferences. Example 2: Imagine that another resident is completing the third of five direct patient care learning experiences, this one in infectious disease, and is judged to be acquiring documentation skills at an acceptable pace. These are the skills specified in educational goal R2.12. Goal R2.12: Document direct patient care activities appropriately. The preceptor checks Satisfactory Progress and enters the following narrative after one of the three objectives for Goal R2.12. OBJ R2.12.1 (Analysis) Appropriately select direct patient-care activities for documentation. OBJ R2.12.2 (Application) Use effective communication practices when documenting a direct patient-care activity. OBJ R2.12.3 (Comprehension) Explain the characteristics of exemplary documentation systems that may be used in the organization s environment. Example 3: Continues to be thorough & concise in documenting SOAP notes. Has added vocabulary for documenting pediatric subjective data. Shows growth in more clearly stating his findings from patient assessments & therapeutic plans. Needs to concentrate on using appropriate abbreviations & condensing descriptions.

Now let s look at a narrative where the resident needs to improve. This last example of a narrative commentary is for a resident in the first direct patient care learning experience. He will have six more direct patient care experiences, but his current preceptor does not judge that his skills in assessing patient progress toward the therapeutic goals during his six weeks in infectious disease has grown sufficiently. The preceptor checks "Needs Improvement" and enters the commentary in the applicable objective of the two that fall under goal R2.10. Goal R2.10: Evaluate patients progress and redesign regimens and monitoring plans. OBJ R2.10.1 (Evaluation) Accurately assess the patient s progress toward the therapeutic goal(s). OBJ R2.10.2 (Synthesis) Redesign a patient-centered, evidence-based therapeutic plan as necessary based on evaluation of monitoring data and therapeutic outcomes. Needs to pay more attention to trends in the monitoring data Needs to pay more attention to factors regarding a particular patient that may make certain of the monitoring data unreliable

Residents' Self-Evaluation of Their Attainment of Educational Goals and Objectives A major expectation of residency program graduates is that they will be capable of evaluating the quality of their own work and, thus, equipped to engage in reflective practice that can lead to expertise in the profession. To learn self-evaluation, residents must be trained to examine each of their job performances and accurately rate them against objective criteria. The acquisition of this skill is a required outcome of PGY1 training: OBJ R.3.1.1 (Characterization) Practice self-managed continuing professional development with the goal of improving the quality of one s own performance through selfassessment and personal change. We recommend giving residents the complete list of PGY1 (or applicable PGY2) criteria and the snapshots (adjusted for PGY2 residents) that are provided in this guide. Preceptors should then require residents to use these tools to complete snapshots of their own performances periodically throughout the learning experience. Once a snapshot is completed the resident should ask the preceptor to review the self-evaluation and discuss how closely the resident's evaluation matches the preceptor s assessment. This approach encourages the resident to internalize the criteria for successful performance, to focus on developing skills, and to grow in the ability to make accurate self-evaluations. Residents' Evaluation of the Preceptor and Learning Experience Principle 4 also requires that residents assess preceptor performance and program quality. The RLS offers a standardized tool for residents to use (see the ASHP website, www.ashp.org for an example). The form is generic and does not need to be modified for individual programs. A form for each of the resident s current learning experiences must be completed by the resident at the end of the learning experience (or at least quarterly for longitudinal learning experiences). Residents are encouraged to discuss the evaluation with the preceptor and must provide their evaluations to the residency program director. From time to time sensitive issues may be raised by the resident via the completion of this form. These may include inadequacies in the performance of a preceptor. For that reason the PGY1 and PGY2 standards allow some flexibility in determining if the form should be transmitted directly to the program director and not via the preceptor. It would then be up to the program director to evaluate the resident s comments both positive and negative and to determine how to make good use of them in shaping preceptor skills and shaping the program.

Chapter 6: Component 4. The RLS Decision Process Component 4 of the RLS, the decision process, is the processes piece of the RLS. It sums up in nine steps the decisions that a program must make in order to produce a system-based training program. All the tools of the RLS are built to facilitate making and recording those decisions. The outputs of the various steps are the products or outcomes surveyed during the accreditation process. Applying the decision process results in the development of a customized version of the RLS that will work appropriately in a specific pharmacy residency program. As such, you can think of the RLS decision process as in a unique position -- the control center for assuring that your program successfully uses the systems approach. Both new and existing programs should use the decision process to incorporate a systems approach into their residency training. New programs can use the decision process to design a program from scratch that will achieve the desired outcomes of training. Existing programs can use the steps to systematically identify the desired characteristics of graduate residents and assure that their training leads to the intended outcomes. In order to derive maximum benefit from the RLS, you should view residency training as a program that requires organization, planning, support, and input from all those involved. Use of the RLS requires that you use a team approach. As the team leader, the program director provides vision, leadership, and guidance while being ultimately responsible for the quality of the program. All pharmacist preceptors, under the guidance of the program director, should reach consensus on a program plan for the training year, customization of the plan for each resident, and revision of each resident's plan based on the resident's performance. We find that the most effective way to create a residency team and to reach consensus is for all preceptors to participate in the design process. The team s design process should be facilitated by the director(s) or someone else on staff who is trained in the RLS decision process. Those who for one reason or another are not direct participants in the design process should be offered the right of review. The Figure 6-1 flow diagram of the nine-step RLS decision process gives you a broad overview of the more detailed discussion that follows. Appendix F of this guide contains a running example of the result of each step of the decision process for a fictional PGY1 program. The principles illustrated in the PGY1 are directly transferable to the design and implementation of a PGY2 residency. First read this chapter s description of the step and how it is performed. Then turn to the appropriate section of Appendix F to see the product of that step. Your examination of each product of the decision process should include an effort to see how the process is cumulative and that each step relies upon the product of the previous step.

Figure 6-1: Flow Diagram of the RLS Decision Identify the program s purpose & outcomes Establish program structure 1 2 Assign educational goals and objectives to specific learning experiences 3 Designate learning activities for each learning experience & write learning experience descriptions 4 Design program assessment strategy, design assessment strategy for each learning experience & design evaluation tools 5 Establish customized training plan for each resident 6 Precept the learning experiences 7 Monitor resident progress 8 Conduct quality improvement activities on the program 9 Step 1: Identify the Residency Program s Purpose and Desired Outcomes The residency program purpose states the type of practice for which residents are to be prepared. For example, will the program focus on training individuals equipped to take faculty positions in academia, clinical providers of advanced patient care services, or individuals to provide administrative leadership? The program s determination of purpose will derive from the residency team s consideration of the site s unique training opportunities. Once the purpose of the program has been decided, the team needs to identify the program s outcomes. The outcomes should reflect the program's philosophy and vision, be consistent with the requirements of the relevant accreditation standard, and reflect the uniqueness of the pharmacy offering the residency. With the introduction of the 2005 PGY1 standard, PGY1 residencies must include six educational outcomes specified in the standard. Those outcomes are: Manage and improve the medication-use process Provide evidence-based, patient-centered medication therapy management with interdisciplinary teams Exercise leadership and practice management skills. Demonstrate project management skills Provide medication and practice-related education/training Utilize medical informatics. In addition, programs are encouraged to consider adding educational outcomes that reflect the strengths of the site and program outcomes necessary to achieve the program s purpose. Appendix C is the COC-approved Required and Elective Educational Outcomes, Goals & Objectives for Postgraduate Year One (PGY1) Pharmacy Residency Standard. The residency team will want to

consider a possible fit for one or more of the outcomes in the electives area. It is also possible that a program may create a suitable outcome that is not included in this listing. Appendix C is an RLS tool that will be useful in making PGY1 Step 1 decisions and also useful in several other decision process steps. For purposes of Step 1, note that each of the approved required and elective educational outcomes, goals, and objectives has been assigned a number. Required educational outcomes, goals, and objectives use the prefix R and electives E. Numbering has been done to facilitate easy location of a particular outcome, goal, or objective. Educational outcomes, goals, and objectives for PGY2 programs are numbered in the same way. While not required by either of the standards, the program will need to develop a brief descriptor of the program to be used in the residency manual, the program listing on the ASHP website, and in marketing materials. Its use is to communicate a concise but accurate description of the program. It should be a carefully worded narrative that incorporates the type of product and program outcomes developed in Step 1. The sample PGY1 purpose statement can be found in Appendix F. Step 2: Establish Program Structure Training structure refers to the organization and length of discrete learning experiences within the residency program. In this step, the challenge to the residency program team is to identify and arrange the resident's learning experiences in a way that will maximize the achievement of the program outcomes. When contemplating a PGY1 structure, the residency program team should keep in mind that the ultimate goals are to accelerate resident growth beyond entry-level professional competence in managing medication-use systems and in activities that support optimal medication therapy outcomes for patients with a broad range of disease states. For PGY2 residencies the goals are to raise the resident s level of expertise in medication therapy management and clinical leadership in the area of focus. The emphasis in training is on the progressive development of competence toward expertise. The growth of clinical judgment is a process begun in the clerkships of the professional school years but requires further extensive practice, self-reflection, and shaping of decision-making skills based on feedback on performance. As depicted in Figure 6-1, professional competence includes the development of problem-solving skills, professionalism, and judgment. The structure of your residency program should facilitate the development of each of these characteristics for the areas of competence that are the focal points of your program. Looking again at Figure 6-1, one can see that developing problem-solving ability involves learning both the content that bears on the problem to be solved and the problem-solving strategies, or procedural skills that can be employed to solve the class of problems into which the problem falls. The Learning Pyramid, discussed in Chapter 4 on instruction, shows the stages of learning through which the resident will pass in moving toward the level of problem-solving ability expected of residency graduates. If residents are to reach culminating integration, they will need time to acquire any new content knowledge, apply it while coached, and then practice the problem-solving skill on their own with preceptor guidance in the background. To complete the process, time must be allowed for the resident to be given patient care responsibilities for sufficient cases and direct

responsibility for management activities to gain confidence and proficiency. Therefore, when creating the program structure, the residency team must give careful consideration to the length of experience required to develop problem-solving skills in any given area of practice. Figure 6-1: Professional Competence Equation. (Source: Nimmo CM, Holland RW. Transitions in pharmacy practice, part 2: who does what and why. Am J Health-Syst Pharm. 1999;56:1981-7.) When using the RLS, it will not be enough to make structure decisions by assuming that an existing program structure will do. Assumptions must be set aside and serious thought given to what learning experiences and what time spent in them will work to provide the needed opportunity to learn. To retain an existing structure while redefining the program s educational outcomes, goals and objectives; approach to instruction; and assessment strategy; is to shortchange the program and short-circuit the systems effect. Choices for structuring learning experiences include the following: Rotation Longitudinal learning experience Concentrated learning experience Extended learning experience

Table 6-1 summarizes the advantages and disadvantages of the four choices. Table 6-1: Comparison of Elements of Structure Rotation ELEMENT DESCRIPTION ADVANTAGES DISADVANTAGES 4 to 6 weeks spent focused on just one area of learning Longitudinal learning experience Concentrated learning experience Regularly scheduled, intermittent experience with a specific focus Less than 4 weeks spent with the same preceptor Distraction minimized Related goal areas can be taught together Short duration facilitates exposure to varied patient populations Scheduling flexibility Enables bonding with medical students Resident has time to build confidence in the practice area Opportunity to integrate elements of a full practice Offers sufficient practice time to develop independent problemsolving skills Patterns of practice become evident Offers depth of experience Sufficient time in place to assess growth in skills Opportunity to learn prerequisites for other learning experiences in a concentrated fashion Efficient way to teach Limits on the practice time needed to develop independent problemsolving skills Difficult to assess growth in skills over short period Pull-out effect may have negative effect on other learning experiences Insufficient time to develop depth of learning Insufficient time to develop independent problemsolving skills

ELEMENT DESCRIPTION ADVANTAGES DISADVANTAGES Extended learning experience 6 to 8 weeks spent with the same preceptor Resident has time to build confidence in the practice area Offers sufficient practice time to develop independent problemsolving skills Facilitates preceptor/resident bonding Sufficient time in place to assess growth in skills Promotes skills in integrating practice Provides sufficient time in place to enable teaching Patters of practice become evident Limits the number of patient populations that can be worked with Scheduling out of synch with students and medical residents

The PGY1 standard is specific in some requirements for structure. These include the following: Structure must include resident opportunity to gain experience in diverse patient populations, a variety of disease states, and a range of complexity of patient problems as characterized by a generalist s practice. If the program is based in a practice setting such as acute care, ambulatory care, hospice, primary care, geriatrics, or pediatrics, the structure must ensure that the program s learning experiences meet the requirements of diversity, variety, and complexity required of all PGY1 programs. During no more than one-third of the twelve-month program can the resident be training with a specific patient population or in a single practice area such as critical care, oncology, cardiology, or drug information. The required resident project, delineated in the required outcome Demonstrate project management skills, must be assigned to a learning experience and that experience must allow sufficient time for the project s completion. Programs will have different learning opportunities upon which to draw. Each program will have to decide for itself what works best. The structure of a health system residency program, in most cases, will mirror the practice model of the institution. Rotations are sometimes used when patient care services are provided by clinical pharmacy specialists. With this structure residents can gain experience with a variety of patient populations as they move from working with one clinical specialist to another and another. Pharmacy departments that use an integrated practice model, integrating clinical services with distribution as their approach to organizing patient care, may use this practice model as the frame for one or more of the resident s learning experiences -- resulting in the choice to utilize the extended learning experience. In this type of learning experience, training can address many of the standard s required program outcomes within a single practice experience. When training residents to provide care to chronically ill patients, a longitudinal learning experience should be considered. For example, residents might spend one afternoon per week for 6 to 12 months learning to care for patients in an ambulatory care clinic. This use of the longitudinal learning experience can facilitate the development of a personal relationship with patients and enable the resident to observe the progress of care over several months. Another appropriate use of the longitudinal learning experience is for completion of the resident s project. Many programs assign the project to its own longitudinal experience, increasing the scheduled time for the learning experience toward the end of the residency year when project demands are at their anticipated highest.

The concentrated learning experience -- a short-duration opportunity to learn in some focused area is particularly useful for acquiring practical skills such as computer facility that under gird practice in multiple areas. Many programs use the concentrated learning experience to provide exposure experiences to areas of practice in which there is no intent to build independent problem-solving skill. Structure decisions must reflect careful consideration of the amount of time required to achieve the program s outcomes. For each learning experience, questions like the following need to be answered before an educationally sound conclusion can be drawn about the appropriate amount of time to assign to the experience. Is there complex problem-solving to be achieved in this learning experience? If so, what level of problem-solving ability (foundation knowledge and skills, application, or culminating integration) do we expect the resident to achieve by the end of this experience in this particular practice area? How long do we anticipate it will take the resident to reach that level? There are three steps to determining the structure of the program s learning experiences. 1. Determine the different learning experiences to be offered. 2. Assign each of the learning experiences to one of the program s outcomes. 3. Determine the duration of each (longitudinal, rotation, concentrated, extended) learning experience. Table 6-2 is another RLS tool for assisting with the decisions and their recording of the first parts of decision Step 2. Note that the template is filled in with the six required PGY1 outcomes. Space is allotted for the inclusion of one more outcome. This not to suggest that adding a specific number of outcomes or adding outcomes at all is necessary. Appendix F provides an example of structuring decisions.

Table 6-2: Generic Template for Structure of PGY1 Learning Experiences Potential learning experiences:

Program Purpose R1: Manage & Improve meduse process R2: Provide evidence-based, pt-centered MTM with interdisciplinary teams R3: Exercise leadership & practice management skills R4: Demonstrate project management skills R5: Provide medication & practice-related education/ training R6: Utilize medical informatics Possible other outcome Learning experience/ type Learning experience/ type Learning experience/ type Learning experience/ type Learning experience/ type Learning experience/ type Learning experience/ type Learning experience/ type

Step 3: Assign Educational Goals and Objectives to Specific Learning Experiences The decisions of Step 3 pinpoint where all aspects of the program s educational outcomes will be taught. This is accomplished by specifying in which learning experience or set of learning experiences the educational goals falling under each of the program s educational outcomes will be addressed. In order to complete Step 3, the program must start by making a clear decision on what educational goals are to be included in the program. For the required outcomes the answer is easy all the goals falling under the six required outcomes must be included. For any added outcomes drawn either from the list of elective outcomes or created by the program the answer is different. Since there is no mandate to include them all, programs are encouraged to choose or create educational goals and objectives for any non-required educational outcome so that the achievement of the sum of the goals under the outcome would establish the resident s capability in that area. Recording goal inclusion decisions for both required and elective can be made easier and also put into a format that facilitates further decision making and recording by using the Table 6-3 RLS tool, a Step 3 recording form.

Table 6-3: RLS Form for Recording PGY1 Step 3 Goal and Assessment Assignment Decisions Directions: Write the names of each of the program s learning experiences at the heads of the empty columns. In the TE/TE+ column write T alongside each goal selected for instruction in the program. In the same row place a T in each learning experience where that goal is to be taught. In the TE/TE+ column add E if the program will evaluate the goal in only one learning experience or E+ if the goal is to be evaluated in more than one learning experience. Note that all goals must be evaluated at least once during the residency year. The rationale for any goals and objectives that are not actively evaluated (are monitored) must be outlined in the residency program s written assessment strategy. Add E after the T in the column for each learning experience where the goal will be evaluated. Goal Emphasis T/TE/TE+ Required outcomes and educational goals and objectives for PGY1 programs Learning Experiences Outcome R1: Manage and improve the medication-use process. R1.1 Identify opportunities for improvement of the organization s medication-use system. R1.2 Design and implement quality improvement changes to the organization s medication-use system. R1.3 Prepare and dispense medications following existing standards of practice and the organization s policies and procedures.

R1.4 Demonstrate ownership of and responsibility for the welfare of the patient by performing all necessary aspects of the medication-use system. R1.5 Provide concise, applicable, comprehensive, and timely responses to requests for drug information from patients, health care providers, and the public. Outcome R2: Provide evidence-based, patient-centered medication therapy management with interdisciplinary teams. R2.1 As appropriate, establish collaborative professional relationships with members of the health care team. R2.2 Place practice priority on the delivery of patient-centered care to patients. R2.3 As appropriate, establish collaborative professional pharmacist-patient relationships. R2.4 Collect and analyze patient information. R2.5 When necessary, make and follow up on patient referrals. R2.6 Design evidence-based therapeutics regimens. R2.7 Design evidence-based monitoring plans.

R2.8 Recommend or communicate regimens and monitoring plans. R2.9 Implement regimens and monitoring plans. R2.10 Evaluate patients progress and redesign regimens and monitoring plans. R2.11 Communicate ongoing patient information. R2.12 Document direct patient care activities appropriately. Outcome R3: Exercise leadership and practice management skills. R3.1 Exhibit essential personal skills of a practice leader. R3.2 Contribute to departmental leadership and management activities. R3.3 Exercise practice leadership. Outcome R4: Demonstrate project management skills. R4.1 Conduct practice-related investigations using effective project management skills.

Outcome R5: Provide medication and practice-related education/training R5.1 Provide effective medication and practice-related education, training, or counseling to patients, caregivers, health care professionals, and the public. Outcome R6: Utilize medical informatics. R6.1 Use information technology to make decisions and reduce error. Elective outcomes and educational goals and objectives for PGY1 programs Outcome E1:Conduct pharmacy practice research. E1.1 Design, execute, and report results of investigations of pharmacy practice-related issues. E1.2 Participate in clinical, humanistic and economic outcomes analyses. Outcome E2:Exercise added leadership and practice management skills. E2.1 Contribute to the development of a new pharmacy service or to the

enhancement of an existing service. E2.2 Understand the pharmacy procurement process. E2.3 Manage the use of investigational drug products (medications, devices, and biologicals). E2.4 Understand the principles of a systematic approach to staff development in pharmacy practice. E2.5 Resolve conflicts through negotiation. E2.6 Understand the process of managing the practice area s human resources. E2.7 Understand the process of establishing a pharmacy practice residency program. Outcome E3: Demonstrate knowledge and skills particular to generalist practice in the home care practice environment. E3.1 Understand the scope of services that might be provided in a typical home care practice.

E3.2 Determine the suitability of individuals patients for home care. E3.3 Understand unique aspects of providing evidence-based, patientcentered medication therapy management with interdisciplinary teams in the home care environment. E3.4 Understand unique aspects of preparing and dispensing medications for home care patients. E3.5 Understand unique aspects if participating in the management of medical emergencies occurring in the home care environment. E3.6 Manage the use, maintenance, and troubleshooting of medication administration equipment and medication-related equipment in the management of home care patients. E3.7 Understand the appropriate relationship between the home care pharmacist and home care suppliers. E3.8 Appreciate the complexity of the financial environment of home care practice. E3.9 Conduct ethical informational and marketing visits to payers, potential referral sources, and patients of the home care organization.

Outcome E4: Demonstrate knowledge and skills particular to generalist practice in the managed care practice environment. E4.1 Maintain confidentiality of patient and proprietary business information. E4.2 Understand the interrelationship of the pharmacy benefit management company, the health plan, and the delivery system functions of managed care. E4.3 Understand unique aspects of providing evidence-based, patientcentered medication therapy management with interdisciplinary teams in the managed care environment. Outcome E5: Participate in the management of medical emergencies. E5.1 Participate in the management of medical emergencies. Outcome E6: Provide drug information to health care professionals and/or the public.

E6.1 Identify a core library, including electronic media, appropriate for a specific practice setting. E6.2 Design and deliver programs that contribute to public health efforts. Outcome E7: Demonstrate additional competencies that contribute to working successfully in the health care environment. E7.1 Use approaches in all communications that display sensitivity to the cultural and personal characteristics of patients, caregivers, and health care colleagues E7.2 Communicate effectively. E7.3 Balance obligations to oneself, relationships, and work in a way that minimizes stress. E7.4 Manage time effectively to fulfill practice responsibilities. E7.5 Make effective use of available software and information systems.

There are cautions to be observed in the goal selection process. The observation of these cautions may even cause a program to reach back and alter its selection of elective outcomes. Limiting the selection of educational outcomes and goals helps a program to focus its efforts on what is most important. Step 3 must be entered into with the conviction that even though a goal is not selected as a program goal, this does not preclude that it will be learned by the resident. Limiting the residency team s selection of goals is always a difficult process. Many of the elective goals may be viewed as important, leading to the inclusion of more goals than necessary to meet the program s outcomes. The more goals chosen, the more diffuse preceptor efforts become in attempting to formally teach and evaluate them all. The individuality of the program s training will be reflected in the additional goals it selects to assure that residents achieve the program s established outcomes. That individuality is, of course, further enhanced and made genuine by subsequent decisions in the RLS decision process, so programs should not be fearful that they will be restricted to a cookie cutter approach when they include all goals the standard requires and add just a few more to fulfill a match with their programs desired outcomes. Another concern in Step 3 is to consider if the collection of educational goals selected or created for elective outcomes represent sufficient focused teaching to assure that the resident has the opportunity to achieve that category s desired outcome. This is a subjective process. It is possible that one goal will fully teach what the team feels the resident needs to learn. Other times the team may not be satisfied until a variety of goals are selected. Once the program s educational goals have been selected to be taught, it is essential to determine where and when they will be evaluated. Table 6-3 will serve to facilitate and record the assessment frequency and location decisions. There are two general types of evaluation espoused in the RLS Model: goals that are actively evaluated during learning experiences (listed on the front of the summative evaluation form), and those goals that are monitored during learning experiences (listed on the back of a summative evaluation form. Some goals will require repeated assessment throughout the year because the skills being taught are complex and require extensive practice. Examples include virtually all of the educational goals falling under outcome R2, Provide evidence-based, patient-centered medication therapy management with interdisciplinary teams. In order to assure that the resident makes ongoing progress toward full achievement of these goals, assessments will be necessary in multiple patientcare learning experiences. Where a goal has been assigned for teaching in just one learning experience, the single assessment point will occur in that learning experience. It is important to note that Principle 4 requires that resident performance of all of the program-selected goals must be assessed. The rationale for any goals and objectives that are not actively evaluated (are monitored) must be outlined in the residency program s written assessment strategy.

Appendix F presents an example of assigning goals to specific learning experiences and determining where the goals will be assessed. At this point in the decision process, it is wise to have the team step back and reflect on the scope of the proposed training program. There are two questions that should be answered before choosing to go on. 1. Do your program and your program s preceptors have the resources and ability to teach all the goals selected? 2. Can you teach all of these things in one year? If the answer to either or both questions is no, then the team needs to examine why and to reconsider if as a program you are ready to offer a residency. Step 4: Designate Learning Activities for Each Learning Experience and Write Learning Experience Descriptors You have identified your program s educational outcomes and goals and placed where they will be taught within your structure. Refer again to Appendix C with its list of both required and elective educational outcomes, goals, and objectives for use in PGY1 residency training. Note the educational objectives falling under each goal. As described in the explanation at the top of this document, goals cannot be directly measured. Instead, we must rely upon resident performance of one or more educational objectives, which can be observed and measured. When the resident can successfully perform the sum of the objectives placed under a goal we can be reasonably assured that the resident has fulfilled the capacity described in the goal. Understanding the educational objectives under the goals assigned to the learning experience will be important in Step 4. In this step the preceptor responsible for a learning experience develops a list of learning activities for the resident. This is an analytical process in which the preceptor addresses each goal selected for his or her learning experience and asks the question, "What can the resident do that will provide the kinds of experiences necessary to achieve this goal?" This is a step that must not be missed, or the notion of designing a program that gains the advantages of working as a system will not be achieved. For effective and efficient training, the preceptor needs to consider the myriad of opportunities that may exist in the learning experience, and then choose those of critical importance to learning the identified goals. This assures that the learning opportunities are a genuine match with each of the learning expectations and that no goals are overlooked. For a time-limited rotation with just a few goals assigned to it, doing a thoughtful job of identifying exactly what the resident will be doing in order to provide the right opportunities to learn will not be difficult. However, trying to capture the details of what those opportunities to learn would be in a longitudinal experience with multiple goals assigned to it can be more of a challenge. The problem lies in both the scope of activities that need to be identified and in the necessity to project a whole year s worth of activities. An RLS tool that can be of use in determining learning activities is contained in Appendix D which you have already reviewed for its assignment of numbers to each of the educational outcomes, goals, and objectives. Notice now that under most educational

objectives there is a list of instructional objectives (IOs). These observable and measurable statements answer the question, What might the resident need to learn about or learn to do that he or she does not already know in order to be able to perform this objective successfully? These IOs can be helpful in determining what might be necessary learning activities. The following is a recommended process for determining learning activities: Select the first educational goal area assigned to the learning experience. Work directly from the goal statement and its educational objectives. Ask the question, What can the resident do in the context of this learning experience that will provide the kinds of experiences necessary to achieve this goal? Construct a list. Refine the list by referring back to each educational objective and instructional objective under the goal until it is obvious that everything has been covered. Keep in mind that the instructional objectives can be of great help in identifying the kinds of activities the resident may have to engage in to learn to do what is called for in the objective. However, no matter how many of the instructional objectives are used to frame learning activities, the full opportunity to learn has not been provided until the resident is given opportunities to practice the task called for in the objective itself. When you have completed work on the first goal, progress to the second. When working on longitudinal experiences, thought needs to be given to how the resident s learning activities will need to change in order to advance learning as the residency year advances. Observation of the preceptor performing tasks of R2 may be appropriate during the first weeks of the first quarter, but over time the appropriate learning activities will need to slide into resident practice with coaching and eventually into resident assumption of independent responsibility for the care of patients. It is critical that the list of learning activities for each learning experience be written down and that the description of each activity be clear enough to be understood by the resident as well as the preceptor. The preceptor can then draw upon the list to craft schedules of resident activities. Sharing the list of anticipated activities with the resident will inform him or her of what he or she should be doing. In addition, the resident will be able to assist the preceptor in keeping the resident s activities on track. Notice that many of the RLS educational objectives address the growth of values and attitudes inherent in the professional socialization of pharmacists. Determining specific activities that would facilitate the growth of values and attitudes can be extraordinarily difficult. For that reason, common sense is the rule in deciding for this type of objective when to write out activities and when not to. Appendix F illustrates how the preceptor for the direct patient care learning experience in the example program identified appropriate learning activities for the resident.

Step 5: Design Program Assessment Strategy, Design Assessment Strategy for Each Learning Experience, and Design Evaluation Tools In Step 5 you will develop a program strategy for each of the program s three areas of assessment including: Preceptor evaluation of resident attainment of educational goals and objectives Resident self-evaluation of their attainment of educational goals and objectives Resident evaluation of the preceptor and learning experience The decisions made by the residency team in this step will form the assessment strategy for the entire program. This will affect the choice of assessment tools available to preceptors and residents, timelines for completion of assessments, and with whom assessments will be shared. When the residency team has completed its assessment strategy design, it will be necessary to identify responsibility for the creation of the designated assessment tools. Preceptor Evaluation of Resident Attainment of Educational Goals and Objectives In regard to preceptor assessment of resident performance, Principle 4 requires that the program s assessment strategy employ the following: The assessments must be criteria-based. The assessments must be conducted at the conclusion of each learning experience. An exception is for longitudinal learning experiences where they may be conducted quarterly. The assessment must reflect the resident s performance at the time conducted at the end of the learning experience. As such it is termed summative because it reflects performance at an end point. The assessment must cover each of the educational goals and objectives assigned to the learning experience. The preceptor must discuss the assessment with the resident and with the program director. All three must document their review. The recommended RLS tool for completing summative assessments in each learning experience is the summative assessment form (Appendix F), discussed at length in Chapter 5. The form is completed for the specific learning experience by entering the educational goals and objectives assigned to that learning experience. Often, when the resident participates in several different direct patient care learning experiences, the program will decide to assign the same educational goals and objectives and the same assessment points to all of these learning experiences. That means that only one summative evaluation form needs to be constructed to cover the needs of all these experiences. The program strategy needs to determine its use of snapshots. You will recall from the Chapter 5 discussion that snapshots provide a means to document criteria-based feedback on a specific resident performance of an educational objective. The use of snapshots is

not required by Principle 4 as a tool for summative assessment. However, preceptors often report that completing one or more selective snapshots at various times during a learning experience can facilitate recalling the specifics needed to appropriately complete summative evaluation. Many programs utilizing RLS tools have decided that they benefit greatly from a highly selective program choice of snapshots for preceptor use. They are then used by preceptors assigned to assess the particular educational objectives covered by the chosen snapshots. The top ten snapshots introduced in Chapter 5 are those reported to be of great use by many programs. They may be used to support routine feedback to the resident during the learning experience and then attached to the summative evaluation form as detailed documentation of resident progress. The top ten snapshots can be found on the ASHP website, www.ashp.org. When discussing the program s assessment strategy, it will be important for the team to assure that all preceptors understand that the summative assessment should measure the resident s performance as it is at the end of the learning experience. Narrative comments can and should refer to progress made during the learning experience, but the ratings on the summative form should reflect resident performance at the end of the learning experience. This is the information that is needed to track resident progress throughout the year and to facilitate the next preceptor being able to pick up with resident learning at the appropriate spot. Note that the appropriate use of this form assures the Principle 4 requirements that assessment be criteria-based, cover each of the educational goals and objectives assigned to the learning experience, and documentation by the three parties are all met. The program assessment strategy must also describe a policy that covers the issues of timing of the assessment and of its transmittal from preceptor and resident to the program director and subsequently to the next preceptor. In discussion of timing, the residency team should keep in mind not only the Principle 4 requirement that assessment be completed at the time the learning experience ends, but also that in order to be useful for monitoring and for guidance to the preceptor with whom the resident will next work, the required speed of transmittal must be rapid.

There should be an agreed-upon policy that preceptors will discuss the summative assessment directly with the resident. The need for this is obvious. The resident needs criteria-based information on performance to guide ongoing improvement. Residents' Self-evaluation of Their Attainment of Goals and Objectives Residents will need to engage in two levels of self-evaluation if they are to achieve competence in this skill. First, practice needs to occur through careful reflection on the performance of individual tasks and then more globally when determining progress toward goal attainment at the end of each learning experience (or quarterly for longitudinal learning experiences.) Principle 4 specifies that the program must include the following in its assessment strategy for resident self-evaluation: 1. A plan whereby each preceptor provides periodic opportunities for residents to practice writing criteria-based, self-evaluations of their performance of the practice tasks they are learning to do. Because the self-evaluation occurs during the learning experience and is meant to help residents identify what needs improvement so they can continue to work on their performance, this is called formative evaluation. 2. A plan in which residents complete a written, criteria-based, summative selfassessment of their level of achievement of the learning experience s assigned educational goals and objectives according to the same schedule as the preceptor s completion of the summative evaluation form. 3. A plan for an end-of-the-residency-year self-assessment of attainment of the program s educational goals and objectives. The RLS tools that support #1 above formative self-assessment -- are the snapshots. It is likely that the residency team will designate the same snapshots chosen for use by preceptors as required for use by residents in the program s assessment strategy. The RLS tool that support #2 above summative self-assessment, is the summative evaluation form. There is no specific RLS tool to support #3. Since Principle 4 requires that the end-of-year self-assessment be criteria-based, it is most readily supported by providing the resident with a complete list of his or her educational goals and objectives and requiring a rating and narrative as one would find on the RLS summative evaluation form. The residency team needs to assure that all preceptors understand the process by which individuals learn to self-assess. 1. Assign the resident to perform the task specified in the educational objective. 2. Ask the resident to independently assess the quality of the work performed using the list of criteria provided by the RLS and record each judgment. 3. The preceptor independently assesses the resident s performance of the same task. 4. The preceptor and resident meet to exchange their assessments and to share why judgments are not the same when they differ.

Residents' Evaluation of the Preceptor and Learning Experience At the end of each learning experience (or quarterly for longitudinal experiences), Principle 4 requires that the resident complete an evaluation of the preceptor and of the learning experience. Residents are urged to discuss these evaluations with the preceptor and must provide their evaluations to the program director. Programs are free to design their own evaluation. An example of this evaluation tool can be found on the ASHP website, www.ashp.org. Design of the Assessment Activities for Each Learning Experience Once the resident team has designed the program assessment strategy, all guides are in place to enable each preceptor to translate that strategy into the specifics of what will occur in his or her learning experience. The use of program-wide snapshots or the choice to use some not being used by the whole program must be made. Other means of measuring resident performance of specific tasks other than snapshots that would be particularly effective in the learning experience need to be considered and brought into the assessment plan. The specifics of how the preceptor will provide self-assessment opportunities must be decided upon. Just what snapshots will be completed and when must they be done and discussed? Appendix F illustrates program and individual preceptor decisions related to step 5. Step 6: Design Customized Training Plan for Each Resident While the residency team has created a set of educational outcomes, goals, and objectives for the program plus a structure and plan for individual learning experiences that constitute the program, residents will enter the program with a wide range of entering knowledge, skills, attitudes, abilities and interests. Customization attends to those differences, producing an individualized approach that maintains the basic program design, but includes minor adjustments to account for the resident s individuality. Customization begins with the collection of information about the resident. Some programs choose to gather much of this information before the resident arrives, while others secure it during resident orientation. Sample tools can be found on the ASHP website, www.ashp.org, including simple and more complex data gathering instruments. A similar complex rating form could be constructed for a PGY2 residency s required objectives. The familiarization with the entering PGY1 resident should include an assessment of those knowledge, skills, abilities, and attitudes that are normally assumed by the program to have been obtained in professional school. If that assessment shows deficiencies, the resident s individualized program must be modified to assure that he or she is trained and attains competence in these areas. These areas must be added to the resident s personalized list of educational outcomes, goals, and objectives to be achieved.

Conversely, it is possible that a resident may enter the program having already fully achieved one or more of the program s educational goals or objectives. If there is reason to believe that is so, the program should assess the resident s performance in those areas using criteria-based assessment procedures and to document the outcome. If the resident is deemed fully achieved in any of these areas, the program is free to drop the educational goals and/or objectives from that resident s individualized plan. Dropping an educational goal and/or objective means that the program is not required to do any further evaluation on the resident s performance in that area. The same sort of assessment of the entering PGY2 resident s knowledge, skills, abilities, and attitudes should be undertaken. In this case the program should be looking for assurance that the resident has achieved the required outcomes of a PGY1 residency. Much of this evaluation can be accomplished through review of the resident s portfolio. Respect should be paid to the resident s specific interests and career goals in considering plan decisions. When sufficient information has been gathered to support a decision regarding modifications to the program s set of educational outcomes, goals, and objectives, a personalized list can be created for the resident. This personalized list can then be applied to consideration of the program s learning experiences including choice of electives, duration and sequence. In the end, the resident s customized plan should include an indication of expected learning and learning experiences in which that learning will take place. The resident s schedule should be the final piece of the resident s individualized training plan. The resulting customized plan must maintain consistency with the program s stated purpose and outcomes. That means no wholesale renovation is allowed. In addition, while the plan should account for resident interests, customization may not interfere with the achievement of the program s stated educational outcomes, goals, and objectives. When the resident s plan is complete, it must be shared with the resident and all of the resident s preceptors. Appendix F contains an example of initial customization for a program s resident. Step 7: Precept the Learning Experiences Resident Orientation Principle 4 requires that the resident be oriented to the program to include its purpose, the applicable accreditation regulations and standards, the evaluation strategy, and the resident s individualized plan. A second needed phase of orientation is to the use of the RLS in their training. This aspect of orientation enables the resident to participate fully in the systems approach to training and maximize the benefits of training under the system. The Resident's Guide to the RLS, Third Edition is a training manual that gives residents a broad-based understanding of the RLS application. Besides providing the resident's guide, on-site training by the residency program director or designee can be offered to orient residents to the RLS. Several years experience with the RLS has shown that resident training in its use should be staged in the following way:

Require that incoming residents study The Resident's Guide to the RLS, Third Edition prior to arrival in the program. Conduct orientation of residents to the residency program and site as usual. About midway in the orientation period, conduct a session in which the following is addressed in order to reinforce what they learned in the pre-reading: 1. What is the RLS, and what exactly does it mean? 2. How does their program operate as a result of using the RLS? 3. What are your educational outcomes, goals, and objectives for this year? 4. What are resident responsibilities under this system? This should be a summary discussion since the residents at this point lack experience with which to connect theory with practice. The preceptors for the residents first learning experiences should orient residents to their learning experiences as specified in the RLS and conduct the learning experience as set forth in the RLS and according to decisions made by the residency program team. Immediately following the end of the first learning experience, conduct an indepth discussion session with the residents in which all questions about use of the RLS are answered and all misunderstandings are rectified. Residents should be encouraged to share their experiences with their preceptors in relationship to use of the system. Residents should leave the discussion fully equipped to use the RLS properly. An offshoot of this discussion will likely be the identification of any problems with preceptors who are not using the RLS properly. Appendix F offers an example of residency program plans for conducting program orientation. Resident Orientation to Each Learning Experience The first step in doing an effective job of precepting during a learning experience is helping the resident understand what to expect. In addition to orientation to the pharmacy, to the residency program as a whole, and to how the RLS works, he or she will need additional orientation to the specifics of each learning experience. Orientation should occur at the beginning of the learning experience. The orientation should answer the following questions for the resident and will be greatly facilitated by the preceptor s written learning experience description: 1. What are the goals with associated objectives for this learning experience? 2. What activities will the resident engage in to accomplish each goal or objective? 3. How will the preceptor evaluate the resident? 4. How will the resident be trained for self-evaluation skills? a. What tools is the resident to use and on what schedule? b. How are interaction and feedback from the preceptor to be arranged?

Preceptor Training of the Resident For the duration of training the preceptor has the following responsibilities: 1. Select the appropriate strategy, method, and technique to correspond to the resident's learning needs. These are discussed in detail in Chapter 4. 3. Provide regular, day-to-day, criteria-based feedback to give the resident information on which to shape his or her task performance. When learning needs justify its use, written formative feedback such as snapshots should be used. In particular, written formative feedback will be called for when verbal feedback is proving to be ineffective. See Chapter 5 for detailed discussion of this. 4. Monitor and facilitate growth in resident self-assessment skills as per plan. 5. Do end-of-learning experience summative evaluations of resident performance using the prescribed plan. Step 8: Monitor Resident Progress The program must have a method for tracking progress throughout the residency training year on achieving each resident s individualized set of educational goals and objectives. An assessment must be conducted at least quarterly. The RLS recommends that a quarterly assessment meeting be held to be attended by all preceptors, the RPD, and, when appropriate, the resident at the time his or her progress is being discussed. Aggregate summative evaluations whose results have been recorded on a central tracking form accompanied by resident discussion of progress and problems should supply the essential information required for decision-making. This is the appropriate time to make any needed adjustments to the resident s customized training plan. Examples of adjustments could include a decision to eliminate further scheduled assessment of the performance of a particular objective because the resident is consistently demonstrating full achievement. Another might a change in a choice of elective because of an emerging focused resident interest. Or, the residency team might choose to eliminate an elective and replace it with a repeat of a required learning experience because the resident has not achieved essential goals and would need the extra time in that area to do so. Any changes made to the plan must be documented and communicated once again to the resident and all preceptors. Examples of quarterly evaluation forms that have been used by programs to facilitate quarterly discussions of resident progress and tracking of results can be found on the ASHP website, www.ashp.org..

Step 9: Conduct Quality Improvement Activities on the Program As the residency program director approaches quality improvement activities for the program, he or she should keep in mind that the objective is to enhance the potential that program residents will achieve the program s choice of educational outcomes. A critical piece of program quality is the quality of precepting offered residents. When setting up the residency, the program director is required by the standard to evaluate potential preceptors based on their desire to teach and their aptitude for teaching. Since residency training rests primarily on the use of practice-based teaching, it would not be sufficient to judge the preceptor potential on the sole basis of their ability to lecture. Rather, the skills desired are those found in the four roles of the preceptor (didactic instruction, modeling, coaching, and facilitation.) Ongoing feedback on preceptor performance will be available through the end-oflearning experience resident evaluations. These will serve as one important source of information to the program director. Others might include, but are definitely not limited to, direct observation of preceptors and residents at work together, discussions with preceptors on how they feel they are doing, and unscheduled discussions with residents. Where problems with preceptor performance occur, it is important to address them as quickly as possible. Training in teaching skills are important options for sustaining and enhancing preceptor performance. At least annually the director and preceptors need to consider their aggregate information on program performance and determine what, if any, program changes are required to enhance quality. An examination of the current residency program plan derived from fulfilling the various steps of the RLS decision process can be beneficial in identifying areas for improvement. Principle 4 requires that programs track their graduates as a means to evaluate whether the residency produces the type of practitioner described in the program s purpose statement. Suggested information to track includes the resident s first post-residency position, changes in employment, achievement of board certification, etc. Appendix F offers an example of residency program plans for quality improvement.

Appendix A: Principle 4 of the ASHP Accreditation Standard for Postgraduate Year One (PGY1) Pharmacy Residency Programs Principle 4: Requirements for the Design and Conduct of the Residency Program (The resident s training will be designed, conducted, and evaluated using a systems-based approach.) To ensure training efficiency and effectiveness, the program must use a systems-based approach to training design, delivery, and evaluation. Such an approach requires that there be a direct correlation among the expectations of resident performance, the type of instruction provided, and the evaluation of resident performance. The requirements in Principle 4 specify the products of a systems-based approach that may be examined during an onsite accreditation survey but, beyond specifying broad RPD and preceptor participation in program decisions do not specify a particular process for producing these products. RPDs are free to develop their own systems-based approach to training or rely on the guidance and tools in the ASHP-endorsed Residency Learning System (RLS) and associated materials. 3,4 Requirements: 4.1 Program Design. The RPD and, when applicable, program preceptors will collaborate to design the residency program. The resulting design will include the following elements: a. The program will document its purpose (the type of practice for which the residents are to be prepared); its outcomes (the residency graduates capabilities); its educational goals (broad, sweeping statements of abilities); and educational objectives (observable, measurable statements of resident performance, the sum of which ensure achievement of the educational goal) for each educational goal. The program s purpose will be reflected in the program s choice of outcomes. For each outcome there must be goals that further explain the capabilities specified by the outcome. For each goal there must be a set of educational objectives that specifies the resident performance to be measured. b. Programs must select all outcomes required by this standard. The required outcomes are as follows: 1. Manage and improve the medication-use process. 2. Provide evidence-based, patient-centered medication therapy management with interdisciplinary teams. 3. Exercise leadership and practice management skills. 4. Demonstrate project management skills. 5. Provide medication and practice-related education/training. 6. Utilize medical informatics.

Programs must include all of the associated educational goals and educational objectives listed with these outcomes. The list of outcomes with their educational goals and educational objectives is published elsewhere. 5 Programs may establish additional program outcomes with associated educational goals that emphasize program strengths. The same reference includes some potential additional (elective) program outcomes with associated educational goals and educational objectives. Interpretation of Requirement 4.1.b: The published Residency Learning System (RLS) lists of outcomes, educational goals, and educational objectives also include instructional objectives to assist, when needed, in teaching. Instructional objectives are not required and are not meant to be evaluated. c. The program will create a structure (the designation of types, lengths, and sequence of learning experiences) that facilitates achievement of the program s educational goals and objectives. The structure must permit residents to gain experience in diverse patient populations, a variety of disease states, and a range of complexity of patient problems as characterized by a generalist s practice. Residency programs that are based in certain practice settings (e.g., acute care, ambulatory care, hospice, primary care, geriatrics, pediatrics) must ensure that the program s learning experiences meet the above requirements for diversity, variety, and complexity. No more than onethird of the twelve-month PGY1 pharmacy residency program may deal with a specific patient population or practice area (e.g., critical care, oncology, cardiology, drug information). The educational goals and objectives, including those for the project, will be assigned for teaching to a single learning experience or a sequence of learning experiences to allow sufficient practice for their achievement by residents. Programs may market the practice strengths they seek to develop as defined by their choice of program structure. d. Preceptors will create a description of their learning experience, and a list of activities to be performed by residents in the learning experience that demonstrates adequate opportunity to learn the educational goals and objectives assigned to the learning experience. e. The program will create a competency-based approach to evaluation of resident performance of the program s educational goals and objectives, resident self-assessment of their performance, and resident evaluation of preceptor performance and of the program. The strategy will be employed uniformly by all preceptors. This three-part, competency-based approach will include provisions for the following: 1. Preceptors conduct and document a criteria-based, summative assessment of each resident s performance of each of the respective program-selected educational goals and objectives assigned to the learning experience. This evaluation must be conducted at the conclusion of the learning experience (or at least quarterly for longitudinal learning experiences), reflect the resident s performance

at that time, and be discussed by the preceptor with the resident and RPD. The resident, preceptor, and RPD must document their review of the summative evaluations. 2. Each preceptor provides periodic opportunities for the resident to practice and document criteria-based, formative self-evaluation of aspects of their routine performance and to document criteria-based, summative self-assessments of achievement of the educational goals and objectives assigned to the learning experience. The latter will be completed on the same schedule as required of the preceptor by the assessment strategy and will include an end-of-the-year component. 3. Residents complete an evaluation of the preceptor and of the learning experience at the completion of each learning experience (or at least quarterly in longitudinal learning experiences.) Residents should discuss their evaluations with the preceptor and must provide their evaluations to the RPD. 4.2 Program Delivery. To achieve systems-based training the program s design must be implemented fully, with ongoing attention to fulfillment of both preceptor and resident roles and responsibilities. In delivering the program the following must occur and be documented: a. The RPD and, when applicable, preceptors will conduct essential orientation activities. Residents will be oriented to the program to include its purpose, the applicable accreditation regulations and standards, designated learning experiences, and the evaluation strategy. When necessary, the RPD will orient staff to the residency program. Preceptors will orient residents to their learning experiences, including reviewing and providing written copies of the learning experience educational goals and objectives, associated learning activities, and evaluation strategies. b. The RPD and, when applicable, preceptors will customize the training program for the resident based upon an assessment of the resident s entering knowledge, skills, attitudes, and abilities and the resident s interests. Any discrepancies in assumed entering knowledge, skills, attitudes, or abilities will be accounted for in the resident s customized plan. Similarly, if a criteriabased assessment of the resident s performance of one or more of the required educational objectives is performed and judged to indicate full achievement of the objective(s), the program is encouraged to modify the resident s program accordingly. This would result in changes to both the resident s educational goals and objectives and to the schedule for assessment of resident performance. The resulting customized plan must maintain consistency with the program s stated purpose and outcomes. Customization to account for specific interests must not interfere with achievement of the program s educational goals and objectives. The customized plan and any modifications to it, including the resident s schedule, must be shared with the resident and all preceptors.

c. Preceptors will provide ongoing, criteria-based verbal and, when needed, documented feedback on resident performance. Documented feedback will be used if there is limited direct contact with the preceptor (e.g., when nonpharmacist preceptors are utilized for learning experiences late in the residency) or verbal feedback alone is not effective in improving performance. d. Preceptors will ensure that all aspects of the program s plan for assessment of resident performance, preceptor performance, and resident self-evaluation are completed. e. RPDs and, when applicable, preceptors will establish a process for tracking residents progress toward achievement of their educational goals and objectives. Overall progress toward achievement of the program s outcomes, through performance of the program s educational goals and objectives, will be assessed at least quarterly, and any necessary adjustments to residents customized plans, including remedial action(s), will be documented and implemented. 4.3 Program Evaluation and Improvement. Program evaluation and improvement activities will be directed at enhancing achievement of the program s choice of outcomes. RPDs will evaluate potential preceptors based on their desire to teach and their aptitude for teaching (as differentiated from formal didactic instruction) and provide preceptors with opportunities to enhance their teaching skills. Further, RPDs will devise and implement a plan for assessing and improving the quality of preceptor instruction including, but not limited to, consideration of the residents documented evaluations of preceptor performance. At least annually, RPDs and, when applicable, preceptors will consider overall program changes based on evaluations, observations, and other information. 4.4 Tracking of Graduates: The RPD should evaluate whether the residency produces the type of practitioner described in the program s purpose statement. (Information tracked may include initial employment, changes in employment, board certification, etc.) References for Principle 4 3. The preceptor s guide to the RLS. 3 rd ed. Bethesda, MD. American Society of Health-System Pharmacists; [in press]. 4. The resident s guide to the RLS. 3 rd ed. Bethesda, MD. American Society of Health-System Pharmacists; [in press]. 5. Required and elective educational outcomes, educational goals, and educational objectives. American Society of Health-System Pharmacists Home Page [resource on World Wide Web]. URL: http://www.ashp.org. Available from Internet. Accessed 2005 September 23.

Appendix B: Principle 4 of the ASHP Accreditation Standard for Postgraduate Year Two (PGY2) Pharmacy Residency Programs Principle 4: Requirements for the Design and Conduct of the Residency Program (The resident s training will be designed, conducted, and evaluated using a systems-based approach.) To ensure training efficiency and effectiveness, the program must use a systems-based approach to training design, delivery, and evaluation. Such an approach requires that there be a direct correlation among the expectations of resident performance, the type of instruction provided, and the evaluation of resident performance. The requirements in Principle 4 specify the products of a systems-based approach that may be examined during an onsite accreditation survey but, beyond specifying broad RPD and preceptor participation in program decisions do not specify a particular process for producing these products. RPDs are free to develop their own systems-based approach to training or rely on the guidance and tools in the ASHP-endorsed Residency Learning System (RLS) and associated materials. 4,5 Requirements: 4.1 Program Design. The RPD and, when applicable, program preceptors will collaborate to design the residency program. The resulting design will include the following elements: f. The program will document: its purpose (the type of practice for which the residents are to be prepared); its outcomes (the residency graduates capabilities); its educational goals (broad, sweeping statements of abilities); and, educational objectives (observable, measurable statements of resident performance, the sum of which ensure achievement of the educational goal) for each educational goal. The program s purpose will be reflected in the program s choice of outcomes. For each outcome there must be educational goals that further explain the capabilities specified by the outcome. For each goal there must be a set of educational objectives that specifies the resident performance to be measured. g. At the beginning of the resident s program, RPDs must document an individualized set of program outcomes, educational goals, and educational objectives for each resident. In doing so, PGY2 residencies in advanced areas of pharmacy practice must draw upon the program outcomes, educational goals, and educational objectives that have been developed by ASHP specifically for that practice area 1 (e.g., critical care, drug information, geriatrics, oncology, primary care). RPDs may establish additional program outcomes, educational goals, and educational objectives that reflect the site s strengths.

For PGY2 residencies in advanced areas of clinical pharmacy practice for which ASHP has not developed a complete set of program outcomes, educational goals, and educational objectives, a generic set of program outcomes, educational goals, and educational objectives (Program Outcomes, Educational Goals, and Educational Objectives for PGY2 Residencies in an Advanced Area of Pharmacy Practice 1 ) is available. This generic set of advanced clinical practice goals and objectives is provided as a required framework for programs that must develop their own Standard-mandated, area-specific, complete set of program outcomes, educational goals, and educational objectives. Also, RPDs for programs in non-clinical practice areas lacking ASHP-developed program outcomes, educational goals, and educational objectives must develop a complete set for their residencies. In both cases, RPDs must provide ASHP s Accreditation Service Division their complete set of program outcomes, educational goals, and educational objectives at the time of application. Interpretation of Requirement 4.1.b: The published Residency Learning System (RLS) lists of outcomes, educational goals, and educational objectives also include instructional objectives to assist, when needed, in teaching. Instructional objectives are not required and are not meant to be evaluated. h. The program will create a structure (the designation of types, lengths, and sequence of learning experiences) that facilitates educational goal and objective achievement. The educational goals and objectives, including those for residents projects, will be assigned for teaching to a single learning experience or a sequence of learning experiences to allow sufficient practice for their achievement by residents. i. Preceptors will create a description of their learning experience, and a list of activities to be performed by residents in the learning experience, that demonstrates adequate opportunity to learn the educational goals and objectives assigned to the learning experience. j. The program will create a competency-based approach to evaluation of resident performance of the program s educational goals and objectives, resident self-assessment of their performance, and resident evaluation of preceptor performance and of the program. The strategy will be employed uniformly by all preceptors. This three-part, competency-based approach will include the following: 1. Preceptors conduct and document a criteria-based, summative assessment of each resident s performance of each of the respective program-selected educational goals and objectives assigned to the learning experience. This evaluation must be conducted at the conclusion of the learning experience (or at least quarterly for longitudinal learning experiences), reflect the resident s performance at that time, and be discussed by the preceptor with the resident and

2. RPD. The resident, preceptor, and RPD must document their review of the summative evaluations. 3. Each preceptor provides periodic opportunities for the resident to practice and document criteria-based, formative self-evaluation of aspects of their routine performance and to document criteria-based, summative self-assessments of achievement of the educational goals and objectives assigned to the learning experience. The latter will be completed on the same schedule as required of the preceptor by the assessment strategy and will include an end-of-the-year component. 4. Residents complete an evaluation of the preceptor and of the learning experience at the completion of each learning experience (or at least quarterly in longitudinal learning experiences.) Residents should discuss their evaluations with the preceptor and must provide their evaluations to the RPD. 4.2 Program Delivery. To achieve systems-based training the program s design must be implemented fully, with ongoing attention to fulfillment of both preceptor and resident roles and responsibilities. In delivering the program the following must occur and be documented: f. The RPD and, when applicable, preceptors will conduct essential orientation activities. Residents will be oriented to the program to include: its purpose; the applicable accreditation regulations and standards; designated learning experiences; and the evaluation strategy. When necessary, the RPD will orient staff to the residency program. Preceptors will orient residents to their learning experiences, including reviewing and providing written copies of the learning experience educational goals and objectives, associated learning activities, and evaluation strategies. g. The RPD and, when applicable, preceptors will customize the training program for the resident based upon an assessment of the resident s entering knowledge, skills, attitudes, and abilities and the resident s interests. Any discrepancies in assumed entering knowledge, skills, attitudes, or abilities will be accounted for in the resident s customized plan. Similarly, if a criteriabased assessment of the resident s performance of one or more of the required educational objectives is performed and judged to indicate full achievement of the objective(s), the program is encouraged to modify the resident s program accordingly. This would result in changes to both the resident s educational goals and objectives and to the schedule for assessment of resident performance. The resulting customized plan must maintain consistency with the program s stated purpose and outcomes. Customization to account for specific interests must not interfere with achievement of the program s educational goals and objectives. The customized plan and any modifications to it, including the resident s schedule, must be shared with the resident and all preceptors. h. Preceptors will provide ongoing, criteria-based verbal and, when needed, documented feedback on resident performance. Documented feedback will be used if there is limited direct contact with the preceptor (e.g., when non-

i. pharmacist preceptors are utilized for learning experiences late in the residency) or verbal feedback alone is not effective in improving performance. j. Preceptors will ensure that all aspects of the program s plan for assessment of resident performance, preceptor performance, and resident self-evaluation are completed. k. RPDs and, when applicable, preceptors will establish a process for tracking residents progress toward achievement of their educational goals and objectives. Overall progress toward achievement of the program s outcomes through performance of the programs educational goals and objectives will be assessed at least quarterly, and any necessary adjustments to residents customized plans, including remedial action(s), will be documented and implemented. 4.3 Program Evaluation and Improvement. Program evaluation and improvement activities will be directed at enhancing achievement of the program s choice of outcomes. RPDs will evaluate potential preceptors based on their desire to teach and their aptitude for teaching (as differentiated from formal didactic instruction) and provide preceptors with opportunities to enhance their teaching skills. Further, RPDs will devise and implement a plan for assessing and improving the quality of preceptor instruction including, but not limited to, consideration of the residents documented evaluations of preceptor performance. At least annually, RPDs and, when applicable, preceptors will consider overall program changes based on evaluations, observations, and other information. 4.4 Tracking of Graduates: The RPD should evaluate whether the residency produces the type of practitioner described in the program s purpose statement. (Information tracked may include initial employment, changes in employment, board certification, etc.)

Appendix C: PGY1 Required & Elective Educational Outcomes, Goals, and Objectives Explanation of the Contents of This Document Educational Outcomes (Outcome): Educational outcomes are statements of broad categories of the residency graduates capabilities. Educational Goals (Goal): Educational goals listed under each educational outcome are broad sweeping statements of abilities. Educational Objectives (OBJ): Resident achievement of educational goals is determined by assessment of the resident s ability to perform the associated educational objectives below each educational goal. Instructional Objectives (IO): Published elsewhere. Instructional objectives are the result of a learning analysis of each of the educational objectives. They are offered as a resource for preceptors encountering difficulty in helping residents achieve a particular educational objective. The instructional objectives falling below the educational objectives suggest knowledge and skills required for successful performance of the educational objective that the resident may not possess upon entering the residency year. Instructional objectives are teaching tools only. They are not required in any way nor are they meant to be evaluated. Required By PGY1 Pharmacy Residency Accreditation Standard Outcome R1: Manage and improve the medication-use process. Goal R1.1: Identify opportunities for improvement of the organization s medicationuse system. OBJ R1.1.1 (Comprehension) Explain the organization s medication-use system and its vulnerabilities to adverse drug events (ADEs). OBJ R1.1.2 (Analysis) Analyze the structure and process and measure outcomes of the medication-use system. OBJ R1.1.3 (Evaluation) Identify opportunities for improvement in the organization s medication-use system by comparing the medication-use system to relevant best practices. Goal R1.2: Design and implement quality improvement changes to the organization s medication-use system. OBJ R1.2.1 (Comprehension) Explain the process for developing, implementing, and maintaining a formulary system. OBJ R1.2.2 (Evaluation) Make a medication-use policy recommendation based on a comparative review (e.g., drug class review, drug monograph). OBJ R1.2.3 (Synthesis) Participate in the identification of need for, development of, implementation of, and evaluation of an evidence-based

treatment guideline/protocol related to individual and population-based patient care. OBJ R1.2.4 (Synthesis) Design and implement pilot interventions to change problematic or potentially problematic aspects of the medication-use system with the objective of improving quality. Goal R1.3: Prepare and dispense medications following existing standards of practice and the organization s policies and procedures. OBJ R1.3.1 (Evaluation) Interpret the appropriateness of a medication order before preparing or permitting the distribution of the first dose. OBJ R1.3.2 (Application) Follow the organization's policies and procedures to maintain the accuracy of the patient s medication profile. OBJ R1.3.3 (Application) Prepare medication using appropriate techniques and following the organization's policies and procedures. OBJ R1.3.4 (Application) Dispense medication products following the organization's policies and procedures. Goal R1.4: Demonstrate ownership of and responsibility for the welfare of the patient by performing all necessary aspects of the medication-use system. OBJ R1.4.1 (Characterization) Display initiative in preventing, identifying, and resolving pharmacy-related patient-care problems. Goal R1.5: Provide concise, applicable, comprehensive, and timely responses to requests for drug information from patients and health care providers. OBJ R1.5.1 (Analysis) Discriminate between the requesters statement of need and the actual drug information need by asking for appropriate additional information. OBJ R1.5.2 (Synthesis) Formulate a systematic, efficient, and thorough procedure for retrieving drug information. OBJ R1.5.3 (Analysis) Determine from all retrieved biomedical literature the appropriate information to evaluate. OBJ R1.5.4 (Evaluation) Evaluate the usefulness of biomedical literature gathered. OBJ R1.5.5 (Synthesis) Formulate responses to drug information requests based on analysis of the literature. OBJ R1.5.6 (Synthesis) Provide appropriate responses to drug information questions that require the pharmacist to draw upon his or her knowledge base. OBJ R1.5.7 (Evaluation) Assess the effectiveness of drug information recommendations.

Outcome R2: Provide evidence-based, patient-centered medication therapy management with interdisciplinary teams. (When provided as part of the practice of direct patient care, this outcome always involves a series of integrated, interrelated steps.) Establish collaborative professional relationships with health care team members Place priority on delivery of patient-centered care to patient Establish collaborative professional pharmacist-patient relationship Collect and analyze patient information When necessary make and follow up on patient referrals Design evidence-based therapeutic regimen Design evidence-based monitoring plan Recommend or communicate regimen and monitoring plan Implement regimen and monitoring plan Evaluate patient progress and redesign as necessary Communicate ongoing patient information Document direct patient care activity Goal R2.1: As appropriate, establish collaborative professional relationships with members of the health care team. OBJ R2.1.1 (Synthesis) Implement a strategy that effectively establishes cooperative, collaborative, and communicative working relationships with members of interdisciplinary health care teams. Goal R2.2: Place practice priority on the delivery of patient-centered care to patients. OBJ R2.2.1 (Organization) Choose and manage daily activities so that they reflect a priority on the delivery of appropriate patient-centered care to each patient. Goal R2.3: As appropriate, establish collaborative professional pharmacist-patient relationships.

OBJ R2.3.1. (Synthesis) Formulate a strategy that effectively establishes a patientcentered pharmacist-patient relationship.. Goal R2.4: Collect and analyze patient information. OBJ R2.4.1 (Analysis) Collect and organize all patient-specific information needed by the pharmacist to prevent, detect, and resolve medicationrelated problems and to make appropriate evidence-based, patient-centered medication therapy recommendations as part of the interdisciplinary team. OBJ R2.4.2 (Analysis) Determine the presence of any of the following medication therapy problems in a patient's current medication therapy: 1. Medication used with no medical indication 2. Patient has medical conditions for which there is no medication prescribed 3. Medication prescribed inappropriately for a particular medical condition 4. Immunization regimen is incomplete 5. Current medication therapy regimen contains something inappropriate (dose, dosage form, duration, schedule, route of administration, method of administration) 6. There is therapeutic duplication 7. Medication to which the patient is allergic has been prescribed 8. There are adverse drug or device-related events or potential for such events 9. There are clinically significant drug-drug, drug-disease, drug-nutrient, or drug-laboratory test interactions or potential for such interactions 10. Medical therapy has been interfered with by social, recreational, nonprescription, or nontraditional drug use by the patient or others 11. Patient not receiving full benefit of prescribed medication therapy 12. There are problems arising from the financial impact of medication therapy on the patient 13. Patient lacks understanding of medication therapy 14. Patient not adhering to medication regimen OBJ R2.4.3 (Analysis) Using an organized collection of patient-specific information, summarize patients health care needs. Goal R2.5: When necessary, make and follow up on patient referrals. OBJ R2.5.1 (Evaluation) When presented with a patient with health care needs that cannot be met by the pharmacist, make a referral to the appropriate health care provider based on the patient s acuity and the presenting problem. OBJ R2.5.2 (Synthesis) Devise a plan for follow-up for a referred patient. Goal R2.6: Design evidence-based therapeutic regimens. OBJ R2.6.1 (Synthesis) Specify therapeutic goals for a patient incorporating the principles of evidence-based medicine that integrate patient-specific

data, disease and medication-specific information, ethics, and quality-of-life considerations. OBJ R2.6.2 (Synthesis) Design a patient-centered regimen that meets the evidence-based therapeutic goals established for a patient; integrates patient-specific information, disease and drug information, ethical issues and quality-of-life issues; and considers pharmacoeconomic principles.. Goal R2.7: Design evidence-based monitoring plans. OBJ R2.7.1 (Synthesis) Design a patient-centered, evidenced-based monitoring plan for a therapeutic regimen that effectively evaluates achievement of the patient-specific goals. Goal R2.8: Recommend or communicate regimens and monitoring plans. OBJ R2.8.1 (Application) Recommend or communicate a patient-centered, evidence-based therapeutic regimen and corresponding monitoring plan to other members of the interdisciplinary team and patients in a way that is systematic, logical, accurate, timely, and secures consensus from the team and patient. Goal R2.9: Implement regimens and monitoring plans. OBJ R2.9.1 (Application) When appropriate, initiate the patient-centered, evidence-based therapeutic regimen and monitoring plan for a patient according to the organization's policies and procedures. OBJ R2.9.2 (Application) Use effective patient education techniques to provide counseling to patients and caregivers, including information on medication therapy, adverse effects, compliance, appropriate use, handling, and medication administration. Goal R2.10: Evaluate patients progress and redesign regimens and monitoring plans. OBJ R2.10.1 (Evaluation) Accurately assess the patient s progress toward the therapeutic goal(s). OBJ R2.10.2 (Synthesis) Redesign a patient-centered, evidence-based therapeutic plan as necessary based on evaluation of monitoring data and therapeutic outcomes. Goal R2.11: Communicate ongoing patient information. OBJ R2.11.1 (Application) When given a patient who is transitioning from one health care setting to another, communicate pertinent pharmacotherapeutic information to the receiving health care professionals. OBJ R2.11.2 (Application) Ensure that accurate and timely medication-specific information regarding a specific patient reaches those who need it at the appropriate time. Goal R2.12: Document direct patient care activities appropriately. OBJ R2.12.1 (Analysis) Appropriately select direct patient-care activities for documentation. OBJ R2.12.2 (Application) Use effective communication practices when documenting a direct patient-care activity. OBJ R2.12.3 (Comprehension) Explain the characteristics of exemplary documentation systems that may be used in the organization s environment.

Outcome R3: Exercise leadership and practice management skills. Goal R3.1: Exhibit essential personal skills of a practice leader. OBJ R.3.1.1 (Characterization) Practice self-managed continuing professional development with the goal of improving the quality of one s own performance through self-assessment and personal change. OBJ R3.1.2 (Characterization) Demonstrate pride in and commitment to the profession through appearance, personal conduct, and association membership. OBJ R3.1.3 (Characterization) Act ethically in the conduct of all job-related activities. Goal R3.2: Contribute to departmental leadership and management activities. OBJ R3.2.1 (Synthesis) Participate in the pharmacy department's planning processes. OBJ R3.2.2 (Comprehension) Explain the effect of accreditation, legal, regulatory, and safety requirements on practice. OBJ R3.2.3 (Comprehension) Explain the principles of financial management of a pharmacy department. OBJ R3.2.4 (Synthesis) Prioritize the work load, organize the work flow, and check the accuracy of the work of pharmacy technical and clerical personnel or others. Goal R3.3: Exercise practice leadership. OBJ R3.3.1 (Synthesis) Use knowledge of an organization's political and decision-making structure to influence accomplishing a practice area goal. OBJ R3.3.2 (Comprehension) Explain various leadership philosophies that effectively support direct patient care and pharmacy practice excellence. OBJ R3.3.3 (Application) Use group participation skills when leading or working as a member of a committee or informal work group. OBJ R3.3.4 (Application) Use knowledge of the principles of change management to achieve organizational, departmental, and/or team goals. Outcome R4: Demonstrate project management skills. Goal R4.1: Conduct a practice-related project using effective project management skills. OBJ R4.1.1: (Synthesis) Identify a topic for a practice-related project of significance for pharmacy practice. OBJ R4.1.2: (Synthesis) Formulate a feasible design for a practice-related project. OBJ R4.1.3: (Synthesis) Secure any necessary approvals, including IRB and funding, for one s design of a practice-related project. OBJ R4.1.4: (Synthesis) Implement a practice-related project as specified in its design. OBJ R4.1.5: (Synthesis) Effectively present the results of a practice-related project. OBJ R4.1.6: (Synthesis) Successfully employ accepted manuscript style to prepare a final report of a practice-related project.

OBJ R4.1.7: (Evaluation) Accurately assess the impact, including sustainability if applicable, of the residency project. Outcome R5: Provide medication and practice-related education/training. Goal R5.1 Provide effective medication and practice-related education, training, or counseling to patients, caregivers, health care professionals, and the public. OBJ R5.1.1 (Application) Use effective educational techniques in the design of all educational activities. OBJ R5.1.2 (Synthesis) Design an assessment strategy that appropriately measures the specified objectives for education or training and fits the learning situation. OBJ R5.1.3 (Application) Use skill in the four preceptor roles employed in practice-based teaching (direct instruction, modeling, coaching, and facilitation). OBJ R5.1.4 (Application) Use skill in case-based teaching. OBJ R5.1.5 (Application) Use public speaking skills to speak effectively in large and small group situations. OBJ R5.1.6 (Application) Use knowledge of audio-visual aids and handouts to enhance the effectiveness of communications. Outcome R6: Utilize medical informatics. Goal R6.1: Use information technology to make decisions and reduce error. OBJ R6.1.1 (Comprehension) Explain security and patient protections such as access control, data security, data encryption, HIPAA privacy regulations, as well as ethical and legal issues related to the use of information technology in pharmacy practice. OBJ R6.1.2 (Application) Exercise skill in basic use of databases and data analysis software. OBJ R6.1.3 (Evaluation) Successfully make decisions using electronic data and information from internal information databases, external online databases, and the Internet. Potential Electives for PGY1 Pharmacy Residency Programs Outcome E1: Conduct pharmacy practice research. Goal E1.1: Design, execute, and report results of investigations of pharmacy practicerelated issues. OBJ E1.1.1 (Analysis) Identify potential practice-related issues that need to be studied. OBJ E1.1.2 (Application) Use a systematic procedure for performing a comprehensive literature search. OBJ E1.1.3 (Analysis) Draw appropriate conclusions based on a summary of a comprehensive literature search. OBJ E1.1.4 (Synthesis) Generate a research question(s) to be answered by an investigation.

OBJ E1.1.5 (Synthesis) Develop specific aims and design study methods that will answer the question(s) identified. OBJ E1.1.6 (Application) Use a systematic procedure to collect and analyze data. OBJ E1.1.7 (Evaluation) Draw valid conclusions through evaluation of the data. OBJ E1.1.8 (Synthesis) Use effective communication skills to report orally and in writing the results and recommendations of an investigation into a pharmacy practice-related issue. Goal E1.2 Participate in clinical, humanistic and economic outcomes analyses. OBJ E1.2.1 (Evaluation) Contribute to a prospective clinical, humanistic and/or economic outcomes analysis. OBJ E1.2.2 (Evaluation) Contribute to a retrospective clinical, humanistic, and/or economic outcomes analysis. Outcome E2: Exercise added leadership and practice management skills. Goal E2.1: Contribute to the development of a new pharmacy service or to the enhancement of an existing service. OBJ E2.1.1 (Evaluation) Appraise a current pharmacy service or program to determine if it meets the stated goals. OBJ E2.1.2 (Synthesis) Participate in the writing of a proposal for a marketable, new or enhanced pharmacy service. OBJ E2.1.3 (Synthesis) Formulate an effective strategy for promoting a proposal for a new service. Goal E2.2: Understand the pharmacy procurement process. OBJ E2.2.1 (Comprehension) Explain the processes and contractural relationships that form the structure of the department s medication procurement system... Goal E2.3: Manage the use of investigational drug products (medications, devices, and biologicals). OBJ E2.3.1 (Application) Manage the use of investigational drug products (medications, devices, and biologicals) according to regulatory requirements, established protocols and the organization s policies and procedures. Goal E2.4: Understand the principles of a systematic approach to staff development in pharmacy practice. OBJ E2.4.1 (Comprehension) Explain the steps in a systematic approach to staff development. OBJ E2.4.2 (Comprehension) Explain the importance of approaching staff development systematically. Goal E2.5: Resolve conflicts through negotiation. OBJ E2.5.1 (Application) Use effective negotiation skills to resolve conflicts. Goal E2.6: Understand the process of managing the practice area's human resources. OBJ E2.6.1 (Comprehension) Explain recruitment strategies for a specific position.

OBJ E2.6.2 (Comprehension) Explain the process used to interview and recommend personnel for employment. OBJ E2.6.3 (Comprehension) Explain the importance of orientation and training for practice area personnel. OBJ E2.6.4 (Comprehension) Explain the components of an employee performance evaluation system. OBJ E2.6.5 (Comprehension) Explain the principles and application of a progressive discipline process. Goal E2.7: Understand the process of establishing a pharmacy residency program. OBJ E2.7.1 (Comprehension) Explain the steps involved in establishing a pharmacy residency program at a particular site. Outcome E3: Demonstrate knowledge and skills particular to generalist practice in the home care practice environment. Goal E3.1: Understand the scope of services that might be provided in a typical home care practice. OBJ E3.1.1 (Comprehension) Compare and contrast the scope of services that might be provided by a typical home care practice for a variety of health systems or stand-alone organizations. OBJ E3.1.2 (Comprehension) Explain the relationship between the scope of services offered by a home care practice and the applicable legal, regulatory, and accreditation issues. Goal E3.2: Determine the suitability of individual patients for home care. OBJ E3.2.1 (Analysis) Collect and organize all patient-specific information needed by the home care pharmacist to determine the suitability of individual patients for home care. OBJ E3.2.2...(Evaluation) Assess patients suitability for home care. Goal E3.3: Understand unique aspects of providing evidence-based, patient-centered medication therapy management with interdisciplinary teams in the home care environment. OBJ E3.3.1: (Comprehension) Explain those aspects of providing evidence-based, patient-centered medication therapy management with interdisciplinary teams that are unique to the home care environment. Goal E3.4: Understand unique aspects of preparing and dispensing medications for home care patients. OBJ E3.4.1: (Comprehension) Explain those aspects of preparing and dispensing medications that are unique to the home care environment. Goal E3.5: Understand unique aspects of participating in the management of medical emergencies occurring in the home care environment. OBJ E3.5.1 (Comprehension) Explain those aspects of participating in the management of medical emergencies that are unique when the medical emergency occurs in a home care setting. Goal E3.6: Manage the use, maintenance, and troubleshooting of medication administration equipment and medication-related equipment used in the management of home care patients.

OBJ E3.6.1 (Synthesis) Solve operational problems related to the use and maintenance of medication administration equipment and medication-related equipment used in the management of home care patients. OBJ E3.6.2 (Analysis) Participate in the development of criteria for selection of medication administration and medication-related equipment. Goal E3.7: Understand the appropriate relationship between the home care pharmacist and home care suppliers. OBJ E3.7.1 (Comprehension) Explain the role of the home care pharmacist in establishing policies for working with the pharmaceutical industry. OBJ E3.7.2 (Comprehension) Explain the role of the home care pharmacist in establishing policies for working with the manufacturers of medicationuse related equipment and supplies used in home care. Goal E3.8: Appreciate the complexity of the financial environment of home care practice. OBJ E3.8.1 (Comprehension) Explain various factors that affect the financial environment of home care practice. OBJ E3.8.2 (Comprehension) Explain the different types of payers in home care and the effect of that mix on the finances of the home care practice OBJ E3.8.3 (Comprehension) Explain the ethical and pharmaceutical issues involved in providing home care to patients with little or no insurance coverage. OBJ E3.8.4 (Comprehension) Explain the effect of patient mix (therapy type) on profitability. OBJ E3.8.5 (Knowledge) Identify resources for financial and reimbursement advice when working in the home care environment. Goal E3.9: Conduct ethical informational and marketing visits to payers, potential referral sources, and patients of the home care organization. OBJ E3.9.1 (Synthesis) Formulate effective strategies for conducting ethical informational and marketing visits to payers, potential referral sources, and patients of the home care organization. OBJ E3.9.2 (Application) Use effective presentation techniques to conduct ethical informational or marketing visits to payers, potential referral sources, and patients of the home care organization. Outcome E4: Demonstrate knowledge and skills particular to generalist practice in the managed care practice environment. Goal E4.1: Maintain confidentiality of patient and proprietary business information. OBJ E4.1.1 (Application) Observe legal and ethical guidelines for safeguarding the confidentiality of patient information. OBJ E4.1.2 (Application) Observe health system policy for the safeguarding of proprietary business information. Goal E4.2: Understand the interrelationship of the pharmacy benefit management company, the health plan, and the delivery system functions of managed care. OBJ E4.2.1 (Comprehension) Explain the health-plan functions of managed care, including benefit design and management, co-pay, formulary coverage, prior

authorization, access, and contract negotiations (medication acquisition and/or network pharmacy). OBJ E4.2.2 (Comprehension) Explain the effect that the health plan has on the delivery functions of managed care. OBJ E4.2.3 (Comprehension) Explain the interrelationship of the health plan and the delivery system functions of managed care. Goal E4.3: Understand unique aspects of providing evidence-based, patient-centered medication therapy management with interdisciplinary teams in the managed care environment. Outcome E5: Participate in the management of medical emergencies. Goal E5.1: Participate in the management of medical emergencies. OBJ E5.1.1 (Evaluation) Exercise skill as a team member in the management of medical emergencies according to the organization s policies and procedures. Outcome E6: Provide drug information to health care professionals and/or the public. Goal E6 1 Identify a core library, including electronic media, appropriate for a specific practice setting. OBJ E6.1.1 (Application) Use knowledge of standard resources to select a core library of primary, secondary, and tertiary references appropriate for a specific practice setting. Goal E6.2: Design and deliver programs that contribute to public health efforts. OBJ E6.2.1 (Comprehension) Explain the pharmacist s role in public health, including specific contributions to public health efforts that can be made by health-system pharmacists. OBJ E6.2.2 (Synthesis) Design and deliver programs for health care consumers that center on disease prevention and wellness promotion. OBJ E6.2.3 (Synthesis) Participate in the development of organizational plans for emergency preparedness. Outcome E7: Demonstrate additional competencies that contribute to working successfully in the health care environment. Goal E7.1: Use approaches in all communications that display sensitivity to the cultural and personal characteristics of patients, caregivers, and health care colleagues. OBJ E7.1.1 (Organization) Demonstrate sensitivity to the perspective of the patient, caregiver, or health care colleague in all communications. Goal E7.2: Communicate effectively. OBJ E7.2.1 (Analysis) Use an understanding of effectiveness, efficiency, customary practice and the recipient's preferences to determine the appropriate type of, and medium and organization for, communication. OBJ E7.2.2 (Complex Overt Response) Speak clearly and distinctly in grammatically correct English or the alternate primary language of the practice site. OBJ E7.2.3 (Application) Use listening skills effectively in performing job functions.

OBJ E7.2.4 (Application) Use correct grammar, punctuation, spelling, style, and formatting conventions in preparing all written communications. Goal E7.3: Balance obligations to oneself, relationships, and work in a way that minimizes stress. OBJ E7.3.1 (Synthesis) Devise an effective plan for minimizing stress while attending to personal needs, maintaining relationships, and meeting professional obligations. Goal E7.4: Manage time effectively to fulfill practice responsibilities. OBJ E7.4.1 (Application) Use time management skills effectively to fulfill practice responsibilities. Goal E7.5: Make effective use of available software and information systems. OBJ E7.5.1 (Application) Successfully search, retrieve, and manage electronic data from internal information databases, external online databases, and the Internet. OBJ E7.5.2 (Application) Exercise skill in the use of the organization s wordprocessing, spreadsheet, and presentation software. OBJ E7.5.3 (Comprehension) Explain how an effectively functioning organizational information system is structured.

Appendix D: A Sample of Numbered and Classified by Taxonomy and Level of Learning Required and Elective Educational Outcomes, Goals, and Objectives Plus Instructional Objectives to Assist with Teaching -- for Use with RLS Explanation of the Contents of This Document Educational Outcomes (Outcome): Educational outcomes are statements of broad categories of the residency graduates capabilities. Educational Goals (Goal): Educational goals listed under each educational outcome are broad sweeping statements of abilities Educational Objectives OBJ: Resident achievement of educational goals is determined by assessment of the resident s ability to perform the associated educational objectives below each educational goal.. Each objective is classified by taxonomy (cognitive, affective, or psychomotor) and level of learning within that taxonomy to facilitate teaching and assessment of performance. Instructional Objectives IO: Instructional objectives (text written in unbolded italics) are the result of a learning analysis of each of the educational objectives. They are offered as a resource for preceptors encountering difficulty in helping residents achieve a particular educational objective. The instructional objectives falling below the educational objectives suggest knowledge and skills required for successful performance of the educational objective that the resident may not possess upon entering the residency year. Instructional objectives are teaching tools only. They are not required in any way nor are they meant to be evaluated. Required By PGY1 Standard Outcome R1: Manage and improve the medication-use process. Goal R1.1: Identify opportunities for improvement of the organization s medicationuse system. OBJ R1.1.1 (Comprehension) Explain the organization s medication-use system and its vulnerabilities to adverse drug events (ADEs). IO Explain the central concepts of systems theory. IO Explain the concept of system error. IO Explain the definitions of the various terms associated with adverse drug events (e.g., medication misadventure, medication error, adverse drug reaction, error, accident, systems error, individual error, latent error).

IO State sources of information on the design, implementation, and maintenance of safe medication-use systems. IO From both the pharmacy department perspective and the organization perspective explain the potential for contribution to the occurrence of adverse drug events by the use of automation and information technology. IO From both the pharmacy department perspective and the organization perspective explain the role that automation and information technology play in preventing adverse drug events. IO Explain the meaning of the term culture of safety. OBJ R1.1.2 (Analysis) Analyze the structure and process and measure outcomes of the medication-use system. IO Explain methods for analyzing a medication-use system s structure. IO Explain how inputs to the medication-use system such as patients, staff, and environment make up its structure. IO Explain methods for analyzing processes within a medication-use system (e.g., root cause analysis, failure mode and effect analysis). IO Explain how the interactions between clinicians and patients constitute processes in the medication-use system. IO Exercise skill in process-mapping, a type of flowchart depicting the steps in a process, with identification of responsibility for each step and the key measures IO Exercise skill in cause-and-effect diagramming. IO Explain the organization s policies and procedures for handling a drug recall. IO Explain the role of medication-use evaluation (MUE) in measuring medication-use processes. IO Explain methods for measuring outcomes of the medication-use system. IO Generate examples of the outcomes of a medication-use process which are changes in patients health status (e.g. length of stay; acuity). IO Explain the characteristics of a clinically significant ADE. IO Explain various methods, including decision trees, for determining the significance of adverse drug events. IO Explain how to categorize medication errors using the ASHP Guidelines on Preventing Medication Errors in Hospitals. IO Explain how to categorize medication errors using the National Coordinating Council for Medication Error Reporting and Prevention's medication error index for categorizing errors. IO Explain how to categorize medication errors using one s own institution s categorization methodology.

IO When a clinically significant ADE is identified, report the event following the organization s policies and procedures. IO Explain the role of the MUE in measuring outcomes of the medication-use process. OBJ R1.1.3 (Evaluation) Identify opportunities for improvement in the organization s medication-use system by comparing the medication-use system to relevant best practices. IO When a clinically significant ADE is identified, participate in determining the presence of any similar potential ADEs. IO Participate in the pharmacy department s ongoing process for tracking and trending ADEs. IO Explain how basic safety design principles such as standardization, simplification, and the employment of human factors training can minimize the incidence of error in the medication-use process. IO Explain safe practices for selecting and securing alternative medications when shortages occur and for adjusting the formulary and notifying prescribers. IO Explain safe practices for the storage, dispensing, administration, and security of pharmaceuticals. IO Use the results of an MUE to identify opportunities for improvement in the medication-use process. IO Explain how to use information on how to design, implement, and maintain safe medication-use systems from external sources to identify opportunities for improvement in the organization s medication-use system. Goal R2.1: As appropriate, establish collaborative professional relationships with members of the health care team. OBJ R2.1.1 (Synthesis) Implement a strategy that effectively establishes cooperative, collaborative, and communicative working relationships with members of interdisciplinary health care teams. IO Demonstrate knowledge of other team members expertise, background, knowledge, and values in all interdisciplinary team interactions. IO Explain the training and expected areas of expertise of the members of the interdisciplinary with which one works. IO For each of the professions with which one interacts on an interdisciplinary team, explain the profession s view of its role and responsibilities in collaborations on patientcentered care. IO Exercise skill in the use of individual roles and processes required to work collaboratively on interdisciplinary teams. IO Define a collaborative professional working relationship.

IO Explain the structures and content of collaborative working relationships that are possible between the pharmacist and the physician and between the pharmacist and other health care professionals. IO Explain the limits that are imposed on possible collaborative relationships by the presence or absence of guidelines, legal and regulatory requirements, and organizational policies and procedures. IO Exercise skill in the use of group techniques to include communication, negotiation, delegation, time management, assessment of group dynamics, and consensus building. IO Explain the principles and applications of negotiation as they apply to interdisciplinary team work. IO Explain the principles and applications of delegation as they apply to interdisciplinary team work. IO Explain the principles and applications of time management as they apply to interdisciplinary team work. IO Explain the principles of group dynamics and how they apply to interdisciplinary team work. IO Explain the principles of conflict management and how they apply to interdisciplinary team work. IO Explain a systematic approach to building consensus. IO Explain how interdisciplinary team members develop unique communication patterns (shared language). IO Explain the importance of adhering to use of an interdisciplinary team s shared language. IO Exercise skill in the coordination and integration of pharmacist s care with the contributions of other members of the interdisciplinary team. Goal R2.2: Place practice priority on the delivery of patient-centered care to patients. OBJ R2.2.1 (Organization) Choose and manage daily activities so that they reflect a priority on the delivery of appropriate patient-centered care to each patient. IO Explain the meaning of patient-centered care and the rationale for its use. IO Explain methods for prioritizing the delivery of care to patients when time or resources prohibit the delivery of full direct patient care services to all patients. Goal R2.3: As appropriate, establish collaborative professional pharmacist-patient relationships. OBJ R2.3.1. (Synthesis) Formulate a strategy that effectively establishes a patient-centered pharmacist-patient relationship. IO Explain the meaning of the term patient-centered and the rationale for its use.

Goal R2.4: OBJ R2.4.1 IO Explain the appropriate sharing of power and responsibility between the pharmacist, patient and caregivers in a patientcentered, pharmacist-patient relationship. IO Explain why it is important that the pharmacist communicate with the patient in a shared and fully open manner in a patientcentered, pharmacist-patient relationship. IO Explain the role of demonstrating respect for the patient s individuality, emotional needs, values, and life issues in a patientcentered, pharmacist-patient relationship. Collect and analyze patient information. (Analysis) Collect and organize all patient-specific information needed by the pharmacist to prevent, detect, and resolve medicationrelated problems and to make appropriate evidence-based, patient-centered medication therapy recommendations as part of the interdisciplinary team. IO IO IO IO IO IO Identify the types of patient-specific information the pharmacist requires to prevent, detect, and resolve medication-related problems and to make appropriate evidence-based, patientcentered medication therapy recommendations as part of the interdisciplinary team. IO IO IO Explain the role of collecting information regarding the patient s culture, emotional needs, preferences, values, and life issues in formulating evidence-based, patient-centered care decisions. Explain patient or disease specifics that would require the pharmacist to collect pharmacogenomic and/or pharmacogenetic information. Explain issues surrounding confidentiality of patient information and the impact of HIPPA regulations on the collection and safeguarding of patient-specific information. Explain signs and symptoms, epidemiology, risk factors, pathogenesis, natural history of disease, pathophysiology, clinical course, etiology, and treatment of diseases commonly encountered. Explain the mechanism of action, pharmacokinetics, pharmacodynamics, pharmacoeconomics, usual regimen (dose, schedule, form, route, and method of administration), indications, contraindications, interactions, adverse reactions, and therapeutics of medications in the treatment of diseases commonly encountered. Explain current trends and issues in nontraditional therapy. Use standard patient medical charts, records and/or internal electronic information databases to collect information that may be pertinent to prevent, detect, and resolve medication-related problems and to make informed evidence-based, patient-centered medication therapy recommendations to an interdisciplinary team. Integrate effective communication techniques in interviews with patients, caregivers, health care professionals, or others so that

the patient-specific information needed by the pharmacist for evidence-based, patient-centered care is collected. IO When presented with a limited time frame (e.g., ambulatory care office visit) use an interview strategy that elicits maximum pertinent information IO Explain effective phone techniques to be used to obtain information for the patient database. IO Explain the impact of having discontinuous or fragmented patient-care information when developing an interview strategy for patients (e.g., patient seeing multiple caregivers, last visit 6 months ago). IO Distinguish the meaning of non-verbal cues in patient encounters (e.g., broken sentences in an asthmatic patient, difficult ambulation in an arthritic patient). IO When appropriate, measure patient vital signs and use appropriate physical assessment skills. IO Determine the most reputable and credible source of required patient-specific information. IO Record required patient-specific information in a manner that facilitates detecting and resolving medication-related problems and making appropriate evidence-based, patient-centered medication therapy recommendations to an interdisciplinary team. IO In a setting where none exists, create an effective organizational system for recording patient-specific data. OBJ R2.4.2 (Analysis) Determine the presence of any of the following medication therapy problems in a patient's current medication therapy: 1. Medication used with no medical indication 2. Patient has medical conditions for which there is no medication prescribed 3. Medication prescribed inappropriately for a particular medical condition 4. Immunization regimen is incomplete 5. Current medication therapy regimen contains something inappropriate (dose, dosage form, duration, schedule, route of administration, method of administration) 6. There is therapeutic duplication 7. Medication to which the patient is allergic has been prescribed 8. There are adverse drug or device-related events or potential for such events 9. There are clinically significant drug-drug, drug-disease, drugnutrient, or drug-laboratory test interactions or potential for such interactions 10. Medical therapy has been interfered with by social, recreational, nonprescription, or nontraditional drug use by the patient or others

11. Patient not receiving full benefit of prescribed medication therapy 12. There are problems arising from the financial impact of medication therapy on the patient 13. Patient lacks understanding of medication therapy 14. Patient not adhering to medication regimen IO Explain psychological, cultural, and economic factors that influence patient compliance with prescribed medications. IO Explain factors to consider when comparing the benefits and risks of an alternative medication therapy. IO Explain factors to consider when trying determining the likelihood that a reaction is occurring because of a medication. IO Assess criteria for assessing the severity of an adverse drug reaction. IO Explain acceptable approaches to the therapeutic management of an adverse drug reaction. IO) Explain mechanisms of determining therapeutic consequence resulting from defective medications or drug products (e.g., exacerbation of asthma due to a defective inhaler). IO Use a functional format to list patients' pharmacotherapy problems. IO Prioritize patients' pharmacotherapy problems. OBJ R2.4.3 (Analysis) Using an organized collection of patient-specific information, summarize patients health care needs. Goal R2.5: When necessary, make and follow up on patient referrals. OBJ R2.5.1 (Evaluation) When presented with a patient with health care needs that cannot be met by the pharmacist, make a referral to the appropriate health care provider based on the patient s acuity and the presenting problem. IO Explain the organization s process for making a patient referral. IO Explain the information needed to make an appropriate referral. IO Explain a systematic process for assessing the acuity of a patient s illness. OBJ R2.5.2 (Synthesis) Devise a plan for follow-up for a referred patient. IO Explain the importance of following up on patients who are referred to other health care providers. IO Explain the importance of integrating follow-up information into the long-term management plan. Goal R2.6: Design evidence-based therapeutic regimens. OBJ R2.6.1 (Synthesis) Specify therapeutic goals for a patient incorporating the principles of evidence-based medicine that integrate patient-specific data, disease and medication-specific information, ethics, and quality-oflife considerations. IO Explain the use of evidence-based consensus statements and guidelines in the setting of patient-specific therapeutic goals. IO Explain how culture influences patients perceptions of desirable outcomes.

IO Explain the importance of the patient's perception of desirable outcomes when setting therapeutic goals for a patient with functional limitations. IO Explain the impact of quality-of-life issues on making decisions about therapeutic goals. IO Explain ethical issues that may need consideration when setting therapeutic goals. IO Compare and contrast the realistic limits of treatment outcomes among the various care settings. IO Explain how a patient's age or mental status might affect the setting of therapeutic goals. IO Explain how goals of others on the interdisciplinary team influence the specification and prioritization of therapeutic goals. IO Explain unique aspects of the patient s role in the ambulatory care setting in determining his/her therapeutic goals. OBJ R2.6.2 (Synthesis) Design a patient-centered regimen that meets the evidence-based therapeutic goals established for a patient; integrates patient-specific information, disease and drug information, ethical issues and quality-of-life issues; and considers pharmacoeconomic principles.. IO Explain the use of evidence-based consensus statements and guidelines in the design of patient-specific therapeutic regimens. IO Accurately interpret best evidence for use in the design of a patient-centered regimen for a specific patient. IO Explain where and how to find the best possible sources of evidence for a specific patient case. IO Explain how to conduct a search for relevant answers to a specific clinical question, including searches of resources that evaluate or appraise the evidence for its validity and usefulness with respect to a particular patient or population. IO Explain how to integrate seemingly applicable findings of best evidence with clinical judgment to arrive at an optimal evidencebased regimen for a specific patient. IO Explain how culture influences patients perception of disease and how this affects responses to various symptoms, diseases, and treatments. IO Explain how patient-specific pharmacogenomics and pharmacogenetics may influence the design of patients medication regimens. IO Explain additional concerns with compliance, cost, and route of administration when making decisions on medication regimens. Goal R2.7: Design evidence-based monitoring plans. OBJ R2.7.1 (Synthesis) Design a patient-centered, evidenced-based monitoring plan for a therapeutic regimen that effectively evaluates achievement of the patient-specific goals.

IO Explain the use of evidence-based consensus statements and guidelines in the design of patient-specific monitoring plans. IO Explain cultural and social issues that should be considered when designing a monitoring plan. IO Explain the importance of considering what is feasible and useful when designing a monitoring plan. IO Compare and contrast various methods for monitoring patient adherence (e.g., refill rates, questioning, return demonstration). IO Determine monitoring parameters that will measure achievement of goals for a therapeutic regimen. IO State customary drug-specific monitoring parameters for medical regimens commonly prescribed. IO Explain the relationship between what are normal value ranges for parameters and the influence on those ranges by a given disease state. IO Identify the most reliable sources of data for measuring the selected parameters. IO Define a desirable value range for each selected parameter, taking into account patient-specific information. IO Explain factors that should influence the frequency and timing of parameter measurements in monitoring plans. IO Explain effective approaches to assuring patient return for followup visits in the ambulatory setting. IO Identify the most appropriate person to collect monitoring data (e.g., family member, nurse, patient). Goal R2.8: Recommend or communicate regimens and monitoring plans. OBJ R2.8.1 (Application) Recommend or communicate a patient-centered, evidence-based therapeutic regimen and corresponding monitoring plan to other members of the interdisciplinary team and patients in a way that is systematic, logical, accurate, timely, and secures consensus from the team and patient. IO Explain the right of patients to refuse a treatment. IO Explain the importance of explicitly citing the use of best evidence when recommending or communicating a patient s regimen and monitoring plan. IO Explain what would be a pharmacist s responsible professional behavior in the circumstance that a patient refuses a proposed treatment. IO Differentiate between circumstances where documenting in the chart is sufficient and when communication to team members requires immediacy. Goal R2.9: Implement regimens and monitoring plans. OBJ R2.9.1 (Application) When appropriate, initiate the patient-centered, evidence-based therapeutic regimen and monitoring plan for a patient according to the organization's policies and procedures.

IO Explain the requirements for a situation in which it is appropriate for the pharmacist to initiate a medication-therapy regimen. IO Explain the organization s policies and procedures for ordering tests. OBJ R2.9.2 (Application) Use effective patient education techniques to provide counseling to patients and caregivers, including information on medication therapy, adverse effects, compliance, appropriate use, handling, and medication administration. Goal R2.10: Evaluate patients progress and redesign regimens and monitoring plans. OBJ R2.10.1 (Evaluation) Accurately assess the patient s progress toward the therapeutic goal(s). IO Gather data as specified in a monitoring plan. IO Explain factors that may contribute to the unreliability of monitoring results (e.g., patient-specific factors, timing of monitoring tests, equipment errors, and outpatient versus inpatient monitoring.) IO Determine reasons for a patient s progress or lack of progress toward the stated health care goal. IO Explain the importance of the analysis of trends over time in monitoring parameter measurements. IO Accurately assess the effectiveness of a patient-specific education program. IO Explain methods for assessing the effects of patient-specific education. OBJ R2.10.2 (Synthesis) Redesign a patient-centered, evidence-based therapeutic plan as necessary based on evaluation of monitoring data and therapeutic outcomes. Goal R2.11: Communicate ongoing patient information. OBJ R2.11.1 (Application) When given a patient who is transitioning from one health care setting to another, communicate pertinent pharmacotherapeutic information to the receiving health care professionals. OBJ R2.11.2 (Application) Ensure that accurate and timely medication-specific information regarding a specific patient reaches those who need it at the appropriate time. IO Explain the importance of effective communication of modifications of the therapeutic plan to the patient and members of the interdisciplinary team. IO Determine instances in which there is urgency in communicating the results of monitoring to the interdisciplinary team. Goal R2.12: Document direct patient care activities appropriately. OBJ R2.12.1 (Analysis) Appropriately select direct patient-care activities for documentation. OBJ R2.12.2 (Application) Use effective communication practices when documenting a direct patient-care activity.

OBJ R2.12.3 (Comprehension) Explain the characteristics of exemplary documentation systems that may be used in the organization s environment. Outcome R5: Provide medication and practice-related education/training. Goal R5.1 Provide effective medication and practice-related education, training, or counseling to patients, caregivers, health care professionals, and the public. OBJ R5.1.1 (Application) Use effective educational techniques in the design of all educational activities. IO Design instruction that meets the individual learner s needs. IO When given a particular patient data base, therapeutic regimen, and monitoring plan, explain the educational needs of the patient for successful implementation of the therapeutic regimen and monitoring plan. IO Explain the concept of learning styles and its influence on the design of instruction. IO Explain the importance of considering the learner s reading level when designing patient education. IO Write appropriately worded educational objectives. IO Design instruction to reflect the specified objectives for education or training. IO Explain the match between instructional delivery systems (e.g., demonstration, written materials, videotapes) and specific types of learning commonly required of patients. IO Design instruction that employs strategies, methods, and techniques congruent with the objectives for education or training. IO Explain effective teaching approaches for the various types of learning required of patients (e.g., imparting information, teaching psychomotor skills, inculcation of new attitudes). OBJ R5.1.2 (Synthesis) Design an assessment strategy that appropriately measures the specified objectives for education or training and fits the learning situation. IO Explain appropriate assessment techniques for assessing the learning outcomes of pharmacist-provided educational or training programs. OBJ R5.1.3 (Application) Use skill in the four preceptor roles employed in practice-based teaching (direct instruction, modeling, coaching, and facilitation). IO Explain the stages of learning that are associated with each of the preceptor roles. OBJ R5.1.4 (Application) Use skill in case-based teaching. OBJ R5.1.5 (Application) Use public speaking skills to speak effectively in large and small group situations.

IO Explain techniques that can be used to enhance audience interest. IO Explain techniques that can be used to enhance audience understanding of one's topic. IO Explain speaker habits that distract the audience. OBJ R5.1.6 (Application) Use knowledge of audio-visual aids and handouts to enhance the effectiveness of communications. IO Use a systematic and educationally sound method for determining when it is appropriate to use handouts or visual aids and for selecting the appropriate aid. IO Explain accepted conventions for the design of visual aids and handouts. IO Exercise skill in the operation of audio-visual equipment.

Glossary Adverse drug event (ADE) -- an injury from a medicine (or lack of an intended medicine). (ASHP. Suggested definitions and relationships among medication misadventures, medication errors, adverse drug events, and adverse drug reactions. AJHP, 1998; 55:165-6.) Culture -- an integrated system of learned behavior patterns that are characteristic of the members of any particular group. It is more than race or ethnicity. Culture includes race or customs, rituals, food, religion, and music; and, in addition, it includes health beliefs and practices, death and birth rituals, structure, and dynamics, social practices and beliefs that define personal space, eye contact, time orientation, and nonverbal communication behaviors. (Randall-David E. Culturally competent HIV counseling and education. Material & Child Health Clearinghouse: McLean, VA: 1994) Cultural competency -- is more than cultural awareness or cultural sensitivity, competency implies skills and expertise to work with and within diverse cultural groups with sensitivity and effectiveness. In its most developed meaning cultural competence includes advocacy. (Randall-David E. Culturally competent HIV counseling and education. Material & Child Health Clearinghouse: McLean, VA: 1994) Evidence-based medicine -- the integration of best research evidence, clinical expertise, and patient values in making decisions about the care of individual patients (Institute of medicine, 2001; Straus and Sackett, 1998). Best research evidence includes evidence that can be quantified, such as that from randomized controlled trials, laboratory experiments, clinical trials, epidemiological research, and outcomes research and evidence derived from the practice knowledge of experts, including inductive reasoning (Guyatt et al., Higgs et al., 2001). Clinical expertise is derived from the knowledge and experience developed over time from practice, including inductive reasoning. Patient values and circumstances are the unique preferences, concerns, expectations, financial resources, and social supports that are brought by each patient to a clinical encounter. (Institute of Medicine. Health professions education: a bridge to quality. Washington, DC: The National Acadamies Press; 2001.) Interdisciplinary team -- a team composed of members from different professions and occupations with varied and specialized knowledge, skills, and methods. The team members integrate their observations, bodies of expertise, and spheres of decision making to coordinate, collaborate, and communicate with one another in order to optimize care for a patient or group of patients. (Institute of Medicine. Health professions education: a bridge to quality. Washington, DC: The National Acadamies Press; 2001.) Leadership -- leadership practices include scanning, focusing, aligning/mobilizing, and inspiring. Scanning: Identify client and stakeholder needs and priorities. Recognize trends, opportunities, and risks.

Look for best practices. Identify staff capacities and constraints. Know yourself, your staff, and your organization values, strengths, and weaknesses. Focusing: Articulate the organizations mission and strategy. Identify critical challenges. Link goals with the overall organizational strategy. Determine key priorities for action Create a common picture of desired results. Aligning/Mobilizing: Ensure congruence of values, mission, strategy, structure, systems and daily actions. Facilitate teamwork. Unite key stakeholders around an inspiring vision. Link goals with rewards and recognition. Enlist stakeholders to commit resources. Inspiring: Match deeds to words. Demonstrate honest in interactions. Show trust and confidence in staff, acknowledge the contributions of others. Provide staff with challenges, feedback and support. Be a model of creativity, innovation, and learning (Management and Leadership Program. Leading and managing framework. Management Sciences for Health, Ballston, VA. 2004.) Management -- management practices include planning, organizing, implementing, and monitoring and evaluating. Planning: Set short-term organizational goals and performance objectives. Develop multi-year and annual plans Allocate adequate resources (money, people, and materials). Anticipate and reduce risks. Organizing: Ensure a structure that provides accountability and delineates authority. Ensure that systems for human resource management, finance, logistics, quality assurance, operations, information, and marketing effectively support the plan. Strengthen work processes to implement the plan. Align staff capacities with planned activities. Implementing: Integrate systems and coordinate work flow. Balance competing demands. Routinely use data for decision making.

Coordinate activities with programs and sectors. Adjust plans and resources as circumstances change. Monitoring and Evaluating: Monitor and reflect on progress against plans. Provide feedback. Identify needed changes Improve work processes, procedures, and tools. (Management and Leadership Program. Leading and managing framework. Management Sciences for Health, Ballston, VA. 2004.) Medical informatics -- the development and application of information technology systems to problems in health care, research, and education. (Institute of Medicine. Health professions education: a bridge to quality. Washington, DC: The National Acadamies Press; 2001.) Medication-use system - Medication use is a complex process that comprises the subprocesses of medication prescribing, order processing, dispensing, administration, and effects monitoring. The key elements that most often affect the medication use process are., patient information; drug information, communication of drug information; drug labeling, packaging and nomenclature; drug storage, stock and standardization; drug device acquisition, use and monitoring; environmental factors; competency and staff education; patient education; and quality processes and risk management. (Institute of Safe Medication Practices web site accessed May 31, 2005 http://www.ismp.org/pages/ismp_faq.html#question%207.) Patient-centered care -- identify, respect, and care about patients differences, values, preferences, and expressed needs; relieve pain and suffering; coordinate continuous care; listen to, clearly inform, communicate with, and educate patients; share decision making and management; and continuously advocate disease prevention, wellness, and promotion of healthy lifestyles, including a focus on population health. (Institute of Medicine. Health professions education: a bridge to quality. Washington, DC: The National Acadamies Press; 2001.) Pharmacy practice research includes all forms of scholarly scientific inquiry that may be performed by pharmacy residents. Broad in scope, it may include prospective or retrospective clinical studies, pharmacokinetic or pharmacodynamic studies, outcome studies, or evaluation of some aspect of pharmacy practice (e.g., impact of a new program or service). Typically, research projects should be applied in nature, using human data, but exceptions may occur. Professional -- the active demonstration of the 10 traits of a professional. 1. Knowledge and skills of a profession. 2. Commitment to self-improvement of skills and knowledge. 3. Service orientation. 4. Pride in the profession. 5. Covenantal relationship with the client.

6. Creativity and innovation. 7. Conscience and trustworthiness. 8. Accountability for his/her work. 9. Ethically sound decision making. 10. Leadership. (Ten marks of a professional working smart. New York, NY: National Institute of Business Management, March 11, 1991;17[5].). Quality -- the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge. (Institute of Medicine. Health professions education: a bridge to quality. Washington, DC: The National Acadamies Press; 2001.) Quality improvement -- identify errors and hazards in care; understand and implement basic safety design principles, such as standardization and simplification; continually understand and measure quality of care in terms of structure, process, and outcomes in relation to patient and community needs; and design and test interventions to change processes and systems of care, with the objective of improving quality. (Institute of Medicine. Health professions education: a bridge to quality. Washington, DC: The National Acadamies Press; 2001.)

Appendix E: A Sample of PGY1 Required and Elective Educational Objectives Criteria for Measuring Resident Performance Outcome R1: Manage and improve the medication-use process. Goal R1.1: Identify opportunities for improvement of the organization s medicationuse system. OBJ R1.1.1 (Comprehension) Explain the organization s medication-use system and its vulnerabilities to adverse drug events (ADEs). Explanation provides accurate definition of terms associated with adverse drug events Explanation of organization s medication-use system is clear and accurate Explanation reflects understanding of the health system s medication-use process as a system Accurately states sources of information on the design, implementation, and maintenance of safe medication-use systems Explanation reflects understanding of system error Explanation reflects understanding of human factors error Cites the potential contribution of automation and technology to preventing medication misadventures at the departmental and at the organizational levels Cites the potential contribution of automation and technology to the occurrence of medication misadventures at the departmental and organizational levels Accurately explains the meaning of the term culture of safety Suggested assessment activity: Resident discussion of the organization s medication-use system and its vulnerabilities to adverse drug events OBJ R1.1.2 (Analysis) Analyze the structure and process and measure outcomes of the medication-use system. Analyses and measurements reflect clear understanding of the differences between structure, process, and outcomes Choices of techniques and tools for analyzing structure and process and measuring outcomes are appropriate Structure, process, and outcomes are accurately analyzed Suggested assessment activity: Examination of resident s analysis of the organization s medication-use system OBJ R1.1.3 (Evaluation) Identify opportunities for improvement in the organization s medication-use system by comparing the medication-use system to relevant best practices. Best practices selected for comparison are relevant

Ideas for improvement have potential to result in significant improvements to the existing process Suggested assessment activity: Evaluation of resident s identification of opportunities for improvement in the organization s medication-use system from comparison with best practice Outcome R2: Provide evidence-based, patient-centered medication therapy management with interdisciplinary teams. Goal R2.1: As appropriate, establish collaborative professional relationships with members of the health care team. OBJ R2.1.1 (Synthesis) Implement a strategy that effectively establishes cooperative, collaborative, and communicative working relationships with members of interdisciplinary health care teams. The relationship reflects mutual respect for the others professional expertise The relationship reflects mutual respect for appropriate delegation of professional responsibilities in providing care for patients The relationship reflects appropriate integration of the pharmacist s care with the contributions of other members of the interdisciplinary team The pharmacist s work within the team reflects skillful application of group process skills such as negotiation, time management, conflict management, communication, and consensus building Suggested assessment activities: Examination of the individual relationship between the resident and a particular physician and health care provider with whom he or she interacts when fulfilling practice responsibilities through a combination of direct observation, anecdotal records, and interviews with staff Goal R2.2: Place practice priority on the delivery of patient-centered care to patients. OBJ R2.2.1 (Organization) Choose and manage daily activities so that they reflect a priority on the delivery of appropriate patient-centered care to each patient. Daily activities consistently show a priority placed on the delivery of patient-centered care Arranges work activities so that the patient-centered care needs of patients are met Suggested assessment activity: Cumulative direct observation of practice; interviews with other staff regarding choice of daily activities

Goal R2.3: As appropriate, establish collaborative professional pharmacist-patient relationships. OBJ R2.3.1. (Synthesis) Formulate a strategy that effectively establishes a patient-centered pharmacist-patient relationship. Patient evidences acceptance of his or her role and responsibilities in the making of care decisions Patient evidences understanding of the pharmacist s role and responsibilities in the making of care decisions Roles and responsibilities attributed to patient and pharmacist are appropriate for a patient-centered pharmacist-patient relationship Patient displays respect for the professional expertise of the pharmacist Pharmacist displays respect for the preferences and expressed needs of the patient Relationship reflects trust from both parties Suggested assessment activities: Examination of the relationship between the resident and a specific patient in his or her care through direct observation and interviews with staff Goal R2.4: Collect and analyze patient information. OBJ R2.4.1 (Analysis) Collect and organize all patient-specific information needed by the pharmacist to prevent, detect, and resolve medicationrelated problems and to make appropriate evidence-based, patient-centered medication therapy recommendations as part of the interdisciplinary team. Information base contains all information needed (demographic, medical, medication therapy, behavioral/lifestyle, social/economic, and administrative [e.g., physician/prescriber, informed consent, pharmacy]) Information base does not contain extraneous information Sources of information are the most reliable available Recording system is functional for subsequent problem solving and decision making Suggested assessment activity: Audit of resident s information base for a specific patient or pharmacy department s patient information base OBJ R2.4.2 (Analysis) Determine the presence of any of the following medication therapy problems in a patient's current medication therapy: 1. Medication used with no medical indication 2. Patient has medical conditions for which there is no medication prescribed 3. Medication prescribed inappropriately for a particular medical condition 4. Immunization regimen is incomplete 5. Current medication therapy regimen contains something inappropriate (dose, dosage form, duration, schedule, route of administration, method of administration) 6. There is therapeutic duplication

7. Medication to which the patient is allergic has been prescribed 8. There are adverse drug or device-related events or potential for such events 9. There are clinically significant drug-drug, drug-disease, drugnutrient, or drug-laboratory test interactions or potential for such interactions 10. Medical therapy has been interfered with by social, recreational, nonprescription, or nontraditional drug use by the patient or others 11. Patient not receiving full benefit of prescribed medication therapy 12. There are problems arising from the financial impact of medication therapy on the patient 13. Patient lacks understanding of medication therapy 14. Patient not adhering to medication regimen All medications used with no medical indication are identified All medical conditions for which there is not a medication prescribed are identified All medications inappropriately prescribed for a particular medical condition are identified All missing immunizations are identified Everything inappropriate in the current medication therapy regimen (dose, dosage form, schedule, duration, route of administration, method of administration) is identified All therapeutic duplications are identified All medications in the regimen to which the patient is allergic are identified Any presence or potential for adverse drug events is identified Any presence or potential for clinically significant drug interactions is identified Any interference with medical therapy by social, recreational, nonprescription or nontraditional medication use is identified Any instance of the patient not receiving full benefit of prescribed medication therapy is identified (e.g., system failure, clinical failure) All problems arising from the financial impact of medication therapy on the patient are identified Any lack of patient (or caregiver) understanding of his/her medication therapy is identified Any lack of patient adherence to medication regimen is identified Nothing is identified as a problem that is not a problem If medication-use problems are found, chart documentation exhibits the following characteristics:

o Written in time to be useful o Follows the health system s policies and procedures, including that entries are signed, dated, timed, legible, and concise Suggested assessment activity: Audit of patient s medical chart, resident s patient information base, or pharmacy department s patient information base OBJ R2.4.3 (Analysis) Using an organized collection of patient-specific information, summarize patients health care needs. List of needs is comprehensive Identification of health care needs integrates all relevant patient-specific and disease-specific information List of needs is concise List contains no irrelevant information Suggested assessment activity: Audit of patient s medical chart, resident s patient information base, or pharmacy department s patient information base. Goal R2.5: When necessary, make and follow up on patient referrals. OBJ R2.5.1 (Evaluation) When presented with a patient with health care needs that cannot be met by the pharmacist, make a referral to the appropriate health care provider based on the patient s acuity and the presenting problem. Referral is the correct option Referral is to the most appropriate category of health care professional to meet the patient s health care need Referral contains all pertinent information required by the recipient to take appropriate action Patient is part of the decision to refer Referral is made according to the health system s policies and procedures Suggested assessment activities: A combination of direct observation of practice; audit of patient s medical chart, resident s patient information base, or pharmacy department s patient information base; interview of patient who has been referred; and interview of health care professional to whom the resident has referred the patient OBJ R2.5.2 (Synthesis) Devise a plan for follow-up for a referred patient. Plan specifies what the follow-up activities will be Plan provides a schedule for follow-up Plan specifies how follow-up information will be integrated into the longterm management plan Suggested assessment activity: Review of the resident s plan for follow-up for a specific referred ambulatory care patient

Goal R2.6: Design evidence-based therapeutic regimens. OBJ R2.6.1 (Synthesis) Specify therapeutic goals for a patient incorporating the principles of evidence-based medicine that integrate patient-specific data, disease and medication-specific information, ethics, and quality-oflife considerations. Goals reflect consideration of all relevant patient-specific information including culture and preferences Goals reflect consideration of the goals of other interdisciplinary team members Goals reflect consideration of the patient's disease state(s) Goals reflect consideration of medication-specific information Goals reflect consideration of best evidence Goals reflect consideration of ethical issues involved in the patient's care Goals reflect consideration of quality-of-life issues specific to the patient Goals reflect integration of all the above factors influencing the setting of goals Goals are realistic Goals are measurable Chart documentation exhibits the following characteristics: 1. Written in time to be useful 2. Follows the health system's policies and procedures, including that entries are signed, dated, timed, legible, and concise Suggested assessment activity: Audit of patient s medical chart, resident s patient information base, or pharmacy department s patient information base OBJ R2.6.2 (Synthesis) Design a patient-centered regimen that meets the evidence-based therapeutic goals established for a patient; integrates patient-specific information, disease and drug information, ethical issues and quality-of-life issues; and considers pharmacoeconomic principles. Regimen reflects the therapeutic goals established for the patient Regimen reflects the patient's and caregiver's specific needs Regimen reflects consideration of compliance Regimen is appropriate to the disease states being treated Regimen reflects consideration of any pertinent pharmacogenomic or pharmacogenetics Regimen reflects consideration of best evidence Regimen reflects consideration of pertinent ethical issues Regimen reflects consideration of pharmacoeconomic components (patient, medical, and systems resources) Regimen reflects consideration of culture and/or language differences Regimen adheres to the health system's medication-use policies Chart documentation exhibits the following characteristics: 1. Written in time to be useful 2. Follows the health system's policies and procedures, including that entries are signed, dated, timed, legible, and concise 3. Recommended plan is clearly presented

Suggested assessment activity: Audit of patient s medical chart, resident s patient information base, or pharmacy department s patient information base Goal R2.7: Design evidence-based monitoring plans. OBJ R2.7.1 (Synthesis) Design a patient-centered, evidenced-based monitoring plan for a therapeutic regimen that effectively evaluates achievement of the patient-specific goals. Parameters are appropriate measures of therapeutic goal achievement Plan reflects consideration of best evidence Selects the most reliable source for each parameter measurement Value ranges selected are appropriate for the patient Parameters measure efficacy Parameters measure potential adverse drug events Parameters are cost-effective Measurement of the parameters specified is obtainable Plan reflects consideration of compliance If plan is for an ambulatory patience, plan includes strategy for assuring patient returns for needed follow-up visit(s) When applicable, plan reflects preferences and needs of the patient Chart documentation exhibits the following characteristics: 1. Written in time to be useful 2. Follows the health system's policies and procedures, including that entries are signed, dated, timed, legible, and concise 3. Recommended plans are clearly presented Suggested assessment activity: Audit of patient s medical chart, resident s patient information base, or pharmacy department s patient information base Goal R2.8: Recommend or communicate regimens and monitoring plans. OBJ R2.8.1 (Application) Recommend or communicate a patient-centered, evidence-based therapeutic regimen and corresponding monitoring plan to other members of the interdisciplinary team and patients in a way that is systematic, logical, accurate, timely, and secures consensus from the team and patient. Recommendation is persuasive Presentation of recommendation accords patient s right to refuse treatment If patient refuses treatment, resident exhibits responsible professional behavior Creates an atmosphere of collaboration Skillfully defuses negative reactions Communication conveys expertise Communication is assertive, but not aggressive Where the patient has been directly involved in the design of the plans, communication appropriately reflects previous collaboration Chart documentation exhibits the following characteristics: 1. Written in time to be useful

2. Follows the health system's policies and procedures, including that entries are signed, dated, timed, legible, and concise 3. Recommended plans are clearly presented Suggested assessment activity: Audit of patient s medical chart, resident s patient information base, or pharmacy department s patient information base where recommendation to the caregiver is done in writing; direct observation of communication process to a specific patient Goal R2.9: Implement regimens and monitoring plans. OBJ R2.9.1 (Application) When appropriate, initiate the patient-centered, evidence-based therapeutic regimen and monitoring plan for a patient according to the organization's policies and procedures. Activity complies with the health system's policies and procedures Therapy corresponds with the recommended regimen Regimen is initiated at the appropriate time Medication orders are clear and concise Activity complies with the health system's policies and procedures Tests correspond with the recommended monitoring plan Tests are ordered and performed at the appropriate time Test orders are clear and concise Chart documentation follows the health system s policies and procedures including that entries are signed, dated, timed, and legible Suggested assessment activity: Audit of patient s medical chart, resident s patient information base, or pharmacy department s patient information base for initiation of therapy and ordering of tests for a specific patient OBJ R2.9.2 (Application) Use effective patient education techniques to provide counseling to patients and caregivers, including information on medication therapy, adverse effects, compliance, appropriate use, handling, and medication administration. Session appropriately reflects the designed plan for patient counseling Clearly conveys the purpose of the counseling session Demonstrates skill in execution of each teaching method employed during the counseling session During the session, adjusts the instruction to appropriately accommodate the patient's or caregiver's previous knowledge of the medication During the session, adjusts the instruction to appropriately accommodate the patient's or caregiver's responses Assures that the information or skills required are learned before ending the session Suggested assessment activity: Direct observation of resident counseling session with a specific patient and/or caregiver

Goal R2.10: Evaluate patients progress and redesign regimens and monitoring plans. OBJ R2.10.1 (Evaluation) Accurately assess the patient s progress toward the therapeutic goal(s). Accounts for all patient data specified in the monitoring plan Interprets each monitoring parameter measurement accurately Accounts for the patient's current status Properly judges the reliability of data (e.g., timing or site of collection, differences in test sites) Where monitoring data are incomplete, makes sound judgments in determining if there are sufficient data upon which to base a conclusion Conclusions drawn reflect consideration of any significant trends in laboratory values or clinical endpoints Conclusions drawn reflect consideration of the safety and effectiveness of the current medication therapy Conclusions drawn reflect consideration of any ineffectiveness in patient counseling Overall conclusions about reasons for patient s progress or lack of progress toward each stated goal is appropriate Chart documentation exhibits the following characteristics: 1. Warrants documentation 2. Written in time to be useful 3. Follows the health system's policies and procedures, including that entries are signed, dated, timed, legible, and concise Suggested assessment activity: Audit of patient s medical chart, resident s patient information base, or pharmacy department s patient information base for resident s interpretation of monitoring data for a specific patient OBJ R2.10.2 (Synthesis) Redesign a patient-centered, evidence-based therapeutic plan as necessary based on evaluation of monitoring data and therapeutic outcomes. (See criteria above for building the information base, designing therapeutic regimens, and designing monitoring plans) Modifications to the plan are effectively communicated to the patient, caregivers, prescriber, and other relevant health care professionals in a timely manner Conditions of urgency for communicating results to the prescriber are honored Suggested assessment activity: Audit of patient s medical chart, resident s patient information base, or pharmacy department s patient information base for resident communication of modifications to a specific patient s plan

Goal R2.11: Communicate ongoing patient information. OBJ R2.11.1 (Application) When given a patient who is transitioning from one health care setting to another, communicate pertinent pharmacotherapeutic information to the receiving health care professionals. Conveys all necessary data in a timely manner Transfers information to all concerned health professionals Is available to clarify any related issues If a chart entry is made, it exhibits the following characteristics: 1. Warrants documentation 2. Written in time to be useful 3. Follows the health system s policies and procedures, including that entries are signed, dated, timed, legible, and concise Suggested assessment activity: Direct observation of resident communication of continuity of care information of a specific patient and, if chart entry is made, review of documentation of communication of information OBJ R2.11.2 (Application) Ensure that accurate and timely medication-specific information regarding a specific patient reaches those who need it at the appropriate time. Information conveyed at the time it is needed Information conveyed is accurate Information conveyed in a format that is usable by the receiver Information is transmitted by means that are accessible to the receiver Suggested assessment activity: Review of resident s communication of patientspecific information at various stages of a patient s care Goal R2.12: Document direct patient care activities appropriately. OBJ R2.12.1 (Analysis) Appropriately select direct patient-care activities for documentation. Activity selected for documentation is one that will effectively contribute to establishing direct patient care outcomes All patient care activities that should be documented are documented Suggested assessment activity: Review of resident documentation of his or her direct patient care activities for a specific period of time OBJ R2.12.2 (Application) Use effective communication practices when documenting a direct patient-care activity. Chart documentation exhibits the following characteristics: 1. Warrants documentation 2. Written in time to be useful 3. Follows the health system's policies and procedures, including that entries are signed, dated, timed, legible, and concise 4. Content includes pertinent subjective and objective data 5. Assessment reflects accurate interpretation of the objective and subjective data

6. Recommended plans are clearly presented and relate to the conclusion Reports of medication-related problems (e.g., ADRs, medication errors, drug interactions) adhere to the health system's policies and procedures Suggested assessment activity: Review of resident documentation of his or her direct patient care activities and of reports of medication-related problems for a specific period of time OBJ R2.12.3 (Comprehension) Explain the characteristics of exemplary documentation systems that may be used in the organization s environment. Discussion reflects a grasp of the significance of documentation Discussion reflects clear understanding of all relevant criteria Suggested assessment activity: Discussion by resident of documentation systems Outcome R5: Provide medication and practice-related education/training. Goal R5.1 Provide effective medication and practice-related education, training, or counseling to patients, caregivers, health care professionals, and the public. OBJ R5.1.1 (Application) Use effective educational techniques in the design of all educational activities. Choice of content for instruction is based on an accurate assessment of the learner s needs Activities are based on behaviorally stated educational objectives Selection of teaching method is based on the type of learning required (cognitive, psychomotor, affective) Content selected for instruction is matched with the intent of the stated educational objectives Content of instructional materials is accurate Content selected for instruction is complete Instruction is properly organized and sequenced Written instructional materials are matched to the learner s reading level Design of instruction includes use of visual aids when appropriate Suggested assessment activity: Review of resident written plan and written instructional materials for delivery of a specific education or training program or other learning activity

OBJ R5.1.2 (Synthesis) Design an assessment strategy that appropriately measures the specified objectives for education or training and fits the learning situation. Plan for assessment will accurately measure the participants attainment of the educational objectives Suggested assessment activities: Review of resident assessment instrument(s) designed to accompany a specific education or training program; review of data collected by the resident through use of the assessment instruments and the resident s conclusions about the level of achievement of learners on the stated objectives OBJ R5.1.3 (Application) Use skill in the four preceptor roles employed in practice-based teaching (direct instruction, modeling, coaching, and facilitation). Provides effective, focused direct instruction when warranted Models problem solving by talking out loud about problem-solving process Coaches according to the learner s current need for degree of intensity Selects appropriate problem-solving situations for independent work by the learner Moves with ease between the four preceptor roles as learner needs change Suggested assessment activity: Cumulative direct observations of resident when engaged in practice-based teaching OBJ R5.1.4 (Application) Use skill in case-based teaching. Case selection is accurately matched with learner needs Case development reflects accurate identification of critical decision points for learner problem-solving Delivery of instruction employs effective use of open-ended and probing questioning strategies Delivery of instruction includes ongoing assessment of learner understanding and corresponding adjustment in instruction Suggested assessment activity: Cumulative direct observations of resident when engaged in case-based teaching

OBJ R5.1.5 (Application) Use public speaking skills to speak effectively in large and small group situations. Secures audience attention at the beginning Style of the speech is matched to its intent (persuasion, information giving, entertainment) Relates remarks to those of previous speakers (if appropriate) Portrays credibility Does not distract with physical mannerisms Does not distract with verbal habits Speech is well organized Uses appropriate transitions when proceeding from one topic to another Summarizes key points at the close Uses phrases, facts, or stories to increase audience interest Paints mental images or pictures to increase audience understanding Uses humor appropriately Appears poised Uses body language to add interest Pauses appropriately to emphasize points Displays energy and enthusiasm Accurately reads the audience s body language and adjusts the speech accordingly Answers questions with ease and confidence Suggested assessment activity: Direct observation of a resident presentation to a large group; direct observation of a resident presentation to a small group OBJ R5.1.6 (Application) Use knowledge of audio-visual aids and handouts to enhance the effectiveness of communications. Uses visual aids when appropriate Operates equipment skillfully Uses visual aids of an acceptable quality for the situation Follows accepted conventions for designing visual aids Suggested assessment activities: Direct observation of resident during a formal presentation; direct observation of resident during an informal presentation

Appendix F: PGY1 Example Outcomes of Application of the RLS Decision Process Background: Overview University Medical Center (UMC) is an 1140 bed tertiary-quaternary care academic medical center. Its centers of excellence include: Level I trauma center Level III NICU Region s largest OB/GYN service Comprehensive cancer referral center including BMT Cardiology and cardiovascular surgery Neurology and neurosurgery Solid organ transplantation Clinics (where pharmacists provide patient care services) - Pediatric specialties - Internal medicine - Family medicine - Anticoagulation - Oncology - Geriatrics assessment - Transplant UMC is the teaching site for a college of pharmacy, offering some 200 clerkship months each year. Clinical pharmacy specialists in the department have appointments in the college. Article I. Pharmacy Services at UMC Management team - Director - Associate director - Assistant director - Clinical coordinator - Evening shift supervisor - Sterile products/purchasing supervisor Sterile products preparation - Centralized - OR satellite Medication distribution system - Cartless - ADCs profiled to hospital information system (including pharmacy)

Associate director oversees clinical services program, and the assistant director oversees medication safety efforts of the department. Information technology (IT) oversees information systems including automation, BCMA, and CPOE Clinical Pharmacy Specialists (19): Critical care: MICU, SICU (2) NICU Oncology, BMT (2) Infectious diseases (2) Internal medicine (4) Trauma/neurosurgery Cardiology/CVICU Transplant: renal/pancreas; liver/small bowel; cardiac/lung (3) Drug information/drug policy Internal medicine ambulatory care Family medicine/anticoagulation Clinical specialists are responsible for quality improvement programs in the department and are mentors for the (PCAP) decentralized pharmacists. Pharmacists on floors in PCAP decentralized integrated practice model 16 hours/day 7 days/week - Medication histories and reconciliation - Identification of medication-related problems - Monitoring drug therapies - Order entry - Targeted drug initiatives and therapeutic interchanges - Patient education - Drug information - Coordinate distribution system - Collect data for MUE, ADR o Cardiology/CVICU o Pediatrics o NICU/obstetrics/newborn nursery o Surgery o SICU o MICU o Internal medicine o Oncology/BMT o Step down units o Geriatrics unit Clinical pharmacy specialists rotate call on a weekly basis. Article II. UMC Residency Program Four residency slots approved Start-up July 1 Clinical coordinator is the RPD

Step 1: Identify the residency program s purpose and desired outcomes Purpose: Prepare pharmacist clinicians for patient care positions, adjunct faculty positions, or for PGY2 training in area of choice Educational outcomes: Outcomes Required by the PGY1 Standard 1. Manage and improve the medication-use process. 2. Provide evidence-based, patient-centered medication therapy management with interdisciplinary teams. 3. Exercise leadership and practice management skills. 4. Demonstrate project management skills. 5. Provide medication and practice-related education/training. 6. Utilize medical informatics. Selected Elective Program Outcomes 7. Conduct pharmacy practice research. 8. Participate in the management of medical emergencies. Program description for inclusion in residency manual: University Medical Center s (UMC) post-graduate year one pharmacy residency prepares its graduates to assume positions as patient care clinicians, to serve as adjunct faculty for a college of pharmacy, or to pursue second year postgraduate training in a focused area of practice. UMC is a tertiary-quaternary care academic medical center providing the unique capability to engage each of our residents in practice research as well as the management of medical emergencies. Other outcomes of our program include: 1) manage and improve the medication use process, 2) provide evidence-based patient-centered medication therapy management with interdisciplinary teams, 3) exercise leadership and practice management skills, 4) demonstrate project management skills, 5) provide medication and practice-related education/training, and 6) utilize medical informatics. A Look at UMC s Step 1 Decision Making With 19 clinical pharmacists as potential preceptors, the UMC RPD decided it would be impractical to gather them all in one place for the amount of time it would take to design the residency program. From volunteers among the staff, she created a 10-person team that represented the scope of possibilities for training and created a process for email review of each of the team s significant decisions. In this way she was able to assure the input of those most likely to be involved in the program and, thus, to get their buy-in to the program s design. There was little controversy in determining the program s purpose. Given the teaching environment of the hospital and the presence of many adjunct faculty from the

nearby college of pharmacy, it seemed obvious that UMC was the ideal place to train both for clinical responsibilities and for serving as adjunct faculty. Inclusion of medical emergencies from the electives list was done because of the medical center s excellence in this area. One of the team members who is noted for writing well was tasked with putting together the program description that would be put in the residency manual and on the ASHP residency web site. Step 2: Establish Program Structure Potential learning experiences: Beginning Ideas on How to Structure the Program Direct patient care internal medicine Cardiology, CVICU MICU, SICU oncology, BMT infectious diseases trauma family medicine ambulatory care abdominal and chest transplant pediatrics, NICU Other -- project service practice management informatics DI and drug policy

Clinicians for pt care Adjunct faculty Prep for PGY2 in multiple areas R1: Manage & improve meduse process R2: Provide evidence-based, pt-centered MTM with interdisciplinary teams R3: Exercise leadership & practice management skills R4: Demonstrate project management skills R5: Provide medication & practicerelated education/ training R6: Utilize medical informatics E1: Conduct research E5: Participate in mgmt of medical emergencies practice mgmt/long direct pt care/rot / long/ext practice mgmt/ long project/ long direct pt care/rot/ long/ext project/ long project/ long direct pt care/rot/ long/ext hospital pharmacy practice/ long hospital pharmacy practice/ long practice mgmt/ long formal presentations practice mgmt/ long hospital pharmacy practice/ long

July Orientation August - May Requirements December Transitional June Elective Hospital Orientation Residency / Into to RLS Hospital Practice Orientation Drug Information Orientation Ambulatory Care Orientation Internal Medicine 6 weeks as first patient care experience Cardiology 4 weeks Critical Care (MICU or SICU) 4 weeks Oncology 4 weeks Infectious Diseases 4 weeks Transplant 4 weeks Ambulatory Care 4 weeks Project 4 weeks Practice Management 2 weeks directly following internal medicine (can accommodate two residents at a time) Projects ASHP Midyear Repeat of any required learning experience or any other rotation offered or arranged with preceptor 4 weeks Longitudinal Experiences Practice Management Drug Information & Drug Policy P&T Article MUE Hospital Pharmacy Practice Ambulatory Care Clinic Project Practice Management Drug Information & Drug Policy Hospital Pharmacy Practice Ambulatory Care Clinic Project Practice Management Drug Information & Drug Policy Hospital Pharmacy Practice Ambulatory Care Clinic Project Practice Management Drug Information & Drug Policy Hospital Pharmacy Practice Ambulatory Care Clinic Project Preceptors: Direct Patient Care: Clinical pharmacy specialists and PCAPs (team-based pharmacy services) Hospital Pharmacy Practice (service): PCAPs Practice Management: director of pharmacy, associate director, assistant director Drug Information & Drug Policy DI clinical specialist and DI staff pharmacists Project Project preceptor chosen by resident

A Look at UMC s Step 2 Decision Making The UMC residency design team was designing a residency from scratch. With no previous design to draw upon, they decided to employ a full three-phase process in determining their program s structure. First they looked over their access to clinical and administrative/management preceptors and made a list of possible options for learning experiences. They were inclusive in this listing, figuring that they were committing to nothing other than creating a shopping cart for further possible use. Next, they used the RLS flow diagram that allowed them to work program outcome by program outcome to determine the overall fit of these potential learning environments with the selected outcomes. Frequently they would find themselves referring to the educational goals and even the educational objectives listed below an outcome in order to assure clarity in their thinking about whether things fit. In some cases, the same learning environment was identified as an option for more than one educational outcome such as practice management for both R1 and R6. Because they had predetermined that to achieve the E1 outcome for the conduct of research meant that the residents projects would always have to be research studies, it was obvious that the project learning experience would fall under just that one outcome. Before completing this phase of designing the structure, they reviewed the diagram for completeness and, where it had not already been done, decided the category of learning experience (e.g., longitudinal, rotation, etc.) that should be assigned to each of their choices in order to maximize resident opportunity to achieve the outcomes. A topic of significant discussion at this point was careful consideration of how much time to allot for each of the focused areas of direct patient care. The team would have liked to make each of these learning experiences an extended one. However, they also wanted to give all residents experience in each of the selected areas. The compromise was to make internal medicine, the required first experience, a six-week extended experience with the expectation the basic skills acquired there would transfer to the subsequent experiences and facilitate speedy pick-up in the new practice area. The team now progressed to the third and final phase of designing their structure, the creation of an overall calendar for the year. While they knew that some programs create a month-by-month calendar, they decided their needs were best met by thinking in terms of blocks of like experiences resulting in four categories. In their two-level diagram, they ran the longitudinal learning experiences across the bottom, making it clear to all that whatever was going on above, these longitudinal experiences would always be absorbing part of residents time. They broke their July orientation into five categories figuring that if residents were oriented in the beginning to each of these areas, they would be more ready to function in the environment of their first clinical learning experience. The group felt strongly that the first learning experience should be an extended learning experience of six weeks in internal medicine, immediately followed by two weeks in practice management. In that way they hoped to give the resident a broad clinical and administrative frame of reference before starting work in more focused areas of clinical practice. The June category offered flexibility to accommodate specific resident interests. To top off the work, they added a general category list of who would be preceptors.

Step 3: Assign Educational Goals and Objectives to Specific Learning Experiences Key to learning experiences: DPC Direct patient care IM Internal medicine Card - Cardiology Onc Oncology ID Infectious diseases TX Solid organ transplant AC Family medicine and amb care internal medicine Peds Pediatrics NICU TR trauma MICU SICU PM/policy Practice management and drug policy activities Service Staffing commitment Talks Formal presentations Project Longitudinal major project required for completion of the residency program Orient Orientation

Emphasis Goal T/TE/ TE+ Required outcomes and educational goals and objectives for PGY1 programs Outcome R1: Manage and improve the medication-use process. DPC Learning Experiences PM/ Service Talks Project Orient Drug Policy R1.1 Identify opportunities for improvement of the organization s medication-use system. R1.2 Design and implement quality improvement changes to the organization s medication-use system. R1.3 Prepare and dispense medications following existing standards of practice and the organization s policies and procedures. R1.4 Demonstrate ownership of and responsibility for the welfare of the patient by performing all necessary aspects of the medication-use system. TE TE TE TE TE TE T TE+ AC - TE TE T

Outcome R2: Provide evidence-based, patient-centered medication therapy management with interdisciplinary teams. T/TE/TE+ DPC PM/ Drug Policy R2.1 As appropriate, establish collaborative professional relationships with members of the health care team. R2.2 Place practice priority on the delivery of patient-centered care to patients. R2.3 As appropriate, establish collaborative professional pharmacist-patient relationships. R2.4 Collect and analyze patient information. R2.5 When necessary, make and follow up on patient referrals. R2.6 Design evidence-based therapeutics regimens. R2.7 Design evidence-based monitoring plans. TE+ TE+ TE+ TE+ TE+ TE+ TE+ IM - TE AC TE MICU - TE IM - TE Card -TE AC TE TX - TE IM - TE AC - TE TX TE NICU - TE AC - TE Card TE TX -TE AC - TE IM - TE Onc TE SICU - TE Card TE Onc TE Service Talks Project Orient TE T T

R2.8 Recommend or communicate regimens and monitoring plans. R2.9 Implement regimens and monitoring plans. R2.10 Evaluate patients progress and redesign regimens and monitoring plans. TE+ TE+ TE+ TX TE BMT TE MICU - TE IM - TE Card-TE Onc - TE ID - TE AC - TE Onc - TE ID TE MICU TE NICU - TE Onc - TE ID - TE TX TE MICU TE SICU TE BMT TE R2.11 Communicate ongoing patient information. R2.12 Document direct patient care activities appropriately. TE+ TE+ AC - TE IM - TE IM - TE AC - TE Onc - TE TE T TE T

Peds - TE Card-TE Outcome R3: Exercise leadership and practice management skills. R3.1 Exhibit essential personal skills of a practice leader. R3.2 Contribute to departmental leadership and management activities. T/TE/ TE+ DPC PM/ Drug Policy TE TE TE TE Service Talks Project Orient R3.3 Exercise practice leadership. TE TE Outcome R4: Demonstrate project management skills. R4.1 Conduct practice-related investigations using effective project management skills. T/TE/ TE+ DPC PM/ Drug Policy Service Talks Project Orient TE+ TE TE Outcome R5: Provide medication and practice-related education/training R5.1 Provide effective medication and TE+ IM-TE TE TE T

practice-related education, training, or counseling to patients, caregivers, health care professionals, and the public. AC-TE TX-TE PEDS - TE Outcome R6: Utilize medical informatics. R6.1 Use information technology to make decisions and reduce error. TE+ MICU - TE TE Elective outcomes and educational goals and objectives for PGY1 programs Outcome E1: Conduct pharmacy practice research. E1.1 Design, execute, and report results of investigations of pharmacy practice-related issues. E1.2 Participate in clinical, humanistic and economic outcomes analyses. TE TE TE+ TE TE Outcome E2: Exercise added leadership and practice management skills. E2.1 Contribute to the development of a new pharmacy service or to the enhancement of an existing service. T/TE/ TE+ DPC PM/ Drug Policy Service Talks Project Orient

E2.2 Understand the pharmacy procurement process. E2.3 Manage the use of investigational drug products (medications, devices, and biologicals). E2.4 Understand the principles of a systematic approach to staff development in pharmacy practice. E2.5 Resolve conflicts through negotiation. E2.6 Understand the process of managing the practice area s human resources. E2.7 Understand the process of establishing a pharmacy practice residency program. Outcome E3: Demonstrate knowledge and skills particular to generalist practice in the home care practice environment. T/TE/ TE+ DPC PM/ Drug Policy Service Talks Project Orient

E3.1 Understand the scope of services that might be provided in a typical home care practice. E3.2 Determine the suitability of individuals patients for home care. E3.3 Understand unique aspects of providing evidence-based, patientcentered medication therapy management with interdisciplinary teams in the home care environment. E3.4 Understand unique aspects of preparing and dispensing medications for home care patients. E3.5 Understand unique aspects if participating in the management of medical emergencies occurring in the home care environment. E3.6 Manage the use, maintenance, and troubleshooting of medication administration equipment and medication-related equipment in the management of home care patients. E3.7 Understand the appropriate relationship between the home care pharmacist and home care suppliers. E3.8 Appreciate the complexity of the financial environment of home care

practice. E3.9 Conduct ethical informational and marketing visits to payers, potential referral sources, and patients of the home care organization. Outcome E4: Demonstrate knowledge and skills particular to generalist practice in the managed care practice environment. E4.1 Maintain confidentiality of patient and proprietary business information. T/TE/ TE+ DPC PM/ Drug Policy Service Talks Project Orient E4.2 Understand the interrelationship of the pharmacy benefit management company, the health plan, and the delivery system functions of managed care. E4.3 Understand unique aspects of providing evidence-based, patientcentered medication therapy management with interdisciplinary teams in the managed care environment. Outcome E5: Participate in the

management of medical emergencies. E5.1 Participate in the management of medical emergencies. T/TE/ TE+ DPC TE+ MICU - TE PM/ Drug Policy Service Talks Project Orient TE T Outcome E6: Contribute to formulary decisions. E6.1 Participate in the organization s formulary process. Outcome E7: Provide drug information to health care professionals and/or the public. E7.1 Identify a core library, including electronic media, appropriate for a specific practice setting. E7.2 Design and deliver programs that contribute to public health efforts. E7.3 Provide concise, applicable, comprehensive, and timely responses to requests for drug information from patients, health care providers, and the public. Outcome E8: Demonstrate additional competencies that contribute to working

successfully in the health care environment. T/TE/ TE+ DPC PM/ Drug Policy Service Talks Project Orient E8.1 Use approaches in all communications that display sensitivity to the cultural and personal characteristics of patients, caregivers, and health care colleagues E8.2 Communicate effectively. E8.3 Balance obligations to oneself, relationships, and work in a way that minimizes stress. E8.4 Manage time effectively to fulfill practice responsibilities. E8.5 Make effective use of available software and information systems. T/TE/ TE+ DPC PM/ Drug Policy Service Talks Project Orient TE+ ALL- TE T T T T TE T TE A Look at UMC s Step 3 Decision Making

As they worked on structure, the UMC residency design team added two of the goals falling under elective outcome E4. One for managing time and the other regarding use of software and information systems. These they decided to teach and evaluate in relevant learning experiences without considering them as part of a separate outcome. Every effort was made to limit the number of times a goal was to be evaluated, however, with great frequency it was recognized that a goal could not be accomplished fully except over repeated refinement of resident performance over the year. This was particularly evident in their discussion of goals falling under R2.and R5. Wherever possible, they chose to limit the number of learning experiences in which a goal was evaluated while maximizing opportunities to teach it. Note that some goals are designated for teaching and monitoring only; such decisions were made taking into account that there were adequate opportunities for formal evaluation at other times during the resident year or that such skills were assumed to be adequately performed by the resident prior to entry into the residency program. The rationale for any goals and objectives that are not actively evaluated (are monitored) must be outlined in the residency program s written assessment strategy.

Step 4 Designate Learning Activities for Learning Experiences and Write Learning Experience Descriptors Internal Medicine is a required learning experience that occurs as the first patient care experience for all residents. There are four internal medicine teaching teams plus private physicians covering private patients for a total of 200 internal medicine beds in the hospital on four different units. Each team includes the attending physician, a fellow (on occasion), one house officer III, two house officer I, two M3 students, two M4 students, the clinical pharmacy specialist, pharmacy students, a respiratory therapist, and a clinical nutritionist. Nurses caring for the patient also participate in patient care rounds. Teams do not round on private patients. The resident is responsible for identifying and resolving medication therapy issues for patients and will work toward assuming care of all patients on the unit throughout the six-week learning experience. Good communication and interpersonal skills are of paramount importance in this setting. The resident must devise efficient strategies for accomplishing the required activities in a limited time frame. While when performing direct patient care the resident will be expected to perform the full sequence of steps involved in pharmaceutical care, specific emphasis will be placed on the performance and evaluation of certain steps as described below in the list of the learning experience s learning activities. Following each activity on the list there is an indication of the goal the activity supports. Activities include but are not limited to: Establishing collaborative professional relationship with health care team (R2.1) Placing priority on delivery of patient-centered care to patients (R2.2) Collect and analyze patient information (R2.4) Designing or modifying therapeutic regimens (R2.6) Recommending the therapeutic plan and regimen (R2.8) Ensuring continuity by communicating pertinent information to the transfer team on transfer to an ICU (R2.11) Documenting direct patient care activities in the medical record and in the pharmacist s care plan as appropriate (R2.12) Providing in-services to the nursing staff as requested (R5.1) Team discussion series in the afternoon when appropriate (R2.1) Common disease states with which the resident will be expected to gain proficiency through literature review, topic discussion, and direct patient care experience include: Neurologic disorders Cardiovascular disorders Renal disorders Pulmonary disorders GI disorders Endocrine disorders Infectious diseases Others as needed based upon patient presentation to the service

The resident is expected to understand the pharmacotherapy related to these disease states as well as other disease states encountered in this setting. Preceptors will be available to the resident throughout the learning experience for consultation and topic discussions. Resident learning is predicated not only on the above responsibilities, but also on acceptance of personal responsibility and dedication to direct patient care and team service. Designated Meetings/Responsibilities: 0700 daily Preparation for rounds; work up patients; meet with preceptor as needed 0800-0930 Work rounds 1000-1100 daily Attending rounds 1100-1200 daily Morning Report 1200-1300 Fridays Grand Rounds 1300-1400 daily Discussion Series 1400-1600 daily - Followup 1500 daily Report to swing shift pharmacists 1600 -? Discussion with preceptors A Look at UMC Internal Medicine Preceptor s Approach to Step 4: Starting with the nine goal areas assigned to his six-week learning experience for teaching and evaluation, the preceptor considered how the internal medicine practice environment could best facilitate residents focused learning in those areas. He recognized that while precepting might be centered on only some of the goals falling under R2, it would be impossible for residents to care for patients except by going through all of the steps under this outcome. For this reason, he chose to call this to the residents attention in the learning experience descriptor lest they think they would only be doing parts of taking care of patients. When he examined his plan for formal evaluation of resident performance during his learning experience, he felt confident that while he would not be formally evaluating the full performance of the interconnected steps in R2, the program s overall design for assessment assured that by the end of the year all aspects of R2 would be formally evaluated through a series of focused evaluations assigned to different preceptors. He thought that functioning within the team approach would provide the opportunity to practice eight of the goals and adding the responsibility for nursing inservices would provide the opportunity to practice the training/education goal. Given the broad spectrum of disease states with which internal medicine deals, the preceptor decided to list those he felt to be most important for resident understanding and added an as needed category to provide leeway for the need to deal with whatever current patients present. Putting a schedule for the daily routine at the end of the description he thought would give residents an idea of the framework for which to prepare themselves.

Step 5: Design Program Assessment Strategy, Design Assessment Strategy for Each Learning Experience, and Design Evaluation Tools UMC Overall Residency Program Assessment Strategy Guidelines Preceptor Evaluation of Residents Attainment of Goals and Objectives Only those goals listed in the program design and those that might be added for an individual resident will be included in the written summative evaluation. Preceptors will provide appropriate orientation to the learning experience, including a review of the educational goals and objectives chosen, learning activities, expectations, and evaluation schedule. Preceptors will provide ongoing, criteria-based feedback throughout each learning experience to assist the resident s skill developmental processes. No fixed schedule of feedback has been established, but a reasonable expectation is 2-3 times weekly, or more often as needed. Written formative evaluation is encouraged. Appropriate formative evaluation instruments may include RLS-snapshots, patient monitoring forms, drafts of newsletters, monographs, DUE s, etc. Any written formative evaluation instrument used should be attached to the summative evaluation. Summative evaluations will be completed by preceptors no later than the last day of the learning experience or by the last day of the quarter for longitudinal learning experiences. They must be discussed with the resident, signed, and dated. Preceptors will check the appropriate rating to indicate resident progress and provide narrative commentary for any goal for which progress is Needs Improvement or Achieved. Narrative comments should relate to criteria developed for achievement of that goal. Please do not provide quantitative commentary it is not helpful to assist in skill development. Signed and dated summative evaluations must be transmitted to the RPD electronically on the day they are completed. A copy should be filed in the resident s notebook. Upon receipt, the RPD will review and sign the summative forms. The evaluating preceptor will invite the oncoming preceptor to attend the resident evaluation session to provide continuity between learning experiences. At the end of the residency year, the residency committee will meet to consider residents progress and ultimate achievement of the program s educational goals and objectives using all assessment and tracking information available. Residents' Self-evaluation of Their Attainment of Goals and Objectives Residents will complete the same summative evaluation instruments at the end of each learning experience or at quarterly intervals for longitudinal learning experiences.

Where snapshots are used, residents will complete the same formative evaluation instruments completed by preceptors on the same schedule. Residents will check the appropriate rating to indicate progress during the learning experience, and should provide narrative comments for any goal for which progress is Needs Improvement. Residents must have evaluation instruments completed to be used in evaluation sessions with preceptor(s). They will be reviewed and discussed with preceptors, and should be signed and dated by the resident and the preceptor. A copy will be kept in the resident s notebook. The original will be forwarded to the residency program director for review and signature. At the end of the residency year, the residents will be provided a list of their educational goals and objectives for the year and asked to self-rate their achievement. Residents' Evaluation of the Preceptor and Learning Experience Residents will complete the program s evaluation form no later than the last day of each learning experience or quarterly for longitudinal learning experiences. Completed evaluations will be discussed with preceptors, signed, and dated by each. Completed, signed evaluations will be forwarded to the residency program director for review on the day of their completion. Assessment Strategy for the Internal Medicine Learning Experience Note: This text would be added to the learning experience description by the preceptor. Formative evaluation: The preceptor will utilize some snapshots as well as written consults completed by the resident as two means of providing criteria-based formative evaluation. The particular snapshots will be selected on the basis of observation of resident skill development needs. Summative evaluation: The preceptor will utilize the attached summative evaluation form used for the UMC residency program for all direct patient care learning experiences. The form will be completed no later than the last day of the learning experience. The preceptor will discuss the summative evaluation with the resident at that time. Resident self-evaluation: Residents will complete snapshots for educational goals R2.2, R2.6, and R2.7 (attached) on the Friday of week 2 and week 3 of the learning experience. The resident should determine which patient s care he or she will be self-evaluating with the snapshots on the day they are completed in order to facilitate the preceptor s concurrent evaluation,

At the preceptor s discretion, additional self-assessment activities may be required. When completed, the resident will be responsible for scheduling time with the preceptor to compare evaluations. Residents will complete the same summative evaluation form for use no later than the last day of the learning experience. A comparison of the ratings between resident and preceptor will be part of the summative evaluation debriefing session. Resident evaluation of the preceptor and learning experience: Resident will complete the UMC residency program form for this purpose no later than the last day of the learning experience. Discussion of this form will be part of the summative evaluation debriefing session. Residents are responsible for forwarding the form to the RPD on the day it is discussed. A Look at the UIMC Residency Design Team and the UMC Preceptor for the Internal Medicine Learning Experience Step 5 Decision-making The UMC residency design team followed the standard with little addition in putting together their program-wide assessment strategy. They decided not to specify the use of any particular snapshots, leaving even the choice to use or not to use them up to the individual preceptor. They decided it would be useful to specify the exact dates when evaluations must be conducted and transmitted, thinking that they could promote having evaluations from the immediately preceding preceptor in the hands of the next preceptor on the first day of the next learning experience. The UMC preceptor for internal medicine crafted his evaluation strategy for the learning experience after the program strategy was decided. He decided he would use snapshots, but left open which ones. He also determined that written patient consults would be prime as a tool for formative evaluation. For resident self-evaluation he decided to be specific, assigning specific snapshots to a specific time frame. He also decided to assign responsibility for managing the self-assessment process including getting the preceptor to do his part to the resident. The summative evaluation form that the internal medicine preceptor will use to evaluate the resident follows. It was constructed using one of the RLS tools provided on the web page a summative evaluation form in which all of the required goals and objectives have been filled in. All that was necessary was to delete those goals and objectives which did not apply to the internal medicine learning experience. Then, in the second category, he entered the time management goal that was drawn from the elective area.

SUMMATIVE EVALUATION INTERNAL MEDICINE ROTATION Resident: Date Completed: Preceptor: Time Period of Learning Experience: This form documents resident attainment of educational goals formally taught and scheduled for assessment during this learning experience in the program s assessment strategy. Evaluation of goal achievement is based on preceptor judgment of resident performance on the associated educational objectives listed below each goal. When used for resident-self assessments, the judgments rendered will be the resident s judgment of his or her performance. The Standard requires that each of the residency program s goals and associated objectives must be evaluated at least once during the residency program. This form provides for three categories of goals for evaluation. 1) Standard-required educational goals that have been designated to be taught and evaluated during this learning experience. 2) Elective program goals designated to be taught and evaluated during this learning experience. 3) Required or elective goals designated for teaching but not evaluation during this learning experience (those that are monitored). The preceptor will provide a narrative commentary for each educational goal that is based on current resident performance level and reflects the aggregate resident activity during the learning experience. NI, SP, or ACH entered opposite the goal statement in the rating column for categories 1 and 2 indicates the level of resident achievement at the end of the learning experience. Key: NI = Needs Improvement SP = Satisfactory Progress ACH = Achieved

Category 1: Goals Required by the Standard and Formally Taught and Evaluated in This Learning Experience EDUCATIONAL GOALS AND ASSOCIATED NARRATIVE COMMENTARY RATING OBJECTIVES Goal R2.1: As appropriate, establish collaborative professional relationships with members of the health care team. OBJ R2.1.1 (Synthesis) Implement a strategy that effectively establishes cooperative, collaborative, and communicative working relationships with members of interdisciplinary health care teams. Goal R2.2: Place practice priority on the delivery of patientcentered care to patients. OBJ R2.2.1 (Organization) Choose and manage daily activities so that they reflect a priority on the delivery of appropriate patient-centered care to each patient. Goal R2.4: Collect and analyze patient information. OBJ R2.4.1 (Analysis) Collect and organize all patientspecific information needed by the pharmacist to prevent, detect, and resolve medication-related problems and to make appropriate evidence-based, patient-centered medication therapy recommendations as part of the interdisciplinary team. OBJ R2.4.2 (Analysis) Determine the presence of any of the following medication therapy problems in a patient's current medication therapy: 1. Medication used with no medical indication 2. Patient has medical conditions for which there is no medication prescribed

3. Medication prescribed inappropriately for a particular medical condition 4. Immunization regimen is incomplete 5. Current medication therapy regimen contains something inappropriate (dose, dosage form, duration, schedule, route of administration, method of administration) 6. There is therapeutic duplication 7. Medication to which the patient is allergic has been prescribed 8. There are adverse drug or device-related events or potential for such events 9. There are clinically significant drug-drug, drug-disease, drug-nutrient, or druglaboratory test interactions or potential for such interactions 10. Medical therapy has been interfered with by social, recreational, nonprescription, or nontraditional drug use by the patient or others 11. Patient not receiving full benefit of prescribed medication therapy 12. There are problems arising from the financial impact of medication therapy on the patient 13. Patient lacks understanding of medication therapy 14. Patient not adhering to medication regimen OBJ R2.4.3 (Analysis) Using an organized collection of patient-specific information, summarize patients health care needs.

Goal R2.6: Design evidence-based therapeutic regimens. OBJ R2.6.1 (Synthesis) Specify therapeutic goals for a patient incorporating the principles of evidencebased medicine that integrate patient-specific data, disease and medication-specific information, ethics, and quality-of-life considerations. OBJ R2.6.2 (Synthesis) Design a patient-centered regimen that meets the evidence-based therapeutic goals established for a patient; integrates patientspecific information, disease and drug information, ethical issues and quality-of-life issues; and considers pharmacoeconomic principles. Goal R2.8: Recommend or communicate regimens and monitoring plans. OBJ R2.8.1 (Application) Recommend or communicate a patient-centered, evidence-based therapeutic regimen and corresponding monitoring plan to other members of the interdisciplinary team and patients in a way that is systematic, logical, accurate, timely, and secures consensus from the team and patient. Goal R2.11: Communicate ongoing patient information. OBJ R2.11.1 (Application) When given a patient who is transitioning from one health care setting to another, communicate pertinent pharmacotherapeutic information to the receiving health care professionals. OBJ R2.11.2 (Application) Ensure that accurate and timely medication-specific information regarding a specific patient reaches those who need it at the

appropriate time. Goal R2.12: Document direct patient care activities appropriately. OBJ R2.12.1 (Analysis) Appropriately select direct patient-care activities for documentation. OBJ R2.12.2 (Application) Use effective communication practices when documenting a direct patient-care activity. OBJ R2.12.3 (Comprehension) Explain the characteristics of exemplary documentation systems that may be used in the organization s environment. Goal R5.1 Provide effective medication and practice-related education, training, or counseling to patients, caregivers, health care professionals, and the public. OBJ R5.1.1 (Application) Use effective educational techniques in the design of all educational activities. OBJ R5.1.2 (Synthesis) Design an assessment strategy that appropriately measures the specified objectives for education or training and fits the learning situation. OBJ R5.1.3 (Application) Use skill in the four preceptor roles employed in practice-based teaching (direct instruction, modeling, coaching, and facilitation). OBJ R5.1.4 (Application) Use skill in case-based teaching. OBJ R5.1.5 (Application) Use public speaking skills to speak effectively in large and small group

situations. OBJ R5.1.6 (Application) Use knowledge of audio-visual aids and handouts to enhance the effectiveness of communications. Category 2: Elective Program Goals Formally Taught and Formally Evaluated During This Learning Experience EDUCATIONAL GOALS AND ASSOCIATED NARRATIVE COMMENTARY OBJECTIVES Goal E8.4: Manage time effectively to fulfill practice responsibilities. OBJ E8.4.1 (Application) Use time management skills effectively to fulfill practice responsibilities. RATING Category 3: Goals Formally Taught but Not Scheduled for Formal Evaluation During This Learning Experience GOALS AND ASSOCIATED OBJECTIVES NEEDS ATTENTION None COMMENTS: Preceptor Signature Resident Signature Program Director Signature

Step 6: Design Customized Training Plan for Each Resident UMC PGY1 Residency Training Plan Customized for Mary Resident July 20XX Background This initial training plan was developed using information provided by the resident via completion of the Entering Resident Interest and Preference Information data gathering instrument and a personal interview. The plan will be reviewed and updated quarterly. Possible entering full achievement of any of the program s educational goals: No indication of sufficient training or experience with any of our program goals to warrant a criteria-based assessment. Interests: Cardiovascular disease Transplant Infectious disease Diabetes Asthma Areas for improvement: Time management skills Developing professional relationships with staff members including working on teams and conflict management Better develop abilities to utilize information systems (Word, PowerPoint, Excel, Access, etc) Areas of strength: Work well independently Good knowledge base Experience as pharmacy technician during pharmacy school. Current and Future Goals: Be actively involved in professional organizations Improve teaching and communication skills for large group settings Possible PGY2 program in cardiology, critical care, transplant, or infectious disease

Initial Training Plan: We will start out with the core program goals. We will be able to focus on areas of interest through core rotation experiences. The required presentations will provide good experience in teaching and communication skills for large groups. Schedule will be the generic schedule set up for the core program. Peter Program Director, Pharm.D. Mary Resident, Pharm.D. A Look at UMC s Step 6 Decision-making UMC is in its second year of offering a PGY1 program. Relatively new at this type of training, the UMC residency design team decided to work with the simplest of data collection approaches, hoping that the information they would gain from the RLS entering data collection form would be sufficient to make initial decisions for customizing residents programs and keep their work at a minimum. The form gave them a reasonable view of the entering resident s interests and generalized self-perception of strengths and weaknesses. Their conclusion was that the opportunities offered in the core program were of sufficient scope to accommodate everything the resident was interested in. They planned to look again at the resident s program at the end of the first quarter with the goal of assessing if there sufficient accommodation to the resident s interests was occurring. The data collection form did not afford sufficient detail on the resident s selfperception of entering capabilities to identify areas that might already be fully accomplished. Step 7: Precept the Learning Experiences UMC Plan for Orientation to the Program Residents will complete the 2-day health-system orientation and then general pharmacist orientation during the first month. During that time, residents will learn all operational aspects of the pharmacy. Each resident will be given the RLS Guide for Residents to review and it will be discussed with the Residency Program Director before the end of orientation. The requirements to complete the program will be reviewed during orientation. Each resident will complete ACLS training during July.

The resident, preceptors, and the Residency Program Director will work to identify resident projects and work will begin during orientation. The project preceptor must be identified during orientation. Each resident will also choose an advisor for the residency year who will work with the resident and the RPD throughout the year. There will be an evaluation at the end of the month; the completed pharmacist orientation checklist will also be reviewed during orientation in order to enable the resident beginning the service commitment in August. UMC Plan for Orientation to Each Learning Experience Orientation will be provided to each learning experience by the preceptor. S/he will review the learning experience description and requirements for that learning experience. All scheduled meetings and specific responsibilities will be outlined. The evaluation schedule will be reviewed. A Look at UMC s Step 7 Decision-making The UMC residency design team decided to clarify each step in the orientation process and to distribute the plans as a type of check-list for the use of each preceptor to assure that the content of the individual learning experiences would be complete. The team determined that there were some key elements to be achieved during the first residency month regarding the resident s project and licensure requirements that they would specify in writing to assure they were done. Step 8: Monitor Resident Progress UMC Plan for Monitoring the Residents Progress The Residency Advisory Committee (RAC) is composed of the residency program director, six representative clinical pharmacy specialists, and two representative PCAPs and is charged with oversight for all aspects of the residency program. The RAC will meet monthly (or more often if needed) to discuss resident progress and overall program effectiveness and planning. The residency program director will schedule the time/place and provide reminders. Each quarter, the resident s advisor, resident, and the residency program director will review all evaluations completed for that quarter for each resident. A summary of each resident s overall progress will be presented by the advisor, and will include an overview of goals in each evaluation category (NI, SP, and ACH). At that time, the individual plan will be reviewed and appropriate changes to the individual plan will be made, if needed.

At least quarterly, the RAC will focus on opportunities to improve resident performance and provide continuity between learning experiences. The agenda for each RAC meeting will also include the progress on the following: the resident project; presentation topics, schedule, and progress; DUE project progress; monograph development progress; performance on the service component; and progress on miscellaneous project completion. Discussions will also identify the needs of preceptors, time requirements/commitments, and any modifications needed. Each resident s individual plan will be reviewed and suggestions may be offered, if needed. If suggestions to resident s individual plans are offered during RAC meetings, the resident s advisor and residency program director will meet with the resident to discuss and plan for implementation, if needed. UMC PGY1 Residency Training Plan Customized for Mary Resident First Quarter Plan Update September 30, 20XX Mary has made acceptable progress in her internal medicine learning experience. In order to accommodate her desire to perfect her speaking skills before large groups, during her next learning experience, practice management, she will be given the opportunity to give a lecture to a group of visiting student nurses and will also provide an inservice training for the pharmacy s technicians. Since Mary continues to think she may wish to pursue a PGY2 residency in cardiology, critical care, transplant, or infectious disease, her schedule has been adjusted so that her next four learning experiences will be these four. This will give her more time to reflect on her choice of elective for the month of June. Mary s schedule has been modified from the core to indicate the named progression of learning experiences. A Look at UMC s Step 8 Decision-making Note that the residency team has laid out a clear plan for tracking resident progress and involved significant representation from the program s preceptors. First quarter evaluations for the resident resulted in a decision to intensify instruction in one area of the resident s interest and to structure the sequence of clinical learning experiences in order to facilitate time for her to reflect and identify in which area she wanted to spend her last month of training.

Step 9: Conducting Quality Improvement Activities for the Program UMC Plan for Quality Improvement Activities Overall responsibility for quality improvement is with the residency program director and the RAC. Last month of program two meetings to review resident progress. Reflect on the quarterly meeting summaries and individualization that occurred. Purpose to assess whether the program outcomes were achieved and if changes should be made in rotation requirements, preceptors, and general program requirements. 1. Between the RPD and resident 2. All preceptors