The Psychiatry Milestone Project: Assessment Tools

The Psychiatry Milestone Project: Assessment Tools A Joint Initiative of The Accreditation Council for Graduate Medical Education and The American Board of Psychiatry and Neurology

Milestone evaluation is completed by the Clinical Competency Committee using resident assessments conducted throughout resident education. These assessments are completed by faculty members, other care providers, and patients. The Psychiatry Milestone Working Group developed sample assessment tools to simplify evaluating the Milestones. These assessment tools are not required. Some of the sample assessment tools below are based on existing tools used in residency programs (e.g., global outpatient psychiatry rotation evaluation), while others are new tools developed for direct observation of particular milestones (e.g., multidisciplinary team meeting evaluation). These assessment tools also provide examples of different formats (e.g., rating resident level of performance on milestones/threads along a developmental continuum versus using a list of specific milestones and rating whether the resident has achieved the particular milestone or not). Residency programs can adapt these tools and formats to best fit their programs, rotations, and educational experiences, or can develop their own assessment tools.

Psychiatry Assessment Tools Outpatient Global Consultation Case Conference Inpatient Admissions PGY-1 and -2 Geriatric Psychotherapy Supervision Multidisciplinary Team Meeting

Version 5/2014 Outpatient Global 1 Psychiatry Outpatient Global Evaluation Form This form is designed for use by an attending physician for global evaluation of a resident s performance in the outpatient setting over an extended period of time (e.g., 3-6 months). Evaluation can be based on direct observation and/or indirect or oversight supervision of many of the following: initial assessments, medication management sessions, psychotherapy sessions, case presentations, treatment and disposition planning, daily care questions posed/answered, review of initial evaluation and ongoing care progress notes, interactions during individual/group resident supervision, etc. For each of the 18 tables, the attending raters should select the level of knowledge, skills, and attitudes (KSAs) that best describe the resident s performance. Select Level 0 if the resident s performance exhibits the deficiencies described in this level For Levels 1-4, selecting a response box in the middle of a level implies that milestones in that level and in lower levels have been substantially demonstrated For levels 1-4, selecting a response box on the line in between levels indicates that milestones in lower levels as well as some but not all milestones in the higher level(s) have been substantially demonstrated If the KSAs in a topic area have not been adequately observed, select the Not Observed option in the top row of the table Please note that: A milestone has been met when the resident demonstrates skills and/or knowledge proficiently, that is, accurately, completely, skillfully, and consistently as called for across patient care situations Levels 3 and 4 indicate more advanced performance that typically will be exhibited by more senior residents only Resident Name Attending Rater Date Setting

Version 5/2014 Outpatient Global 2 1. Patient Care: History Taking and Examination Skills Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Obtains history and History taking is complete, relevant collateral accurate, relevant, and information; screens for efficient, with flexibility patient safety, including appropriate to clinical suicidal and homicidal setting and workload ideation; performs mental demands; uses hypothesisdriven status examination information (PC1:1.1, 1.2, 1.3) gathering (PC1: 3.1, 3.2,3.4) History and collateral information are inconsistently obtained or inaccurate; does not screen for patient safety; mental status examination incomplete, not well performed History taking is efficient, accurate, relevant and customized to patient complaint; shows sufficient knowledge of and assesses safety (suicide, homicide); shows sufficient knowledge of and can perform a mental status examination relevant to the patient s complaints (PC1: 2.1, 2.2, 2.4; MK2: 2.2, 2.3) Routinely identifies subtle, unusual findings and follows clues to relevant information in complex clinical situations; uses own emotional responses as a diagnostic tool (PC1: 4.1, 4.2, 4.4) 2. Patient Care: Rapport and Therapeutic Alliance Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Establishes rapport; Engages patients/family displays compassion, members well; develops honesty, genuine interest, and sustains therapeutic and respect for patients relationships in awkward, and their families (PROF1: complicated, and 1.1, ICS1: 1.1) conflicted situations (ICS1: 3.1, 3.2) Has difficulty engaging patients and establishing a therapeutic alliance Establishes rapport and obtains information that is sensitive and not readily volunteered by the patient; develops a therapeutic alliance in uncomplicated situations; manages simple patient/family conflicts (PC1: 2.3; ICS1: 2.1, 2.3) Establishes rapport and maintains therapeutic relationships when evaluating and treating challenging patients, including during transitions of care (ICS1: 4.1)

Version 5/2014 Outpatient Global 3 3. Patient Care: Laboratory and Diagnostic Testing Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Knows availability of and can order common laboratory, neuroimaging, neurophysiologic, and neuropsychological tests (MK3: 1.1, 1.2) Unable to order common laboratory, neuroimaging, neurophysiologic, and neuropsychological tests; test ordering is unnecessary, inappropriate, harmful Knows indications for and appropriately orders structural neuroimaging (cranial CT & MRI), neuropsychological, and neurophysiological testing (e.g., EEG, evoked potentials, sleep studies) (MK3: 2.1, 2.3) Selects laboratory and diagnostic tests appropriate to the clinical presentation; knows indications for ordering diagnostic studies that are targeted to the patient s presentation; recognizes the significance of abnormal findings in routine diagnostic and neurodiagnostic test reports in psychiatric patients (PC1: 3.3; MK3: 3.1, 3.2) Explains the significance of routine laboratory, neuroimaging, neurophysiological and neuropsychological testing abnormalities, neurobiologic hypotheses, and genetic risks to patients/their families; (MK3: 4.1, 4.4)

Version 5/2014 Outpatient Global 4 4. Patient Care: Case Presentation, Diagnosis, Differential Diagnosis, and Formulation Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Organizes, accurately reports, and summarizes information obtained from the patient evaluation; develops working diagnosis (PC2: 1.1, 1.2) Unable to give an accurate, organized case presentation Recognizes clinician s emotional response has diagnostic value; includes in the formulation predisposing risk factors, precipitating stressors or events, perpetuating and protective factors, and prognosis; generates diagnostic hypotheses based on typical patterns for common presentations; assesses patient safety (suicidal/homicidal) ideation; uses developmental concepts in case formulation (PC1: 2.5; PC2: 2.1, 2.2. 2.3; MK1: 2.4) Develops full differential diagnosis; avoids irrelevant/unlikely diagnoses and premature closure; organizes formulation around phenomenological models, etiology, and developmental tasks and transitions; describes the influence of psychosocial factors and medical/neurological illness on personality (PC2: 3.1, 3.2; MK1: 3.1, 3.2) Interprets and synthesizes all information into a concise but comprehensive presentation and formulation; adjusts differential diagnosis in response to new information and subtle, unusual, conflicting findings; uses emotional responses as diagnostic tool; describes the import of acquisition/loss of developmental capacities in the expression of psychopathology in outpatients (emerging adults elderly) (PC1: 4.3; PC2: 4.1, 4.2; MK1: 4.1)

Version 5/2014 Outpatient Global 5 5. Patient Care: Treatment Planning and Management Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Recognizes patient readiness for treatment and patients in crisis or who are acutely ill; identifies potential treatment options (PC3: 1.1, 1.2, 1.3) Is not an active participant in treatment planning and patient management decisions; does not offer own ideas; is unaware of the patient s role in treatment; unable to identify potential treatment options Sets treatment goals in collaboration with the patient/family; monitors treatment adherence and response; manages crises with supervision; recognizes the effect of co-morbid conditions and side effects on treatment; incorporates a clinical practice guideline or treatment algorithm when available (PC3: 2.1, 2.2, 2.3, 2.4, 2.5) Links treatment plan to formulation; applies understanding of psychiatric, neurologic, and medical comorbidities to treatment selection; re-evaluates and revises treatment approach based on new information and response to treatment; incorporates manual-based treatment when appropriate; recognizes need for consultation/supervision in complex/refractory cases; describes the differences among, indications /contraindications, risks/ benefits of the three core psychotherapies and couples, family, and group therapies (PC3: 3.1, 3.2, 3.3, 3.4, 3.5; MK4: 3.4) Designs individualized treatment and transition plans for patients with complex presentations; modifies techniques and flexibly applies practice guidelines to fit patient needs; integrates multiple modalities and providers in a comprehensive treatment approach; selects psychotherapeutic modality, tailoring it on the basis of the formulation; seeks consultation about and manages treatment impasses; practices costeffective, high-value care; employs prevention and risk reduction strategies in clinical care (PC3: 4.1, 4.2, 4.3; PC4: 4.4, 4.6; SBP2: 4.1; SBP3: 4.2)

Version 5/2014 Outpatient Global 6 6. Patient Care: Somatic Therapies Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Has beginning Manages drug understanding of how to interactions, monitors prescribe commonly used relevant laboratory psychopharmacologic studies, and uses agents to target specific augmentation strategies psychiatric symptoms with supervision (PC5, 3.1, (e.g., depression, 3.2, 3.3) psychosis); reviews with the patient (and family, where appropriate) general indications, dosing parameters, and common and serious side effects for commonly prescribed psychopharmacologic agents (PC5, 1.1, 1.2) Has difficulty selecting appropriate medications and reviewing indications, dosing, and side effects in patients with common problems; does not offer own ideas Appropriately prescribes commonly used psychopharmacologic agents and obtains basic physical and laboratory studies needed to initiate treatment; incorporates basic knowledge of proposed mechanisms of action of commonly prescribed psychopharmacologic agents in treatment selection and explains rationale to patient/family; seeks consultation and supervision regarding potential referral for ECT (if available) (PC5, 2.1, 2.2, 2.3, 2.4) Titrates dosage and manages side effects of multiple medications; appropriately selects evidence-based somatic therapies for patients whose symptoms are partially- or nonresponsive to treatment (PC5, 4.1, 4.2)

Version 5/2014 Outpatient Global 7 7. Patient Care: Psychosocial Treatments Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Demonstrates a Utilizes elements of professional interest in supportive psychotherapy and curiosity about the (PC4: 2.4) patient s story; accurately identifies patient emotions (sadness, anger, fear); takes into account the patient s social situation, social supports, and available community resources (PC4: 1.1, 1.3; SBP3: 1.2) Has difficulty accurately identifying patient emotions; uninterested in patient s story; little appreciation of patient s social situation or social supports Manages treatment alliance and provides core psychotherapies (supportive, psychodynamic, cognitivebehavioral) to patients with uncomplicated problems; balances autonomy with need for supervision; manages emotional content and the feelings aroused during outpatient care; integrates psychotherapy with other treatment modalities and providers (PC4: 3.3, 3.4, 3.5, 3.6) Provides supportive psychotherapy and at least one of either psychodynamic or cognitive-behavioral therapy to patients with moderately complicated problems (PC4: 4.3)

Version 5/2014 Outpatient Global 8 8. Medical Knowledge: Psychopathology Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Identifies the major psychiatric diagnostic system (DSM) (MK2, 1.1) Cannot identify the major psychiatric diagnostic system; poor knowledge regarding common psychiatric diagnoses Demonstrates sufficient knowledge to identify and treat common psychiatric conditions in adults in outpatient settings (e.g., major depression, anxiety disorders, bipolar disorder, schizophrenia) and to identify common medical conditions in psychiatric outpatients (MK2, 2.1, 2.4) Demonstrates sufficient knowledge to identify and treat most psychiatric conditions through the life cycle (emerging adults elderly) in an outpatient setting, to weigh risk/protective factors of danger to self/others and to include psychiatric manifestations of medical illness and relevant medical and neurological conditions in the differential diagnosis of psychiatric outpatients (MK2, 3.1, 3.2, 3.3, 3.4) Demonstrates sufficient knowledge to identify and treat atypical and complex psychiatric conditions in psychiatric outpatients through the life cycle (emerging adult elderly); to determine the most appropriate level of care; and to diagnose and ensure appropriate care of common medical disorders in psychiatric patients in collaboration with other medical providers (MK2, 4.1, 4.2, 4.4)

Version 5/2014 Outpatient Global 9 9. Medical Knowledge: Somatic Treatment Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Describes general indications and common side effects for commonly prescribed psychopharmacologic agents (MK5: 1.1) Cannot describe general indications or common side-effects for commonly prescribed pharmacologic treatments Describes the indications/ contraindications, including less frequent but serious adverse effects, initiation requirements, and time course of action, for commonly prescribed psychopharmacologic agents; describes indications for second- /third-line agents (MK5: 2.1, 2.2, 2.3, 2.4) Demonstrates an understanding of psychotropic selection based on current practice guidelines or treatment algorithms for common psychiatric disorders and knowledge of pharmacokinetics/ pharmacodynamics (MK5: 3.1, 3.2) Integrates knowledge of the titration and side effect management of multiple medications, monitoring the appropriate lab studies, and how emerging physical and laboratory findings impact somatic treatments; describes use of multiple medications (polypharmacy, augmentation) (MK5: 4.1, 4.2)

Version 5/2014 Outpatient Global 10 10. Professionalism: Sensitivity to Diversity and Patient-centered Care Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Recognizes that patient diversity affects patient care (PROF1: 1.2) Does not recognize the role of diversity in patient care or the importance of patient-centered care Demonstrates capacity for self-reflection, curiosity about and openness to different beliefs and points of view and respect for diversity (PROF1: 2.1) Elicits beliefs, values, and diverse practices of patients and their families, and understands their potential impact on patient care; routinely displays sensitivity to diversity in psychiatric evaluation and treatment (PROF1: 3.1, 3.2) Develops a mutually agreeable care plan in context of conflicting physician and patient and/or family values and beliefs; aware of ways in which own background/beliefs affect interactions with patients (PROF1: 4.1, 4.2)

Version 5/2014 Outpatient Global 11 11. Professionalism: Adherence to Ethical Principles Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Maintains appropriate professional boundaries; maintains confidentiality; obtains informed consent (PC4: 1.2; PROF1: 1.3) Does not consistently maintain confidentiality, understand or obtain informed consent, or maintain appropriate professional boundaries Maintains boundaries while preserving emotional responsivity; lists situations that mandate reporting or breach of confidentiality; recognizes ethical conflicts in practice and seeks supervision to manage them (PC4: 2.2; MK6: 2.2; PROF1: 2.3) Recognizes and avoids potential boundary violations; recognizes ethical issues in practice and is able to discuss, analyze, and manage these in common clinical situations (PC4: 3.2; PROF1: 3.3) Anticipates and appropriately manages potential boundary crossings (PC4: 4.2)

Version 5/2014 Outpatient Global 12 12. Professionalism: Accountability and Responsibility for Patient Care Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Exhibits professional Notifies team and enlists Resident, as well as behavior (e.g., punctuality, back-up when fatigued or patient, family and staff, timeliness, reliable ill so as to ensure good recognize the resident as attendance, appropriate patient care; accepts the the patient s primary professional attire, follow role of the patient s psychiatric provider through on assigned tasks) physician and takes (PROF2: 3.4) and openness to feedback; responsibility (under introduces self as patient s supervision) for ensuring physician (PROF2: 1.3, 1.4, that the patient receives 1.5) the best possible care (PROF2: 2.1, 2.3) Does not consistently exhibit professional behavior Displays increasing autonomy and leadership in taking responsibility for ensuring that patients receive the best possible care (PROF2: 4.4) 13. Interpersonal and Communication Skills: Relationship Development and Teamwork Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Cultivates positive Actively participates in Sustains working relationships with team-based care; supports relationships in the face of outpatient staff members; activities of other staff conflict (ICS1: 3.2) collaborates as a member members and of a care team; recognizes communicates their value communication conflicts in to the patient and family work relationships (ICS1: (ICS1: 2.4) 1.1, 1.2, 1.3) Does not cultivate positive work relationships or collaborate as a team member Sustains working relationships during complex and challenging situations, including transitions of care (ICS1: 4.1)

Version 5/2014 Outpatient Global 13 14. Interpersonal and Communication Skills: Information Sharing and Record Keeping Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Written record (e.g., Written and oral Uses easy-to-understand notes, discharge communication with language in all phases of summaries) accurate, patient/family is communication, including complete, and timely; organized; consistently working with interpreters; ensures transitions of care demonstrates patient/family decisionmaking are accurately communication strategies is shared; documented; engages in to ensure patient/family identifies situations in active listening, teach understanding; which communication is back, and other strategies demonstrates appropriate difficult (ICS2: 3.1, 3.2, 3.3) to ensure patient face-to-face interaction understanding; maintains while using EMR; (ICS2: boundaries during 2.1, 2.2, 2.3) electronic communication (ICS2: 1.1, 1.2, 1.3, 1.4) Written records inaccurate, incomplete, or late; does not accurately document transitions of care; does not ensure patient understanding of their treatment Demonstrates effective written and verbal communication, with patients, colleagues, and other health care providers, that is appropriate, efficient, concise and pertinent; uses discretion and judgment in electronic communication and in the inclusion of sensitive patient material in the medical record (ICS2: 4.1, 4.2, 4.3, 4.4) 15. Practice-based Learning and Improvement: Life-long Learning Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Recognizes limits of one s Seeks and incorporates Uses appropriate knowledge and skills and feedback to improve evidence-based seeks supervision; uses performance; researches information tools (e.g., feedback from teachers, answers to current clinical PubMed searches, Up-Tocolleagues, and patients to questions (PBLI1: 2.1, 2.3) Date, Cochrane, DARE) to assess own level of answer clinical questions; knowledge and expertise critically appraises (PBLI1: 1.1, 1.2) research literature (PBLI1: 3.2, 3.3) Unaware of own gaps in knowledge and areas for improvement; does not ask for help/supervision; does not use feedback for improvement and/or little evidence of performance improvement following supervision Independently improves clinical practice through use of evidence-based information; routinely conducts relevant reviews of evidence when delivering patient care, reads and applies with discrimination; uses

Version 5/2014 Outpatient Global 14 information technologies to support decision making (PBLI1: 4.1, 4.3, 4.4; SBP: 4.1) 16. Practice-based Learning and Improvement: Teaching Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Recognizes role of physician as teacher (PBLI3: 1.1) Uninterested in the role of teacher Teaches medical students or other early learners; communicates goals and objectives; evaluates early learners and provides feedback (PBLI3: 2.1, 2.2, 2.3) Organizes content and methods for individual instruction of early learners (PBLI3: 3.2) Gives formal presentations to groups (e.g., grand rounds, case conference, journal club) (PBLI3: 4.1)

Version 5/2014 Outpatient Global 15 17. Systems-based Practice: Patient Safety and Resource Management Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Follows institutional safety Describes systems and policies, including procedures that promote reporting of problematic patient safety; coordinates behaviors and processes, patient access to errors and near misses community and system (SBP1: 1.3) resources (SBP1: 3.1; SBP2: 3.1) Unaware of or does not follow institutional safety policies Consistently uses structured communication tools to prevent adverse events (e.g., checklists, safe hand-off procedures and briefings); actively participates in conferences focusing on systems-based errors in patient care; recognizes disparities in health care provided on outpatient setting; knows the relative cost of care provided in one s own system (SBP1: 2.2, 2.3; SBP2: 2.1, 2.2) Develops content for and facilitates a patient safety presentation or conference focusing on systems-based errors in patient care (such as M&M conference); practices cost-effective care (SBP1: 4.2; SBP2: 4.1)

Version 5/2014 Outpatient Global 16 18. Systems-based Practice: Community-based Care Not Observed Level 0 Level 1 Level 2 Level 3 Level 4 Knows that community Coordinates care with Incorporates disorderspecific mental health, self-help community mental health self-help support groups, and social agencies, including groups and advocacy networks exist and can collaboration with case groups in clinical care (e.g., name them (SBP3: 1.2) managers (SBP3: 2.1) AA, NA, NAMI) (SBP: 3.1) Does not know about resources commonly employed within the department or institution nor those with whom one customarily collaborates in the community Employs patient-centered principles of care; routinely uses self-help, community resources, social networks, rehabilitation referrals, and recovery programs in patient care (SBP3: 4.1, 4.3, 4.4)

Version 5/2014 Consultation 1 Consultation This form is designed for use by an attending physician to assess a resident s performance in a single consultation or in multiple consultations throughout a rotation based on direct observation. The attending should select one of the following response options by checking in the space for each milestone listed below: Yes (Y): Performs proficiently and reliably; if multiple observations conducted, the resident consistently demonstrates this attribute Partially met (PM): Performs, but not entirely proficiently; some aspects of the performance may be unskillful, incomplete, or inaccurate; if multiple observations conducted, the resident demonstrates attribute(s) proficiently on some occasions No (N): Performs, but significant improvement is needed or fails to perform even when called for Not observed (NOB): The encounter did not require demonstration of this milestone Resident Name Date Attending Rater Milestone # Milestone Yes Partially Met No Not Observed SBP4 (2.2/B) Provides consultation to other medical services SBP4 (2.3/C) SBP4 (3.1/C) SBP4 (3.2/C) SBP4 (3.3/C) SBP4 (4.2/C) PC3 (1.2/B) PC3 (2.3/A) Clarifies the consultation question Assists the primary treatment team to identify unrecognized clinical care issues Identifies system issues in clinical care and provides recommendations Discusses methods for integrating mental health and medical care in treatment planning Manages complicated and challenging consultation requests Recognizes patient in crisis or acute presentation Recognizes co-morbid conditions and impact on treatment

Version 5/2014 Consultation 2 Milestone # Milestone Yes Partially Met No Not Observed PC5 (3.1/A) Manages pharmacokinetic and pharmacodynamics drug interactions when using multiple medications concurrently MK2 (1.3/C) Gives examples of interactions between medical and psychiatric symptoms and disorders MK2 (3.2/B, 4.2/B) Accurately determines need for hospitalization and appropriate level of care MK2 (3.3/C) Identifies and treats common psychiatric manifestations of medical illness MK2 (3.4/C) Includes relevant medical and neurological conditions in differential diagnosis MK2 (4.1/A, 4.3/C, 5.3/C) MK3 (2.3/C) MK3 (4.3/C) ICS1 (2.2/A) Identifies and treats atypical and complex psychiatric conditions in patients with medical and neurological disorders Describes psychiatric comorbidities of less common neurologic disorders Develops working relationships across specialties ICS2(4.2/A,B) Demonstrates appropriate, efficient, concise and pertinent written communication with other health care providers

Version 5/2014 Case Conference 1 Case Conference This measure is intended for completion by attending psychiatrists while observing resident-presented case conferences. The intent is to record the milestones below that are observed (with no expectation that ALL would be observed in a given conference). The first part of the form is for all types of case conferences. The second part is an example of milestone ratings in a specific type of case conference. The attending should assess performance by marking each small box with a Y, PM, N, or NOB for each milestone listed below: Yes (Y): Performs proficiently and reliably Partially met (PM): Performs some aspects proficiently, but other aspects may be unskillful, incomplete, or inaccurate No (N): Performs, but significant improvement is needed or fails to perform even when called for Not observed (NOB): The encounter did not require demonstration of this milestone Resident Name Date Attending Rater Subcompetency Thread Milestones PC2/A Organizes and summarizes findings and generates differential diagnosis 1.1/A Organizes and accurately summarizes, reports, and presents to colleagues information obtained from the patient evaluation 2.2/A Develops a basic differential diagnosis for common syndromes and patient presentation 3.1/A Develops full differential diagnosis while avoiding premature closure 4.1/A Incorporates subtle, unusual, or conflicting findings into hypotheses and formulations 1.2/A Develops a working diagnosis based on patient evaluation PC2/B 2.3/B Describes 4.2/B Efficiently Identifies contributing factors

Version 5/2014 Case Conference 2 Subcompetency Thread Milestones and contextual features and creates a formulation precipitants, predisposing, perpetuating, and protective factors synthesizes all information into concise but comprehensive formulation PC3/A Creates treatment plan 1.1/A Identifies potential treatment options 3.3/A Links treatment to formulation PC3/C Monitors and revises treatment when needed PBLI1/B Evidence in the clinical workflow 3.5C Re-evaluates and revises treatment approach based on new formulation and/ or response to treatment 3.3/B Critically appraises different types of research, e.g., RCTs, systematic reviews, metaanalyses and practice guidelines PROF1/B Compassion, reflection, sensitivity to diversity 2.1/B Demonstrates selfreflection, curiosity about and openness to different beliefs and points of view and respect for diversity

Version 5/2014 Case Conference 3 Subcompetency Thread Milestones ICS2/A Accurate and effective communication with the healthcare team PBLI3/A Development as a teacher PBLI3/B Observable teaching skills 2.1/A Organizes oral information for presentation to colleagues 3.2/B Organizes content and methods for individual instruction for early learners 4.1/A Presentation is appropriate, efficient, concise, and pertinent 4.1/A Gives formal didactic presentation to groups (e.g., grand rounds, case conferences, journal club)

Version 5/2014 Case Conference 4 Conference-specific Content: Example for Neuropsychiatry Case Conference Create the second half of this form according to the milestones being assessed in the specific type of case conference. Examples of specific types of case conferences might be diagnostic, treatment (inpatient, outpatient, or community treatment, neuropsychiatric, morbidity and mortality, clinical EBM conference). Subcompetency Thread MK3/A Neurodiagnostic testing Milestones for Neuropsychiatry Case Conference (Example of adaptation of form for specific types of case conference) 2.1/A Knows indications for structural neuroimaging (cranial CT and MRI) and neurophysiological testing (EEG, evoked potentials, sleep students) 3.1/A Recognizes the significance of abnormal findings in routine neurodiagnostic test reports in psychiatric patients 4.1/A Explains the significance of routine neuroimaging, neurophysiological and neuropsychological testing abnormalities to patients MK3/B Neuropsychological testing MK3/C Neuropsychiatric comorbidity MK3/E Applied neuroscience 2.2/B Describes common neuropsychological tests and their indications 2.3/C Describes psychiatric disorders comorbid with common neurologic disorders and neurological disorders frequently seen in psychiatric patients 2.4/E Identifies the brain areas thought to 3.3/B Knows indications for specific neuropsychological tests and understands meaning of common abnormal findings 4.3/C Describes psychiatric comorbidities of less common neurologic disorders and less common neurologic comorbidities of psychiatric disorders 4.5/E Demonstrates sufficient

Version 5/2014 Case Conference 5 Subcompetency Thread Milestones for Neuropsychiatry Case Conference (Example of adaptation of form for specific types of case conference) be important in social knowledge to and emotional behavior incorporate leading neuroscientific hypotheses of emotions and social behaviors into case formulation

Version 5/2014 Inpatient Admissions PGY1/2 1 Inpatient Psychiatry Evaluation for Admission(s) for Early Residents (PGY-1 and -2) This form is designed for use by an attending physician to assess a PGY-1 or -2 resident based on direct observation (in person, by videotape, through one-way mirror, etc.) of a single inpatient psychiatric admission or several admissions within a short period of time (e.g. a day or a week). The attending should select one of the following response options by checking in the box for each milestone listed below: Yes: Performs proficiently and reliably; if multiple observations conducted, the resident consistently demonstrates this attribute Partially Met: Performs some aspects proficiently, but other aspects may be unskillful, incomplete, or inaccurate; if multiple observations conducted, the resident demonstrates attributes on some occasions No: Performs, but significant improvement is needed or fails to perform even when called for Not Observed : The encounter did not require demonstration of this milestone Resident: Date: Attending Rater: Milestones No Partially Met Yes Not Observed Patient Care Obtains general medical and psychiatric history and mental status examination (PC1, 1.1/A) Acquires efficient, accurate, and relevant history customized to the patient complaints (PC1, 2.1/A) Obtains information that is sensitive and not readily offered by the patient (PC1, 2.3/B) Uses hypothesis driven information gathering techniques (PC1, 3.4/B) Screens for patient safety, including suicidal and homicidal ideation (PC1, 1.3/C) Assesses patient safety, including suicidal and homicidal ideation (PC1, 2.4/C) Performs a targeted examination (MSE), including neurological examination, relevant to the patient s complaints (PC1, 2.2/A) Performs efficient interview and examination with flexibility appropriate to the clinical setting and with attention to time constraints (PC1, 3.2/A) Selects laboratory and diagnostic tests appropriate to the clinical presentation (PC1, 3.3/B) Organizes and accurately summarizes, reports, and presents to attending information obtained from the Copyright (c) Pending. The Accreditation Council for Graduate Medical Education, and the American Board of Psychiatry and Neurology. All rights reserved. The copyright owners grant third parties the right to use the Psychiatry Assessment Tools on a nonexclusive bases for educational purposes.

Version 5/2014 Inpatient Admissions PGY1/2 2 Milestones No Partially Met Yes Not Observed patient evaluation (PC2,1.1/A) Develops a working diagnosis based on patient evaluation (PC2, 1.2/A) Develops a basic differential diagnosis for common syndromes and patient presentations (PC2, 2.2/A) Develops a full differential diagnosis while avoiding premature closure (PC2, 3.1/A) Describes patients symptoms and problems, precipitating stressors or events, predisposing life events or stressors, perpetuating and protective factors, and prognosis (PC2, 2.3/B) Organizes formulation around comprehensive models of phenomenology that take etiology into account (PC2, 3.2/B) Recognizes patient in crisis or acute presentation (PC3, 1.2/B) Asks about treatment adherence and response (PC3, 2.5/C) Manages patient crises with supervision (PC3, 2.4/B) Identifies potential treatment options (PC3, 1.1/A) Applies understanding of psychiatric, neurologic, and medical co-morbidities to treatment selection (PC3, 3.2/A) Links treatment to formulation (PC3, 3.3/A) Maintains appropriate professional boundaries (PC4, 1.2/B) Demonstrates a professional interest and curiosity in patient s story (PC4, 1.3/C) Lists commonly used psychopharmacologic agents and their indications to target specific psychiatric symptoms (e.g., depression, mania, psychosis) (PC5, 1.1/A) Appropriately prescribes commonly used psychopharmacologic agents (PC5, 2.1/A) Uses augmentation strategies with supervision when primary pharmacological interventions are only partially successful (PC5, 3.3/C) Medical Knowledge Demonstrates sufficient knowledge to include relevant medical and neurological conditions in the differential diagnosis of psychiatric patients (MK2, 3.4/C) Professionalism Demonstrates behaviors that convey caring, honesty, genuine interest, and respect for patients and their families (PROF1, 1.1/A) Copyright (c) Pending. The Accreditation Council for Graduate Medical Education, and the American Board of Psychiatry and Neurology. All rights reserved. The copyright owners grant third parties the right to use the Psychiatry Assessment Tools on a nonexclusive bases for educational purposes.

Version 5/2014 Inpatient Admissions PGY1/2 3 Milestones No Partially Met Demonstrates capacity for empathy and curiosity about and openness to different beliefs and points of view and respect for diversity (PROF1, 2.1/A) Exhibits core professional behavior (PROF2, 1.3/B) Introduces self as patient s physician (PROF2, 1.5/C) Displays openness to feedback (PROF2, 1.4/B) Interpersonal and Communication Skills Cultivates positive relationships with patients and families (ICS1, 1.1/A) Develops a therapeutic relationship with patients in uncomplicated situations (ICS1, 2.1/A) Develops therapeutic relationships in complicated situations and sustains these relationships in the face of conflicts (ICS1, 3.1/A, 3.2/B) Demonstrates communication strategies to ensure patient and family understanding (ICS2, 2.2/B) Uses easy-to-understand language in all phases of communication, that may include working with interpreter (ICS2, 3.1/A,B) Yes Not Observed Copyright (c) Pending. The Accreditation Council for Graduate Medical Education, and the American Board of Psychiatry and Neurology. All rights reserved. The copyright owners grant third parties the right to use the Psychiatry Assessment Tools on a nonexclusive bases for educational purposes.

Version 5/2014 Geriatric 1 Geriatric Rating Form This is a supplementary assessment form for geriatric clinical rotations. The forms for specific settings (e.g., inpatient or outpatient) should be used in addition to this form. This form is designed primarily for use by an attending physician for global evaluation of a resident s performance during his or her geriatric rotation. Evaluation can be based on direct observation and/or indirect or oversight supervision. This form also can be used for assessment following observation of a single patient encounter or several encounters within a short period of time (e.g., a day or week). The attending should assess performance by marking each small box with a Y, PM, N, or NOB for each milestone listed below: Yes (Y): Performs proficiently and reliably; if multiple observations conducted, the resident consistently demonstrates this attribute Partially met (PM): Performs some aspects proficiently, but other aspects may be unskillful, incomplete, or inaccurate; if multiple observations conducted, the resident demonstrates attributes proficiently on some occasions No (N): Performs, but significant improvement is needed or fails to perform even when called for Not observed (NOB): The encounter did not require demonstration of this milestone Resident Name Date Attending Rater Rating of performance for: (check one) Clinical rotation Patient encounter(s) Subcompetency: Thread MK1/A Knowledge of human development In late life MK1/B Knowledge of pathological and environmental influences on 1.1/A Describes the basic stages of normal physical, social, and cognitive development in late life 2.1/A Describes normal age-related neural changes in late life 2.2/A Recognizes deviation from normal age-related neural changes, including arrests and regressions at a basic level 2.3/B Describes the effects of emotional and sexual abuse on the Milestones 3.1/A Explains developmental tasks and transitions into late life using multiple conceptual models 3.2/B Describes the influence of psychosocial factors (gender, ethnic, 4.1/B Describes the influence of acquisition and loss of specific capacities Copyright (c) Pending. The Accreditation Council for Graduate Medical Education, and the American Board of Psychiatry and Neurology. All rights reserved. The copyright owners grant third parties the right to use the Psychiatry Assessment Tools on a nonexclusive bases for educational purposes.

Version 5/2014 Geriatric 2 Subcompetency: Thread development in late life MK1/C Incorporation of developmental concepts in understanding geriatric patients MK3/B Neurophysiological testing in geriatric patients PC1/B Collateral information gathering and use MK6/A Ethics 1.2/B Obtains relevant collateral information from secondary sources (PC1) development of personality and psychiatric disorders in late life at a basic level 2.4/C Utilizes developmental concepts in case formulation (MK1) 2.3/B Describes common neuropsychological tests and their indications in late life (MK3) 2.3/B Obtains information that is sensitive and not readily offered by the patient (PC1) Milestones cultural, economic), general medical and neurological illness on personality development in late life 3.3/C Utilizes appropriate conceptual models of development in case formulation (MK1) 3.2/B Displays knowledge of, and the ability to weigh risk and protective factors for danger to self and/or others in late life and the ability to determine the need for acute psychiatric hospitalization (MK3) in the expression of psychopathology in late life 4.2/B Gives examples of geneenvironment interaction influences on development and psychopathology 4 in late life 2.2/A Lists situations that mandate reporting or breach of confidentiality (MK6) ICS1/A Relationship with patients; ICS1/B Conflict management ICS2/A Accurate and effective communication with health care team; ICS2/B Effective 1.1/A Cultivates positive relationships with patients, families, care givers, and team members (ICS1) 2.3/B Negotiates and manages simple patient/family/care giver-related conflicts (ICS1) 3.1/A,B Uses easy-to-understand language in all phases of communication, including working with interpreters (ICS2) 4.1/A,B Demonstrates effective verbal communication, with patients, colleagues and other health care providers, that is appropriate, efficient, concise, Copyright (c) Pending. The Accreditation Council for Graduate Medical Education, and the American Board of Psychiatry and Neurology. All rights reserved. The copyright owners grant third parties the right to use the Psychiatry Assessment Tools on a nonexclusive bases for educational purposes.

Version 5/2014 Geriatric 3 Subcompetency: Thread communications with patients ICS2/B Effective communications with patients 1.3/B Engages in active listening, teach back, and other strategies to ensure patient understanding 2.2/B Consistently demonstrates communication strategies to ensure patient understanding Milestones 3.2/B Consistently engages patients, families, and caregivers in shared decision making and pertinent (ICS2) SBP2/A Costs of care and resource management SBP3/A Communitybased programs SPB3/C Prevention 2.1/A Coordinates care with community health and mental health agencies, including collaboration with case managers (SBP3) 3.2/A Coordinates patient access to community and system resources (SBP2) 2.3/C Describes individual and population risk factors for mental illness (SBP3) Copyright (c) Pending. The Accreditation Council for Graduate Medical Education, and the American Board of Psychiatry and Neurology. All rights reserved. The copyright owners grant third parties the right to use the Psychiatry Assessment Tools on a nonexclusive bases for educational purposes.

Version 5/2014 Psychotherapy Supervision 1 Psychotherapy Supervision This form is designed for use by an attending physician to assess a resident based on direct observation (in person, by videotape, through one-way mirror, etc.) of a single or multiple psychotherapy sessions. The attending should select one of the following response options by checking in the appropriate space for each milestone listed below: Yes: Performs proficiently and reliably; if multiple observations conducted, the resident consistently demonstrates this attribute Partially Met: Performs some aspects proficiently, but other aspects may be unskillful, incomplete, or inaccurate; if multiple observations conducted, the resident demonstrates attributes proficiently on some occasions No: Performs, but significant improvement is needed or does not perform even when called for Not Observed: The encounter did not require demonstration of this milestone Resident: Date: Attending Rater: Psychotherapy Modality: Subcompetency Thread PC4/A Empathy and process ICS1/A Relationship with patients PC4/B Boundaries Milestone Yes Partially Met 1.1/A Accurately identifies patient emotions, particularly sadness, anger, and fear 2.1/A Identifies and reflects the core feeling and key issue for the patient during a session 3.1/A Identifies and reflects the core feeling, key issue, and what the issue means to the patient 3.5/A Manages the emotional content of and feelings aroused during sessions and in response to the patient. 4.1/A Links feelings, behavior, recurrent/central themes/schemas, and their meanings to the patient as he or she shifts within and across sessions 4.1/A Sustains therapeutic and working relationships during complex and challenging situations, including transitions of care 1.2/B Maintains appropriate professional boundaries 2.2/B Maintains appropriate professional boundaries in psychotherapeutic relationships, while being responsive to the patient 3.2/B Recognizes and avoids potential No Not Observed Copyright (c) Pending. The Accreditation Council for Graduate Medical Education, and the American Board of Psychiatry and Neurology. All rights reserved. The copyright owners grant third parties the right to use the Psychiatry Assessment Tools on a nonexclusive bases for educational purposes.

Version 5/2014 Psychotherapy Supervision 2 Subcompetency Thread PC3/A Creates treatment plan PC4/C The alliance and provision of psychotherapies PC4/D Seeking and providing psychotherapy supervision ICS2/B Effective communication with patients PROF1/A Compassion, reflection, Milestone Yes Partially Met boundary violations and crossings 4.2/B Anticipates and appropriately manages potential boundary crossings and avoids boundary violations 4.1 Develops a treatment plan linked to a specific type of psychotherapy 1.3/C Demonstrates a professional interest and curiosity in patient s story 2.3/C Establishes and maintains a therapeutic alliance with patients with uncomplicated problems 2.4/C Utilizes elements of supportive therapy in treatment of patients 3.3/C Establishes and maintains a therapeutic alliance with and provides psychotherapy to patients with uncomplicated problems 3.5/C Integrates the selected psychotherapy with other treatment modalities and other treatment providers Recognizes and manages patient s resistance and other obstacles to change in psychotherapy 4.3/C Provides psychotherapy to patients with moderately complicated problems 4.4/C Selects a psychotherapeutic modality and tailors the selected psychotherapy to the patient on the basis of an appropriate case formulation 4.5/C Successfully guides the patient through the different phases of psychotherapy, including termination 4.6/C,D Recognizes, seeks appropriate consultation about, and manages treatment impasses 5.1/C Provides psychotherapies to patients with very complicated and/or refractory disorders/problems 5.2/C Personalizes treatment based on awareness of one s own skill sets, strengths, and limitations 2.2/B Consistently demonstrates modality specific communication strategies to ensure patient understanding 2.1/A Demonstrates capacity for self reflection, curiosity, and openness to different beliefs and points of view and respect for No Not Observed Copyright (c) Pending. The Accreditation Council for Graduate Medical Education, and the American Board of Psychiatry and Neurology. All rights reserved. The copyright owners grant third parties the right to use the Psychiatry Assessment Tools on a nonexclusive bases for educational purposes.

Version 5/2014 Psychotherapy Supervision 3 Subcompetency Thread sensitivity to diversity PROF1/B Ethics ICS2/C Maintaining professional boundaries in communication PC4/D Seeking and providing psychotherapy supervision diversity Milestone Yes Partially Met 1.3/B Displays familiarity with basic ethical principles including confidentiality and informed consent 4.3/C Uses discretion and judgment in the inclusion of sensitive patient material in the medical record 3.6/D Balances autonomy with needs for consultation and supervision 5.1/D Provides basic psychotherapy supervision to others No Not Observed Copyright (c) Pending. The Accreditation Council for Graduate Medical Education, and the American Board of Psychiatry and Neurology. All rights reserved. The copyright owners grant third parties the right to use the Psychiatry Assessment Tools on a nonexclusive bases for educational purposes.

Version 5/2014 Multidisciplinary Team Meeting 1 Multidisciplinary Team Meeting This form is designed as an encounter form, for use by an attending physician to assess a resident based on direct observation of a single team meeting or several team meetings within a short period of time (e.g., a day or week). This will most commonly occur on inpatient, but could also occur in certain community practice settings. It is anticipated that residents will require multiple rating cycles (with feedback and suggestions for improvement based on performance) to complete all milestones. The attending should select one of the following response options by checking in the space for each milestone listed below: Yes: Performs proficiently and reliably; if multiple observations conducted, the resident consistently demonstrates this attribute Partially Met: Performs some aspects proficiently, but other aspects may be unskillful, incomplete, or inaccurate; if multiple observations conducted, the resident demonstrates attributes proficiently on some occasions No: Performs, but significant improvement is needed or fails to perform even when called for Not Observed: The encounter did not require demonstration of this milestone Resident Name Date Attending Rater Milestone ICS1 2.4 PROF1 2.1 ICS1 1.1 ICS2 2.1 ICS2 4.1 SBP1 2.2 PROF2 2.3,3.4 Behavior Actively participates during meeting and supports activities of other team members Seeks and is open to feedback and diversity of perspective from team members; is curious about different beliefs and others points of view Demonstrates respect for other team members and cultivates positive relationships Organizes both written and oral information to be shared with team Verbal communication with team is appropriate, efficient, concise, and pertinent Consistently uses structured communication to prevent adverse events (e.g., checklists, safe hand-off Yes Partially Met procedures, and briefings) Displays attitude of ownership acceptance of role as patient s primary psychiatric provider; takes responsibility for performance of assigned tasks/roles Items below this line would be expected of more advanced residents, No Not Observed Copyright (c) Pending. The Accreditation Council for Graduate Medical Education, and the American Board of Psychiatry and Neurology. All rights reserved. The copyright owners grant third parties the right to use the Psychiatry Assessment Tools on a nonexclusive bases for educational purposes.