SELF-STUDY FOR GRADUATE PROGRAMS IN HEALTHCARE MANAGEMENT EDUCATION

SELF-STUDY FOR GRADUATE PROGRAMS IN HEALTHCARE MANAGEMENT EDUCATION Self-Study Year: 9/2014 6/2015 Site Visit Date: October 28-30, 2015 Submitted electronically on September 2, 2015 to: The Commission on Accreditation of Health Management Education in support of an application for re-accreditation by: Portland State University Programs in Health Management and Policy; Division of Public Administration College of Urban & Public Affairs; Mark O. Hatfield School of Government October 20, 2015 1

Contents I.A.1.1. Program's Mission... 1 I.A.1.2. Program's Vision... 5 I.A.1.3. Program's Values... 6 I.A.1.4. Statements of University and College/School mission... 7 I.A.1.5. Relation of Program mission to College/School mission... 9 I.A.2.1. Ongoing evaluation of the program... 10 I.A.2.2. Goals, objectives and expected performance outcomes... 12 I.A.2.3. Strengths and weaknesses of evaluation process... 17 I.A.3.1. Monitoring and review process for health systems & University environments... 18 I.A.3.2. Examples from monitoring process... 19 I.B.1.1. Special resources available to the Program and barriers to utilization... 20 I.B.1.2. Relation of non-accredited activities regarding allocation of resources... 21 I.B.1.3. Budgetary allocation administrative procedures... 23 I.B.1.4. Program expenditures... 24 I.B.1.5. Program revenues... 25 I.B.1.6. Administrative support available... 26 I.B.1.7. Program's ability to meet objectives as related to current resources... 27 I.B.2.1. Authority and responsibilities of Program Director... 28 I.B.2.2. Organizational Chart... 30 I.B.2.3. Organization of the Program and its relationships... 30 I.B.2.4. Suitability of Structural location... 31 I.B.3.1. Resources and accommodations associated with large classes... 32 I.B.3.2. Additional resources for online or blended instruction... 34 I.B.3.3. Policies for TA's and co-teaching... 35 I.B.3.4. Majority of instructional time and qualified faculty... 36 I.B.4.1. Availability of University wide academic resources... 36 I.B.4.2. Barriers to access... 37 II.A.1.1. Information available to students... 37 II.A.1.2. Information available to other stakeholders... 38 II.A.1.3. Publication of student achievement measures... 39 II.A.2.1. Target applicant and annual recruitment goals... 39 II.A.2.2. Recruitment activity outcomes... 41 II.A.2.3. Characteristics of entering students... 43 II.A.2.4. Assessment of recruitment and admissions process... 43 2

II.A.2.5. Distribution of enrolled students... 44 II.A.2.6. Criteria used in the student selection process... 45 II.A.2.7. Policies and procedures for exceptions in the Program's selection criteria... 46 II.A.2.8. Recruitment efforts directed at diversity... 47 II.A.3.1. Academic and career advising... 47 II.A.3.2. Evaluation of advising... 49 II.A.3.3. Financial Aid... 49 II.A.3.4. Other support services... 51 II.A.4.1. Stakeholder involvement in Program decision making... 51 II.A.5.1. Career achievement assessment... 52 II.A.5.2. Self-assessment... 53 II.A.5.3. Completion rates... 54 II.A.5.4. Employment rates... 56 II.A.5.5. Employment settings and Program goals... 57 II.A.5.5. Employment settings and Program goals... 58 III.A.1.1. Competencies... 59 III.A.1.2. Competency development and review... 61 III.A.1.3. Competency Coverage across the Curriculum... 62 III.A.1.4. Design of courses and activities relating to competencies... 65 III.A.2.1. Curriculum definition of healthcare management... 66 III.A.2.2. Identification of essential health system & healthcare mgmt knowledge areas... 68 III.A.2.3. Degree s... 69 III.A.2.4. Integration for courses outside the Program... 74 III.A.2.5. Core course waivers... 75 III.A.2.6. Implementation of the competencies... 76 III.A.3-6.1. Core competencies... 76 III.A.3-6.2. Core competencies in the curriculum and activities... 80 III.B.1.1. Time spent on higher vs. lower teaching and learning methods... 81 III.B.1.2. Balance between higher vs. lower teaching and learning methods... 84 III.B.2.1. Major team based activities... 86 III.B.2.2. Interprofessional activities... 87 III.B.2.3. Team building & interprofessional activities for programs with online instruction. 88 III.B.3.1. Exposure to health professionals... 88 III.B.3.2. Exposures relating to students learning... 89 III.B.3.3. List of health organizations... 90 3

III.B.4.1. Integrative experiences relating to learning objectives... 92 III.B.4.2. Primarily integrative activities... 93 III.B.4.3. Field based applications relating to Program goals and objectives... 94 III.B.4.4. Monitoring field based applications... 96 III.B.4.5. Nature of required major paper, thesis or research project... 97 III.C.1.1. Percent of student evaluations using higher vs. lower assessment methods... 99 III.C.1.2. Balance between higher vs. lower level assessment methods... 102 III.C.2.1. Body or Person(s) responsible for ongoing evaluation... 103 III.C.2.2. Methods of evaluation... 104 III.C.3.1. Measurement tools for student progress toward mastery... 105 III.C.3.2. Results of measurements... 106 IV.A.1.1. Assessment of Program's ability to meet objectives... 107 IV.A.1.2. Faculty - Report/list... 108 IV.A.1.3. Faculty teaching responsibilities... 113 IV.A.2.1. Faculty Diversity... 114 IV.A.2.2. Faculty profile/race/ethnicity... 115 IV.A.3.1. Admission decisions procedures... 115 IV.A.3.2. Awarding degrees... 116 IV.A.3.3. Designing curriculum and defining content... 117 IV.A.3.4. Course content and curriculum structure... 117 IV.A.4.1. Faculty appointments... 118 IV.B.1.1. Listing and Description of Program Research and Scholarship Activity... 119 IV.B.1.2. Content and quantity of scholarship/achievement... 122 IV.B.1.3. Relationship of achievement activities to Program mission/goals/objectives... 125 IV.B.2.1. Plan for faculty scholarship development... 126 IV.B.2.2. Individual faculty development... 127 IV.C.1.1. Pedagogical Improvement... 128 IV.C.1.2. Teaching improvement goals... 128 IV.C.1.3. Faculty development activities... 129 IV.C.1.4. Pedagogical Development... 130 IV.C.2.1. Research and scholarship... 130 IV.D.1.1. Community service policies and procedures... 131 IV.D.1.2. Current community service projects by faculty... 133 IV.D.2.1. Community service activities... 133 4

I.A.1.1. Program's Mission The Program will have statements of mission, vision, and values that guide the Program's design, evaluation and quality improvement efforts. The mission, vision, and value statements should define the focus of the healthcare management program in terms of the target audience to be served and the career field(s) for which students will be prepared. A mission statement defines the purpose and direction and any unique aspects of the Program. A vision for the program is a statement that communicates where the Program aspires to be, and serves to motivate the Program to move towards this ideal state. The Program's values are an abstract generalized principle of behavior to which the Program feels a strong emotionally-toned commitment and which provides a standard for judging specific acts and goals. The mission, vision and values will provide the basis for reviewing the Program and for assessing Program effectiveness. In addition, these statements will provide direction for student selection, curriculum design, and scholarly activity of the faculty and will be considered in relation to the mission of the University regarding graduate education, research and service. Program mission may be encompassed in those of a larger unit, such as department or center, and the program should describe and demonstrate how the Program mission relates to the mission of the University and of the parent College/School. Provide the Program's mission statement. Program Mission: The Program in Health Management and Policy builds upon the mission of the Division of Public Administration, translating the broader domain of public service into the specific context of health management and policy. The Program prepares current and future health leaders with the knowledge, skills and competencies needed for progressively responsible careers in health systems management across a range of institutional settings including hospitals, health systems, government, research, advocacy, and nonprofit community-based organizations. I.A.1.2. Program's Vision The Program will have statements of mission, vision, and values that guide the Program's design, evaluation and quality improvement efforts. The mission, vision, and value statements should define the focus of the healthcare management program in terms of the target audience to be served and the career field(s) for which students will be prepared. A mission statement defines the purpose and direction and any unique aspects of the Program. A vision for the program is a statement that communicates where the Program aspires to be, and serves to motivate the Program to move towards this ideal state. The Program's values are an abstract generalized principle of behavior to which the Program feels a strong emotionally-toned commitment and which provides a standard for judging specific acts and goals. The mission, vision and 5

values will provide the basis for reviewing the Program and for assessing Program effectiveness. In addition, these statements will provide direction for student selection, curriculum design, and scholarly activity of the faculty and will be considered in relation to the mission of the University regarding graduate education, research and service. Program mission may be encompassed in those of a larger unit, such as department or center, and the program should describe and demonstrate how the Program mission relates to the mission of the University and of the parent College/School. Provide the Program's vision statement. The vision of the Division of Public Administration s Programs in Health Management and Policy is to be a leader in health management and policy education and be widely recognized for our contributions to the health community though a dynamic, innovative and rigorous curriculum; commitment to excellence in scholarship; and collaborative, inclusive and inspirational leadership among our graduates. I.A.1.3. Program's Values The Program will have statements of mission, vision, and values that guide the Program's design, evaluation and quality improvement efforts. The mission, vision, and value statements should define the focus of the healthcare management program in terms of the target audience to be served and the career field(s) for which students will be prepared. A mission statement defines the purpose and direction and any unique aspects of the Program. A vision for the program is a statement that communicates where the Program aspires to be, and serves to motivate the Program to move towards this ideal state. The Program's values are an abstract generalized principle of behavior to which the Program feels a strong emotionally-toned commitment and which provides a standard for judging specific acts and goals. The mission, vision and values will provide the basis for reviewing the Program and for assessing Program effectiveness. In addition, these statements will provide direction for student selection, curriculum design, and scholarly activity of the faculty and will be considered in relation to the mission of the University regarding graduate education, research and service. Program mission may be encompassed in those of a larger unit, such as department or center, and the program should describe and demonstrate how the Program mission relates to the mission of the University and of the parent College/School. Provide the Program's values statement. We recognize that health management professionals work within a complex set of institutional, sociocultural, clinical, and legal structures. We are guided by the following values which encompass our commitment to integrated, rigorous and relevant teaching, learning, and scholarship, all geared toward the compassionate, effective, and efficient use of resources to address individual and population health issues. 6

We believe that the integration of theory and practice is essential for successful management and leadership in health organizations, and that reflective practice enhances this integration. We believe that a health management education that integrates ethics and experience is essential. We seek to model in ourselves and develop in our students values of public service and provide mechanisms through which students can explore potential roles for engaging the communities they serve in their work. We are committed to addressing the life-long learning needs of our graduates and career health service professionals. We value scholarship that is both rigorous and relevant. We value Portland State University s commitment to community-engaged teaching and scholarship. We are committed to collaborating with public and nonprofit health system organizations and communities to create rich learning experiences that both assist those organizations and communities, and prepare graduates for productive health management and policy careers. We believe that health management professionals must understand the ways in which a multiplicity of interests help to shape, and in turn are shaped by, the underlying values of our health system and by our systems of democratic governance. We believe that understanding the dynamic social relationships in diverse communities is essential for health system leaders and managers. I.A.1.4. Statements of University and College/School mission The Program will have statements of mission, vision, and values that guide the Program's design, evaluation and quality improvement efforts. The mission, vision, and value statements should define the focus of the healthcare management program in terms of the target audience to be served and the career field(s) for which students will be prepared. A mission statement defines the purpose and direction and any unique aspects of the Program. A vision for the program is a statement that communicates where the Program aspires to be, and serves to motivate the Program to move towards this ideal state. The Program's values are an abstract generalized principle of behavior to which the Program feels a strong emotionally-toned commitment and which provides a standard for judging specific acts and goals. The mission, vision and values will provide the basis for reviewing the Program and for assessing Program effectiveness. In addition, these statements will provide direction for student selection, curriculum design, and scholarly activity of the faculty and will be considered in relation to the mission of the University regarding graduate education, research and service. Program mission may be encompassed in those of a larger unit, such as department or center, and the program should describe and demonstrate how the Program mission relates to the mission of the University and of the parent College/School. Provide statements of the University and College/School missions and provide the relevant URLs. 7

Each of the three academic divisions within the Mark O. Hatfield School of Government (Public Administration, Political Science, and Criminology and Criminal Justice) has established a mission statement that is unique, while the College of Urban and Public Affairs mission and vision statements provide overarching direction for all three schools within the college. Thus the HSOG has not established its own unique statements of mission or vision. PA Division Mission: The Division of Public Administration at Portland State University is dedicated to preparing individuals for ethical, competent and effective public service in a range of roles in policy, management and leadership. We seek to improve practice by facilitating learning through community engagement promoting scholarship, and encouraging reflection as we develop and work with leaders representing diverse communities across all domains of public service. PA Division Vision: The Division of Public Administration s vision is to be an agent of change to develop and enhance public leadership through education, scholarship and service. Building upon Portland State University s commitment to community-engaged scholarship and service, the Division of Public Administration is uniquely placed to prepare current and emergent leaders in local, regional, national and global communities. We recognize that solutions to contemporary problems require innovative approaches and alliances among governments, nonprofits and businesses, and encourage our faculty and students to engage in multidisciplinary and collaborative approaches to advance the public interest. We aim to contribute to the integrity, effectiveness, and transparency of the next generation of trusted public leaders. College of Urban and Public Affairs Mission: The College of Urban and Public Affairs provides students interested in the study of urban problems and processes with the resources of an urban university. The Oregon approach to land-use planning, transportation, environmental aspects of urban growth, community policing, low-income housing, solid waste recycling, and community engagement is a model for the nation and a powerful combination with the college s groundbreaking work on government, health and urban issues. College of Urban and Public Affairs Vision: To remain a major force in shaping the future of cities. Through our research and teaching, we aspire to contribute to a comprehensive understanding of the issues and the challenges facing a rapidly urbanizing world. Portland State University Mission: To enhance the intellectual, social, cultural and economic qualities of urban life by providing access throughout the life span to a quality liberal education for undergraduates and an appropriate array of professional and graduate programs especially relevant to metropolitan areas. The University conducts research and community service that support a high quality educational environment and reflects issues important to the region. It actively promotes the development of a network of educational institutions to serve the community. The University s statements of vision, mission and values may be found at: http://www.pdx.edu/our-mission. Portland State University Vision: To be an internationally recognized urban university known for excellence in student learning, innovative research, and community engagement that contributes to the economic vitality, environmental sustainability, and quality of life in the Portland region and beyond. 8

I.A.1.5. Relation of Program mission to College/School mission The Program will have statements of mission, vision, and values that guide the Program's design, evaluation and quality improvement efforts. The mission, vision, and value statements should define the focus of the healthcare management program in terms of the target audience to be served and the career field(s) for which students will be prepared. A mission statement defines the purpose and direction and any unique aspects of the Program. A vision for the program is a statement that communicates where the Program aspires to be, and serves to motivate the Program to move towards this ideal state. The Program's values are an abstract generalized principle of behavior to which the Program feels a strong emotionally-toned commitment and which provides a standard for judging specific acts and goals. The mission, vision and values will provide the basis for reviewing the Program and for assessing Program effectiveness. In addition, these statements will provide direction for student selection, curriculum design, and scholarly activity of the faculty and will be considered in relation to the mission of the University regarding graduate education, research and service. Program mission may be encompassed in those of a larger unit, such as department or center, and the program should describe and demonstrate how the Program mission relates to the mission of the University and of the parent College/School. Comment and demonstrate how the Program mission relates to the mission of the University and of the parent College/School. PSU s Programs in Health Management and Policy are administratively housed within the Division of Public Administration. The Division reviewed and updated its mission, vision, and value and belief statements in fall 2011 as part of a strategic planning effort, and to assure consistency with the broader missions of the College and the University, as well as direction from NASPAA, CEPH and CAHME. The Programs' mission, vision and values statements are consistent with the PA Division, College and University statements with regard to student learning that is achieved through community-engaged scholarship, academic and professional integrity, and the application of knowledge to service in the specific context of health. Thus, we ensure that our Program is responsive to these broader missions, while developing skills and knowledge that are relevant to graduate-level academic preparation for positions in health management and policy. The mission, vision and values of the Division of Public Administration are consistent with, and complementary to, the mission, vision and values of both CUPA and PSU. At every level, these statements embody the PSU motto: Let knowledge serve the city. This motto resonates with our students, the majority of whom seek to build careers in nonprofit, health services delivery and public health sectors. Also inherent in this motto is the concept of community engagement, which is further evident in the mission, vision and values of the University, College and Division. PSU s programs in health management and policy are proud of the extensive network and collaborative relationships we have developed with health programs and service providers across all of these sectors. In fact, it is our culture of collaboration and community engagement that attracts many of our students. 9

A unique vision statement was developed for the Health Management and Policy Programs in response to a CAHME site visit team recommendation in fall 2012. The new vision statement reflects the Program's shared commitment to community engagement, a rigorous curriculum, relevant research and inspirational leadership. In addition, the Program adapted the PA Division Value and Belief statements to further define and clarify Program goals. I.A.2.1. Ongoing evaluation of the program The Program will establish goals, objectives and performance outcomes that are action-based, observable, and measurable. CAHME will seek evidence that specified goals and objectives and expected performance outcomes provide direction and criteria for evaluating ongoing program and curriculum enhancement. Goals must support the mission, vision and values statement by identifying specific areas of emphasis within the Program. Objectives serve to operationalize the goals and should include separate educational, research, service, and other appropriate subcategories of goals and objectives. Prepare a narrative describing how the on-going evaluation of the Program is carried out in light of its mission, goals and objectives. The narrative should demonstrate how the various constituencies of the Program, e.g., students, graduates, faculty, preceptors, and advisory groups, relate to this process. Briefly describe the results of the ongoing evaluations in the last two years and note what improvements, if any, have resulted from the evaluation results. The Program utilizes multiple modes of formal and informal evaluation of its mission, goals and objectives. Stemming from the Program faculty s extensive professional relationships, informal modes include ongoing contact and feedback from the leadership of community-based organizations, including many who are graduates of PSU. The Program also seeks advice about specific strategic and operational issues through periodic interviews, structured reflection sessions and/or focus groups with alumni, field supervisors, local health leaders and other stakeholders. Information gleaned from these efforts regularly results in program changes. Oregon is in the midst of substantial health systems transformation, and it is essential that our students understand the evolving current context of health services delivery. As a result, we have enhanced the content in a number of classes to ensure coverage of examples such as Oregon s Coordinated Care Organizations (CCOs); Patient Centered Primary Care Home Program (PCPCH); and changes in health care reimbursement through Medicare, Medicaid, and the Oregon Health Plan. Based on feedback suggesting an unmet demand for applied health outcomes research, the Program has hired a recent UC Berkeley graduate. The new faculty member has a strong background in quantitative methods and applied population health outcomes research. The Program also seeks informal assessment of strengths and opportunities from students. Division faculty regularly attend informal events hosted by the Public Administration Students Association (PASA), the PSU & OHSU Chapter of the IHI Open School, and other student groups to share information 10

about proposed Program enhancements, changes and opportunities. A particular strength of these regularly scheduled meetings, which occur weekly to monthly during most of the academic year, is the opportunity for students to share information about applied projects, internships and field placement projects and receive on-the-spot advice from faculty about these projects. More formal evaluations of the Program are conducted through course evaluations, student surveys, and student representation on standing and ad hoc administrative committees. Student representatives attend monthly faculty meetings during which proposed changes to the curriculum, competency assessment methods and other program planning and evaluation issues are discussed. Student feedback on course evaluations plays an important role in the development of teaching methods that facilitate attainment, retention and integration of knowledge, skills and competencies. In addition, both formal and informal student feedback is used in the evaluation of faculty effectiveness and the identification of situations that warrant further professional development or counseling of faculty. A student is a voting member of every formal faculty tenure and promotion committee, as well as search committees for new faculty. Program alumni also provide a valuable contribution to ongoing Program assessment and improvements. A substantial number of alumni have established careers in health management and policy positions throughout the greater Portland metropolitan area and in Oregon state government, and regularly serve as points of contact for introducing students to a wide variety of practitioner settings, as well as attending classes and student events as guest speakers. Informal feedback and formal surveys of alumni are used to assess curriculum relevance and applicability, as well as information about competencies and professional development needs. The 2015 Alumni Survey found that 72% of graduates remained in the Portland area, most commonly in supervisory/managerial positions (38%), professional/research positions (25%), or clinical positions (11%). Increased interest in applied research and policy among our students reflects the dynamic health system reform environment in Oregon, and was a salient factor in our decision to seek a new faculty member with these skills. Organizational Experience (PAH 509) preceptors also provide an important dimension of the overall assessment of student competencies. Preceptors complete an assessment of the student who has worked with them, and a survey of their satisfaction with the experience of being a preceptor. Survey results are used to identify needed improvements, such as the creation of a Field Supervisor Guidebook that is now provided to all site-based preceptors. Preceptor assessments of professionalism, work quality, and competency attainment are factored into final grades for student s PAH 509 projects and, in the aggregate, also provide useful information about program strengths and areas for improvement. We have had mixed success over the years with a formal external advisory group as a source of input for program assessment. A Division of Public Administration Advisory Council was reactivated in May 2012, and served in an advisory capacity for all four of the Division s master s level programs. However, the Council did not meet formally in AY 2014-2015. In lieu of the broad input from the PA Division Advisory Council, the Program utilized the OMPH Advisory Committee. This health-focused group does meet regularly, and we receive feedback relevant to both degree programs from this group. Given the work during 2014-2015 on the School of Public Health, as well as evolving plans within the College of Urban and Public Affairs which may affect the structure of the Division and the Hatfield School, our faculty chose not to convene an external advisory group during the self-study year since developments were in process and we wanted to be sure we could present correct information about future directions. Once CUPA solidifies its plans, we will convene our advisors in order to share plans with them and seek their advice on future directions. 11

The SPH Initiative convenes its External Advisory Committee (EAC) regularly, and input from that group is offered to the program through SPH Executive Leadership Council (Dr. Gelmon is a member) and the Academic Program and Priorities Committee (Dr. Wallace and Dr. Gelmon are members). The SPH-EAC serves as a source of information about external trends and provides relevant feedback on the broad scope of public health, including health management and policy, issues. It is comprised of approximately 20 public health professionals representing various geographic regions and specific populations of Oregon, and provides a valuable perspective on public health practice to the Program. The EAC will be an important group for providing input to the developing SPH and MPH:HMP degree, and also provides a useful perspective for the MPA:HA program. Attachments: 2015 Student Survey Report 2015 Student Survey Instrument 2015 Alumni Survey Report 2015 Alumni Survey Instrument I.A.2.2. Goals, objectives and expected performance outcomes The Program will establish goals, objectives and performance outcomes that are action-based, observable, and measurable. CAHME will seek evidence that specified goals and objectives and expected performance outcomes provide direction and criteria for evaluating ongoing program and curriculum enhancement. Goals must support the mission, vision and values statement by identifying specific areas of emphasis within the Program. Objectives serve to operationalize the goals and should include separate educational, research, service, and other appropriate subcategories of goals and objectives. Program response is not required for this. An attachment should be uploaded as support. Supporting Material Use attached template to illustrate goals, objectives, benchmarks, and outcome assessments used routinely by the Program. Evaluate the extent to which each Program objective is met. Indicate actual performance against set targets. 12

I.A.2.2 Program Goals, Objectives and Performance Measures GOAL 1: EDUCATION: THE PROGRAMS WILL PROVIDE COMPREHENSIVE EDUCATION, PROFESSIONAL DEVELOPMENT AND EXPERIENTIAL OPPORTUNITIES THAT PREPARE STUDENTS FOR PROGRESSIVELY RESPONSIBLE MANAGEMENT AND POLICY POSITIONS IN A VARIETY OF HEALTH SERVICE SETTINGS. Measurement Recent Benchmark Status Measurement Results Actions Taken Source Assessment Objective 1.1: The programs will admit students who are well-qualified for graduate studies. At least 80% of admitted students will achieve combined quant/verbal score of 299 on the GRE. GRE scores AY 2014-15 and Fall 2015 At least 90% of admitted students have Review of AY 2014-15 minimum undergraduate GPA of 3.0 transcripts and Fall 2015 Met At least 30% of admitted students will have substantive health system Application files AY 2014-15 Met experience. Objective 1.2: The programs will admit students who represent a range of demographics. At least 20% of admitted students will Review of Partially AY 2014-15 be male. admissions files Met At least 20% of admitted students will identify as racial/ethnic minority. Review of admissions files AY 2014-15 Not Met Partially Met MPH: AY15 = 68%; Fall15 = 63% MPA: AY15 = 38%; Fall15 = 64% MPH: AY15 = 91%; Fall15 = 86% MPA: AY15 = 100%; Fall15 = 92% MPH: 90% w/experience MPA: 75% w/experience MPH: 19% male MPA: 25% male MPH: 14% minority MPA: 27% minority Standard met for verbal, not met for quantitative, mixed for analytic. (Conditional admit &/or stats course required.) Continue to monitor. Continue to monitor. Continue to monitor. Continue to monitor. Results discussed with PA Division EDI Committee; developing action plan. Objective 1.3: Students will be able to access multiple sources of information to progress through the programs. At least 60% of newly admitted students will attend new student orientation. Log of Attendance Fall 2014 Met 61% attendance Continue to monitor. At least 80% of students will have at least one advising session each year. Student Survey AY 2014-15 Met 86% met with advisor Continue to monitor. The Program s website will be updated Records of Numerous updates and ongoing AY 2014-15 Met at least twice per year. Updates revision of website. Continue to monitor. At least 80% of students will report that Partially 81% accessed information on the Add satisfaction with website the website provides easy access to Student Survey Spring 2015 Met website; satisfaction not measured. to future student surveys. relevant information. Objective 1.4: The programs will provide students with didactic learning opportunities appropriate to progressively responsible health management and policy positions. At least 90% of overall course evaluations for health courses with scores 3.5 on average. (5-pt scale) 80% of graduating students who are seeking employment will accept positions within 3 months of graduation. Review of course evaluations Alumni survey Each quarter 2014-15 Met Annually Partially (Summer Term) Met Fall14 met for all but one MPA:HA course (not taken by any HA students) Winter15 met for all courses Spring15 met for all courses 2014 = 92%; 4-6 mos = 96% 2013 = 77%; 4-6 most = 92 2012 = 73%; 4-6 mos = 86% Continue to monitor. Continue to monitor; consider effort to increase visibility of and expand career advising 13

Measurement Recent Benchmark Status Measurement Results Actions Taken Source Assessment Objective 1.5: Students will achieve competence in health management and policy through didactic and experiential learning experiences. Students competency self-assessment Competency Progression evident; all but scores by number of credits completed Each quarter Partially evaluations competency #8 (6.9 average) meet Continue to monitor. will progress to score of 7+ (9 pt. scale) 2014-15 Met (survey report) or exceed target. upon completion of 48 credits. Total average competency score for all courses will be at least 3.5 (5 pt. scale). Course evaluations Quarterly Unsure Data analysis ongoing; available onsite during site visit. N/A Student ratings of course contributions to competency attainment are aligned with competency/curriculum matrix. Competency attainment surveys Quarterly Unsure Data analysis ongoing; available onsite during site visit. N/A 85% of students program competency self-evaluations score at least 7 (9 pt. scale) on each competency upon completion of PAH 509/512 Total average for PAH 509 preceptors evaluations of student competency scores at least 2.5 (3 point scale) Competency selfassessment Preceptor evaluation of students Each quarter 2014-15 Each quarter 2014-15 Met Met PAH 509 student self-assessments range from 7.5 8.6; 90% at 7+. For AY2014-15: PADCs 4 & 5 = 2.5 PADCs 1, 2, 3, 8, 9 & 10 = 2.8 PADCs 6 & 7 = 3.0 Continue to monitor. Continue to monitor. 85% of alumni will be report being prepared for careers based on attainment of PA Division competencies. Alumni survey Every three years Met 90-100% of alumni reported competency-based career preparation. Continue to monitor. Objective 1.6: The program will engage alumni and key stakeholders in program review and development. The Advisory Board will meet at least once per year. Alumni will be surveyed at least once every three years. Log of board meetings Spring 2015 Not Met The PA Division Advisory Board did not meet during AY 2014-15. Utilized SPH Advisory Group, alumni, and formal/ informal feedback from preceptors and community partners. Alumni survey Winter 2015 Met Alumni Survey Report Continue to monitor. 14

GOAL 2: RESEARCH AND SCHOLARSHIP: THE PROGRAMS WILL PROMOTE AND SUPPORT COLLABORATIVE AND APPLIED RESEARCH AND SCHOLARSHIP AMONG FACULTY, STUDENTS, PRACTITIONERS AND COMMUNITIES. Benchmark Measurement Source Recent Assessment Status Measurement Results Actions Taken Objective 2.1: Core faculty will develop, conduct, and communicate an active research and scholarship agenda. 80% of core faculty receive external funding for research/ scholarship related activities each year. 80% of core faculty create and disseminate at least one product of research or scholarship each year (e.g. publications, reports, manuscripts, curricular innovations, or other academic products) 80% of core faculty participate in at least one collaborative or community-engaged scholarship project each year. 80% of core faculty participate in at least one professional development activity each year. Review of faculty CVs AY 2014-15 Met Review of faculty CVs AY 2014-15 Met Review of faculty CVs AY 2014-15 Met Review of faculty CVs AY 2014-15 Met 100% of core faculty received external funding for research activities. 100% of core faculty produced peerreviewed scholarly works. 100% of core faculty engaged in community-based and/or collaborative projects. 100% of core faculty engaged in one or more professional development activities. Continue to monitor. Continue to monitor. Continue to monitor. Continue to monitor. Objective 2.2: Students will develop an appreciation for, and engage in, applied research & scholarship to advance health management & policy. 100% of students will participate in at least three applied health management or policy projects during their program of study. 75% of core/required courses will require students to participate in a collaborative problem-solving project. Review of GO-12 s against inventory of applied projects Inventory of applied projects in courses AY 2014-15 AY 2014-15 Met Met 100% of students participated in three or more course-related applied projects. Percent of courses requiring collaborative problem-solving project Continue to monitor. Continue to monitor 15

GOAL 3: SERVICE: THE PROGRAMS WILL PROVIDE, ENHANCE AND SUPPORT SERVICE ACTIVITIES AMONG FACULTY AND STUDENTS. Benchmark Measurement Source Recent Assessment Measurement Results Actions Taken Objective 3.1: Faculty will engage in service activities that benefit the Division, School, College, University, community, and/or profession. 75% of core health faculty participate in two or more service activities within the University each year. 75% of core health faculty participate in two or more community and/or professional service activities each year. Review of faculty CVs AY 2014-15 Met Review of faculty CVs AY 2014-15 Not Met 100% of core faculty engaged in four or more institutional committees, including serving in leadership roles. 75% of core (PSU health program) and 63% of participating (all PSU & OHSU) faculty participated in 2 community/ professional service activities. Objective 3.2: Students will engage in service activities that benefit academic and/or external communities. The Program or affiliated student organizations will offer at least three opportunities for students to participate in a service activity. 50% of students participate in at least one service activity prior to graduation. Review of student service opportunities (PASA, IHI, OMPH) AY 2014-15 Met Student survey AY 2014-15 Met A combined total of five community engagement opportunities were offered. 86% of graduates participated in at least one service activity. Continue to monitor. Continue to monitor; explore support and/or re-consider metric for non-tenured faculty mix. Continue to monitor. Continue to monitor. GOAL 4: LEADERSHIP: THE PROGRAMS WILL PROVIDE LEADERSHIP AND SERVE AS A RESOURCE FOR ADDRESSING HEALTH MANAGEMENT AND POLICY ISSUES. Objective 4.1: Faculty will demonstrate leadership intended to improve the health of communities. 75% of core faculty annually serve on relevant committees, commissions and/or boards. 75% of core faculty conduct two or more seminars, workshops or presentations on health and public managementrelated topics each year. 75% of core faculty provide professional consultation or technical assistance to education, government and health service organizations each year. Review of faculty CVs AY 2014-15 Met Review of faculty CVs AY 2014-15 Met Review of faculty CVs AY 2014-15 Not Met 100% of core PSU health faculty serve in community leadership roles. 75% of core PSU health faculty, 60% of participating PSU faculty, and 75% of all participating faculty presented. 50% of core health faculty provided professional consultation. Objective 4.2: The programs will facilitate student opportunities to develop and demonstrate leadership skills. 50% of students participate in one or more relevant professional activities or student organizations. Inventory of student activities (survey and/or CV/resume) AY 2014-15 Partially Met 76% of alumni reported participation; not asked in 2014-15 student survey Continue to monitor. Consider roles for other participating/junior faculty. Assess level of support for faculty travel; continue to monitor. Continue to monitor; assess allocation of faculty responsibilities. Continue to monitor; add question to student survey. 16

I.A.2.3. Strengths and weaknesses of evaluation process The Program will establish goals, objectives and performance outcomes that are action-based, observable, and measurable. CAHME will seek evidence that specified goals and objectives and expected performance outcomes provide direction and criteria for evaluating ongoing program and curriculum enhancement. Goals must support the mission, vision and values statement by identifying specific areas of emphasis within the Program. Objectives serve to operationalize the goals and should include separate educational, research, service, and other appropriate subcategories of goals and objectives. Assess the Program's evaluation process highlighting strengths and/or problems. Suggest desired changes in the process and identify steps and a timeframe for making changes The Program s evaluation process is comprehensive, deriving information through a variety of sources and modes. At the operational and strategic levels, recommendations are incorporated into the Program s design and operations and contribute to their interdisciplinary, cooperative nature. The effectiveness of these efforts is reflected in the growing national reputation of the Programs over the past several years and successful initial CAHME accreditation in 2013. At the same time, we have recognized the need to augment and systematize formal data collection efforts. For example, tracking of students job placement and career progression following graduation remains an ongoing challenge due to the number of students who work while attending classes then transition to new roles upon graduation, as well as the fact that the university de-activates psu.edu email addresses after graduation. Although the PA Division maintains a detailed longitudinal database to track the career progression of students and graduates that staff have utilize to develop customized reports for the CAHME accreditation self-study, as well as for responding to periodic requests for data from AUPHA, CEPH and NASPAA, these tracking efforts require significant staff time. During the self-study year and ad hoc committee of faculty, students, and administrative support staff reviewed and recommended minor changes to data tracking activities reflected in the Program s Goals and Objectives (See Supporting Materials in I.A.2.2.) The modifications were subsequently accepted by the Program faculty. Changes included establishment of higher benchmarks for several measures, elimination of measures that consistently and substantially exceeded the established benchmark, the addition of measures related to student diversity and satisfaction with the program, and refinement of measures related to faculty scholarship. Because the modifications were made during the course of the self-study year and required new data collections strategies, data analysis for several of the new measures will not be completed in time for inclusion in this self-study report, however the results will be available for review during the site visit. A final point of evaluation and potential modification is related to the evaluation metrics themselves. We have established a rigorous model that varies slightly from the CAHME template, but which addresses all four (teaching, service, research, and engagement) dimensions of our mission. While robust, the data collection and analysis effort to support programmatic evaluation is considerable. Given the resource constraints we face, the data collection and analysis timeline for several measures has been modified from a quarterly to an annual basis. Although the longer interval raises the possibility that issues requiring corrective action will not be identified as quickly, given the program s long history and seasoned faculty, we believe the risk for such an occurrence is relatively low. 17

I.A.3.1. Monitoring and review process for health systems & University environments The Program will monitor changes in the health system, the University environment, and management theory and practice and adjust its mission, vision, goals, objectives and competency model as necessary. Utilization of market research of the health system and strategic planning assessment tools will enable the Program to identify the changing needs for essential competencies in graduates. This should include appropriate strategic plans, five year academic plans and evidence of external stakeholder input into the curriculum. Programs will be expected to demonstrate a big picture examination of the market, including population served and demonstrate changes effected in response. Describe the process for monitoring the health system and University environments, such as strategic plans, and the process for incorporating this information into the process of Program review and change. The core Program faculty and collaborating OHSU faculty are an important source of information for monitoring the health system through their various committee and board appointments, and through their research and consulting activities. All faculty are actively involved in the Oregon Health Research and Evaluation Collaborative (OHREC) and work regularly with the Office for Oregon Health Policy and Research (OHPR), both within the Oregon Health Authority, and regularly provide consultation and conduct contracted research on the system-level health services environment and state health policy developments. Core faculty are also actively engaged with professional associations (AUPHA, APHA, AcademyHealth, Oregon Public Health Association, Academy of Management, etc.) at the local, state and/or national levels. All of these activities contribute to the Program s ability to monitor both the University and broader health system environments. Program students also provide valuable insights and information about the University, health system and local environment. Many students work in health service organizations and are astute observers of the factors that affect service delivery, system change, policy, innovation, and the fiscal, social and political factors that contribute to dynamic academic, health, and social service environments. They bring these experiences into class discussions, and frequently identify opportunities for student and/or faculty participation through postings to the program listservs. Adjunct faculty members are also a key source of information about the health services environment. As working professionals, these colleagues bring salient expertise and perspectives to the classroom and the programs. Many adjunct faculty have been affiliated with the Program for a number of years and in some cases are Program alumni, offering a stable and insightful resource for program and curriculum development and evaluation. Finally, we draw frequently upon health system practitioners as guest speakers and preceptors, and invite their input into Program activities where they share information and insights regarding current issues in health services management directly with students and faculty. To complement the extensive informal monitoring, formal mechanisms such as the Advisory Council, alumni surveys, and student and preceptor evaluations are used to solicit feedback about the Program. Because all PAH 509 preceptors are working professionals, their assessments of student competency and professional performance provide useful information for evaluation of course-level competencies and the overall curriculum. The PA Division and OMPH Advisory Councils include members from the health, public, and nonprofit sectors, thus provide a broad perspective on the value that Program graduates bring to the community. All of these efforts highlight the community-engaged culture of the Division, and the continuous 18