MASTER OF PUBLIC ADMINISTRATION PROGRAM

Transcription

1 CALIFORNIA STATE UNIVERSITY, SAN BERNARDINO NATIONAL ASSOCIATION OF SCHOOLS OF PUBLIC AFFAIRS AND ADMINISTRATION MASTER OF PUBLIC ADMINISTRATION PROGRAM SELF-STUDY REPORT VOLUME I DEPARTMENT OF PUBLIC ADMINISTRATION COLLEGE OF BUSINESS AND PUBLIC ADMINISTRATION 5500 UNIVERSITY PARKWAY, JACK BROWN HALL SAN BERNARDINO, CA AUGUST 15, 2009

2 TABLE OF CONTENTS PROGRAM SUMMARY STANDARD 1.0 ELIGIBILITY FOR PEER REVIEW AND ACCREDITATION Eligibility Institutional Accreditation Professional Education Program Length... 1 STANDARD 2.0 PROGRAM MISSION MISSION STATEMENT A. Background... 2 B. Mission Statement... 7 C. Mission Process D. Variation from Standards Assessment Guiding Performance A. Guiding Performance B. Program Changes STANDARD 3.0 PROGRAM JURISDICTION Governance and Mission Administrative Organization Recognized Program Program Administration Scope of Influence A. General Program Policy and Planning B. Degree Requirements C. New Courses and Curriculum Changes D. Admissions E. Certification of Degree Candidates F. Course Scheduling and Teaching Assignments G. Use of Financial and Other Resources H. Appointment, Promotion and Tenuring of Program Faculty STANDARD 4.0 CURRICULUM Purpose of Curriculum Curriculum Components A. Background Information Credit System Length of Term Number of Terms Full-Time Status Time Limitation... 54

3 6. Class Contact Hours Numbering System B. Course Distribution C. Capable Professionals D. Assessment and Guiding Performance E. Graduate Courses F. Required Prerequisites Common Curriculum Components A. Required Graduate Courses B. Ethical Action C. Curriculum Coverage Additional Curriculum Components A. Elective Design B. Elective Courses for Broad Advanced Training C. Elective Courses for Specializations General Competencies Minimum Degree Requirements A. Degree Credit Hours B. Degree Length C. Concluding Requirements D. Course Formats E. Student Transcript Analysis Internships A. Internship Design B. Internship Status C. Internship Supervision D. Internship Placement STANDARD 5.0 THE FACULTY Faculty Nucleus A. Critical Mass B. Core Faculty Summary Listing C. Teaching Assignments D. Normal Faculty Course Load E. Other Full-time Faculty Professional Qualifications Practitioner Involvement A. Instruction by Practitioners B. Practitioner Involvement Faculty Quality A. Faculty Data Sheets B. Promotion and Tenure Instruction A. Quality of Instruction B. Workload Policy... 95

4 C. Class Sizes D. Actual Credit Hours Taught Research Experience and Service Faculty Diversity A. Diversity Plans B. Diversity Data C. Faculty Searches STANDARD 6.0 ADMISSION OF STUDENTS Admission Goals and Standards A. Mission, Assessment and Guiding Performance B. Admission Process C. Representation Baccalaureate Requirement Admission Factors A. Admission Factors B. Admissions Record C. Probationary Student Assessments D. Enrollment/Size of Program STANDARD 7.0 STUDENT SERVICES Student Services and Mission Advisement and Appraisal A. Advising System B. Financial Assistance` C. Student Attrition Placement Service A. Placement Assistance B. Follow-up of Graduates STANDARD 8.0 SUPPORT SERVICES AND FACILITIES Budget A. Budget Process B. Budget Sufficiency C. Salary Information Library Services A. Library Support B. Library Program Role Support Personnel Instructional Equipment A. Computer Support B. Audio-Visual Support Faculty Offices Classrooms Meeting Areas STANDARD 9.0 OFF-CAMPUS AND DISTANCE EDUCATION

5 9.1 Definition and Scope Program Mission, Assessment and Guidance Program Jurisdiction Curriculum Faculty Admission of Students Student Services Support Services and Facilities

6 PROGRAM SUMMARY Item Data 1. Title of Degree (1.3). Master of Public Administration 2. Off-campus locations (9.1). Palm Desert Campus 3. Number of credits normally 48 required for degree (4.3-A). 4. Total credits in required courses 36 (4.21-B). 5. Total credits in elective courses 12 (4.22-A). 6. Specializations advertised as None available (4.22C) 7. Number of credits which can be None reduced for prior undergraduate education ((4.3-B). 8. Number of credits which can be None reduced for significant professional experience (4.3-B). 9. Number of faculty nucleus (5.1-6 B). 10. Number of students in degree Full-time 129 (2 or more courses in fall program (6.3-D). term) Part-time 29 (1 course in fall term) Total Is a thesis or major professional Thesis or comprehensive exam is report required? (4.3-C). required 12. Is a comprehensive examination Comprehensive exam or thesis is required? (4.3-C). 13. Is an internship available? Is it required? (4.4-B). required. Yes, but required only for pre-service students. 1

7 STANDARD 1.0 ELIGIBILITY FOR PEER REVIEW AND ACCREDITATION 1.1 Eligibility The Master of Public Administration program meets the criteria in Standard 1.0 and is eligible for peer review and accreditation. 1.2 Institutional Accreditation Institutional accreditation: The Western Association of Schools and Colleges (WASC). Most recent accreditation was March The University received a 10-year accreditation. There are no separate accreditation procedures for the Palm Desert Campus. 1.3 Professional Education A. Leadership and Management Objective The Master of Public Administration (MPA) is a broad program designed to prepare students for a career in administration in the public sector. The major objective is to provide the student with a generalist perspective of public administration, including tools of decision-making, understanding of the total public administrative system, and a capacity for working within the public environment. The degree is intended for graduate students and career employees seeking to develop and strengthen academic and professional skills for admission and advancement in the professional practice of public administration. B. Degree Specification The Master of Public Administration degree, Department of Public Administration, California State University, San Bernardino. 1.4 Program Length The MPA program has been in operation since The Department has received accreditation twice before; this is the Department s third accreditation cycle. The date of the program s first admission is Fall

8 STANDARD 2.0 PROGRAM MISSION 2.1 Mission Statement A. Background California State University San Bernardino (CSUSB) Institutional Setting California State University, San Bernardino is located in the City of San Bernardino and has a satellite campus in the City of Palm Desert in the Coachella Valley, about 72 miles from the main campus. It is one of 23 campuses of the California State University (CSU) system. The system extends from Humboldt State in Arcata in the north to San Diego State in the south. In the system served more than 450,000 students. CSU is the largest system of higher education in the country. The system offers about half of the bachelor s degrees, a third of the master s degrees, and 82% of the degrees in public administration granted in California. CSUSB was founded in 1965 and enrolled more than 17,500 students in and employs more than 2,100 faculty and staff. The San Bernardino campus is the most diverse in California, with a student enrollment so unique that there is no majority ethnic group. CSUSB is designated a Hispanic Serving Institution with at least 35% of the student body self-identified as Hispanic. California State University, San Bernardino serves a cachement area comprised of San Bernardino and Riverside Counties, a geographic area extending more than 27,000 square miles. This area is referred to as the Inland Empire, and includes highly urbanized areas as well as large undeveloped tracts of desert, extending east from the cities of San Bernardino and Riverside to the Arizona and Nevada borders. To the west are the urban counties of Los Angeles and Orange. Bordering on its southern flank are the urban County of San Diego and agricultural Imperial County. Demographic Characteristics The U.S. Census Bureau 2007 current population estimate for Riverside County is 2,073,571. The population for San Bernardino County is listed at 2,007,800. The 2007 estimate for California as a whole is 36,553,215. 3

9 Table 1 shows population demographic characteristics for Riverside and San Bernardino counties. Table Population Estimate Race/Ethnicity for Inland Empire Counties Race/Ethnicity Riverside San Bernardino White alone 1,739,693 1,616,417 Black or African American alone 137, ,635 Asian alone 114, ,807 American Indian & Alaskan Native alone 27,902 28,348 Native Hawaiian & Pacific Islander alone 7,548 7,615 Hispanic or Latino origin 896, ,798 (Source: U.S. Census Bureau, Statistical Abstract of the United States: 2009 (128 th Edition) Washington, DC, 2008.) Historical Development: CSUSB Master of Public Administration Program The Master of Public Administration (MPA) program is housed in the Department of Public Administration (DPA), one of five departments in the College of Business and Public Administration (CBPA) at California State University, San Bernardino. The other constituent departments of the College are the departments of Marketing, Information and Decision Sciences, Management, and Accounting and Finance. In addition to offering the MPA program, the Department of Public Administration offers a Bachelor of Arts degree in Administration, with a Concentration in Public Administration. The College also offers the Master of Business Administration (MBA) degree, which along with the undergraduate concentrations is accredited by AACSB. 4

10 The MPA program was first established in 1975, eight years after the founding of the University. The National Association of Schools of Public Affairs and Administration (NASPAA) first accredited the MPA program in The current reaccreditation process is NASPAA s third extensive program review. The 2002 re-accreditation was for the full seven-year cycle. Enrollment Growth and Ethnicity Average enrollment in the MPA program is 137 students with a low of 110 in 2006 and a high of 158 students in Of the 2008 fall term enrollment, 25% were full-time and 50% were female. Table 2 Fall Term Student Enrollment 2002 to (Source: CSUSB, Office of Institutional Research, Statistical Factbook, 2009) Like the overall University student population, there is no majority racial/ethnic student group enrolled in the MPA program. The largest group in Fall 2008 is White, with 39% of the total, followed by Latino with 27%. Table 3 Master of Public Administration Program Ethnicity Fall Term 2008 & 2007 Ethnicity Fall 2008 Fall 2007 Native American 1 3 Black Latino Asian/Pacific Islander 4 5 5

11 White Unknown No Report 3 1 Total (Source: CSUSB, Office of Institutional Research, Statistical Factbooks 2009 & 2008) Faculty Size The current faculty nucleus is comprised of David Baker, Michael Clarke, Guenther Kress, Anna Ni, Montgomery Van Wart and Clifford Young. In Fall 2009, Dr. Kimberly Collins will join the faculty. Dr Van Wart was hired to be the DPA Chair. He has served in this capacity since Since the last NASPAA review several changes occurred in the composition of the faculty nucleus. In AY 2002/03, the faculty nucleus included Drs. David Bellis, Charles Christie, Michael Clarke, Guenther Kress, Audrey Mathews, Brian Watts and Clifford Young. Five of seven of these senior-level faculty members now are retired and one is deceased From AY 2003/04 to AY 2005/06, Bellis, Christie, Clarke, Kress, Mathews and Watts retired from full-time employment. All with the exception of Watts and Bellis continued their instruction with Faculty Early Retirement Program (FERP) status. In Fall 2004, Dr. Joely Proudfit (formerly de la Torre) joined the faculty and resigned in Summer She transferred from another CSU campus to join the CSU- San Bernardino Department of Public Administration. Among other conditions of her appointment to the department, de la Torre was tasked to develop and teach a specialization on Native American government. Faculty Transition, Special Assignments, and Adjunct Faculty In addition to full retirements, CSU faculty may continue part-time participation in the Faculty Early Retirement Program. The CSU FERP program provides up to fiveyears of partial salary compensation for faculty who desire to continue their participation 6

12 as a faculty member. They may elect to teach as well as assume other responsibilities during this period. At the start of AY 2005/06, Bellis retired and withdrew from teaching. He continued informal participation in department activities until his death a year later. Christie began FERP status at the start of AY 2004/05 and assumed significant MPA program responsibilities that included chairing the annual review process of parttime faculty. He fully retired after a four-year commitment at the end of AY2007/08. Kress began FERP status in AY 2005/06 and Clarke began his FERP obligation in AY2006/07. Both assume significant responsibility for MPA program administration beyond their teaching responsibilities. Kress will be required to end FERP status at the end of AY 2009/2010. Clarke must end his FERP status in June In Fall Quarter 2007, Mathews assumed FERP status. In 2008, Mathews changed to faculty emeritus status and continues to teach as an adjunct faculty. She has remained active as a faculty guest at numerous faculty meetings and as a participant in graduation ceremonies. Dr. Clifford Young has had a variety of special assignments over the years. Recently (starting in 2006), he has managed a large Army-funded research project. He continues his obligations to the department and the MPA program through faculty meetings, committee assignments, and participation in the Comprehensive Exam grading process. Complementing the faculty nucleus are public sector practitioners. MPA Curriculum Changes Since the last NASPAA review, major changes that have occurred include the addition of online class options for all core classes and some elective classes, as well as the elimination of two specialties: Water Resource Management and Tribal Management and Governance. These details changes are discussed at the end of this standard. B. Mission Statement What follows is a statement of the Department s mission, vision, and values, along with its objectives and strategies. These core principles and values convey our intent to deliver a high quality graduate education and effective program operation. The 7

13 Mission Statement, Objectives and Strategies take into consideration current external institutional and internal University dynamics that affect the MPA program. Formal Statements of Mission, Vision and Values The mission of the Master of Public Administration program at California State University, San Bernardino is to provide exemplary and affordable face-to-face and online learning opportunities to students with diverse backgrounds primarily from the Inland Empire region of Southern California. We are committed to helping our students enhance their employment opportunities, career advancement, leadership roles, ethical practices, commitment to democratic values, and service to their communities. To sustain and constantly enrich our educational offerings, we encourage and support faculty to remain current in their fields through active professional development and to conduct applied and basic research on critical issues in public administration. Our faculty is committed to serving the University, governmental and nonprofit organizations in our service area and elsewhere, and professional associations. Vision Statement: As an exemplary educational institution, the Public Administration Department strives to promote an engaged democratic public administration that values accountability, equity, efficiency, effectivenes, and transparency in the management of public and nonprofit organizations. Statement of Shared Values: The Department of Public Administration s Mission and Vision are supported by these shared values that form the foundation of all our activities as educators and researchers in a democratic society. We Value Our Students. All of our actions are judged on the effects they have on the education and career achievement of the students. We value the inputs and points of view presented by the diversity of our student body. We strive to provide excellence in classroom teaching and in the advising of our MPA and undergraduate students. We Value Our Local Government Community. We are committed to serve local, state and federal governments as well as nonprofit entities. We strive to be a regional resource center for expertise in public and nonprofit management. We Value Our Faculty and Staff. The Department provides a collegial atmosphere for the interchange of ideas and diverse points of view. We provide professional development opportunities for both faculty and staff and strive to be a 8

14 productive unit within the greater University. We recognize the value of the faculty s efforts in teaching, in professional development, and in University and community service. We Value Our Alumni. We strive to keep our alumni involved in the Department by encouraging their service on the Program Advisory Board and by recognizing their long-term accomplishments. We Value our College of Business and Public Administration and the University. We are committed to furthering the mission and goals of the College and the University. Care is taken to align the Department's strategic plan with those of the College and the University and to serve as "good citizens" of both. Mission, Objectives and Strategies The MPA program s objectives and strategies are organized to mirror the elements of the mission statement. This statement and the associated objectives and strategies are the result of our strategic planning process. Among these objectives are those that were the result of a special SWOT analysis conducted in the fall of Rapidly changing conditions in the MPA program environment created a need to identify critical issues and priorities for action in the self-study and upcoming years. Already, a clear understanding of the significant challenges we face is guiding our decisions. Simultaneously, our bottom line remains the same. It is to provide students with the ability to creatively solve problems through the application of knowledge and skills they have mastered through their participation in the program. The MPA program objectives and strategies fall into three areas of emphasis: (1) teaching, (2) active professional development, and (3) service to the University, governments, professional associations, and nonprofit organizations. 9

15 Mission Part 1: Provide exemplary and affordable face-to-face and online learning opportunities to students with diverse backgrounds primarily from the Inland Empire region of Southern California. We are committed to helping our students enhance their employment opportunities, career advancement, leadership roles, ethical practices, commitment to democratic values and service to their communities. Mission Part 1: Objectives 1. Enhance CBPA and University support for achievement of the Department s Mission. Advocate for resources to implement the Department s succession plan to increase the number of faculty members to continue the delivery of a high quality graduate education and a balanced faculty workload. 2. Ensure that the MPA program delivers a quality graduate education, curriculum review, and an adequate knowledge and skills-based focus. 3. Ensure the development of a diverse MPA student population and Department faculty. 4. Enhance student recruitment by creating a brand for the MPA program as the Provider of Choice with the best value. 5. Enhance the opportunities for Department faculty and MPA students to assume advisory and leadership roles and general participation in regional public sector associations, programs, and voluntary service projects, as well as faculty sponsored service and projects. 6. Reinforce democratic values and ethical practices in the curriculum, teaching, and Department operations. Mission Part 1: Strategies Objective 1 Enhance CBPA and University support for achievement of the Department s Mission. Advocate for resources to implement the Department s succession plan to increase the number of faculty members to continue to deliver a high quality graduate education and a balanced faculty workload. 1.1 Continue to communicate the need for faculty lines to maintain both program quality and accreditation. 10

16 The Chair will continue to discuss faculty nucleus changes with the Dean to secure her support for an orderly succession. Also, the faculty will explore opportunities within the University to increase full-time faculty participation in the delivery of the MPA program. 1.2 Engage Department Advisory Board members and faculty to establish linkages between the public sector and the University to highlight the positive contributions the MPA program makes to the public sector and service area communities. Continue work with Advisory Board members to advocate on behalf of the positive contribution the MPA program makes to the public sector and service area communities. Leverage the faculty s extensive public service activities to advocate on behalf of the MPA program. 1.3 As soon as tenure track faculty become tenured, nominate them to serve on strategic College and University committees. Objective 2 Ensure the MPA program s delivery of a quality graduate education, curriculum review, and an adequate knowledge and skills-based focus. 2.1 Maintain current MPA program value-added practices (high efficiency levels) and identify others. 2.2 Review MPA program administrative procedures to strengthen quality control of the curriculum and courses. This is an element of a two-year curriculum and course content review to start in AY2009/ Structure a two-year procedure to review the MPA curriculum and course content for opportunities to strengthen coverage of knowledge and skills related to contemporary management and communication advancements. Objective 3 Ensure the development of a diverse MPA student population and Department faculty. 3.1 Use the Department s web site and recruitment print material to include information on University support services and facilities for persons with diverse backgrounds, including physical disabilities. Use the information in recruitment for faculty and students. Increase user-friendly access to the web site. 11

17 3.2 Aim to recruit a diversity of students and faculty. Objective 4 Enhance student recruitment by creating a brand for the MPA program as the Provider of Choice with the best value. 4.1 Develop appropriate marketing identity and material. Create an identity as Provider of Choice and the best value. 4.2 Increase user-friendly access and MPA program information on the Department s web site. Objective 5 Support opportunities for Department faculty and MPA students to assume advisory and leadership roles as well as general participation in regional public sector associations, programs, voluntary service projects as well as faculty sponsored service and projects. 5.1 Continue to engage the Department s Advisory Board membership in the identification of public sector and non-profit managers to participate in formal and informal activities designed to highlight opportunities for research, practice and public service. 5.2 Develop and maintain a central database of useful contacts in the public and nonprofit management community in the University s service area. Objective 6 Reinforce democratic values and ethical practices in the curriculum, teaching, and Department operations. 6.1 Develop a Department policy on transparency and openness with regard to MPA program performance and Department operations. Identify relevant information and/or links to post on the Department s web site. 6.2 Ensure the democratic values and ethical practices in teaching through the use of such practices as informal anonymous feedback, timely and concrete feedback on assessments and adherence to University policies. Ensure that the curriculum addresses and enforces democratic values. 12

18 Mission Part 2: To sustain and constantly enrich our educational offerings, we encourage and support faculty to remain current in their fields through active professional development activities and to conduct applied and basic research on critical issues in public administration. Mission Part 2: Objectives 1. Establish professional development goals that link faculty intellectual and teaching growth with curricular growth. Create the potential for faculty to identify and plan their professional development activities. 2. Generate sources for independent external funds to support research activities. Take full advantage of CBPA and University-wide resources to support professional development activities. 3. Enhance faculty cohesion. Mission Part 2: Strategies Objective 1 Establish professional development goals that link faculty intellectual and teaching growth with curricular growth. Create the potential for faculty to identify and plan their professional development activities. 1.1 Develop and maintain research, teaching, and service agendas. 1.2 Provide support for research, curriculum development, and service via reassign time where possible. 1.3 Provide support for research, curriculum development, and service via funded travel where possible. 1.4 Provide support for research, curriculum development, and service via other types of support such as graduate assistants. Objective 2 Generate independent external funds to support research activities. Take full advantage of CBPA and University-wide resources to support professional development activities. 2.1 The faculty will exert initiative to secure funds from the College s program for professional development funds and reassign time for the publication of peerreviewed journal articles as well as other University programs, such as those from 13

19 the Teaching Resource Center and the Provost s Office for Sponsored Projects seed money initiatives. 2.2 Explore possibilities for funds external to the University, such as federal monies related to military and veteran initiatives. Objective 3 Enhance faculty cohesion. 3.1 Hold annual faculty retreats to establish common goals and develop mutually enforcing strategies. 3.2 Explore joint research and service opportunities. 3.3 Hold quarterly faculty brown bag lunches to discuss research and teaching agendas. Mission Part 3: Our faculty is committed to serving the University, governmental, nonprofit organizations, and professional associations in our service area and elsewhere. Mission Part 3: Objectives 1. Reinforce the Department s effectiveness in shared governance within the University and CBPA community. 2. Maintain close relationships with governmental entities and nonprofit management community in the University s service area. 3. Enhance the Department s presence in the local public sector by engaging in public service activities as well as strengthening the alumni connection. Mission Part 3: Strategies Objective 1 Reinforce the Department s effectiveness in shared governance within the University and CBPA community. 1.1 Nominate qualified faculty to University and CBPA governing committees. Objective 2 Maintain close relationships with governmental entities and nonprofit management community in the University s service area. 2.1 Develop and maintain a central database of useful contacts in the public and nonprofit management community in the University s service 14

20 area. Engage the Department s Advisory Board membership in the identification of public sector and non-profit managers to participate in formal and informal activities designed to highlight opportunities for research, practice and public service. 2.2 Engage members of the DPA program s constituencies in formal and informal activities to enhance the relationship between them and the Department. Objective 3 Enhance the Department s presence in the local public sector by strengthening the alumni connection. 3.1 Develop an outreach plan to strengthen the alumni connection. The faculty will invite and work with an Advisory Board member to participate in this task. MPA Program Environment This section discusses both the strengths and opportunities of the CSUSB MPA program/department on one hand, and the weaknesses and concerns on the other. Strengths and Opportunities For several years (academic years , and ) the faculty and chair engaged in a strategic issues discussion resulting in an evolving list of issues designed to focus the Department s attention on areas for improvement. Although the initial list each year was Chair originated, it generated healthy and robust debate, and was significantly altered by the faculty before being endorsed. Then, as a part of the faculty governance and strategic planning functions in , the faculty conducted a comprehensive, multi-day SWOT analysis. This work by the faculty as a whole not only provided the process product of much enhanced faculty communication and strategic awareness, but also resulted in the end product of both a concrete set of strengths and opportunities, as well as list of critical issues (weaknesses and threats). In its self-assessment by the Department, the following strengths and opportunities were identified (which are discussed subsequently): The breadth of faculty experience and the coherence of their joint mission. The overall quality of the teaching throughout the Department. The special quality and consistency of online teaching in the Department. 15

21 The increasing level of student quality through higher admissions standards and programmatic rigor. The connectedness of the faculty and Department to the community. The enhancement of administrative capacity in the Department to ensure the support of programmatic goals. The faculty is a major strength of the Department because of their shared perceptions of the mission focus and their diverse, but highly complementary experience. While MPA programs can do well when housed in departments of political science, business management, or multi-disciplinary settings, it is often a constant challenge because of the dissimilarity of professional purpose and disciplinary interests. Existing in a free-standing Department of Public Administration, the CSUSB MPA program gets the necessary critical, central attention of the Department and is not relegated an inferior status. As a management-oriented public administration department, the placement of the Department in a business college is appropriate. To the degree that there is competition for resources within the Department between the MPA and the undergraduate program, there is the relatively easy give-and-take of like-minded colleagues who balance the fundamental needs of each. The faculty members have a complementary perspective in that they share a common teaching focus, while maintaining research interests. The faculty has experience and interests in a wide variety of management and policy areas, which is appropriate given the diversity of subjects that must be routinely covered in an MPA curriculum. The diversity of the faculty is also a plus, with the only significant gap being a core Hispanic faculty member. Fortunately, the addition of Collins in 2009, with her Border Studies focus and fluency in Spanish, will mitigate this modest weakness. Finally, the faculty share both a noteworthy entrepreneurialism and healthy pragmatism. While they have taken advantage of strategic opportunities to expand or improve the curriculum, they have also kept the curriculum manageable by eliminating specialties when resources are no longer available. A second, tremendous strength of the faculty is the overall quality of the teaching noted in the Department. While all departments can attest to a student evaluation process and inclusion of teaching as an area of concern for junior faculty, the focus on teaching in the Department of Public Administration is particularly evident in two areas. First, the 16

22 part-time faculty is rigorously reviewed each year, not only with class evaluations, but class visitations by core faculty as well. In addition, a core faculty review of the parttime faculty occurs each spring, in which student issues, complaints, and commendations are shared, before any part-time faculty members are renewed. Part-time faculty get a jointly-issued report each year, so they know that it is not just the Chair who is monitoring performance. Second, good professional education takes an enormous discipline in terms of providing students with numerous opportunities to practice various competencies and get individualized feedback. A recent syllabus review of the courses in the Department confirmed the uniform policy of providing at least five graded activities, and in the case of online classes, sometimes in excess of ten separately evaluated elements. The quality of teaching is also carefully monitored as a part of the exit survey, providing a global measure of performance. This is an area and in which the Department rates highly. A third, related area that deserves special comment is the quality of online teaching. Online teaching can be robust, convenient, and highly individualized, but it can also fail to engage students, be diluted with redundant methods (e.g., excessive print based methods), and feel mass produced and anonymous. Here, the Department of Public Administration does an outstanding job. It has worked diligently to understand the unique dynamics of online education, to master the appropriate technologies (including video and audio methods), and to be disciplined in the practices necessary for successful classes. The faculty was much aided in this quest by federal funding which subsidized the delivery of state-of-the-art classes that were, in fact, ahead of their time. With substantial amounts of technical assistance, the department developed an exceptional level of expertise and was afforded the opportunity to offer showcase classes as well as conduct pilot classes. This commitment to the support of online education is also demonstrated by the in-house technical support that has consistently been provided by the Department. A firm consensus of what it takes to teach online classes evolved through several years of extensive discussions in faculty meetings. The importance of the online teaching experience is further demonstrated by the development of a special online teaching evaluation tool that helps distinguish and bring to light good teaching practices. 17

23 A fourth area of strength in the Department is the increasing level of student quality. Noticing that student rigor was languishing after the last accreditation, a major faculty transition was beginning to take place, and Departmental leadership was distracted by illness, the Department set a new, multi-year course of higher standards. First, there was an effort to curb excessive grade inflation. This was done merely by reporting departmental averages back to faculty, along with their own grade distribution by class (the individual grade distribution of other faculty are not public). In the case of part-time faculty, significant advising now occurs to ensure that they give a reasonable distribution rather than simply assigning all A s as a few might normally do. This means that a significant number of students are put on probation after each quarter, and normally one or two students are dismissed as well. This has also reduced the number of classes that students take, allowing students to more clearly focus on their course of study. Starting in the fall of 2009, students wanting to take three or more classes must have the Graduate Advisor s permission, which is primarily based on GPA as a proxy for their ability to handle the work. Also starting in fall 2009, the Department has raised the GPA requirement from 2.5 to The timing is fortuitous since it matches the national increase in MPA applications. Notwithstanding, the Department has still seen an increase in applications. Finally, the comprehensive examinations have been rationalized to provide a more consistent and coherent experience for all students. In the past, students were provided copies of old exams but did not know what would be asked in the actual four-hour exam. Questions would vary considerably among faculty members with different foci and textbooks, and the hand-written exams were often challenging to read. The latter was addressed by moving the exam to a computer lab setup. Today, students do not have internet access and must use a Departmental disk to save their answers. The former was addressed by providing a 32-question annual study guide. Today, students know exactly what they must prepare (16 questions selected from half the areas), that the quality of writing must be near term-paper quality, and that the Department sees them writing at a professional level in a proctored setting. While some students with solid GPAs in their program of studies have failed the comprehensive exam in the past, we are now realizing a far greater consistency of performance between course work and examination results. This is also a testament to increased grading rigor. 18

24 A fifth strength is the connectedness of the faculty and Department to the community. Three of the recent core faculty members have had major appointed positions in the Inland Empire: Audrey Mathews on the San Bernardino County Planning Commission, Cliff Young as a San Bernardino County Supervisor, and David Baker as the Court Executive Officer for the County of San Bernardino. In addition, many part-time faculty members have appointed positions as city managers, directors, or serve in other senior administrative capacities. The core and part-time faculty are also active in local public sector organizations. The local chapter of ASPA is a particularly important example because of its extremely strong ties to CSUSB. Core faculty members have served as Inland Empire ASPA Chapter President, with Department Chair Montgomery Van Wart the most recent example, serving in CSUSB students are the recipients of the greater portion of awards and scholarships provided by ASPA, and the organization itself relies heavily on the University for support and ideas. Baker and Van Wart have close ties to the surrounding network of community colleges. Connections between the cities and counties and the Department are very strong, whether they be through the Advisory Board, or through teaching, guest lectures, or frequent site visits. For example, the County Administrator for San Bernardino is a part-time instructor and the County Administrator for Riverside is an alumnus of the MPA program. Particularly close ties exist with the City of San Bernardino, which provides internships to those students who are unable to find them elsewhere. The Department also has especially strong ties with Economic Development Agencies in the area, with many of the senior level managers being MPA alumni from CSUSB. Additionally, faculty members use the region as a research laboratory, which further establishes and deepens complementary relationships. Dr. Anna Ni has prepared local case studies on contract cities, privatized library services, and public-private freeway toll lanes. Dr. Montgomery Van Wart has used local human resource examples for his textbook and has just completed a study on municipal Wi-Fi with the City of Riverside as a primary example. A sixth area of strength is the administrative capacity of the Department to ensure programmatic goals. Procedures and practices have been substantially improved with additional staffing and reassign time for undergraduate advising. Increased staffing has allowed for good practices to become institutionalized, such as thorough recordkeeping, 19

25 proactive student progress monitoring, and student services. Students are assisted with program plans on a proactive basis, and participation at the student orientation is required. Students receive an update of the program rules and news every quarter, and comprehensive exam reviews are provided on a regular basis. Additionally, the Graduate Advisor visits some classes every quarter to get a sense of the students issues. Although staffing levels must be slightly reduced in light of the financial exigency in which the University finds itself, it is hoped that the quality of student services and recordkeeping will not noticeably deteriorate. Weaknesses and Threats Weaknesses and threats are constantly evolving. In , the faculty took a snapshot of the long-term critical issues and priorities of the Department during a series of retreats in an attempt to comprehensively examine the areas needing sustained attention. Since that time, the only factor that has significantly altered this list is the ferocity of the economic downturn. Understandably, this crisis has heightened a number of the concerns. The issues and priorities are: The need to maintain CBPA and University support for the achievement of the Department s Mission, especially in the face of the deteriorating fiscal environment. The need to increase the number of faculty members, reduce faculty workload, and effectively manage faculty nucleus changes. The need to maintain the Department s presence in the local public sector, increase local market exposure, and strengthen the alumni connection. The need for enhanced DPA faculty cohesion. The need to continue integration of the MPA on-line and regular curriculum and ensure that the curriculum is relevant to contemporary public sector management challenges. The need to generate independent external funding. All departments and programs need attention and champions to be highly successful, and MPA programs are certainly no exception. Because of its unusual ties to 20

26 the community, its tendency toward entrepreneurialism, and its service obligations within the College, the DPA needs to stay in sync with the College and University. For example, the Department funnels enormous resources into its undergraduate service course, Government-Business Relations (PA 315), which must have a very high percentage of academically qualified faculty, but not at the expense of the MPA program. As another example, University priorities for areas such as water resource management and tribal governance can be Departmental priorities as well, but not when the resources for these specialized initiatives are removed. Because of the Department s unusually high visibility (not atypical of MPA programs), College and University support is critical to carrying out its mission. Fortunately, the College and University support has been strong to date, from the provision of resources to the participation of the President and Provost at Advisory Board meetings. The Department also needs to ensure that the need for an adequate number of faculty members and a reasonable load (given its diverse service and graduate students demands) is addressed. Recent retirements and an unexpected resignation have caused the Department to rely excessively on part-time faculty. In the last accreditation, there were seven faculty members. The departmental core faculty will resume being seven again in the fall of 2009 with the addition of Dr. Collins. Three of the faculty have reduced teaching loads (Clarke, Kress and Young), so that the net effect is a smaller core faculty. While the expectation of additional faculty would normally be pro forma, the financial crisis has put hiring on hold. The Department was fortunate in being able to hire one of only seven new faculty across the entire university this last year, and hopes to soon address its long-term needs, given the regular undergraduate program, the undergraduate service course (PA 315), and the size of the MPA program. The PA 315 (about 14 sections a year) requires staffing primarily by academically qualified core faculty members. In the short-term, reduced reassign times and increased teaching loads can be employed to deal with the fiscal crisis. Eventually, the Department must have additional resources to ensure quality and avoid burnout. As the faculty nucleus evolves, as do their internal responsibilities, attention needs to be paid to managing this evolution carefully and pragmatically to ensure that a relatively small faculty can cover a wide array of duties and assignments. 21

27 The Department must maintain its presence in the local public sector, maintain its local market exposure, and strengthen the alumni connection. While currently a strength, maintaining a presence in the community is an ongoing function that requires the diligence and attention of all faculty members. Such a presence provides a market exposure to ensure that the program gets the most desirable candidates for its program. The fiscal exigency and temporary stop-gap measures may cause some diminution of activity. While the Department has excellent relations with many alumni, consistent outreach and annual functions with all alumni would add considerably to its presence in the community, as well as to desirable development opportunities. Another area of priority is enhancing faculty cohesion. Given the Department s limited resources and high demands, it is critical that the Department take advantage of the faculty s unified vision to ensure that it has a high level of esprit de corps. The Department has risen admirably to various challenges, such as the development of a highquality online curriculum, new hiring, and maintenance of an active Advisory Board. The extraordinary amount of time spent in faculty meetings and annual retreats has assured that the faculty is highly involved in most details. Improving quality control, balancing priorities with limited resources, and maintaining a tight focus is tough work for a faculty. Although the level of professionalism and collegiality in the Department remains extremely high, enhancing collegiality is something towards which we can always strive. Another concern is to maintain the Department s contemporary general curriculum and strong quality of online classes. The curriculum innovations have generally been with electives and time-to-time specialties. The Department is undertaking a two-year curriculum review to examine, discuss, and decide if significant shifts in some fundamental core subjects would be useful. The accreditation process is an integral and timely part of this consideration. The rapid evolution of technology has increasingly shifted course design responsibilities for online classes from design experts to faculty. Manually updating material in online classes, as well ensuring that feedback is frequent, rich, and timely, is ever-present. Finally, there is the need to ensure external funding to assist with requisites and goals of the Department and faculty. As the University does not generally provide an 22

28 annual professional development stipend except to probationary faculty, faculty generate funds for travel and research through successful scholarly activity, auxiliary projects, grants, and out-of-load teaching programs. Special federal projects and cohorts with supplemental fees have done a commendable job in this regard. Indeed, cohorts continue to raise extra money necessary to generate professional development funds and support additional staffing. The Policy Analysis Project with the Army continues to provide auxiliary funds for faculty. Currently, the Department is in serious negotiations for an undergraduate cohort at March Airbase and will implement an online supplemental fee initiative, providing core faculty with funds and the Department with online staff support. MPA Student Population The MPA program s Mission, Objectives, and Strategies meet the requirements and needs of three distinct target groups: (1) in-service government personnel; (2) nonprofit agency managers; and (3) pre-service students. Alumni with an MPA degree work in the public sector in local and state government. The 2008 survey of alumni graduating about five-years ago reported that 100% of those surveyed held positions in the public sector and the 2009 survey indicated that all but two were in public sector organizations. Most are employed in local government municipalities, counties, and special purpose districts. Others are employed by federal and state agencies. In terms of job positions, most students occupy supervisory and mid-management levels, with a smaller percentage at the senior management level, including department heads and elected officials. Many pre-service students come from various undergraduate majors such as Public Administration, Management, Human Services, Sociology, Economics, or Political Science. Most pre-service students have previously worked in the private sector and are transitioning to public agency employment. According to the 2009 Statistical Factbook, 25% of MPA students pursue the degree in full-time status, normally taking two graduate courses per quarter (8 units). These are four-hour block classes, meeting one evening per week from 6:00 PM to 9:50 PM. If students take two of these classes, they come to the University two nights per week. There are some Summer School offerings that allow MPA students to finish the 23

29 program in 18 months (six quarters) if they take two courses per regular quarter, excluding prerequisites and the comprehensive examination. The MPA offered at the City of Corona utilizes a cohort approach, whereby all students move through the program together. This affords the students an opportunity to take classes together for mutual benefit and learning. They really come to know one another and bond, resulting in very positive learning outcomes for such exercises as inclass presentations, team research projects, and group study in general. As documented elsewhere in this report, the MPA program maintains a strong commitment to diversity among its students. Latinos comprise 27%; African Americans, 13%; Asian/Pacific Islanders, 4%; White, 39%; Unknown 13%, and No Report 3%. Women comprise 50% of all MPA students. Other Public Service Programs There are two Western Association of Schools and Colleges (WASC) accredited public higher education institutions in the cachement area: University of California, Riverside and California State University, San Bernardino. WASC accredited private institutions include California Baptist University, La Sierra University, Loma Linda University and the University of Redlands. State-approved institutions include American Heritage University, California Southern Law School, and CapStone University. The University of La Verne, with the main campus located outside of the cachement area, grants bachelor and master degrees through satellite programs offered in Rancho Cucamonga, a campus located in the cachement area. Chapman University, providing similar services, maintains three campuses in the cachement area: Moreno Valley, Ontario and Coachella Valley. On-line WASC accredited higher education institutions include the University of Phoenix and National University. Both maintain satellite education centers in the cachement area. The University of Phoenix has three centers: Murrieta Learning Center, Ontario Learning Center and San Bernardino Learning Center. National University has two campuses: Ontario and San Bernardino. 24

30 C. Mission Process This current reaccreditation constitutes NASPAA's third intensive external program review under NASPAA s mission-driven standards. The Mission statement, as well as the objectives and strategies developed pursuant to the statement, are the result of a broad-based, bottom-up strategic planning process that involved all major MPA program stakeholders. The process began in AY2005/06 with a more focused strategic issues approach and intensified in the years up to AY2007/08. In that year, a thorough, formal, and systematic process was instituted to revise the overarching plan. The strategic planning process was completed in AY2008/09. The formal strategic planning process included information from several sources and included a special SWOT analysis as well as the input of an external consultant. Under the leadership of the MPA faculty, an orderly process was used to revise the mission statement, objectives, and strategies to reflect the findings of the external and internal program analyses. The strategic planning process represents both the informal and ongoing efforts to improve the program from an evolving plan, as well as a formal effort to simultaneously review all major elements of the Department in order to get an overall, balanced, big picture. Regular and special faculty meetings as well as annual retreats have been used to ensure effective, ongoing improvement (e.g., from 2005 to 2007 with an annual strategic issues document, as well as in the self-study year when the Department recreated its overall strategic plan (detailed below)). This process was culminated by a visit from Dr. Meredith Newman in the self-study year. All assessment elements are useful in strategic planning to some degree. The most powerful ones either provide a broad scope on performance or ensure that the program is consistent with its environment. Some of the most important elements providing a foundation for the strategic planning process were: Student exit surveys of their academic experience (one by University and one by Department) MPA curriculum and course content review Feedback from the Public Administration Program Advisory Board via meetings and a Board survey Structured surveys of alumni and employers 25

31 External consultation provided by Dr. Meredith Newman on NASPAA accreditation standards What follows describes some of the more important elements of the strategic planning process. Regular meetings, special faculty meetings, and retreats The Department Mission statement, goals, and strategies evolved in large measure through extensive faculty discussions during regular and special faculty meetings and retreats, with the entire faculty operating as a "committee-of-the-whole". Also present at a number of the meetings were Department clerical staff and adjunct professors. In this manner, mission, objectives, and strategies went through numerous iterations and drafts. Typically, these draft missions, objectives, and strategies are marked-up and edited by all faculty. Special meetings were scheduled to pursue issues highlighted by the process mentioned above, and, that hold significance for the achievement of MPA program goals and objectives. Two meetings were held to engage in a SWOT analysis and identify critical issues, objectives and strategies. The ensuing discussions focused on key constituencies (employers, students, and government agencies), departmental resources in terms of MPA curriculum, budget, faculty, and staff, CBPA support, and then back to the Mission, goals, and strategies themselves. It was thus a systematic process of laying out where the program has been, where it is now, where we want it to go in the future, and how we will determine/assess whether or not we have achieved our goals. Faculty meetings are scheduled monthly for regular business and are generally scheduled for two hours although they are sometimes scheduled for three hours or run over. Special meetings for recruitment and special topics are not infrequent. At the start of the academic year, the Department holds an all-day faculty retreat. All-day retreats between one and two days in length are a traditional and yearly event. Where appropriate, some retreats were attended by office staff and adjunct professors. The sessions were in a relaxed atmosphere at faculty members' homes or an appropriate offcampus facility where the butcher paper was unrolled. This allowed an even more intensive discussion, review, and critique of strategic planning results from the regular 26

32 faculty meetings. It should also be noted that the College has all-cbpa faculty meetings about three or four times a year as well. These meetings include all the Departments and staff in the College on issues of broad importance. In sum, the faculty of the Department of Public Administration spend a substantial amount of time meeting, discussing, and deciding, and thus the faculty governance process is extremely robust. Current Student Survey (University) The University s Office of Institutional Research usually administers a spring survey to students who graduate in the current year of the survey s administration. The Survey was not done for AY2008/09. However, past surveys exist and have data relevant to the current self-study. This survey addresses the overall quality of students experience while enrolled in the MPA program. It is comprehensive and addresses multiple factors relevant to the quality of the education as well as several other factors. Among them are questions related to advisement, community or professional service, program administration, or student relationships. The final question solicits responses regarding overall satisfaction with the program s performance. However, the survey s overall utility is somewhat diminished by a small number of respondents and by mixing the undergraduate and graduate students in the same N. Therefore, the Departmental exit survey (below) is the more powerful tool. Graduating student exit surveys (Department) Since the 1980s, every MPA program graduate has been surveyed using an exit questionnaire. This survey is used to assess the adequacy of support facilities, job related knowledge and skills requirements, the quality of the faculty, the quality of the internship, program administration, minority and women s issues, and the students overall satisfaction with the MPA program. These data are reported and analyzed more completely elsewhere in this report. Suffice to say here that this information, obtained from students who have just finished their comprehensive examination or graduate research project, has been useful in developing curricular aspects of the mission, objectives, and strategies as well as in academic advisement and the culture of the program. The information has also provided faculty and students with general feedback 27

33 on student perceptions of the MPA program s value to their careers. Do they believe, generally, that the program has been of value to them in terms of enhanced skills, competencies, and abilities, and where career advancement is concerned? While the overall answer to this question has been strongly affirmative, the survey provides a powerful profile allowing faculty to decide on what priorities need additional attention on an annual basis. The specific value of this information in the strategic planning process was to inform faculty that the program is generally on the right track, and major deviations from the present course are not demanded at this time from the students perceptions. MPA curriculum and course review An element of the self-study year strategic planning process included an overarching review of the MPA curriculum and course content with limited resources, as no major expansion of the curriculum is envisioned. However, the preliminary assessment indicated that a more in-depth and comprehensive review would be useful and is scheduled to occur next year. An in-depth syllabus review was conducted this year to kick off that process. Program Advisory Board Meetings The PAB meets twice a year to discuss MPA program curricular and administrative issues, special events, and other subjects the Board wishes to address. This information is formally documented in Board minutes. The Board Chair, Department Chair and faculty PAB liaison hold joint responsibility to initiate action on Board decisions. Membership on the Board includes practitioners who are former MPA students in the program and members who are government administrators without having been CSUSB students. As senior managers, their feedback on the philosophy and exact content of the MPA program has been invaluable in assisting faculty with developing a clear mission articulated through achievable goals and action. Feedback from the Public Administration Program Advisory Board 28

34 An Advisory Board survey was conducted to solicit formalized feedback for the strategic planning process. It was an survey and proved to be quite effective in soliciting valuable information. The survey was conducted in Fall This survey generated important information about effectiveness of the PAB in program governance, Board diversity, opinions of on-line course offerings, educational partnerships, and identification of contemporary knowledge and skills for the public sector. The survey also asked for their general opinion (favorable, unfavorable) of the MPA program. Following these surveys, Department faculty conducted a structured focus group exercise with the leadership cadre of the PAB (n = 10), to review the results of the surveys and to provide additional perspectives on MPA program operations. Employer Survey This survey of employers of our MPA graduates is also conducted every seven years as an integral part of the reaccreditation self-study process. The Department administered it. The Survey was conducted in Spring The Survey ascertains information on the strengths and weakness of the MPA program, satisfaction with the job preparedness of graduates, performance qualities of MPA graduates, and a rating of knowledge and skills exhibited by MPA graduates. Alumni Survey The Office of Institutional Research conducts this survey every year. The respondents are those who graduated five years from the date of the current annual distribution. This survey measures students satisfaction with their educational experience in the MPA program, effectiveness in achieving specific learning outcomes, effectiveness of their graduate education in enhancing the achievement of professional goals, and employment-related outcomes. The most recent survey includes the AY2003/04 graduating class. Dr. Meredith Newman, special consultation Upon start of the self-study year, the faculty invited Dr. Meredith Newman, Professor and Chair, Department of Public Administration, Florida International 29

35 University, to the campus to review the MPA program and consult on NASPAA standards. Of particular interest to the Department was gaining a sound understanding of the standards and their application. Dr. Newman is a recognized expert on strategic planning and mission driven concepts and processes. Her observations were very useful to the development of the mission, objectives, and strategies. To close the planning/action feedback loop, information from all the above research has been fed back into the program strategic planning process. Exit survey data have continually demonstrated that students believe that as they advance to more senior positions, those skills, competencies, and abilities developed during their MPA studies will become more important. This indicates that the Department's curriculum is appropriate for our constituency, and students find that it will be useful as they move up the career ladder. Obviously, faculty meetings and extended off-site retreats have provided the forum for refinement of the Department's Mission, objectives, and strategies for goal achievement. Overall, this orderly, inclusive strategic planning process has focused heavily on developing a clear educational philosophy and Mission and generating appropriate, achievable strategies and objectives consistent with that Mission, resources, and constituencies. The process has given faculty a much better sense of the program's elements and their relation to the whole, and has enabled the Department to better assess program operations and standards. The Mission, objectives and strategies are the result of hours of discussion, deliberation, and refinement of critical issues related to the old planning questions: (1) Where we come from? (2) Where we are? (3) Where we want to go? (4) How we know once we ve arrived? D. Variation from Standards None 2.2 Assessment The first part of this section covers how data are collected (Assessment Procedures) and the second section (Assessment Plan) describes how those procedures 30

36 are used in conjunction with the strategic plan generally and to provide continuous improvement more specifically. Assessment Procedures The Department uses a rigorous and comprehensive framework to assess the accomplishment of its Mission and objectives. At the core of the assessment process is the faculty s evaluation of progress, identification of areas for improvement and methods to enhance performance. Throughout an academic year, there are monthly faculty meetings, usually two to three hours, where items for action or discussion are considered. Added to the monthly meeting are special ones set to address issues of heightened importance. Information for this process is gathered from quarterly, annual and periodic surveys administered by both the University and Department. Survey respondents include MPA program stakeholders: students, alumni, employers, and Advisory Board members. Important additional information is gathered from student teaching evaluations of all instructors; and, peer teaching evaluations of both adjunct and full-time faculty. We also obtain significant information through informal faculty and staff contacts with students and other relevant stakeholders, University and College colleagues, and officials. A number of assessment procedures have already been discussed in the previous section and will only be listed here including: student exit surveys of their academic experience (one by University and one by Department); MPA curriculum and course content review; feedback from the Public Administration Program Advisory Board via meetings and a Board survey; structured surveys of alumni and employers; and external consultation provided by Dr. Meredith Newman on NASPAA accreditation standards. Additional assessment instruments and procedures are as follows: Student Opinions of Teaching Effectiveness (SOTEs) Course evaluations are conducted under the oversight of the Associate Provost for Academic personnel at the end of each quarter. The Student Opinions of Teaching Effectiveness (SOTE) process is designed as one of several sources of information to be used in performance review and periodic evaluation of faculty teaching. In addition, faculty should 31

37 find SOTE information helpful in pursuing the normal professional goal of maintaining and improving instructional effectiveness. All full-time faculty members must have at least two courses evaluated per year. All courses taught by part-time instructors are also evaluated. Each evaluation assesses the overall quality of instruction, instructor performance, and various aspects of classroom experience. SOTEs play a major role in the Retention, Promotion and Tenure and compensation awards processes. Student teaching evaluations Students enrolled in a course evaluate the instructor and faculty is evaluated twice a year. The questions ask for ratings on faculty teaching, knowledge of the subject, quality of interaction and student feedback. The rating scale is a six-point Likert-type scale. Evaluation results are discussed in another section of this report. Peer teaching evaluations A member of the regular faculty evaluated every part-time and assistant professor teaching in the MPA program. Part-time evaluations are discussed annually as a committee-of-the-whole. The information is useful in identifying needs for teaching improvements. Evaluation results are discussed in another section of this report. Comprehensive Examination This exam, which most of our graduate students take, provides the faculty with important feedback on program performance in terms of students ability to conceptualize, write well (informatively, persuasively, etc.), and master content domains at a professional level. Students have a new question bank each year composed of four broad questions in eight areas (i.e., 32 in all) aligned with the eight core classes. In the exam, they may select four areas but do not know which question will be asked in each area. Because students have a clear idea of what to expect, writing is expected to be near that of term-paper quality. The exam is generally taken in the last quarter of a student s program of study, but can take it a quarter early with permission. The Comprehensive Exam (PA 999) is zero units so, although required for graduation, it does not constitute a part of the student s 48 unit requirement. The exam may only be taken twice. 32

38 Students do have the option of a Graduate Research Project, which is a master s thesis-type alternative. In order to exercise this option, students must approach the Graduate Advisor in the first half of their program of study with a formal proposal and work outline. No more than one or two students per year select this option. Ad Hoc Assessment Instruments and Procedures It should be noted that the MPA program has used additional assessment procedures and instruments in conjunction with the strategic planning process discussed above. These assessment efforts (Advisory Board focus group and student focus group exercises) were aimed at determining the external opportunities and constraints, as well as the internal strengths and weakness, of the program (SWOT analysis). MPA program faculty members also use course syllabi, faculty data sheets, and department and program files to monitor program performance. In addition, the Department did an audit of all MPA courses to identify the type and number of in-class and out-of-class student assignments; and, initiated a curriculum review. Faculty members use these information sources to assist in the assessment of the program s performance and Mission. The assessment efforts are used to make appropriate changes in program operations, as will be detailed below. Our experience in assessment is in-depth and very useful. The Student Exit Survey is a core instrument, and survey results guide our decisions regarding program changes. The Department for decades has used the Student Exit Survey. Over the last three years, exit results show an upward, positive trend. This reflects the faculty s decisions to improve performance based on the exit surveys. Likewise, use of the Advisory Board survey results reinforces our approach to achieve a balance of face-toface and on-line instruction as the MPA program endeavors to respond to the program delivery needs of our constituencies. In-depth analysis of our assessment experience is presented in other sections of this report. In sum, the MPA program has in place useful and timely procedures and instruments to assess its performance. Multiple and overlapping assessment procedures are used to enhance the reliability and usefulness of findings. As a result, assessment has 33

39 become an integral part of the MPA program s management. Assessment procedures and instruments detailed above are integrated into the following program assessment plan. Assessment Plan The Department of Public Administration has designed a detailed and comprehensive assessment plan in order to systematically track and measure the performance of the MPA program. Department faculty members use this assessment information in an ongoing process of program improvement. Our outcomes assessment plan goes beyond the identification and measurement of simple program outputs, such as the number of graduates or the type and level of knowledge attained by our students. Instead, the plan focuses on desirable outcomes the program is expected to achieve in accordance with the stated mission and program goals. The Department's assessment plan addresses three pertinent questions regarding program operations: (1) What are we doing? (2) Why are we doing it? (3) How well are we doing? Specific plan features are detailed below. For each part of the Mission Statement, and for each objective, the plan specifies the following: measurable performance indicators (both for outputs and outcomes) and information sources. Where appropriate and possible, the assessment plan calls for multiple measures to provide information about a specific performance indicator. Use of multiple measures, whether quantitative or qualitative, enhance the accuracy and reliability of performance measurement. Mission Part 1: Provide exemplary and affordable face-to-face and online learning opportunities to students with diverse backgrounds primarily from the Inland Empire region of Southern California. We are committed to helping our students enhance their employment opportunities, career advancement, leadership roles, ethical practices, commitment to democratic values and service to their communities. 34

40 Table 6 Mission Part 1 Assessment Plan Mission Part 1: Objectives Indicators Information Source 1. Enhance CBPA & University support for achievement of the Department s Mission 1.1. Number and success of issue-based discussions taking place between CBPA and DPA Chair ad hoc reports Faculty meeting minutes Faculty/Dean meeting minutes 2. Ensure the MPA program s delivery of a quality graduate education, curriculum review, and an adequate knowledge and skills-based focus 1.2. Number of Advisory Board members promoting benefits of MPA program in appropriate CBPA, University and community venues 1.3. Number of DPA faculty represented on strategic CBPA and University committees 2.1a. Ongoing student satisfaction with curriculum and faculty quality 2.1b. Professional evaluation of teaching effectiveness 2.1c. Ongoing monitoring of acceptable grade distribution 2.2a. Advisory Board satisfaction with MPA 2.2b. Employer satisfaction with MPA education 2.2c. Assessment of learning outcomes 2.3a. Completion of course syllabi review Advisory Board minutes Faculty meeting minutes Faculty meeting minutes (committee reports) Faculty Data Sheets (CVs) Student Exit Survey SETE evaluations Peer teaching evaluations (class visitations and review of faculty files Quarterly grade distribution review by class and the Department Annual open discussion forums with the Board Advisory Board survey (with accreditation cycle) Employer survey (with accreditation cycle) Comprehensive exams (review of pass rate, etc.) Course syllabi review report 35

41 3. Ensure the development of a diverse MPA student population and Department faculty 4. Enhance student recruitment by creating a brand for the MPA program as the Provider of Choice with the best value 2.3b. Strength of match between course activities, knowledge and skills gained 2.3c. Curriculum review structured & initiated 3.1. Update website and materials to reflect an interest in diversity 3.2a. Students admitted from students of diverse backgrounds: race/ethnicity/ gender/physical disabilities/ sexual orientation 3.2b. Composition of faculty by race/ethnicity/ gender/physical disabilities/ sexual orientation c) Employment of adjunct faculty of diverse backgrounds: race/ethnicity/gender/physical disabilities/sexual orientation 4.1 Agreement upon positioning strengths and brand image 4.2 Use of positioning strengths and brand image in communications Employer survey Advisory Board survey Student exit survey Curriculum review All-day faculty retreat minutes Faculty meeting minutes Curriculum review meeting minutes Website Recruitment materials Admissions records Recruitment material Department employment records Adjunct faculty employment records Recruitment material Faculty meeting minutes Web site review Advertising documents 36

42 5. Enhance the opportunities for Department faculty and MPA students to assume advisory and leadership roles as well as general participation in regional public sector associations, programs, voluntary service projects and faculty sponsored service and projects 6. Reinforce democratic values and ethical practices in the curriculum, teaching, and Department operations 5.1a. Number of faculty engaged in public service activities 5.1b. Number of active students engaged in public service activities 5.2 Use of the Advisory Board to provide connections for students and research 5.3 Development of a central database of organizations and individuals with whom to stay in contact and utilize for development 6.1a. Courses including topics on democratic values and ethical practices 6.1b. Learning outcomes including knowledge and skills relevant to democratic values and ethical practices 6.1c. Students satisfied that essential democratic practices are used in classes and office operations 6.2 Web site includes information that provides transparency and openness regarding standards, expectations, etc. Department files Faculty Data Sheets (CVs) Faculty meeting minutes Graduate student orientation Faculty ad hoc reports Department bulletins and notices Advisory Board minutes Central database Curriculum review reports Syllabi Comprehensive Exams Student Exit Survey SETE results Website Quarterly graduate advisor letter Quarterly undergraduate information letter Mission Part 2: To sustain and constantly enrich our educational offerings, we encourage and support faculty to remain current in their fields through active professional development activities and to conduct applied and basic research on critical issues in public administration. 37

43 Table 7 Mission Part 2 Assessment Plan Mission Part 2: Objectives Indicators Information Source 1. Establish professional development goals that link faculty intellectual and teaching growth with curricular growth. Create the potential for faculty to identify and plan their professional development activities 1.1. Quality of professional development plans and research agenda 1.2a. Number of reassign times for research and curricular development activities, special projects Faculty activity reports Faculty CVs Department course schedules Faculty activity reports 1.2b. Number and amount of University and externally funded travel for research, instructional improvements and presentations Faculty Data Sheets (CVs) Curriculum review reports Faculty meeting minutes 2. Generate sources of funds to support research activities, curricular and service activities. Take full advantage of CBPA, University-wide, and external resources to support professional development activities 1.2c. Number and amount of other types of support for teaching, research, and service activities (e.g., reassign time for service, funding for graduate students, etc.) 2.1a. Number of applications and approvals for CBPA Peer Review Journal (PRJ) program 2.1b. Number of applications and acceptances for University mini-grants 2.2. Faculty submissions for support from external sources Faculty Data Sheets (CVs) Faculty activity reports Faculty Data Sheets (CVs) CBPA Peer Review Journal notification Teaching schedule Faculty Data Sheets (CVs) Teaching schedule Faculty Data Sheets (CVs) Office of Research & Sponsored Programs records 3. Enhance faculty cohesion 3.1. Number of faculty retreats Faculty meeting minutes 38

44 3.2. Number of joint projects conducted by faculty 3.3. Number and success of brown bag lunches Faculty CVs Faculty minutes (reports) Mission Part 3: Our faculty is committed to serving the University, governmental and nonprofit organizations in our service area and elsewhere, as well as professional associations. Table 8 Mission Part 3 Assessment Plan Mission Part 3: Objectives Indicators Information Source 1. Reinforce the Department s effectiveness in shared governance within the University and CBPA community 2. Maintain close relationships with governmental entities and nonprofit management community in the University s service area 3. Enhance the Department s presence in the local public sector by strengthening the alumni connection 1.1. Number of qualified faculty serving on strategic University and CBPA committees 2.1. Database of contacts established Increase representation of governmental entities & nonprofit sector in DPA meetings and/or functions 3.1. Alumni participation in DPA events grows Faculty meeting minutes CBPA notifications University Senate notifications Department records Advisory Board meeting minutes Faculty ad hoc reports Department files Faculty meeting minutes Department records Advisory Board Survey Advisory Board meeting minutes Alumni surveys Employer Survey 2.3 Guiding Performance A. Guiding Performance As reported in the Assessment section of this report, the Department uses evaluative information to guide program performance. The information that is used more routinely than other sources is the Student Exit Survey, Student Opinions of Teaching Effectiveness (SOTE) Survey, faculty evaluations of adjunct teaching effectiveness, peer 39

45 evaluations of full-time faculty and comprehensive examinations. On a more informal basis, faculty members bring their own observations into performance discussions. Use of Information in Guiding Performance Our experience in using assessment information to guide program performance has been a core activity of the Department for decades. The Student Exit Survey is a core instrument and our use of survey results is quite effective. Over the last three years, exit results generally show an upward, positive trend. This reflects the faculty s decision to focus attention on specific program elements to improve performance. Likewise, our use of the Advisory Board survey results have reinforced our approach to achieve a balance of face-to-face and on-line instruction as the MPA program endeavors to respond to the program delivery needs of our constituencies. These are examples of the results the Department has enjoyed due to the systematic collection and use of program data. Indepth analysis of our assessment experience is presented in other sections of this report. Use of Information, Identifying Issues and Results There is consensus across all sources of information that the MPA program is more than adequately preparing our students for management positions in the public sector. For example, exit interviews with MPA graduates and our employer survey underscore the usefulness of the curriculum for the preparation of public managers. Additionally, almost all of the employers surveyed expressed high satisfaction with the performance qualities of the Department's graduates. Overall, our current students and graduates appear to be extremely satisfied with their experiences in the MPA program. Almost all would recommend the MPA program at CSUSB to a friend or colleague. The number of pre-service students requiring an internship is relatively low because most MPA students have had previous administrative experience and, therefore, have the internship requirement waived. Of those who have been enrolled in an internship, most felt that it provided them with an invaluable experience. One hundred percent of the alumni surveyed in 2008 are employed in the public service. They agree that their MPA degree has led to significant career advancement. 40

46 The upward mobility of our graduates appears to be strongly related to the many qualities they bring to their jobs as a result of their graduate education. Specific qualities observed by employers include having very good theoretical underpinnings, knowing the operations of local government, being versatile, and displaying solid analytical skills. When asked to rate the performance qualities of a person with an MPA degree, the large majority of employers rated those qualities as very effective. Our graduates seem to excel across multiple job-related skills and job-related knowledge. The only skill that was not uniformly rated as average or above average was writing. All other measures of performance were either rated average or above average with the latter receiving a majority of responses. These are highlights taken from the surveys conducted during the self-study year. Of particular interest are the results of the Employer Survey. The questionnaire asked for ratings of a number of job-related skills and specific knowledge. Another source of evaluative information comes from the faculty itself. In the self-study year, the faculty directed its attention to the adequacy of the curriculum. They looked at it in terms of NASPAA s Standard 4.21 Curriculum Coverage and also in terms of the recent addition of faculty with special expertise in the use of technology in management. The results of the Employer Survey also will be useful in the two-year curriculum review. The review will consider course content as well as course linkages. Our graduates rate the effectiveness of the MPA program very highly. On the five-point scale of the Student Exit Survey, 2008 graduates overall impression of the MPA program was rated at 4.4. The high rating is the result of the use of the Student Exit Survey to increase satisfaction levels with the MPA program. Similar responses were obtained from alumni who have been in the field for a longer period of time. Eighty-seven percent of the students from the 2009 five year alumni survey and one hundred percent of public administration respondents in the 2008 survey responded that they were Very Satisfied or Satisfied with the overall quality of their graduate experience. Graduates also indicated a high level of satisfaction with the scheduling of classes, a factor that certainly contributes to their overall rating of program effectiveness. 41

47 Students currently enrolled in the program also rate the effectiveness of our teaching program very high. A review of student opinions of teaching effectiveness (SOTE) reveals very strong ratings for the overall quality of the courses offered. Both the Student Exit Survey and the Current Student Survey ask for opinions on the treatment of minority students by gender and ethnicity. Female and minority students feel adequately treated in the program. Women generally agreed that the program was responsive to their needs, that they were treated fairly, and that the program is free of sexual harassment. Their responses also indicate a desire for more contact with women faculty and women public sector practitioners. Minority student responses affirm the program's responsiveness to the special needs of minority students. They feel treated fairly and without prejudice. Again, several indicated a desire for more contact with minority faculty and practitioners. The faculty is cognizant of this issue. In response, the Department intends to direct more attention to identification of minority and/or female adjunct faculty to contract as instructors or bring into the classroom as guest speakers. In addition to the diversity of the full-time faculty, there are many male Hispanics as well as female and male African-American adjunct instructors. Overall, the MPA full-time faculty is strong and well-rounded, with doctorates from highly regarded universities, and an appropriate mix of practical experiences and competencies. Exit interviews with recent graduates rated faculty quality at 4.7. Respondents felt that faculty exposed them to a variety of points of view, that faculty members were well prepared and knowledgeable, and accessible to students outside of the classroom. Some respondents indicated a desire for more faculty assistance in gaining appropriate employment, which has become more critical with the collapsing job market (3.5 rating). Additional suggestions include the use of more practitioners as guest lecturers and the strengthening of academic and career advising (3.8 rating). In sum, the design and operation of the MPA program at California State University, San Bernardino reflect a strong commitment to high professional standards, creativity in program management, including off-site course offerings and sequencing of classes, and sensitivity to the needs of its students and service area constituents. The program rests on a solid foundation of core courses and electives. Numerous employers indicated that the program is serving the Inland Empire region well. In their opinion, the 42

48 program has had a significant impact on the professionalization of the public service in the Inland Empire. This is a record of achievement reached through the use of evaluative information to enhance the program. The findings of our current program assessment also point to some weaknesses in program operations: a) There was some need expressed for more intensive career advising for promotional purposes as well as for getting started in public service careers. b) Some pre-service students have expressed a desire for more faculty assistance in gaining employment or in making career changes. These issues were included in our strategic planning process and mission, objectives and strategy revisions. B. Program Changes The most significant change has been the development of an on-line MPA curriculum. Faculty members have been fully trained in on-line technology and teaching pedagogy, computer software and attendant skills. These changes are discussed in detail elsewhere in this report (see Standard 4.22C). Other significant changes since the last accreditation have been the elimination of the Water Resources and Native American Tribal Management specializations. The decision to eliminate these specializations was part of the Department s overall strategic planning process. The analysis took into consideration possible options to continue the specializations. Various models were discussed. The result was to keep one Water Resources course, PA 620, Regulatory Structure, Policies and Process in Water Law because there is solid student demand for PA620 (some of which comes from another masters program). However, the plan is to allow all the Native American Tribal Management courses to be discontinued with the loss of Dr. Proudfit, who had been a special hire instigated by President Karnig. Data on the Native American specialization did not support its viability, as there was little demand for it and the expertise to staff the program would need to be entirely external to the Department. The demands for efficient resource allocations must ultimately be paramount in such a highly constrained environment. The expertise of faculty has been captured in the addition of several classes to the curriculum: PA 501, Administrative Leadership by Van Wart, and PA 550, Public 43

49 Management Information Systems by Ni. Because of the Navy-sponsored project, two new courses were added to the curriculum: PA661, Government Systems Acquisitioning and Contracting, and PA671, Defense Acquisition Program Management. With the completion of that project, only PA 661 remains active. Short and Long-Term Changes This upcoming year ( ) the Department will begin a curriculum discussion that may or may not result in a significant change to the core curriculum. The time will be ripe with another new faculty member coming on board. While the Department cannot currently consider a specialization due to the need for faculty to focus on the relatively wide range of current course offerings, the curriculum review process can provide an opportunity to plan for the future, when faculty staffing is once again increased and a specialization may well be plausible. Additional changes will occur from the shock waves of the fiscal crisis. The Department will need to adapt to the more constrained staffing environment with the elimination of one temporary full-time equivalent position. Further, the Department will need to try to maintain its long-term research and service functions in light of temporary reassigned-time reductions. Depending on the severity and length of the fiscal crisis, other temporary adjustments may absorb much of the Department s energy. Yet despite the temporary resource constraints, the Department needs to be active in its outreach to the community. Of particular interest to the Department and Advisory Board are some workshops that would involve the public sector community on topics of wide interest. 44

50 STANDARD 3.0 PROGRAM JURISDICTION Governance and Mission The MPA program was established in 1975, eight years after the founding of California State University, San Bernardino. In 1987, the program received its first accreditation, and the second in Throughout the duration of the MPA program, the Department of Public Administration, a department of the College of Business and Public Administration, has held primary responsibility for the program s administration. The original administrative structure remains the same. Administrative stability has made it possible to deliver a quality graduate education that remains relevant to the educational needs of the program s major stakeholders. Our mission is to provide a graduate education to a diverse student population primarily from the Inland Empire, to enhance their opportunities for employment, career success and leadership in their chosen fields and service to the community. While assessment procedures have been in place since the program s start, it was with the first accreditation that a systematic assessment plan was envisioned and put into place and further refined over the years. The plan has proved its usefulness. Over time, the MPA degree has undergone changes related to assessment information. Most recently, the curriculum was put online in order to satisfy a growing student demand. In general, there is a clear link between the Department s program jurisdiction, administration, type of graduate program it administers, and our mission. The particular mission of the MPA program relates to its: graduate standing, professional management orientation, development of analytical skills, focus on the issues of practical governance, student body, comprised primarily of public administrators, and, efforts to build a diverse student body. The program has its own MPA director, who makes decisions regarding individual student program plans, and generally provides program leadership. 45

51 Public administration faculty come together, both formally and informally, to make decisions regarding policy, schedules, and other matters that concern the MPA program. Faculty recruitment is based on the advice of the public administration faculty. Recruitment, promotion and tenure decisions are made according to the Faculty Senate procedures of the University, College and Department. Special emphasis is given to the wishes of the public administration faculty in recruitment. Search committees are always composed of core faculty members with the input of the entire faculty. This situation is compatible with the mission of the MPA program, since the public administration faculty is an identifiable unit with substantial responsibility and authority for designing and implementing the program. 3.1 Administrative Organization Dr. Albert Karnig is the President and Chief Executive Officer on our campus. The Vice president and Provost for Academic Affairs reports directly to the president, is responsible for the delivery of academic programs, and is to whom the Dean of the College of Business and Public Administrations reports. The organization of the College of Business and Public Administration uses a traditional department structure, with a number of support programs that serve all departments in a matrix format. There are five departments in the College: Accounting and Finance, Information and Decision Sciences, Management, Marketing, and Public Administration. The Department Chair is a faculty member selected from a list of instructional factulty recommended by the department and appointed by the Provost and reports directly to the Dean. The Dean of the College of Business and Public Administration is appointed by the University President following positive recommendation from a Dean's Advisory Selection Committee, comprised of faculty representatives from each of the departments within the College, as well as staff and student representatives, and normally chaired by another academic dean. The Committee prepares the advertisement, screens applicants, selects and interviews the top candidates, and forwards a recommendation to the campus Provost who in turn gives his or her recommendation to the President. 46

52 DEPARTMENT CHAIR M. VAN WART ADMIN. SUPPORT; IRENE GONZALES AND DEBORAH GRIJALVA MPA PROGRAM GRADUATE ADVISOR M. VAN WART M. VAN WART PROFESSOR C. YOUNG, PROFESSOR M. CLARKE, PROFESSOR G. KRESS, PROFESSOR A. NI, ASSISTANT PROFESSOR D. BAKER, ASSISTANT PROFESSOR The department has the following formal and informal agreements with other departments on campus that are reflected in their curriculum: Health Services Administration: Sixteen units of Public Administration coursework are listed as acceptable for elective credit. Master of Environmental Science: PA 621 (Water Law) is a required class. National Security Studies: Strongly recommends taking PA 671, Defense Management and Budgeting, as an elective. Informally, should students from the Public Administration Department want to take courses in, for example, Criminal Justice or Health Services Administration, they will be advised of appropriate electives depending on their career interests. 3.2 Recognized Program The structure of the Department is traditional. Below are the current positions within the Department. 3.3 Program Administration 47

53 Primary administrative responsibility for the program lies with the MPA Advisor. At present, the MPA Graduate Advisor also serves as the Department Chair. The Graduate Advisor manages all facets of MPA program administration. His responsibilities include recruitment, selection, orientation, information dissemination (electronic and published), advisement, student guidance through the program, and maintenance of program records with support from the Administrative Support Coordinator and Administrative Support Assistant. The Graduate Advisor and the faculty, working as a committee-of-the-whole, decide upon all program admission requirements, degree requirements, and curriculum changes. A committee-of-the-whole is responsible for adjunct faculty reviews and decisions on qualifications to teach in the MPA program. Working as a committee-of-the-whole, the faculty selects questions for the comprehensive examination from a list constructed by a sub-committee. Examinations are read and evaluated by the entire faculty in a double-blind review process. 3.4 Scope of Influence The Department Chair and the public administration faculty exercise initiative and substantial determining influence in decisions regarding: A. General program policy and planning Subject to the approval processes of the Department, College, and University, public administration faculty take the initiative in modifying the MPA program, and formulate any proposals to be considered by the appropriate review bodies. B. Degree requirements From the Department, proposals go to the College Curriculum Committee and the University Curriculum Council, the final body giving approval. At the Department level, the Chair certifies degree candidates by reviewing and signing off on program plans, comprehensive examination results, and research projects. C. New courses and curriculum changes 48

54 The faculty carries out the primary responsibility for developing academic programs, maintaining quality, and promoting an institutional character. Academic programs are developed by curriculum committees composed of elected faculty at the College and University levels. The Department initiates curriculum changes as a committee-of-the-whole and meets regularly to act on faculty proposals and review the curriculum. If approved, an item(s) is forwarded to the College Graduate Committee before going to the College Curriculum Committee. The College Curriculum Committee is composed of elected faculty members -- one from each department. The College Committee reviews and approves/disapproves all course and program proposals from the departments and periodically reviews the curriculum and programs. Approved items are forwarded to the College Dean for approval/disapproval. Any item disapproved at any level is referred back to the person/group submitting the proposal. If approved by the Dean, the item is forwarded to the University Graduate Committee before going to the University Curriculum Committee. The final step in the process is approval/disapproval by the Faculty Senate. For the establishment of new courses, the Department committee must approve a new course, then submit it to the College and University Curriculum Committees, where members serve as representatives on all decision making levels. Degree requirements that are not a part of the graduate school s requirements are the responsibility of the Department committee. D. Admissions After the University has admitted a student, the MPA Graduate Advisor makes admittance decisions on behalf of the department faculty, using authorized standards. Students who meet all entrance requirements, except course prerequisites, may be admitted to the MPA program in a "conditionally classified" status until those prerequisite deficiencies are removed, at which time they are advanced to "classified graduate standing" and advanced to candidacy in the MPA program. The requirements for part-time students are the same as those for full-time students seeking admission to classified graduate status in the MPA program. There are 49

55 no special admission policies for non-traditional backgrounds, minorities, transfers, etc. Further, all admission policies, standards, and practices for part-time and full-time graduate students at the Palm Desert Center off-campus program are identical to those for admission to the San Bernardino program. E. Certification of degree candidates Students who have achieved classified status may be advanced to candidacy for graduation by filing a Graduate Program Plan with the MPA Graduate Office. Students are eligible to apply for candidacy after taking 20 graduate level units in the MPA Program and fulfilling all prerequisites. When a student is advanced to candidacy, they must file a Graduation Requirement Check with the office of Admissions and Records. F. Course scheduling and teaching assignments Course schedules and faculty teaching assignments are determined by the dean after consultation with the department chair and/or the individual faculty member as as required by the Collective Bargaining Agreement (CBA); in practice, scheduling of assignments is generally handled by the Department Chairperson in consultation with individual faculty members. Others may determine non-teaching assignments. For example, the Dean determines reassigned time awards for publications and may award such time for research purposes to be scheduled as appropriate by the chair. All awarded reassigned time, including from grants and contracts, must be coordinated with the Department Chair and is dependant on the needs of the program (See Appendix x). G. Use of financial and other resources The Dean prepares the College budget for consultation with the administrative team that includes the Chair of Public Administration. The Department Chair formulates the Department s own budget requests which are presented to the Dean. While most funds are allocated on a formula basis, the Department Chair has some flexibility in spending. The Department Secretary functions as bookkeeper and accounts payable clerk under the direction of the Chair. 50

56 H. Appointment, promotion, and tenuring of program faculty Appointments. The Graduate Advisor of the MPA Program is officially appointed by the Chairperson of the Department in consultation with the Dean of the College and all tenured faculty members in the Department. In recent years, the positions of Chair and Graduate Advisor have been assigned in tandem. The faculty appointment criteria are administered by the Provost and implemented by Departmental Recruiting Committees, Department Chairs, and College Deans. The process of selecting and hiring new faculty is structured by both the Agreement between the California Faculty Association (CFA) and the CSU system, and by Faculty Senate procedures. Section of the CBA Agreement requires each department to elect a recruitment committee of tenured faculty members for selection purposes, including the department chair. Recommendations for hiring must originate at the department level. Recruitment. It is the policy of the University to recruit a faculty that is balanced and diverse in terms of gender and ethnicity. Therefore, at the beginning of the recruiting process, the Department of Public Administration, in consultation with the Dean of the College of Business and Public Administration, determines what kinds of individuals to search for in order to maintain or bring about balance and diversity among the faculty. The Department Chair is expected to monitor the recruiting process, provide clerical support and participate in all aspects of reviewing applicant files. Department Recruiting Committee members (all Department faculty) participate in all meetings and in all interviews when possible. Selection and Hiring. In selecting faculty, the Department Appointment, Retention, Tenure and Promotion document states that each Search Committee is elected from among the faculty, as required by the CBA. In hiring new full time faculty, the Public Administration faculty is charged with communicating their needs through the MPA Director to the full Department. Promotion and Tenure. Decisions and nucleus faculty participation regarding promotion and tenure are made in accordance with the University's "Procedures and Criteria for Performance Review and Periodic Evaluation" Revisions II, April The Department faculty elects a departmental evaluation committee to review non-tenured faculty and those eligible for promotion. The Department Evaluation Committee is 51

57 composed of three tenured, elected faculty, two of whom must be at the rank of Professor The third must be a Professor or an Associate Professor not presently being considered for promotion. The committee then elects a chair who must hold the rank of Professor. As part of the evaluation process, a faculty member will be visited in class by two tenured departmental colleagues of a rank equal to, or higher than, that of the faculty member to be visited. If necessary, visitors may come from related academic disciplines. After the visit, each visiting colleague completes the Classroom Visitation Report form and presents it to the faculty member visited. At this time, a post-visit discussion takes place concerning the contents of the report and suggestions for improvement. The faculty member and visiting colleague both sign the Classroom Visitation Report. The report of the Evaluation Committee then goes to the College, and enters the faculty member s file. Retention. Junior, non-tenured faculty members are reviewed each year for retention. The College review originates in a Department committee and subsequently goes to a College committee, and then on to the College Dean, Provost and Vice President for Academic Affairs, and, finally, the President. 52

58 STANDARD 4.0 CURRICULUM 4.1 Purpose of Curriculum The MPA curriculum is designed to correlate with the program s mission to provide exemplary and affordable face-to-face and online learning opportunities to students with diverse backgrounds In addition, the curriculum draws upon our commitment to help our students enhance their employment opportunities, career advancement, leadership roles, ethical practices, commitment to democratic values, and service to their communities. To these ends, the program consists of core common curriculum components, electives, and a comprehensive examination. The eight-course core covers basic areas of public administration, including administrative processes (budgeting, finance, personnel, and administrative regulation), disciplinary perspectives (public administration in theory and practice), and reasoning and analytical skills (public policy analysis, and research methods in administration). In addition, since the program is strongly oriented toward practicing public administrators, a core course on management of public organizations is included. This course enables students to integrate management knowledge and skills in the context of a specific organizational environment. Electives have also been designed to follow the same direction, but students are free to choose electives without the restriction of a concentration. The program allows students to choose between completion of a research project, equivalent to four course units, or course work and a comprehensive examination, equivalent to four units of graduate credit. The Department s experience has been that the majority of students take the course work option, and the preparation (for the comprehensive examination) is valuable as a capstone experience in-and-of-itself. However, certain students are in the position, and have the desire, to complete a substantial research project, and the program allows them the flexibility to do so. There are no deviations from Standard 4.0 based on the program s mission. 4.2 Curriculum Components A. Background Information The information for this section is taken from the University Bulletin of Courses. 1. Credit System The University and the Masters in Public Administration program use the quarter system. 53

59 2. Length of Term The length of term from the first class meeting to final examination is eleven (11) weeks. 3. Number of Terms There are three (3) terms in the academic year, fall, winter and spring. A small roster of courses is offered during the summer quarter; thus, students have the opportunity to attend classes year round. 4. Full-Time Status All MPA classes are four units. The minimum to be considered full-time for financial aid is 8 units; however, most full time students take three or four classes. To enroll in more than 16 units in any one quarter, a student must have approval of the graduate advisor of his/her graduate program. In addition, the graduate advisor can, and does, limit registration for students whose academic performance indicates that more than eight units may be too much. 5. Time Limitation The program must be completed within a seven-year period. No more than seven years may elapse between registration for the earliest course listed on the program and completion of all requirements for the degree. While students do occasionally get permission for an additional quarter, several students each year request extensions after eight or more years and are denied. 6. Class Contact Hours The contact hours per week for each four (4) unit class for the MPA is four hours, with a twenty minute break, with the exception of PA 548 Management of Computers in the Public Sector. PA 546 is divided into 3 class hours and 2 laboratory hours. Final examinations during the eleventh week are generally 2 hours long. This equates to 42 hours per quarter. 7. Numbering System The course number system for lower division are , upper division , upper division and graduate , and exclusively graduate courses B. Course Distribution The MPA program offers two options toward completion of degree requirements; the comprehensive examination track and the graduate research project track. The following table reflects the distribution of course credits in fulfillment of MPA degree requirements. The source 54

60 of the information provided for this section (B. Course Distribution) is from the University Bulletin of Courses. Distribution of Course Credits Course Level Lower Division 100 or 200- level Upper Division 300 or 400- level Upper Division & Graduate 500 or 600-level Exclusively Graduate 600- level Common Curriculum Components Required Required Prerequisites Courses Additional Curriculum Components 4 4* 4 4* Total *A prerequisite course is not counted towards completion of the degree. Total 1 Students who wish to take up to two 300- or 400-level courses must petition the College s director of graduate studies for approval in advance. For each course approved, they will be required to enroll simultaneously in a two-unit independent study course directly related to the course selected. These independent study units cannot be applied to any graduate degree unit requirements. (In practice, this option has not been used in many years.) 2 All students may take 500-level courses without getting permission. Only conditionally classified and classified students (or those who have the consent of the MPA Director) may enroll in 600 -level courses. All students taking the comprehensive examination need at least one 600-level elective. Students may not include more than 12 quarter units in transfer credit from other colleges. California State University, San Bernardino will not consider for transfer credit course work from any institution that will not accept that work in its own advanced degree program. Students pursuing the research track may take no more than twelve units at the Department of Public Administration 500 or upper division course level, of the sixteen unit elective requirement, because students earn 4 units of elective credit when they are enrolled in PA 695 Graduate Research Project. 3 The comprehensive examination requirement is taken on a CR/NR basis. 55

61 The research project track requires that 4 units of credit must be granted to the research project and students may choose 12 units of electives from any Department of Public Administration 500 or 600 level courses. The common curriculum component does not differ depending upon the track chosen; however, all required Public Administration core courses are exclusively graduate level courses. The Public Administration electives may be upper division/graduate or exclusively graduate level courses. C. Capable Professionals The Master of Public Administration is a broad program designed to prepare students for a career in administration in the public sector. The major objective is to provide the students with a generalist perspective of public administration. The courses are designed to provide tools for decision-making in an administrative position (budget, finance, personnel, and local government management). The courses provide an understanding of the total public administrative system (research methods, theory and practice, and administrative regulations). Capacity for working and understanding within the public environment is gained through management of public organizations and public policy analysis. The specific design of the curriculum components provides the capable professional with tools for creative analysis and communication, and the opportunities to practice and hone their decision-making and management skills. Finally, involving students in course projects, faculty research, and public service activities within the parameters of the course provides additional depth and breath to the curriculum. D. Assessment and Guiding Performance The common and additional curriculum components are assessed as to their quality and consistency with the stated mission of the program. This occurs through the development and implementation of a comprehensive evaluation design for total program assessment using feedback from rigorous student evaluations of the program and instructor, advisory board and employer surveys (See Appendices for copies of these documents). The tools employed include MPA student exit surveys, regular program review by the Public Administration Department Program Advisory Board, employer surveys, student evaluations of teacher effectiveness (SOTE s), and periodic review of non-tenured, tenured faculty, the chair and adjunct faculty. Core (except full professors) and adjunct faculty classroom visits occur on a periodic basis. 56

62 The assessments, suggestions, and recommendations are incorporated into the strategic planning, strengths, weaknesses, threats and opportunities (SWOT) analysis processes. Evaluation information also is used to select adjunct faculty. Examples of how these results were incorporated into the program and curriculum include the development of increasing state-of-theart competencies in budgeting, E-government, policy formation, policy analysis, program evaluation, strategic management, leadership, values analysis, and negotiating skills. Student and Advisory Board evaluations also have contributed to the Department s development of enhanced use of online teaching elements. The program has developed specific cohorts at sites convenient to the students working/living locations, yet in close proximity to the main campus. E. Graduate Courses The information provided for sections E and F is taken from the University Bulletin of Courses. The required 32 units of graduate level core courses fulfill the common curriculum components. Six hundred level elective courses are offered only to graduate students in order to fulfill the additional curriculum elective requirements. In addition to 600-level courses, students may take three 500-level courses to fulfill 12 of the 16 unit elective requirement. The minimum number of credits that can be earned in a core or elective course is 4 units. F. Required Prerequisites Following is a list all prerequisite subjects and the undergraduate credits for each, which are required of each student, but for which no graduate credit is given. Required Prerequisite Courses Management 306: Writing related to business and public administration including Expository documented research reports, summaries and analytical papers. Writing for Revision and rewriting will be required. Course fulfills the CSU Administration graduation requirement in writing proficiency. Prerequisites: English 101 and a minimum of 90-quarter (60 semester) units of college credit. Students may elect to take an examination instead of the course. Also, students who have graduated from another California State University campus may have taken an upper division writing course or a writing examination prior to graduation. This may 57

63 And Administration 575: Internship in Administration And Management Science 210: Applied Business Statistics Or Math 305Statistics: Hypothesis Testing and Estimation count toward satisfying the writing requirement. Supervised work and study in private or public organizations. May be repeated once for credit, and used toward satisfying program elective requirements. Graded credit/no credit. (May be waived dependent upon related work experience). Prerequisites: consent of instructor and College. An introduction to statistical reasoning and application of primary statistical techniques used in solving managerial problems. Covers both descriptive and inferential statistics. Credit may not be received for more than one of the following courses: Management Science 210, Psychology 210, or Social Sciences 215. After a brief introduction to descriptive statistics, course emphasizes hypothesis testing and estimation, using packaged computer programs. May not be taken for credit by students who have received credit for Management Science 210, Mathematics 262 or 350, Psychology 210, or Social Science 215. Prerequisite: one term of college mathematics. There is a three course prerequisite for the program: MGMT 306 Expository Writing for Administration; MGMT SCI 210 Applied Business Statistics; and PA 575 Internship in Public Administration (waived for in-service students). The MGMT 306 prerequisite is a University requirement. It hones the writing skills of graduate students through intensive instruction in syntax, grammar, punctuation, organization, and writing style. Since most graduate seminars require written projects, the MGMT 306 course provides skills that lead to higher quality course papers. Substitutions from other universities and waivers based on a challenge (the submission of an original research paper) or special conditions (another masters degree) are only authorized by the Graduate Dean. A little less than half of the students are required to take the writing course as a part of their program of study. The statistics prerequisite courses, Math 305 or MGMT SCI 210 are useful in improving student interpretation of the empirical literature in public administration and related fields, especially works presenting evaluation results, studies in public finance and budgeting, and research methodologies. Statistical methods taught in these prerequisite courses are also applicable to specific courses in the MPA curriculum. These classes are PA 603 Research Methods in Public Administration; PA 630 Program Evaluation; PA 615 Public Financial 58

64 Management; and PA 663 Public Budgeting and Finance. Student performance in these classes improves through taking the statistics classes. In reality, no waivers are allowed for this prerequisite. Only those who have documented statistics coursework on their transcript are waived from the requirement, but this is in reality a substitution when that coursework is from another institution. The final prerequisite, PA 575 Internship in Public Administration is discussed fully in Standard 4.4. For pre-service students, this experience exposes them to the real world of public agency management and services delivery. This knowledge and experience provide a solid foundation for better understanding of course material, and allow pre-service students to compete on a more level basis with their in-service classmates. Because the majority of our students are already working in the public sector, waivers in this category are frequent. Also, students may document an appropriate internship at the undergraduate level to meet this requirement Common Curriculum Components A. Required Graduate Courses PA 603 Research Methods in Administration 4 units PA 611 Public Administration Theory and Practice 4 units PA 615 Public Financial Management 4 units PA 662 Human Resource Administration 4 units in the Public Sector PA 663 Public Budgeting and Finance 4 units PA 664 Management of Public Organizations 4 units PA 672 Administrative Regulation 4 units PA 680 Public Policy Analysis 4 units Core course substitutions are very rare. The one exception was for students who participated in the Water Resource Management Program. In those cases, students were required to substitute PA 620 (Regulatory Structure, Policies, and Process in Water Law) for PA 672 (Administrative Regulation) and PA 621 (Water Resources Agency Management) for PA 664 (Management of Public Organizations). 59

65 B. Ethical Action The curriculum enhances student values, knowledge, and skills to act ethically and effectively through courses that are designed to improve and support management decision-making skills, analytic capabilities, sensitivity to political relations, understanding of the dynamics of diversity in an organization, appreciation of the role of the public sector in a democracy, and ethical behavior. The core ethical competencies that are addressed relate to the subject areas of management of public service organizations, techniques of analysis, and responsibilities of leadership, public policy formation and organizational environment. Students examine issues involving the exercise of discretionary authority to promote public interest. The issues include discrimination and harassment, public's right to know the public's business, the importance of involving citizens in policy decision-making and making decisions that may not be popular regarding budget, finance, personnel, and management concepts. Simulations, case studies or role playing is used to assist students to gain respect for the Constitution and the law, learn to demonstrate personal integrity by taking responsibility for their own errors and conduct official acts without partisanship. The courses are also designed to teach how to promote ethical organizations through open communication, creativity, accountability, dedication and professional development. C. Curriculum Coverage The design strategy used to implement the MPA program curriculum is geared to build upon skills already developed and provide state of the art knowledge and competencies to a diverse student population. This approach provides the student body with the tools necessary to advance their employment opportunities, careers, leadership roles, and service to the community while engaging in ethical behavior. In order to understand the level of effectiveness of this strategy, the Department uses an evaluation system to gather external feedback from the region s employment sector, Department Advisory Board, and student evaluations. Recommendations from this community are incorporated into the curriculum strategic planning processes and into the program objectives and strategies. As a result, the generalist approach of the curriculum is based on disciplinary objectives and the feedback of the MPA program s major constituencies. 60

66 The knowledge and competencies related to management of public service organizations is presented in classes that are geared specifically to focus on the commonly recognized components of this area: i.e., PA662 Human Resources Management in the Public Sector, PA615 Public Financial Management, PA663 Public Budgeting and Finance, and PA550 Public Management Information Systems. Previously, PA548 Management of Computers in the Public Sector was the course focused on computerized management systems and overall literacy. In the past two years, the Department has updated the subject matter of this course and established it as PA550 Public Sector Management of Information Systems. Contemporary technologies are presented via modules on E-government (web-mediated communication and information dissemination), and the use of various software in project planning, graphics, network diagramming, and lifecycle planning. These topics are presented in PA615 Public Financial Management, PA516, Public and Nonprofit Project Management, PA617 Management of Local Economic Development, and PA663 Public Budgeting and Finance. Similarly, the application of quantitative and qualitative analytic techniques is presented in courses that specifically focus on these topics: PA603 Research Methods in Administration, PA680 Public Policy analysis, and PA630 Program Evaluation. Other courses such as PA663 Public Budget and Finance and PA 615 Public Financial Management also include quantitative analytical techniques. Additionally, these courses present a variety of decision-making and problem solving models. Subject matter relevant to public sector political, legal, economic and social institutions and organizational and management concepts are presented in PA611 Public Administration Theory and Practice, PA619 Diversity and Public Organizations, PA628 Intergovernmental Administration, PA664 Management of Public Organizations, and PA672 Administrative Regulation. Course descriptions provide for common, or additional, curriculum components, and prerequisites reflect the inclusion of core competencies on a course-by-course basis Additional Curriculum Components A. Elective Design 61

67 The MPA program offers two options toward completion of degree requirements; the comprehensive examination track and the graduate research project track. The elective design of the curriculum, with respect to additional curriculum components, provides the student with 22 potential selections. Students on the comprehensive examination track option are required to select 16 units, with no more than twelve of these units at the 500 or upper division course level. The research project track requires that 4 units of credit must be granted to the research project. Students may choose 12 units of elective credit from any of the 500 or 600 level courses. The common curriculum component does not differ depending upon the track chosen. B. Elective Courses for Broad Advanced Training Course objectives provide students an opportunity to enhance the breadth and depth their knowledge and competencies in specific areas of public management beyond that gained through the common curriculum. This reinforces our mission to help our students enhance their employment opportunities, career advancement, leadership roles, ethical practices, commitment to democratic values, and service to their communities. The number of allowable undergraduate credits a graduate student may take is twelve (12). Elective Courses List PA 501 Administrative Leadership PA 514 Management of Private Non-Profit Organizations PA 516 Public and Non-Profit Project Management PA 521 Principles of Planning PA 550 Public Management Information Systems PA 557 Public Sector Labor Relations PA 564 Local Public Administration PA 568 Accounting for Government and Not-for-Profit Organizations PA 575 Internship in Public Administration PA 590 Seminar in Public Administration PA 595 Independent Study PA 612 Productivity Measurement in the Public Sector PA 616 Comparative Public Administration PA 617 Management of Local Economic Development 62

68 PA 618 Local Development Finance PA 619 Diversity and Public Organizations PA 628 Intergovernmental Administration PA 630 Program Evaluation PA 670 Emergency Management PA 671 Defense Budgeting and Management PA 690 Advanced Topics in Public Administration PA 695 Graduate Research Project C. Elective Courses for Specializations The MPA program does not offer specializations. Students customize their electives with the guidance of the Graduate Advisor General Competencies Curriculum components are designed to develop a student s general competencies in line with the MPA program s mission. The curriculum is designed and implemented to provide state of the art knowledge, skills and abilities to a diverse student population primarily from the Inland Empire. Students receive a graduate education outfitted with the necessary tools to advance their employment opportunities, careers, and leadership roles, and a committed enrichment to democratic values, ethical practices and service to their communities. In preparing syllabi and other course materials, the focus is on providing a graduate education that is skills-based, with appropriate presentation of theory, concepts, analytic techniques, and state-of-the-art technology. Competencies for each course are specified in course syllabi and vary according to subject. In general, these may be identified as follows: Understanding of state-of-the art theory, concepts and techniques, as well as their historical context Ability to apply concepts in a systematic fashion in practical, structured research projects, case studies, simulations and discussion Enhanced proficiency in working with task-based small groups, consensus building, negotiation, conflict resolution, group discussion and group or individual presentations 63

69 Ability to draw logical inferences based upon analysis of local contemporary administrative/management structures, processes and environments and other jurisdictions in the United States and internationally Ability to identify and evaluate problems and construct innovative, as well as practical, alternatives for their resolution Ability to integrate multiple factors into a logically consistent model Develop and sharpen quantitative and qualitative analytic skills with emphasis on those common to public management Sharpen oral and written public management communication skills Become knowledgeable of state-of-the-art technology for the public sector Become proficient in use of contemporary technology for public management Gain insight into public administration issues in the broader context of economic, political, legal and social institutions and processes. Deepen understanding of political and societal democratic values and goals Ability to identify and resolve ethical dilemmas common to the public sector The final curriculum element is either a comprehensive examination or a research project. In either case, the student is expected to competently bring together what they have learned in the various courses and apply it to specific set of questions. In this exercise, the student must demonstrate competency and confidence in analyzing major questions in contemporary public administration and relate elements of the arguments and evidence regarding these questions. The expectation is that student will express ideas and explain concepts by writing in a clear, grammatical, and effective manner, and, where applicable, express the possibilities of flexible adaptation and creativity in the application of what they have learned by exercising their imagination and breadth in thinking. 4.3 Minimum Degree Requirements A. Degree Credit Hours The minimum number of degree credit units is 48-quarter units of acceptable graduate level work (with a grade-point average of 3.0), with at least 36 units completed at this university. The program must be completed within a seven-year period. No more than seven years may elapse 64

70 between the time of registration for the earliest course listed on the graduate program and the completion of all requirements for the degree. There is no reduction of credit, based upon undergraduate preparation, significant professional experience or internship. B. Degree Length It is possible for a well qualified, full-time student to complete the program in 18 months. Most full-time students take two years. Part-time students typically take 2 and one half to three years to complete their degrees. C. Concluding Requirements The graduate research project is defined as an advanced field study in public administration, including both literary and empirical research. This requirement is met by a student who enrolls in PA 695 during the last quarter of work in the program, and upon completion of the project to the satisfaction of the student s advisor. Students will earn 4 units of credit upon completion of the graduate research project. The Comprehensive Examination is a set of questions based on materials covered in the common curriculum courses. Students with a GPA below 3.0 ( B ) are not permitted to take the examination. A no credit (NC) designation is entered on the students transcript if he/she fails the examination. Students who successfully complete the comprehensive exam receive a CR on their transcript. Students are expected to write their answers in essay form, demonstrating an appropriate descriptive, persuasive, analytic, or argumentative style. The examination is designed to demonstrate high-quality writing and conceptual skills in a proctored environment. All core faculty members grade the examination; two, independent graders read the examinations, blind of student identification or the other s grading. When not unanimous, a third reviewer is solicited. The examination committee decides a consensus grade and one of the following grades is assigned to the student: High Pass (HP), a superior performance on the examination; Pass (P), an acceptable performance on the examination; No Pass (NP), a failing performance on the examination. About 10 to 15% fail their first comprehensive examination. D. Course Formats 65

71 There are three formats for the MPA common courses: face-to-face, partially online, and fully online. All MPA required courses are presently 4 units. The face-to-face course is scheduled to meet for 4 academic hours, one evening per week. Online classes have weekly assignments in lieu of class time; in some cases they may have a virtual class night, used for synchronous discussions or concurrent timed examinations. E. Student Transcript Analysis Following is a list of program graduates for the twelve months preceding this report. Since the total number of graduates for this period does not exceed 40, all students graduating during the academic year totaling 23 are used for the student transcript analysis. List Of Program Graduates For Student Transcript Analysis 8582 Kassick, Erick J 5041 Christensen, Diane 9474 Kimble, Sheilah 8823 Conley, Cynthia 5990 McBride, Gary Dean 0882 Escobedo, Christopher 5288 Uhley, Jason Edward 9718 Ezinwa, Chinonyerem 9792 Vedrode, Shelley 2119 Green, Sophia 6175 Yakutis, Eva 8743 Hawel, Laurie 1733 Amaya, Sylvia 4826 Ibarra, Kristen 4784 Buncheck, Stacy 3200 Martinez, Tome 0858 Gomez, Michael 2508 Monroe, Clay 2331 Adams, Summer 0819 Vallangca, Jivi 4157 Carreon, Luis 6367 Wilken, Janet 4541 Carroll, Susan 66

72 67