INTERIM REPORT TO CEPH ACCREDITATION REVIEW

Transcription

1 INTERIM REPORT TO CEPH ACCREDITATION REVIEW Criterion V.D. This criterion was partially met with the following concerns noted: A. Systems needed to monitor student achievement of stated learning objectives B. Strategies needed to improve student graduation rates A number of steps have been taken to address both criticisms. Policies have been adopted and administrative procedures have been instituted to better monitor student achievement and progress toward completion of the degree program. The specific steps to fully meet CEPH criterion V.D. are listed below. A. Monitoring student achievement of stated learning objectives Review and revision of MPH competencies --The Review for Accreditation noted significant variability across faculty regarding quality assurance. This was viewed in part as an outcome of vaguely written competencies that could lead to wide interpretation. To address this issue and to respond to the current national reform led by ASPH, the competencies for the MPH program have been modified and implemented. The process was led by the UTSPH Academic Council and involved the SPH faculty in each Division. This work resulted in a new and expanded list of competencies and subcompetencies that are more specific and less open to individual interpretation. After a lengthy process, this set of MPH competencies and the evaluation form (see Appendix 1, MPH Competencies and Form ) were approved and effective as of the fall semester, The new MPH competencies and form are required of all students matriculating in the fall semester, 2006 or thereafter and is available for use by any of the more advanced students in the MPH program. The new form, which measures student progress by not only the set of competencies but by the degree to which each is met, was used by students and faculty at the fall semester 2006 evaluation committee meetings. Core course waiver and oversight -- At the time of the CEPH site visit, completion of core public health requirements was monitored by the student s advisory committee. Documentation of a waiver for one or more requirements consisted of a memorandum generated by the student s advisor and placed in the student record. A new system has been approved and implemented (Appendix 2, MPH Students and the Core Requirement ) and consists of a policy statement and a waiver form documenting how each of the MPH competencies has been met. This form is completed by the student and advisor and is signed by the academic advisor, the Director of the Division in which the course is offered, and the Associate Dean for Academic Affairs. There have been 8 waivers for a core course in the last three years. Letter alerting students and committee of questionable grades -- Previous to the CEPH accreditation review, student progress was monitored solely by the student s advisory committee. Since fall 2002, the Office of Student Affairs has monitored student grades and alerts the student and his/her advisory committee of any C s or F s so that academic difficulties may be recognized and addressed (Table 1). The student is directed to meet with his/her advisory committee at the earliest opportunity to identify the source of the problem and to devise a plan for remediation. If the student is placed on academic probation by the advisory committee, the meeting is summarized in a form and placed in the student record. The policy document and form are included in Appendix 2. Table 1. The number of letters sent to students experiencing academic difficulties over the past 3 academic years. Letter for: C C s F s

2 Notification of doctoral students at 3 years following admission to candidacy -- The UTSPH Catalog has long included the following language; however, before fall 2002 this expectation was not acted upon. Any student who has been admitted to candidacy for a doctoral Dr.P.H. degree (i.e., following successful completion of the qualifying examination) is expected to complete the degree within three years from the date of admission to candidacy. Otherwise, the dissertation committee will review the case at the end of the three-year period and annually thereafter, and will consider such recommendations as (1) modifications necessary in the research protocol, analyses, or interpretations, (2) additional coursework, or (3) dismissal. Recommendations of the dissertation committee are forwarded to the Dean, who is the final authority on all academic matters. Currently, doctoral student records are monitored by the Office of Student Affairs to identify students who have not graduated within three years of admission to candidacy. A letter is sent to each student and copied to the student s advisor. If the student intends to complete the degree program, a memorandum including a plan and timetable for completion is prepared and signed by the student and advisor. The memorandum is sent to the Associate Dean for Academic Affairs who reviews the plan and, if approved, extends the degree program by one year (Table 2). A second extension, if well justified, may be approved. An administrative hold is placed on enrollment for students who do not have an approved extension. Students who do not finish the degree program at the end of the extension period are dismissed from the school. Table 2. The number of approved extensions for students taking longer than three years to graduate after admission to candidacy. Data are for the indicated academic years DrPH 1 st Ext nd Ext PhD 1 st Ext nd Ext Auditing of student progress and degree certification -- After receipt of the CEPH Review for Accreditation, administrative oversight was instituted for documentation of student progress and degree certification, both previously carried out by the student s advisory committee. The administrative function is carried out by the Office of Student Affairs under the auspices of the Office of Academic Affairs. For doctoral students, the student record is audited at the time of the qualifying examination. The academic advisor and chair of the student s qualifying committee submits a signed memorandum to the Office of Academic Affairs attesting to the successful completion of the qualifying examination. In addition, the Office of Student Affairs verifies that course and credit hour requirements have been met before the student is admitted to candidacy. The completion of graduation requirements is documented by the Office of Student Affairs and the student s advisory committee. The Office of Student Affairs verifies that all degree requirements have been completed and documented in the student record. Each member of the student s advisory committee signs a form certifying that the student has met the requirements for the degree (Appendix 3-4).

3 Practicum -- A student-oriented website was launched during the summer of This site serves as a guide for students, community preceptors and faculty sponsors. It includes specific and general practica information as well as contact information, announcements and links to external resources. An online data base of current practica contains 150 sites. Students are able to search for specific opportunities by location, organization or discipline. Beginning with the Fall 2005 semester, SurveyMoneky has been used to electronically collect practicum-related evaluation data from students and community preceptors. Information collected includes: types of organizations where students were placed; topics; final product; time commitment; assessment of students and preceptors satisfaction with the experience; and, student achievement of specific practice-related competencies. The average annual response rates for students and community preceptors for AY were 82% and 76% respectively. Each semester an executive summary of the evaluation is disseminated to community preceptors and posted on the Office of Public Health Practice website ( ). For the academic year, we completed one school-wide, five division and four regional campus evaluation reports. These reports were disseminated to the appropriate administrator and practice coordinator and posted on the Office of Public Health Practice home page. A practicum waiver form and procedures (Appendix 5) were implemented during 2006 and are posted on the Office of Public Health Practice website ( ). Three waivers were approved during All documentation and administrative monitoring of waivers are managed by the Associate Dean of Public Health Practice. B. Strategies needed to improve student graduation rates The CEPH Review for Accreditation document expressed concerns for the overall and degreespecific graduation rates. While many factors may influence the graduation rate, some stand out in importance. These factors include: the high percent of part time students; the need for public health training not leading to a degree; the need for course scheduling and availability; and the plan to offer options for the culminating experience. The school was in the process of addressing several of these factors before and during the time of the site visit review. Steps taken to address each of these factors are given, as well as new policies and procedures designed to encourage all students to maintain appropriate progress toward degree completion. The effect of these policy and procedural changes are shown below. Number of graduates and duration of degree programs Since master s and doctoral programs have time limits of 5 and 7 years, respectively, sufficient time has not passed to fully assess the impact of changes in policies and procedures on graduation rates. However, improvement is evidenced by the number of graduates, which increased by approximately 43% between FY2003 and FY2004 and has remained at the higher level thereafter (Figure 1). This increase is attributed to several policy and administrative changes described in the above sections. It should be noted that while the number of graduates increased by more than 40%, enrollment from FY2001 to FY2006 increased approximately 4%. Examined in another way, the number of graduates compared to the total student body has increased from approximately 14.5% before 2004 to approximately 21% after Further, the duration of degree programs has decreased (Table 3). The median time in the MPH and MS degree programs has dropped from 3.0 and 3.3 to 2.7 and 2.3, respectively, in the last three years.

4 Figure 1. Number of graduates (all degree programs) for FY2000-FY Graduates Table 3. Duration of degree programs for master s students graduating academic years Values represent the number of years in the degree program. Graduating MPH MS in: N= Median N= Median FY FY FY Part-time students -- A large majority of UTSPH students obtain their education while holding full-time jobs or having other major responsibilities (Table 4). However, in the past four years, a steady trend toward a lower percentage of part-time students has been seen. This trend should positively impact the graduation rate since full-time students take more courses per semester and progress at a faster pace than part-time students. Table 4. Students (all degree programs) enrolled as full or part-time. Fall semester Total enrolled # Full time # Part time % Part time Course availability -- Part-time students often take only one or two classes per semester and in the past have been limited by the lack of adequate class availability at times that are convenient to full-time employees. Significant changes have been made in course availability, scheduling, and the method of delivery. The school has made an effort to increase the number of courses offered in the late afternoon/evening (after 4:00 p.m.). The following data show the number of courses offered in each semester of the last three years in Houston and at the regional campuses (Table 5).

5 Table 5. The number of courses offered after 4:00 p.m. Values indicate the total number of courses offered per semester over a three-year period. Data do not include online courses. Academic year Fall Spring Summer A schedule of all core courses is projected for a two-year period to give students and advisors better information with which to plan their academic program. This information is posted on the school s website ( ) for easy access and use. Further, the UTSPH Academic Council set a goal to have a core course from each discipline available to students at every campus, every semester. These courses are delivered online, by ITV, or by faculty on location. The data below show that this goal was reached in the 2006 academic year (Table 6). That is, core courses were available to all students in all semesters and locations, if they wished to enroll. Table 6. Core course delivery per Division and Regional Campus for each semester in academic years Symbols are defined as: + = course offered, students enrolled; + (shaded) = course offered, none enrolled; dark shaded = no course offered. Division & Campus* Fa Sp Su Fa Sp Su Fa Sp Su Houston Biostatistics Env & Occ Hlth Sci Epi & Dis Control Hlth Prom & Behav Sci Mgmt, Policy & Com Hlth Brownsville Biostatistics Env & Occ Hlth Sci Epi & Dis Control Hlth Prom & Behav Sci Mgmt, Policy & Com Hlth Dallas Biostatistics Env & Occ Hlth Sci Epi & Dis Control Hlth Prom & Behav Sci Mgmt, Policy & Com Hlth El Paso Biostatistics Env & Occ Hlth Sci Epi & Dis Control Hlth Prom & Behav Sci Mgmt, Policy & Com Hlth San Antonio Biostatistics Env & Occ Hlth Sci Epi & Dis Control Hlth Prom & Behav Sci Mgmt, Policy & Com Hlth *The Austin Regional Campus is very recent addition and was not included in the analysis.

6 The school has developed online courses to further address the issue of course availability. The first online course was offered in the spring 2006 semester and four more were offered in the summer, completing the complement of core courses for each of the five public health disciplines. Courses were typically capped at students unless the instructor was willing and able to accommodate additional students. The early goal was to offer each online course at least one time per year. However, from the first, the courses filled very quickly (with an additional waiting list of students) each time they were offered. The immediate success of these courses has led to a school-wide goal of offering each online core course in each semester, including the summer. The following information documents the progress to date in this effort (Table 7). Table 7. Online core course offerings and enrollments Course # Core Course Title Enrollment Spring 2006: 2610 Introduction to Epidemiology 24 Summer 2006: 1110 Social and Behavioral Aspects of Community Health Introduction to Biostatistics Man s Impact on the Environment Introduction to Epidemiology Community-based Health Assessment 28 Fall 2006: 1110 Social and Behavioral Aspects of Community Health 30 Spring 2007: 1110 Social and Behavioral Aspects of Community Health Introduction to Biostatistics Man s Impact on the Environment Introduction to Epidemiology 34 Non-enrollment policy A 2006 amendment to the UTSPH Catalog ( ) changed the non-enrollment policy from two years to one year and reads: After absences for duration of one or more calendar years (three or more consecutive semesters), the student is automatically dismissed from the School. To complete the degree the student must be readmitted to the degree program. All applicants for readmission must meet the admission standards as described in the current SPH catalog. Readmission requires a review of the applicant s record while previously enrolled at the SPH. Following the review and decision by the Division to which the student wishes to be admitted, the Divisional recommendation will be forwarded for subsequent evaluation and approval of the application by the school s Admissions Committee. This policy allows for better monitoring of degree programs. Non-enrolled students and their advisors are contacted in the second non-enrollment semester and reminded of the need to register for courses. Advisors then have the opportunity to work with students who are experiencing problems that are interfering with their education. Students either re-enroll, request a leave of absence, or withdraw from the degree program. The policy is too recent to provide any meaningful data. Public health training -- The UTSPH continues to draw a large number of students from the public health workforce and health care professions. In the past, these students were forced to matriculate into a degree program in order to obtain public health education since few non-degree public health training opportunities existed. The courses that they completed were valuable for

7 their work, but the actual completion of the degree was not always seen as necessary or even advantageous to their careers, and many did not complete the degree program. This contributed to the low graduation rate. Since 2003, the school has worked to develop a variety of opportunities that allow individuals to obtain public health education without admission to a degree program. These include a general non-degree program, formal educational collaborations, and a Certificate of Public Health. Successfully completed courses may be counted toward a degree if the student applies to and matriculates in a degree program within five years of taking the course(s). Persons admitted to the general non-degree program may take up to three courses. These courses may be any courses for which the prerequisites have been met. In addition, instructor approval must be obtained. Enrollees include students and junior faculty from other UT Health Science Center schools, alumni returning for continuing education, post graduate medical trainees, and individuals interested in exploring public health education, among others. UTSPH has entered into educational collaboration agreements with other institutions to provide a specified curriculum for students enrolled in their degree programs. These agreements include Baylor College of Medicine (MS in Clinical Investigation) and The University of Texas at San Antonio (PhD in Applied Statistics and Demography) whose students take up to eight UTSPH courses in a specific curriculum. The Certificate in Public Health program was initiated in the fall semester, 2005, for the purpose of providing basic public health education to the public health workforce and other interested persons. Students admitted to the Certificate in Public Health complete the core requirement in each of the five public health disciplines prior to the award of a certificate of completion. The number of non-degree students involved in these programs has increased substantially over the past few years (Table 8). Further, as the number of non-degree students has increased, the number of students in the MPH degree program has decreased proportionally. Thus, these nondegree programs are expected to have a positive effect on the MPH (and the school s overall) graduation rate since the denominator will no longer include persons seeking a more limited public health education. Table 8. Enrollment in non-degree programs in the fall semesters. Non-degree program General & Edu Collaborations Certificate NA NA NA Total * *Decrease due to implementation of clinical research degree program by UT Southwestern. Time limits on degree programs -- In past years, UTSPH had a philosophy that the student had primary responsibility for his/her education, set the pace of the degree program, and made other decisions affecting the duration and completion of the degree. Thus, little pressure was placed on the student to complete his/her education in a certain period of time, particularly in the professional degree programs. The culture of the school is changing, and a new policy placing time limits on all UTSPH degree programs was approved for new students, effective as of the fall 2006 semester. The policy was added to the UTSPH Catalog ( and reads: Students are expected to complete master s degree programs (M.P.H. and M.S.) within five (5) years and doctoral degree programs (Dr.P.H. and Ph.D.) within seven (7) years. In case of extenuating circumstances, the student may request a one-year extension provided there is adequate justification. The possibility of a second year of extension exists for extraordinary

8 circumstances. Students who do not graduate within the approved time limit will need to be readmitted to the school in order to complete the degree program. MPH culminating experience -- At the time of the self-study and CEPH site visit, the faculty was considering several options for the MPH program, including a project designed around the practice experience. Prior to that time, the only accepted culminating experience was the research thesis. Options designed to maintain a high-quality experience consistent with the student s career path were developed and formally adopted. These options are described in detail in the Culminating Experience Guidelines, which is available to students on the school s website ( ). These options provide students with culminating experiences that better fit their educational and career goals. Options other than the research thesis, which typically includes primary data collection, should shorten the time needed to complete the project. To further assist the students in preparing their culminating experiences, theses and dissertations, pre-formatted templates have been developed. The following templates are available on the school s website ( ): MPH Thesis MPH Practice-based MPH Policy Analysis MPH Systematic Review MS Thesis DrPH Dissertation PhD Dissertation Each of the above templates includes versions for students preparing a document with or without a peer-reviewed journal submission.

9 Criterion IX.C. This criterion was partially met with the following concerns noted: A. Need for a proactive advising system B. Improvement of student satisfaction with advising procedures The UTSPH has been very active in addressing the above criticisms. Changes have been made in many areas, including: the accessibility of information on academic policies, practices and expectations; scheduling of regular meetings of advisory committees; and administrative oversight and dissemination of results from student evaluation of advising. The specific steps taken to fully meet CEPH criterion IX.C. are listed below. A. Enhancement of a proactive advising system Access to academic policies and procedures -- In the past, there has been no centralized location beyond the SPH Catalog for academic policies and procedures. Two important steps have been taken to address this issue. First, a website has been created for the Office of Academic Affairs. This site contains information for the internet (public) audience, but more importantly, for the intranet audience of faculty and current students. The internet (public) site includes sections for reports (including annual Student Exit and Alumni/Employer Surveys), an archive of SPH Catalogs, and links to important external agencies, such as ASPH, CEPH, Texas Education Code, and others. The intranet site includes sections for: guides, including the Guide to Student Advising among others; internal reports and academic program data; full policy documents, including background, rationale, and new policy; and policies and procedures A-to-Z, which provides succinct information and link(s) to additional text. The intranet sections are constantly updated for new materials and policies so that faculty and students have easy access to current information. Second, a Guide to Student Advising has been written and approved by the Academic Council and endorsed by the Executive Council. The Guide, which is a collection of academic policies, procedures and expectations, is primarily written for faculty advisors but may also be utilized by students. A copy of the Guide is attached (Appendix 6) and is also posted on the Academic Affairs website. The Guide is regularly updated with new policies and procedures. Scheduling of MS, DrPH and PhD advisory committees -- At the time of the CEPH site visit, advisory committees for the MS, DrPH and PhD programs met irregularly on an as needed basis. These meetings were not documented in the student record. A policy regarding advisory meetings was approved by the Academic Council and endorsed by the Executive Council. The policy provides for regularly-scheduled meetings for all degree programs as a useful tool for assisting students toward the timely completion of their degrees. The policy stipulates that meetings for MPH, MS, DrPH, and PhD students are held at the end of fall and spring semesters. The UTSPH Office of Student Affairs schedules meetings for each student and advisory committee. Evaluation meetings for students in all degree programs include a form documenting student progress, plans, and any issues that arise. Competencies for each of the degree programs and majors are available from the Student Affairs website (MPH Competencies Form) or the student s Division (MS, DrPH, and PhD programs) so that progress in the degree program can be assessed. The Office of Student Affairs tracks the number of meetings held for each degree program and places the completed meeting summary forms in the student record. Data given below report the number of MS, DrPH and PhD committee meetings the first semester after the policy change (Table 9).

10 Table 9. Advisory committee evaluation meetings for the fall 2006 semester (all degree programs). Total # Students Canc d* Mtgs sched d % Mtgs Occur d % MPH Houston MPH Regional Campuses MS DrPH PHD TOTAL *Scheduled meetings are not necessary when graduation is imminent, students are actively working on the CE/thesis/dissertation (i.e. regularly meeting with committee), or when other justifiable reasons are provided. B. Student satisfaction with advising procedures The advising problems identified in the 2004 Self-Study and CEPH site visit have been actively addressed and are being resolved. In the most recent surveys of current and graduating students (see below), fewer problems were documented involving fewer faculty members. The vast majority of students report that they are satisfied with their faculty advisors and the advising process. Evaluation of advisors and reporting of results-- Important changes have been made in the evaluation of advisors and advisory committees. The evaluation of advisors and the advising process include evaluations of advisors survey by current students and an completed by graduating students. Specific changes in each survey are detailed below. At the time of the CEPH site visit, the evaluation of advisors survey was carried out in the spring semester of each year. Results, which tended to have a low response rate, were collected and summarized per advisor by the Office of Student Affairs. Summary results (per faculty member) were sent to individual faculty. Summary results did not include all advisors and thus were not shared among faculty or with students. Further, results for an individual faculty member were not administratively reviewed. Since then, significant changes have been made in the process and distribution of results and include the following: A new instrument has been developed, reviewed by faculty, and approved by the Academic Council. Results from all respondents advised by a faculty member are summarized and sent to the advisor. A school-wide summary of results, including de-identified student comments is distributed by the Office of Academic Affairs for review by the Academic Council, the Dean, and Associate Deans. In addition, results for individual faculty advisors are sent to the faculty s Division Director and/or Regional Dean. In addition, Student Exit Surveys are completed by graduating students, who fill them out online. The surveys are anonymous. The results are reported annually by the Office of Student Affairs in summary form to the Office of Academic Affairs, which in turn provides the data and individual student comments (minus personal identifiers) along with an executive summary to all faculty and students. The opinions of these graduating students inform the faculty and administration of problems and/or issues that need to be addressed. These issues are placed on the agenda of the appropriate Council (i.e. Academic Council, Practice Council, Research Council) for deliberation and action. New policies or changes in existing policies approved in the Councils are forwarded to the Executive Council for endorsement. Endorsed policies are then implemented by the appropriate administrative office.

11 A number of changes have been made to improve the advising process (see above sections), including the creation of new tools, such as the Guide to Student Advising among others. Student satisfaction with advising New forms and procedures have been developed and put into practice for advisor surveys carried out in the spring and fall 2006 semesters. The spring 2006 advisor s survey was provided to students via the UTSPH Student Affairs website for completion. Of the 791 degree-seeking students, 245 (31%) participated in the survey by evaluating 92 (70%) of the 132 advisors in the school. The majority (75% or more) of students rated advisors in the highest categories (4 or 5). Results from two of the questions indicated a need for improvement for some advisors in their availability and knowledge of policies and practices. Results from the spring 2006 survey were informative and led to several changes. The low response rate was problematic and led to efforts to encourage more students to complete the fall 2006 advisor survey. While positive comments outweighed negative comments by 3 to 1, the negative comments reflected continuing problems with some advisors and the need for an advising guide that could be utilized by advisors and students alike. In an effort to increase the response rate, the advisor evaluation survey was folded into the course evaluation process, which is an outsourced, web-based system. Instead of having to be reminded to complete surveys on two web sites at different times during the semester, students access one screen that includes all classes taken as well as all faculty serving on their advisory committees. Further, each committee member is evaluated per their role on the committee. Another factor in the low response rate has been the lack of effect that the evaluation has had in changing faculty behavior. Division Directors and Regional Deans are now encouraged to include the information in their annual meetings with individual faculty, and the review of results by the Associate Dean for Academic Affairs allows another opportunity to identify repeated problems, which can then be addressed by divisional and/or regional deans. Revisions were made to the questions by the Academic Council in order to focus on the most important aspects of the advising process. Also, the evaluation was set up to indicate the faculty role on the committee so that students are able to comment on each faculty member in each advisory role. Further, the evaluation scale was reset to 1-4 (worst to best). Since the 2006 evaluation of advisors was completed, a Guide to Student Advising (discussed in an earlier section) was approved and placed on the Academic Affairs website. This document, which includes academic policies and procedures, is for use by faculty and students and provides a convenient repository for continuous updating of information. The fall 2006 advisor survey, done in concert with course evaluations, resulted in a significantly increased overall response rate from 31% to 49% (390 respondents out of 790 enrolled) and increased number of advisors evaluated from 70% to 87% (110 evaluated out of 126 advisors). The majority (88%) of students rated faculty in the highest categories (3 or 4) for all questions (Table 10). Data indicate that the quality of advising is very high, has improved over time, and is addressing the needs of students.

12 Table 10. Percent of students who rated advisors in the highest two categories. Data are from the spring and fall semesters of Question Spring 06 Fall 06 Satisfied with advisor* Responsive to requests* Works well with other committee members* Helpful to my research/culminating experience* Available when needed Clarity of communications Understood educational/career goals Knowledge of policies and procedures** Knowledge of degree programs and curricula** Support Flexibility Advocacy Usefulness of information Degree of thoroughness Degree of patience Respect for my input *New questions included in fall 2006 survey **Knowledge of degree programs, curricula, policies and procedures combined for fall 2006 survey Not included in fall 2006 survey Student Exit Survey ratings are provided by graduating students. One section of the survey is devoted to the advising program. The responses for these questions indicate a high degree of satisfaction overall with advisors and the advising system (Table 11). Table 11. The percentage of students who agreed or strongly agreed with the statements regarding advising. Data were taken from the Student Exit Survey in academic years 2004 (n = 213), 2005 (n = 195) and 2006 (n = 207). Advising statements The academic advising system for students was clearly explained to me. My academic advisor was readily accessible to me My advisor was sufficiently familiar with the degree programs and curricula to guide me in selecting my course of study. My advisor understood my career goals The Student Exit Survey also asks graduating students for open-ended comments about the greatest strengths and weaknesses of the school. Among the 207 graduates in academic year 2006, only 15 comments (7.2%) reported problems with advisors or the advisory committee. In comparison, in academic year 2004, 25 (11.7%) of the 213 graduates reported problems with advisors or the advisory committee. Thus, the percentage of complaints about the advisory program is low, and there has been an improvement as evidenced by the decrease in the number and percent of negative comments over time.