Comprehensive Assessment Plan (CAP) Report The University of Alabama. Executive Summary Report for the Public

Transcription

1 Comprehensive Assessment Plan (CAP) Report The University of Alabama Executive Summary Report for the Public (Adopted by Faculty - January 31, 2014 (Approved by the Counselor Education Advisory Committee April 17, 2014) 1

2 I. CAP Procedures Used: Reporting: The Program in Counselor Education at The University of Alabama engages in continuous systematic program evaluation (CACREP Accreditation Manual, 2009, p. 87) each year. Each September, after all data from the previous academic year (AY) are collected and analyzed, a comprehensive CAP report (all-inclusive data) is distributed to faculty and the Department Head. This all-inclusive CAP report is an overall program evaluation, considering data across all assessments presented in the CAP Manual. The annual CAP reports are kept on file in the Counselor Education office. From the overall program evaluation (CAP report), faculty review and generate an annual summary report (CAP Executive Report) in October and set goals for the current (ongoing) academic year. In Spring semester, highlights from the CAP Executive Report are distributed to specific university officials (e.g., Dean of the College of Education, Dean of Graduate School, Provost, University President). External constituencies (e.g., site supervisors, employers, program graduates, advisory committee) are informed of the availability of the data on the Counselor Education website. Current students, stakeholders, prospective students, and the general public are also invited to review the findings on the Counselor Education website. Note: This AY Executive Report is a summary of the Comprehensive Assessment Plan (CAP) Report presented in September The Executive Summary (summarized in this document) was published in October 2013 after faculty review and input from the all-inclusive CAP Report. This Executive Report offers program data from Academic Year (AY) The subsequent shaded boxes present summaries from CAP, followed by 10 significant findings derived from CAP, and goals for AY based the AY report. Distribution: After faculty adopt the Executive Summary Report each fall, university officials and the named external constituencies (described above) receive this abbreviated version of this CAP Executive Report and the Program Goals during Spring semester. The report is placed on the Counselor Education website for public view for current students, stakeholders, and the general public. 2

3 II. C A P R e s u l ts: Ext e r n a l I n f l u e n c e s: 1. Program Policies and Procedures Updates to program policies and procedures for the Program in Counselor Education authorized by the University, the Graduate School, the College of Education, and/or the Department of Educational Studies are discussed below: Updates: New Policies and Procedures Program: CAP implementation in Counselor Education for third year Department: Goals evaluated in Sept 2012 and set for department values and principles (Department Handbook) College: Dean s office updating policies 2. National, State, and Regional Influences Updates to external national, state, and regional influences related to accreditation, licensure, certification, credentials, professional organizations, and registry, each of which is critical to the program s assessment plan and ultimate success are discussed below. Each has unique considerations and expectations, and many require assessment, evaluation, and documentation. Other external bodies include: Accreditation (NCATE [CAEP], SACS), State Licensure (Alabama Board of Examiners in Counseling), State Certification (Alabama Department of Education), Credentialing (National Board of Certified Counselors), Professional Counseling Associations (e.g., American Counseling Association), and the International Registry (The International Registry of Counsellor Education Programs (IRCEP) Updates Joined the International Registry of Counsellor Education Programs (2010), renewed membership in Department working on SACS accreditation and the College of Education is currently working on NCATE accreditation 3

4 3. Advisory Committee An essential external program influence, the Counselor Education Advisory Committee, is described below. The Counselor Education Advisory Committee members are graduates, representing the CACREP tracks in Counselor Education. As graduates, the committee has a vested interest in the program, while offering an outside perspective. They often serve as site supervisors, licensed counselors, and leaders in the field. Typical committee meetings relate to program decisions, needs, goals, and recommendations (e.g., curricular changes, Comprehensive Assessment Plan [CAP], policy, student outcomes). Future goals are to extend the advisory committee membership beyond graduates to the community. AY Counselor Education Advisory Committee met in April Faculty added three additional members to committee to be more representative (n=6). Committee offered input related to CAP, Executive Report, status of program, faculty search. Committee feedback was taken back to faculty and used to make programmatic decisions. 4. CAP Data AY *See subsequent Tables Data were used to evaluate and make programmatic, curricula and other program- related modifications. The following data were used in AY Summary reports of Tables 1-19 are shown below from CAP: This section describes the external data collected under the auspices of the Comprehensive Assessment Plan (CAP). The CAP involves data collections across the University, College, Department, and in Counselor Education. Since AY , data has been collected and reported annually (see CAP for details). 4

5 CAP data were used to evaluate and make programmatic, curricula and other program-related modifications. The following data were used in AY CAP Summary reports of data collected are shown below: 1. College- Wide Data Results College- wide data are completed each semester (e.g., number of graduates, those taking comprehensive examinations, doctoral students defending dissertations). These data, disaggregated by specific tracks in Counselor Education, inform the faculty on such matters as: the number of applicants to accept each semester, how students are matriculating through the program, track- specific information (e.g., CMHC or SC), graduation trends, and scholarly activities of doctoral students. These data also shed light on several CACREP doctoral- level standards (e.g., Doctoral, B.1; F.3, F.4) and inform faculty related to such standards in Section I - - Evaluation: AA.5 the use of data findings to inform program modifications. Internal data in Counselor Education (Microsoft Access 10 database) also offers additional data. AY Results: Enrolled Students: The number of doctoral students increased this year, while SC student numbers decreased and CMHC remained about the same. Numbers need to increase across all programs. 2. Departmental Data Results Data generated by the Department of Educational Studies is also used in the Counselor Education program. For example, a follow- up student survey sheds light on characteristics of applicants (Evaluation AA.1), as well as their perceptions and evaluations of the program (AA.2) (see Table 2). The electronic survey is available to all students at the end of each semester (Section I - Evaluation BB.). Data offer formal program information (i.e., effectiveness of the program, student perceptions, and evaluations of major aspects of the program [Section I - Evaluation: AA.2], recommendations, and concerns). Exit interviews (i.e., focus groups) are also conducted to further inform the program about student opinions and suggestions. AY Results: Students believe that their graduate education has contributed to knowledge, skills, and personal development stronger agreement for writing skills, scientific methods of inquiry, information gathering, comprehension skills Assessment of department and faculty: highest ratings on faculty care about academic success, encouragement to be actively involved, access to advisement, feedback 5

6 Strengths: engaged faculty, accessibility, advocates for our success in the field, many faculty have published books Suggestions: greater assistance with publishing, writing assignments that lend to publications, some alienation in online courses, avoid online format as much as possible 3. College of Education (NCATE) Disposition Results Data collection began in Fall 2013 and will be reported in CAP AY Program Evaluation Data Results in Counselor Education During the past two decades, program evaluation data were collected every three years (Section I -- Evaluation: AA.2, AA.3, and AA.4). Data were sought from four significant sources: (a) site supervisors for external placements (e.g., practicum, internship), (b) employers of program graduates, (c) program graduates, and (d) current students. Data were collected using: (a) rating scales reflecting knowledge and skill rubrics for the various levels of graduate preparation, and (b) narrative comments from respondent sources. Beginning in Fall 2012, program evaluation data were collected and reported annually. The Counselor Education program uses a program/track- specific survey to collect program evaluation data (Section I - Evaluation: AA.2, BB.]). Program evaluation results are published on the Counselor Education website. Additionally, specific university officials (e.g., Dean of the College of Education, Provost, University President) and external constituencies (e.g., site supervisors, employers) are informed (Section I -- Evaluation: and DD) of the results. Current students, program graduates, and prospective students (see Counselor Education website) are also invited to review these findings (Section I - Evaluation: and AA.6). These data are available to the public on the Counselor Education website. Summary data findings located through the links represent four Counselor Education programs: Master of Arts Programs (SC and CMHC) and Doctoral Program. AY Results: Site Supervisor Results CMHC overall ratings were above average SC overall ratings were above average Doctoral - overall ratings were above average Employers Results - CMHC overall ratings were above average SC overall ratings were all above average 6

7 Doctoral - overall ratings were above average Graduate Results CMHC overall ratings were above average to excellent SC overall ratings were all above average to excellent Doctoral - overall ratings were above average to excellent Current Results CMHC - Majority rated program Above Average SC - Majority rated program Above Average Doctoral Majority rated program Excellent 8. University Course Evaluation Results Student course evaluations are another important external influence on the program and the Comprehensive Assessment Plan (CAP) (Section I - Evaluation: CC.). Students systematically evaluate course instructors near the each of each term. The rating forms are made available to the students in an electronic format. Instructors receive their student feedback during the subsequent semester. AY Results - Students systematically evaluate course instructors near the each of each term. The rating forms are made available to the students in an electronic format. The Office of Institutional Research and Assessment manages the evaluation process. All faculty data are stored on the Alabama Online Profile System (UAOPS). Data are confidential Annual Review of Courses and Annual Assessment of Mission and Program Objectives: Faculty assess curricular offerings, the mission and objectives, the programs, and the characteristics of program applicants (Section I Evaluation: AA and AA.1). For example, each fall semester entails the annual review of curricular offerings (Section I -- Evaluation: AA.1.) in the program. Each course is reviewed and the status and/or revisions are reported, with faculty charged with making necessary updates. Each spring semester (beginning Spring 2011), involves the annual review of the program mission and program objectives, as well as an individualized review of each track (i.e., School Counseling, Clinical Mental Health, and Counselor Education and Supervision) (Section I Evaluation: AA:1). These steps ensure that program objectives are measured, met, and needed changes are communicated. When faculty members note that modifications are indicated, discussions are carried out in faculty meetings (faculty meeting minutes reflect evidence of program evaluation, analysis, and modification) 7

8 (Section I Evaluation: AA: 5). Overall ratings of < 2 require further review and remediation. Program assessment and evaluation are illustrated as regular and critical aspects of quality review for the Program in Counselor Education. Such evaluations inform faculty about curricular needs, policy matters, communication concerns, and other aspects of the program. Faculty review additional data to assess curricular offerings, the mission and objectives, the programs, and the characteristics of program applicants (Section I Evaluation: AA and AA.1). For example, each fall semester entails the annual review of curricular offerings (Section I - Evaluation: AA.1.) in the program. Each course is reviewed and the status and/or revisions are reported, with faculty charged with making necessary updates (see Table 9). Faculty members also discuss course rotations each term, as well as semester scheduling of courses. This information offers insight about which courses are needed and helps to eliminate scheduling conflicts. AY Results: Course/Curricular Review: Courses were reviewed in Fall All CACREP courses were reported as under revisions to comply with CACREP Standards. BCE has been given a new title Counseling Skills and a group component was added to BCE 616. A new course, BCE 533, was initiated. Due to faculty retirement, three courses were suspended BCE 620, BCE 653 and BCE 654. Review of Program Mission and Objectives: Program Mission and Program Objectives were reviewed in Spring Program Mission and all Program Objectives were rated 3 or higher. Review of Program Mission and Objectives (CMHC, SC, Doctoral): CMHC - Program Mission and Program Objectives were reviewed in Spring Program Mission and all Program Objectives were rated 3 or higher. SC = Program Mission and Program Objectives were reviewed in Spring Program Mission and all Program Objectives were rated 3 or higher. Doctoral - Program Mission and Program Objectives were reviewed in Spring Program Mission and all Program Objectives were rated 3 or higher. 12. Applicant Decisions AY Results: Standardized criteria are used to review program applicants (Section I Evaluation: AA:1). These data offer feedback about the applicants and inform 8

9 future admission decisions. Applicant decisions are made after multiple, systematic decision- making opportunities occur among the faculty. Data are confidential. 13. Standards for CMHC and SC Tracks AY Results: Total Counseling was used to track all CACREP entry- level standards, beginning in January See Total Counseling Reports for full details. 14. Remediation Plans (CMHC and SC) AY Results: There were no formal remediation plans during this academic year for master s level students. 15. Student Evaluations by Faculty (CMHC and SC) AY Results: All master s level students received recommendations for Fall 2012 and Spring Standards for Doctoral Track AY Results: Total Counseling was used to track all CACREP doctoral- level standards, beginning in January See Total Counseling Reports for full details. 17. Remediation Plans (Doctoral) AY Results: There were no formal remediation plans during this academic year for doctoral students. 18. Student Evaluations by Faculty (Doctoral) AY Results: All doctoral level students received recommendations for Fall 2012 and Spring 2013 based on the Student Evaluation form. 19: National Counselor Exam and Other Exams NCE Examination - Student scores reported scores were above the national median. Praxis II - School Counseling and Guidance Students scores were above the state median. 100% pass rate. 9

10 PROGRAM/STUDENT OUTCOMES Vital Statistics as Shown on Website: Number of Graduates this Academic Year Program Completion Rate Licensure or Certification Pass Rate for Graduates In Field Job Placement for Graduates Clinical Mental Health Counseling 4 95% 100% 100% School Counseling 4 95% 100% 80% Counselor Education and Supervision 2 90% % III. C A P R e s u l ts: Int e r n a l I n f l u e n c e s: The Counselor Education program is impacted by significant internal influences such as the Comprehensive Assessment Plan (CAP). This CAP plan offers continuous and systematic program evaluation across numerous variables (those previously described, the data previously shown, as well as the following influences). 1. CAP Section I Evaluation AA1-6, BB, CC and DD (CACREP Manual, 2009, pp ) are the compelling forces behind the CAP. Nonetheless, the overall plan involves assessments and necessary evaluative steps from many internal and external influences (e.g., department, graduate school, university, advisory committee, student input, faculty input). AY Results: This report is the CAP summary. See program for full details of all data collected this year. 2. Faculty Input to the Program Faculty members proffer levels of experience, training, skills, credentials, professional service, research skills, leadership, and visibility. Faculty contributions affect how the 10

11 overall assessment plan is put forth, and the related decision- making, planning, and goals for the future. AY Results This CAP report is an example of faculty input and decision- making. The benchmark data from was replaced with Total Counseling. AY is the second full year to use Total Counseling. 3. Guidelines, Procedures, and Policies Accreditation bodies, licensure boards, and certification offices all require programmatic guidelines, procedures, and policies for the program to be in compliance. The Counselor Education program uses such mediums as the UA Student Handbook (see website) and the Counselor Education website to extend the guidelines, polices, and procedures to others (e.g., student handbook and website offer admission dates, portfolio deadliness, checklists to graduation, scholarship opportunities, remediation steps, and so forth). AY Results Website was updated CAP into third year of implementation, Total Counseling used from Spring 2012 to present Active Chi Sigma Iota Examined yield of admittees Course modifications based on data Total Counseling results considered and used for curricular decisions Total Counseling modifications were made based on data results Advisory Committee increased in size to be more representative 4. Faculty Output to Students Faculty output to the students relates to the professional practices offered during formal teaching and other interactions with students. Faculty output involves: clinical skills, training, teaching, knowledge, mentoring, research opportunities, supervision, and student evaluations. This section is interrelated with many components of assessment and evaluation. AY Results: CACREP Knowledge Preparedness and CACREP Skills and Practices are evaluated from the beginning of coursework to graduation. Student evaluations are completed on each student (in spring and fall semesters). Academic skills, clinical skills, ethical and clinical behavior, and personal characteristics are discussed. 11

12 Significant findings, derived from the information in the AY report: 1. Three programs need to do recruit and increase admissions (Table 1, Table 12) 2. Recruitment of minority students should continue (Table 12) 3. Students rated Counselor Education positively (Tables 3, 4-7) 4. Scores above average on NCE and Praxis II. (Table 19) 5. Program mission and program objectives appropriate based on faculty feedback (Tables 10 and 11) 6. Total Counseling has been effective way to track and assess standards 7. There were no remediation plans after the Fall/Spring Student Evaluations (Tables 15 and 18) 8. CAP is in third year of implementation 9. Course modifications based on data were evident in

13 CAP Goals for Student- Centered Professional Development: a. Recruiting Masters and Ph.D. students to become more involved in research projects b. Encourage more students to present at conferences c. Search for funding for Ph.D. students 2. Continue recruiting efforts for Master s and Doctoral programs: a. Continue displaying student inquiry meetings on digital monitors in Graves and Carmichael Halls b. Recruit through Chi Sigma Iota c. Recruit through Graduate School resources, including McNair Scholars d. Letters to HBCUs in Fall and Spring e. Keep track of the number of applicants that applied, were accepted, and enrolled f. Follow- up with new admits (survey and data collection - admitted students who do not enroll) g. Follow up with newly admitted students to welcome and offer encouragement 3. Continue recruiting efforts for minority students across all programs: a. Continue displaying student inquiry meeting on digital monitors in Graves and Carmichael Halls b. Recruit through Chi Sigma Iota c. Recruit through Graduate School resources, including McNair Scholars d. Letters to HBCUs in Fall and Spring e. Keep track of the number of applicants that applied, were accepted, and enrolled f. Follow- up with new admits (survey and data collection - admitted students who do not enroll) g. Follow up with newly admitted students to welcome and offer encouragement 4. Maintenance of databases: a. Refine Databases b. Track Job Placement and NCE data for Master s and Doctoral tracks c. Use of database to keep current distribution list for communication 13

14 d. Adoption of crimson account to communicate with active students in program 5. Comprehensive Assessment Plan (CAP) a. Present CAP Comprehensive report from all data sources to faculty (each September) b. Present Executive CAP Summary and Goals for AY (with faculty input each October) c. Present Overview of Executive CAP to Advisory Committee each spring, send out overview to external constituents (each Spring) d. Refine and modify, as needed (e.g., add NCATE, COE Dispositions) Adopted - October 22, 2013 Reviewed in January 2014 and April