Form 201BC: Assessment Report Form for Instructional Programs

Transcription

1 Form 21BC: Assessment Report Form for Instructional Programs Program Clinical Mental Health Counseling Graduate Program Department Psychology Student Learner Outcomes List the outcome(s) being assessed. Upon Graduation from the Counseling Graduate Program Students will be able to: 1. Integrate knowledge of human growth and development into the diagnostic and counseling process. 2. Appropriately use counseling assessment, research, and evaluation protocols. 3. Use the analysis of the dynamics of the helping relationship in the counseling process. 4. Prepare and utilize effective educational and career planning in school/clinical mental health counseling settings, including settings with the complex challenges of urban youth, families and communities. 5. Effectively respond to professional and ethical issues. 6. Implement an appropriate plan of intervention to use in effectively teaching and counseling clients, including urban youth. 7. Appropriately respond to social and cultural issues in school/clinical mental health counseling settings. 8. Integrate counseling knowledge and skills into effective performance as a school/clinical mental health counselor. 9. Effectively coordinate services with other professionals in the school/clinical mental health counseling setting. Method of Assessment List the assessment instrument(s) and explain the administration procedures. Unless stated and explained otherwise, all instruments in the assessment plan must be administered and reported. a. First Assessment - Alumni and Exit Survey This survey is given to students at the end of their final semester to determine how well they think that they have been prepared in a comprehensive set of content and skills related to counseling. (assesses # 1,2,3,4,5,6,7,8, 9) b. Second Assessment--Content Assessment #2 - Comprehensive Examination Part One Objective Exam Results: This objective exam is administered by the Counseling Graduate Program faculty to students enrolled in Practicum (assesses outcomes # 1,2,3,4,5,6,7,8,9).

2 c. Third Assessment--Assessment of Planning - Comprehensive Examination Part Two Case Conceptualization and Treatment Planning Essay Results: This essay exam is administered by the Counseling Graduate Program faculty to students enrolled in Internship and involved the conceptualization of a case study and the development of a theoretically based, appropriate plan of assessment and treatment (assesses outcomes # 1,2,3,4,5,6,7,8,9). d. Fourth Assessment--Clinical Assessment - Practicum Assessment (Field Supervisor Rating Form): This evaluation form is completed by the field supervisor during practicum (assesses outcomes # 3,4,5,6,7,8,9). e. Fifth Assessment--Assessment of Impact on Student Learning or Providing a Supporting Learning Environment - Internship Assessment (Field Supervisor Rating Form): This evaluation form is completed by the field supervisor during internship (assesses outcomes # 3,4,5,6,7,8,9). f. Sixth Assessment (Determined by Program, Unit or SPA) - Portfolio and Faculty Supervisor Form: This evaluation is completed at the end of the program and involves and rating of the portfolio and an exit survey (assesses outcomes # 1,2,3,4,5,6,7,8,9). g. Seventh Assessment (Determined by Program, Unit or SPA) -- Candidacy Evaluation Results: Program faculty evaluate the academic performance, written skills, verbal skills, ethical behavior, emotional maturity, professional development, and personal stability of all students during Pre-Practicum (assesses # 3,5,7,8). h. Supervisor/Employer Program Evaluation Survey This short survey is distributed to all supervisors of practicum and internship students during these classes as well as employers of graduates when available. (assesses # 3,4,5,7) Assessment Findings/Interpretations/Conclusion (Answer all questions.) Actual assessment data for the 11/12 as well as the 9/8 through and 1/11 year cycles are reported on the attached Form 21C. The data suggest that students are performing well on the required program assessments. Since the institution of an objective comprehensive examination, which prepares students for the School Counseling Content Area Exam and the National Counselor Examination, students have performed better on these exams. In addition, study and practice for the exam is incorporated into the Practicum class, so all students are prepared as they matriculate through the program. Only one failed the exam, but passed prior to taking internship and completing the program. Most students report that they are more than adequately prepared for this exam by the course content of COUN 578. Similarly, students scores on Part One of the comprehensive examination have also continued to improve. Item analysis of Part One of the comprehensive examination has demonstrated specific content areas in which students struggle. Faculty are given the results of the analysis and asked to improve student mastery of this content. Students continue to have more difficulty in the areas of research/statistics and career. This has been discussed with faculty, who are examining the material that is not adequately mastered and are developing additional strategies for discussing and reviewing this material. In addition, there are plans to apply more research tools and career planning during practicum and internship.

3 A new Study Guide was developed in the internship classes to prepare students for Part Two of the comprehensive examination. In addition, students are required to write case reports, requiring the same conceptualization and planning skills in all of the clinical courses leading up to internship. Student writing and planning have continued to improve over the last few years as a result. During Practicum, students are also starting to actually use their applied practice skills. Supervisors rating these skills over the last three academic years have given satisfactory ratings to the students. Faculty are starting to have bi-weekly consultation with site supervisors during practicum to better assess and strengthen student performance. Faculty are continuing to enhance strategies for application in clinical courses. Students are now developing a Developmental Counseling Plan in PSYC 578, which they are expected to implement with appropriate revisions during their internship. This allows the internship supervisors to better assess the impact on student learning in more direct ways. The complete evaluation process for of the Developmental Counseling Plan was implemented last year, when the plans of at least some internship students will have been already assessed using the appropriate rubric while taking PSYC 578. Finally, the logs and forms always required in clinical courses have been expanded and formalized into student portfolios, allowing a comprehensive representation of student work in their final semester. As appropriate rubric addressing standards were developed to assess these portfolios, better assessment data will be available in the future to examine more specific areas of student learning. Trend data does suggest that the previous improvements to the program are paying off. The program completed the Self-Study for the CACREP re-accreditation and the site visit was completed in the Fall of 21. Ultimately eight-year accreditation was granted. Feedback from this process was further used to make improvements in the program. The program has already changed program curriculum and added emphasis on emergency preparedness, addictions, and psychopathology to better meet the 29 accreditation standards. In addition, the Community Counseling Program has been revised and now meets the CACREP standards for Clinical Mental Health Counseling. To this end, the program was renamed and increased to 6 hours along with increasing the required internship hours from 6 to 9, thus meeting the current CACREP standards for Clinical Mental Health Counseling. An entire manual was developed to assess all of the CACREP and ISBE standards through rubrics assessing student learner outcomes within each program. The program Student Learner Outcomes were revised and a curriculum map was developed to demonstrate that all courses are relevant and all standards are covered and assessed, increasing in complexity as student matriculate through the program. In the Fall of 213, the two main school counseling courses were modified to reduce redundancy and better prepare students for practice as a school counselor. All students will have to take a course, Advanced School Counseling Applications (COUN 578). Decision-making Using Findings Additional outcome measures are being developed for use in individual courses to better assess the many CACREP standards. CACREP now is requiring evidence of assessment of every standard. To this end, a comprehensive system of student outcome measures was developed for each course in all programs. Following the re-accreditation when additional

4 curriculum changes can be made in the program, the faculty decided and approved curriculum changes to improve the program. Students are encourages to take a course in Counseling Across the Lifespan as an elective to give them more strategies for working with children and adolescents. Students will create an American School Counseling Association (ASCA) Counseling Program in practicum. The increased emphasis on outcome assessment is expected to improve student performance and understanding of research, career, and appraisal including content knowledge as well as application. More technology is used in all of the program courses, but increases in budget are needed to update equipment in the Counseling Laboratory and make Smart Classrooms available to all students in every class. The equipment in the lab is particularly dated and having performance issues. Requests for such funding have been discussed and administrators have been notified.. Demonstrating Improved Learning Student learning continues to improve. Students are more consistently receiving high ratings from their supervisors and the employer/supervisor program evaluation ratings have improved. Although there were no serious areas of concern in the present trend data report, past reports indicated some problem areas. The current assessment process is much more comprehensive at this point and the student performance is higher and more consistent. Fewer students are failing the certification exam or the comprehensive examinations. Feedback from field supervisors continues to get higher over time. In addition, all faculty have a better understanding of the CACREP and IBSE standards and how these are assessed in their specific courses, allowing them to track student performance and make curriculum adjustments as indicated. Publicizing Student Learning A brief summary of assessment results is available to students and prospective students on our website. Assessment requirements and results are also presented in New Student Orientation and Clinical Course Orientation. Results are ed to faculty and a summary is discussed New Student Orientation and now also posted on the program website. Brochures mention assessment and give instructions for obtaining more information. In addition, the report is published along with all of the other CSU assessment data on the web. Accomplishments and Challenge The program assessment plan has been extensive for years and has been repeatedly revised. Significant changes have been made in program curriculum and examination preparation in response to the plan. Completion of the self-study for re-accreditation was a massive accomplishment. The program was granted two-year accreditation in 211 and was granted the full eight year accreditation following the next CACREP board meeting the following January. Limited budget for supportive software or clerical assistance as well as continued decrease in reimbursement for the assessment coordinators are very challenging. The program has been trying to hire additional faculty for three years, but timely provision of offers and hiring freezing have resulted in only one hires. Although quality has been maintained, less is provided each year and more is asked of the assessment process. More resources and release time should be provided to assure that the process has depth and meaning. Without adequate support of assessment, there is inherent risk that the process will become perfunctory.

5 CHICAGO STATE UNIVERSITY COUNSELING GRADUATE PROGRAM Graduate Studies Clinical Mental Health Counseling Alumni Survey Results Directions: Please indicate, using the following scale, the extent that the program prepared you to meet the following objectives. Comments would be helpful to the department. Inadequate Adequate Exceptional Understand basic principles of human behavior Understand developmental concepts of childhood, adolescence and adulthood Demonstrate an understanding of the dynamics of mental health and abnormal behavior Understand principles of motivation and learning theory Understand and apply techniques of assessment, research, and evaluation within school and community settings Utilize statistical concepts in test construction, interpretation, and application of test results Develop proficiency in application of appraisal techniques in counseling Assist individuals in making career and educational decisions based on test results, occupational information, and changing gender roles Understand concepts of group dynamics for the purpose of assessment and counseling in groups Understand fundamental principles of educational, social, personal, and vocational counseling of both individuals and groups Become proficient in interview and individual case reports Be able to analyze and prepare individual case study reports Function as a behavioral specialist/consultant to education and community agencies Develop an awareness of special needs populations including high risk

6 and exceptional individuals as well as multicultural populations Demonstrate an understanding of the helping process and its application in counseling Utilize various counseling intervention techniques and procedures in crisis situations Understand and apply knowledge of legal and ethical principles to individual and group counseling Continue their professional orientation and development through familiarization with professional organizations, journals and professional credentialing. Overall Average 4.27 N = 17

7 Clinical Mental Health Counselors Employer Survey ( ) SD = Strongly Disagree (1) D = Disagree (2) N = Neutral or NA (3) A = Agree (4) SA = Strongly Agree (5) Average Score 4.74 A. Counselor possesses sufficient knowledge and organizational skills to adequately perform the tasks of a counselor B. Counselor is able to work well with clients, co-workers, administrators, students, and/or parents C. Counselor demonstrates effective counseling skills and implements individuals and/or group counseling effectively in your setting D. Counselor effectively intervenes in crises E. Counselor demonstrates a commitment to clients and improving their behavior or learning environment F. Counselor is sensitive to aspects of ethnicity, culture, and special needs. Overall Average = 4.44 N = 23

8 FORM 21C: SUMMARY OF TREND [COE/CAS Education Programs] Program & Department Clinical Mental Health Counseling Graduate Program, Psychology Assessment Coordinator Lindsay Bicknell-Hentges Directions: Assessment trend data is recorded in 3-year cycles. Provide the information requested in each column along with a summary of actions taken. May 213 Report May 214 Report May 215 Report Instruments 211/212 Results Actions Taken 212/213 Results Indirect Required Assessments Graduation Rates / Program Completer Rates [UG/PB/MAT] 1 st Year Retention Rates COE/CAS Education Program Self-Survey Report Exit Survey 4.44/5. Avg. Direct Key Unit/Program Required Assessments COE 1 Licensure Basic Skills, Content, APT: IL Type 73 in School Counseling Certification Exam Results for Program Completers COE 2 Content Knowledge: Comprehensive Exam Objective Part One COE 3 Planning: Faculty are reviewing weaker areas and altering related curriculum 4.4/5. Avg. Actions Taken 213/214 Results Greater emphasis and application will be added to the assessment and research courses 4.27/5. NA NA NA Fail 2 Pass 8 Pass Hi 1 9 % PASS Fail Pass 1 Faculty plan to revamp Part One of comps and have students purchase study materials at program onset Students will continue to have Fail Pass 4 Pass Hi 3 1 % PASS Fail Pass 6 Curriculum is being revised in courses with lowest average scores New course on case Fail 2 Pass 9 Pass Hi 4 87 % PASS Fail Pass 12 Actions Taken Research and assessment will be offered in variety of formats to better match student learning styles. Students are instructed several semesters prior to this exam to start a program of study New course on case Summary of Actions Taken Faculty have examined weaker areas and responded with changes in curriculum and instructional formats. Greater emphasis is being placed on developing a comprehensive study plan early in the program. Faculty have continued to place

9 Comprehensive Exam Part 2 Essay- conceptualization & treatment planning Pass Hi 8 PASS extensive clinical writing infused in all courses Pass Hi 1 PASS conceptualization was developed and will be required Pass Hi 3 1 % PASS conceptualization and treatment is being offered and required in new program greater emphasis on case conceptualization and clinical writing throughout program. COE 4 Effective Practice: Practicum Field Supervisor Rating Form = 18 (3.5/4. avg.) 1 % Students are given individual feedback on specific improvement goals for internship = 7 (3.43/4. avg.) 1 % Students will continue to be given individual feedback on specific improvement goals for internship = 15 (3.28/4. avg.) 1 % Students will continue to be given individual feedback on specific improvement goals for internship Areas of weakness are specifically addressed with students during and following practicum for all students. COE 5 Impact on Student Learning/ Learning Environment: Internship Field Supervisor Rating Form = 18 (3.71/4. avg.) Review is being made of results to identify any patterns across students = 7 (3.64/4. avg.) Faculty discuss past areas with the weakest performance in program interns with current students = 15 (3.72/4. avg.) 1 % Faculty will compare individual performance in practicum with ratings in internship to best prepare student for practice Prior to completion of internship, instructors will address any areas of weakness and suggest plans of remediation. COE 6 Professional Portfolio & Dispositions: = 18 Use of digital portfolios will be examined = 7 Program is examining the use of Livetext for all courses = 15 Faculty are digitizing several of the instruments and forms included in the portfolio. A several phase plan is being started to digitize the portfolio and assessment process. 7- Candidacy Evaluation Results Accepted = 18 - All students are given feedback on any week areas of evaluation Accepted = 7 - New forms for documentation of recommendations are being utilized Accepted = 15 - Satis. All students are given feedback on any week areas of evaluation The process of Candidacy remains a critical component of the program s gatekeeping function for the profession and is diligently maintained. 8 - Supervisor/Employer Survey 4.7/5. Review is being made of results to identify any patterns across raters 4.8/5. Results are closely examined to assure that levels of performance 4.44/5. Results remain fairly consistent, but are reviewed closely to identify program In general, results are consistently positive, but reviewed closely for any changes.

10 remain consistent weaknesses.

11