Assessment and Outcomes for Counselor Education Programs Marquette University, the Department of Counselor Education and Counseling Psychology (CECP) and Counselor Education Program Committee (CEPC) are committed to assessing student learning and to using data to inform the continued improvement and development of our programs. Toward this goal, several key learning outcomes have been developed by the program for students who graduate with degrees in Clinical Mental Health Counseling (CMHC) and School Counseling (SC). There are four core learning outcomes that are common to all counseling students and then two learning outcomes that are unique to the CMHC and SC students. These learning outcomes are listed below: Core Counselor Education Learning Outcomes 1. Apply knowledge of bio-psycho-social-cultural foundations of behavior and evidence-based counseling approaches to diverse individuals and groups. 2. Apply professional, ethical, and legal standards in their counseling practices. 3. Assume advocacy roles for the mental health care of underserved individuals and groups in urban settings. 4. Integrate self-awareness, counseling roles and reflective practices into a professional counseling identity. CMHC and SC Specific Learning Outcomes 5a. Provide clinical mental health counseling prevention and treatment services for diverse individuals and groups in community settings. (CMHC graduates) 5b. Lead the development and implementation of critical interventions of a Comprehensive School Counseling Program* in culturally diverse, urban PK-12 schools. (SC graduates) Given these identified learning goals, the program developed an assessment plan comprised of measures of student performance on national standardized examinations [i.e., Counselor Preparation Comprehensive Examination (CPCE), National Counselor Exam (NCE), PRAXIS II, Professional School Counselor Exam) and student, supervisor and employer survey data. The Assessment Plan can be accessed here. In the following sections, data on recent performance on national examinations is presented as well as evidence that students are meeting important learning outcomes as identified above. Performance on National Standardized Exams Clinical Mental Health Counseling Program Students in the CMHC program are required to complete the CPCE as their comprehensive examination and most students elect to take the NCE prior to graduation. Both the CPCE and NCE are national standardized examinations that have been developed by the National Board of Certified Counselors (NBCC) to assess key areas of knowledge for prospective professional counselors. These key areas are informed by the CACREP (2009) core knowledge standards for counselors. Tables 1 and 2 below provide the aggregate results for the last three years for each examination.
Table 1 Comparison Data on the Counselor Preparation Comprehensive Examination (CPCE) for CMHC Students and CACREP Approved Program Students for 2011, 2012 and 2013 (Scores range from 0 to 136) Year Marquette University Pass Rate for CMHC Students Mean CPCE Score for Marquette University CMHC Students National Mean for CACREP Programs 2013 Not currently available 2012 16 of 17 94.19 79.62 2011 21 of 23 103.57 94.18 Table 2 Comparison Data on the National Counselor Examination (NCE) for CMHC Students and CACREP Approved Program Students for 2010, 2011 and 2012 (Scores range from 0 to 160) Year NCE Pass Rate for Marquette University CMHC Students Mean NCE Score for Marquette University CMHC Students Mean NCE Score for CACREP Programs 2012 23 of 24 120.21 111.98 2011 18 of 19 124.71 116.44 2010 34 of 34 125.03 111.77 As can be seen from the results of the CPCE and the NCE, CMHC students perform at a high level comparatively to national means. As such, these results indicate that CMHC students are mastering the prerequisite knowledge to perform well as a professional counselor. School Counseling Program Students from the SC program are required to complete the PRAXIS II, Professional School Counselor Examination which was developed by the Educational Testing Service (ETS) and required for school counselor licensing by the Wisconsin Department of Public Instruction. The exam was designed to assess prospective school counselor knowledge of contemporary school counseling practice. The SC student performance results can be found in Table 3 below for the three most recent years. Table 3 Results for SC Students on the PRAXIS II, Professional School Counselor Examination for 2011, 2012 and 2013 (Scores range from 100 to 200; Passing Rate for Licensure in Wisconsin is 156) Year PRAXIS II Pass Rate for SC Students Mean PRAXIS II Score for CMHC Students National Median for the PRAXIS II 2013 5 of 5 179.8 Not Currently Available *2012 8 of 8 696.25 660 *2011 6 of 6 701.67 600 *Note: The PRAXIS II was revised in 2013. In 2011 and 2012 the PRAXIS II, School Guidance and Counseling Examination was administered. The score range was 250 to 990 and Wisconsin s passing rate was 560.
SC programs students have a 100% pass rate on the PRAXIS II for several years and thus immediately meet the examination qualification for licensing as Professional School Counselor in Wisconsin. Additionally, the results indicate that SC students are mastering the prerequisite knowledge to perform at a high level as professional school counselor. Assessment of Program Learning Outcomes In addition to results from national examinations, the program developed an assessment strategy using subscale scores from national examinations to determine if students are meeting expected learning outcomes. The assessment strategy includes expectations (benchmarks) for graduating student performance on each measure. The student score on each measure (e.g., the student exit survey, the CPCE subscale scores, and Praxis II subscale scores) is converted to one of four levels of performance: Expectations Not Met, Developing towards Expectations, Meets Expectations and Exceeds Expectations. It is important to note that the scores required to meet program expectations are higher than the state or national passing scores. The levels of performance scaling for each assessment measure are described in Table 4. Table 4 Levels of Expectation of Marquette Master's Degree Counseling Students Assessment Measure Expectations Not Met Developing Meets Expectations Exceeds Expectations Student Exit Survey NA Rates item moderate confidence CPCE Exam Praxis Exam Rates item no confidence or little confidence Score below.5 SD below the national mean. Score is below the national 50% confidence band scores from minus.5 SD below the national mean up to the mean NA Score at national mean to plus.49 SD from the national mean Score is within the national 50% confidence band NCE Fail NA Pass NA Rates item much confidence to complete confidence Score is plus.5 SD or more above the national mean Score is above the national 50% confidence band The eight sub-scores of the CPCE exam and the sub-scores of the Praxis II exams, both national standardized tests, provide a robust way to look at student learning across the core areas of the counseling curriculum and the counseling programs. Currently, the CMHC students take the CPCE exam and the SC students take the Praxis II. Effective Spring 2014, students in CMHC and SC programs will complete the CPCE as the program comprehensive exam of their mastery of the CACREP (2009) core and for our assessment of the strengths and weaknesses in the core. Clinical Mental Health Counseling Program The program faculty members consider only scores that are at or above the national means as meeting our expectations. Each year the percentage of students who met or exceed program expectations on each of the CPCE subscales is calculated. The percentage of students meeting program expectations is examined in two ways:
Faculty look across the eight sub-scores on the CPCE to determine areas that need improvement. Faculty look at the percentage of students meeting expectations on each subscale across the years to determine if changes made in the curriculum and courses are indeed improving student learning in that area. The graph in Figure 1 displays the percentage of students that met or exceeded expectations on the CPCE subscales in Spring 2011, 2012 and 2013. (Meets = scoring at or above the national mean for that sub-score.) Figure 1 Percentage of Students Meeting and Exceeding Program Expectations on the CPCE Sub-scores Examining the graph of CPCE sub-scores; the following observations can be made: 1. Core areas of consistent student strengths are helping relationships, social-cultural and group work. Core areas in which student performance has been inconsistent or consistently lower are career, appraisal and research. 2. Overall, the students who graduated in 2013 (the first group to complete the programs' CACREP aligned core), show an increase in the percentage of students who met or exceeded department expectations for the sub-scores. Over 90% of the students scored at or above the national mean in: group work (97%) socialcultural (97%), research (94%), human development (97%) and professional issues (94%). Student performance on career and professional issues and ethics improved over the prior two years and the helping relationships remained at the same level of performance around 80%.
While it will take a few more years of CPCE scores to confirm that indeed the changes to the core curriculum improved student learning, the noticeable increase for the students graduating in 2013 is gratifying evidence for improvement. School Counseling Program School Counseling students complete the national Praxis II Exam in the spring of their second year. For each Praxis II subscale Educational Testing Service (ETS) provides the band of scores that represent the middle 50% of all national test takers. Unfortunately the titles and composition of the subscales changed with the new version of Praxis II so we cannot make the same comparisons across years. Table 5 School Counseling Students Meeting or Exceeding Expectations on the Praxis II Subscales 2011 N = 7 2012 N = 6 Revised Subscales 2013 N = 5 Counseling and Guidance 100% 100% Foundations 100% Consulting 80% 100% Delivery of Services 100% Coordinating 100% 100% Management 100% Professional Issues 80% 100% Accountability 100% Listening 80% 100% The evidence from the past few years of PRAXIS II results suggests that school counseling students are performing at a very high level, with all students meeting or exceeding established benchmarks for learning outcomes. It is important that we continue to monitor school counseling student performance to determine if any discernible areas for improvement emerge. Additionally, as students begin completing the CPCE in the Spring of 2014 the possibility emerges that new areas for improvement may be identified. Program Improvements Based on Assessment Data During the course of each year department faculty review the formal assessment reports and the informal feedback to identify areas of strengths and weakness in the program and make suggestions to improve the assessment measures and/or curriculum and courses. The Counselor Education Program Committee is responsible for using the data and department faculty comments to form recommendations to either gather more data or to take action and make specific changes. The actions taken to improve the program over the past two years are summarized in the list below presented as Table 6. Table 6 Actions to Improve the Counseling Programs, AY 2011-12 & AY 2012-13 Action Assessment Data Indicating Status of Action Area to Improve Developed required Ethics course CPCE scores Effective Fall 2011, COUN 6010 Professional Ethics and Legal
Hired a counselor education faculty member with career counseling expertise CPCE scores, student exit surveys Issues required Effective Summer 2012, COUN 6080 Career Development and Counseling taught by Dr. Kevin Tate Added coursework to strengthen professional identity CPCE scores, student exit surveys Effective Fall 2012, COUN 6000 Introduction to Counseling course required for all students Restructured course sequence for counseling skills development Added a module on licensure and supervision to foundations course Student exit surveys Student exit surveys Effective Fall 2012, basic skills are taught in COUN 6000 Introduction to Counseling and continued in 6965 and 6970 Effective Spring 2013, module added to COUN 6003 Foundations in Clinical Mental Health Counseling