YSU Program Student Learning Assessment Report Due Date: Thursday, October 31, 2013

Transcription

1 2013 Academic Assessment REPORT Template 1 YSU Program Student Learning Assessment Report Due Date: Thursday, October 31, 2013 As part of our ongoing accreditation by the Higher Learning Commission (HLC), all academic degree programs at YSU participate in assessment processes and practices. This process is facilitated with help from the YSU Office of Assessment (OOA). The purpose of the student learning assessment process is to provide ongoing opportunities for faculty to discuss student learning, review student artifacts, and improve educational programs through departmental discussions. The assessment planning and reporting process is about program self- evaluation and collective reflection for improvement of student learning within a program; it is not an evaluation of individual students, courses, faculty, or departments. Assessment is an ongoing activity to improve student learning and student success. When is the report due? The deadline for all academic program assessment reports is Thursday, October 31, What is crucial to include in these reports? Assessment reports should focus on data and explanations of how data are used from the previous academic year ( ); strengths and challenges of students learning; and realistic action steps for continuous improvement. It is the vitality of the program s assessment processes, not results, the Office of Assessment will evaluate in its review. Reports should also update the status of action steps A Robust Assessment Process Closing the Loop resulting from previous years assessment processes. We encourage you to describe your assessment activities and findings in ways that are useful to you. As explained in the instructions, we will accept reports in other formats that include all necessary components. Do I have to complete an assessment plan and an assessment report? NO! Each program may choose to complete either an assessment plan or an assessment report. Do not complete both. If a program wishes to realign or redesign assessment activities, they may choose to submit an assessment plan (in lieu of a report) using the plan template found on our website. Note: programs should not complete a plan two years in a row. Where do I send the report? Your report should be submitted through our online reporting form at

2 2013 Academic Assessment REPORT Template 2 What s new this year? Combined Workshops: As part of the streamlining process the OOA has combined the plan and report workshops into one. Workshops will be held in late September. If you would like to sign up for a workshop, or would like to access the workshop materials, please visit web.ysu.edu/assessment/templates. Report Preparation Open Forums: In addition to workshops open forums for academic departments will be held in October. These drop- in forums at the OOA office will allow departments to ask specific questions or brainstorm ideas. The forums are for those who do not need to attend a training workshop but would like to discuss their reports. See the OOA calendar at for forum dates. Online Reporting Process: In 2012 the Office of Assessment moved to an online reporting process. This allowed the office to streamline the reporting process and will allow for more powerful analysis of plans and reports. A walkthrough of the form will be provided at the workshops and open forums, or you may contact our office for assistance. If you are experiencing extreme difficulties with this system we will allow submissions. How will the reports be reviewed? A team from the YSU Assessment Council will review the reports and provide feedback based on a rubric (a set of evaluative criteria). Feedback will be returned separately for each degree program, even if a combined report is submitted. To ensure that our focus is on the vitality of program self- appraisal and improvement of student learning outcomes (and not the form itself), note that questions on the rubric are weighted differently. You can view the rubric and specific question values on the OOA website. The Office of Assessment also uses volunteers to assist the Council in report reviews; this is a great, short- term university service opportunity. If you would like to volunteer please contact our office. Do I need to keep a record of this report? Faculty need to maintain their own department s student learning archive for 10 years. The archive can be electronic and/or paper and must include: all summaries of data on student learning, representative examples of student work at various levels of performance, and rubrics and/or criteria used to evaluate student learning. The archive can also include newsletters, minutes of meetings on assessment, representative examples of the tools used to measure student learning, assessment plans and reports ( ), and any other relevant materials. For accreditation purposes the OOA maintains a record of all plans and reports. Assessment plans and reports will be posted on a secure site, currently the OOA Word Press site, to provide programs a wider distribution of their results and recommendations. The OOA also seeks to provide information back the campus community in useful ways; for example, the OOA Use of Data Chart shares how other programs are collecting and using student learning data for improvement purposes. Contact our office if you would like to access the report archive. How do I know what you are looking for in the report? For definitions of important terms, as well as tips on filling out the report, please see the Assessment Report Tips and Hints document on the OOA website at web.ysu.edu/assessment/templates. In addition, a link to the rubric that will be used to review and evaluate this report is provided at web.ysu.edu/assessment/templates. A workshop and open forum series will be conducted by the Office of Assessment; please see the website for more information.

3 2013 Academic Assessment REPORT Template 3 What should happen if our program s Student Learning Outcomes (SLOs) have changed? If your department has made changes to the learning outcomes, since the 2012 SLO review, please send a separate copy of all the SLOs to both the Office of Assessment (ysuassessment@ysu.edu) and to either Deborah Lowe for undergraduate learning outcomes (dlowe01@ysu.edu), or Angie Urmson- Jeffries for graduate learning outcomes (ajurmsonjeffries@ysu.edu), by October 31, SLO revisions are incorporated into future editions of the Undergraduate and Graduate Bulletins. Report Instructions To Complete the Report: Please answer all questions regarding your department s assessment activities for the prior academic year, Include data on only two Student Learning Outcomes (SLOs) per program. Each SLO will, in turn, include two methods (for a total of two SLOs and four methods in a report). For programs with well- developed assessment processes, or those programs with low numbers of majors, it is allowable (but does not mandate the need) to supply multiple years data and evaluation for a particular SLO. Departments with more than one degree program: Separate reports may be submitted for each degree program or certificate program with 18 or more semester hours. In some cases it is possible to combine multiple programs into one report. There are several considerations when combining reports: Multiple programs if SLOs are the same or are substantially similar for multiple programs in your department (e.g., a BA and BS in Mathematics), you may include related programs in one report. However, if you choose this option, you must disaggregate data on assessment outcomes for each program. Multiple degree levels similar programs at different levels (e.g. an AA and BA program) may be combined into one report; however, you should break out data for each degree level. Multiple Tracks when a program has multiple tracks, but these all result in the same degree (e.g., teacher education has multiple tracks, including elementary and secondary education), the department may rotate through the tracks on a department- developed timeline, but should cover all SLO s within a 3-4 year cycle Note: tracks are possible areas of concentration in a degree program; tracks are not equivalent to a minor. Your report could, depending on the programs in your department, include a combination of different programs, tracks, or various degree levels of the program. As long as you rotate and assess the SLO s in all programs over the course of your cycle, the way they are completed in the report is a departmental decision. SECTION 3 (Learning outcome data) INSTRUCTIONS: PAY CAREFUL ATTENTION TO SECTION 3: While you have the option to turn in just one report for your department, you should report on two Student Learning Outcomes for each program on this template. (e.g. the Foreign Languages Department would submit TWO SLOs for EACH of their programs, French, Italian, and Spanish a total of six SLOs). If multiple programs share the same SLOs, you could complete just one Section 3, as long as you show data for each group (so, if all Foreign Languages programs share 2 SLOs, just in different languages, they could then complete one Section 3 with data for each program. Also, if you are submitting a combined report and need more than the table provided in section three, just copy and paste. Note that you will have this option in the online form as well. Alternate to Completing the Form: To create a meaningful process, the Program Student Learning Assessment Report Form is designed to be flexible. If you prefer to write a narrative to describe your assessment process or if you have produced an accreditation report/self- study within the past year (or produce accreditation assessment documents yearly), you can include such document(s) as attachment(s) to the Program Student Learning Assessment Report and then provide page

4 2013 Academic Assessment REPORT Template 4 numbers from your attachment on the report form. Mapping the page numbers on the report form enables reviewers to readily identify how your narrative addresses each question as it is asked on the Report form. Additionally, highlighting that information on the corresponding page within the narrative or accreditation document will help the Assessment Council quickly locate responses for which they are looking. Please also read the form and form instructions before completing an alternate report so that all necessary components may be included. Assessment process image credit, page one:

5 2013 Academic Assessment REPORT Template 5 Section 1 Identifying and Contact Information Note: Save time and don t fill out this section until you are completing the online form! Department: Name: Address: Campus Phone: Degree Program(s) Covered by This Report: Level (associate, bachelor, undergraduate or graduate certificate, master, doctorate): List of All Program Tracks in Program (if applicable): Specific Program Tracks Included in Report: (if applicable): Number of attachments included with this report (not including this report) Section 2 Outside Accreditation Counseling, Special Education, and School Psychology Matthew J. Paylo, Ph.D. mpaylo@ysu.edu Counseling Program Master of Science Degree in Education (M.S. Ed.) Clinical Mental Health Counseling, School Counseling, Addictions Counseling, and Student Affairs and College Counseling Clinical Mental Health Counseling, School Counseling, Addictions Counseling, and Student Affairs and College Counseling 0 1. Does any program covered by this report undergo program- specific accreditation? If yes, please list the program(s). If no, proceed to Section Please provide the name of your accrediting agency. If multiple programs with various accrediting agencies are covered in this report, please reference the name of each agency to the appropriate program. 3. How often does your accreditation occur, and when is your next self- study, site visit, and/or interim report due? Include all dates as necessary for each program Yes All Program Tracks: Clinical Mental Health Counseling, School Counseling, Addictions Counseling, and Student Affairs and College Counseling Council for Accreditation of Counseling and Related Educational Programs (CACREP) Accreditation visits occur every 8 years; the next accreditation visit will occur in Fall 2014 and the self- study is due January 2014.

6 2013 Academic Assessment REPORT Template 6 included in this report. 4. How often do you submit interim reports to your primary accreditation agency or agencies? Interim Reports are submitted every four years

7 2013 Academic Assessment REPORT Template 7 Section 3 Assessment and Evaluation of Student Learning Outcomes Note: This form only contains a table for one program/two learning outcomes. If you are combining multiple programs into one report, space will be provided in the online form for additional program and learning outcome information. For programs drafting in this template, just cut and paste the table as needed. Student Learning Outcomes for Counseling Program (program name) 1. State the student learning outcome assessed: 2. How did you assess student performance? Please include: Method of collection- please provide no more than two methods for each learning outcome. One method must be a direct measure. Where data was gathered Number of students assessed Performance criteria Data summary- What patterns emerged as you studied the data with regard to this particular student learning outcome. Learning Outcome 1 Students will demonstrate a professional identity, including professionalism, and fitness to practice as a professional counselor 1. (Direct Measure). The Counselor Preparation Comprehensive Examination (CPCE): Section 8 (Professional Orientation and Ethics section). The CPCE is our program s comprehensive examination and our students are required to pass it before they are approved to start Practicum and Internship. The CPCE is sent A total of 23 students took the CPCE examination in the last academic year ( ). Of these, 91.3% (21) passed the examination and 8.7% (2) did not pass the examination. Passing the CPCE requires scores within ones standard deviation of the mean; this is a general practice for counseling programs throughout the country. Subsequently, both students passed on their second attempt. Within that examination, there are eight specific sections that align with CACREP s (Council for Accreditation of Counseling and Related Educational Programs) eight core areas. Section 8 of the examination addresses the Professional Orientation and Ethics standards. Our students exceeded the national average (M=11.14) on this section and performed higher (M Learning Outcome 2 Students will demonstrate knowledge and skills specific to their program areas (i.e., Addiction, Clinical Mental Health, School, & Student Affairs and College Counseling) 1. (Direct Measure). The Counselor Preparation Comprehensive Examination (CPCE): Section 8 (Professional Orientation and Ethics section); Praxis (School Guidance and Counseling exam; School counselor Licensure Examination); National Counselor Examination (NCE; 1 st Tier Counselor Licensure Examination); and National Clinical Mental Health Counselor examination (NCMHCE; 2 nd Tier Counselor Licensure Examination). A total of 58 students took the CPCE (23), Praxis (12), NCE (8), and the NCMHCE (15). These examinations are utilized to measure this standard because the knowledge and skill specific domains within each on of our tracks (Addiction, Clinical Mental Health, School, & Student Affairs and College Counseling) are evaluated by one of these examinations. Our performance criteria for the counseling program is an 85% first time pass rate. The academic year data on these four examinations:

8 2013 Academic Assessment REPORT Template 8 =11.47) in this knowledge area of Professionalism and Ethical behaviors, than the other sections. CPCE = 91.3% Passing rate (21 pass; 2 fail) PRAXIS = 100% Passing Rate (12 pass) NCE= 87.5 Passing Rate (7 pass; 1 fail) NCMHCE = 86.7 Passing Rate (13 pass; 2 fail) The data reviewed that on the CPCE (general core knowledge and skills) our students performed consistently well (at or above national mean averages). The school- counseling track seems to prepare students extremely well for the praxis examination. The NCE and NCMHCE were lower pass rates (yet still within our performance criteria) and mirrored state and national examination results. 2. (Direct Measure). Site Supervisor s Final Semester Evaluation. All 87 Site Supervisor s Final Semester Evaluations ( ) were reviewed. Additionally, these evaluations have qualitative questions, which include a short statements of their judgment of the intern as a prospective counselor, their perception of the interns fitness to practice, the intern s professionalism, and the intern s overall performance). These forms are kept and maintained in the student s clinical files behind a lock door (3 rd floor Beeghly). After reviewing all of the evaluations, no complaints or issues were made specifically to the competency, professionalism and fitness to practice as a counselor, and overall performance was high (i.e., average of of a potential 100). 2. (Direct Measure). Site Supervisor s Final Semester Evaluation. All 87 Site Supervisor s Final Semester Evaluations were reviewed (Questions including intern s specific track knowledge, skills, competencies, and the intern s overall performance). After reviewing all of the evaluations, interns scored extremely high throughout and their overall performance was extremely high (i.e., average of of a potential 100).

9 2013 Academic Assessment REPORT Template 9 3. What successes and challenges do you see in students learning as a result of these assessments? NOTE: This question specifically looks for you to assess the students strengths and weaknesses based on the data. It does not ask for an assessment of pedagogy or curriculum. 4. How did you use the data: e.g. what recommendations and action steps to the program have resulted from reviewing this data and where is the department in this process? Note: Data may not indicate the need for change; this is fine. In this case, just explain your department s conclusion resulting from the data. Strengths: The vast majority of our students have developed adequate knowledge around professional practice, ethics, and professionalism. In addition, all of the site- supervisors reported interns not only possessed professional knowledge, but displayed professional skills and ethical behaviors throughout their internship experience. Weakness: A small number of our students are lacking in the general knowledge to pass the CPCE examination on the first attempt; subsequently, passing on the second examination. During this review of our comprehensive exam (CPCE) data, the faculty determined that we needed to change textbooks in COUN 6972 Career Counseling and COUN 6976 Social and Cultural Issues in Counseling to provide students with more of the content knowledge necessary to increase their performance on the CPCE. Strengths: The vast majority of our students have developed specific knowledge and within their track of student (i.e., Addiction, Clinical Mental Health, School, & Student Affairs and College Counseling). These students not only meet the academic rigor of the program s coursework, but a majority of the students display their specific knowledge and skills by pass their professional examination on their first attempt. In addition, all of the site- supervisors reported that these interns possessed specific knowledge and skills within their track and rated them overall extremely high. Weakness: A small number of our students are lacking in the general knowledge and skills to pass the CPCE, NCE, and NCMHCE. While we are consistent with state and national averages, some of our students are lacking in the specific knowledge and skills required to be licensed and successful in their chosen profession. During the review of our SLO 6 data, faculty decided to increase the assessment process of the specific knowledge and skills within each track. Therefore, faculty reviewed and established new critical tasks with assessment rubrics for every course within our entire program, which highlights the core CACREP standards and specific CACREP track (i.e., addiction, clinical mental health, school, and student affairs and college counseling) standards that are being addressed. This will be done through the use of TaskStream (i.e., which provides faculty a means to manage all of this additional data.

10 Section 4 Use of Data 2013 Academic Assessment REPORT Template 10 Complete this section only once, regardless of the number of programs included in the report. NOTE: It is of critical importance to the Higher Learning Commission that programs make strong connections between the data they are collecting on Student Learning Outcomes (Section 3) and using that data to: reflect upon the current state of student learning; make changes to the curriculum (when necessary); share that data with stakeholders (both internal and external). 5. How is your department working to engage all faculty in the assessment process? For example, it is recommended that all departments meet once a year to discuss assessment processes the highlights of such a meeting could be included here. 6. How are you sharing the results of the data discussed in section three with your students, your college, and other stakeholders? Stakeholders should include groups internal (e.g., students) & external (e.g., Career Services, college- wide assessment committees) to your department. During faculty meetings (which convene once per month), curriculum committee meetings (which convene twice per year), and community advisory board meetings (which convene each spring), faculty evaluate the counseling program (all tracks) in light of the program s mission statement, objectives, and student learning outcomes. Additionally, The Counseling Program faculty have rigorously reviewed SLOs, survey data, Comprehensive Examinations (CPCE), National Counselors Examinations (NCE), Praxis examinations, critical tasks in coursework and consulted with administrators, alumni, employers, site supervisors, and students. All faculty full time faculty members are present for these meetings aimed at program assessment. Systematic program evaluations are made available to all faculty members, to students, to the college, and to the community/alumni. The Department Annual Report, which includes the discussion of progress on our student learning outcomes, is available for student and faculty review in the departmental office. Summary reports of assessment data are uploaded on the Counseling Program website. An annual announcement that these reports are available for student review is posted to the Counseling Program listserv. Faculty, students, and the external college and community are encouraged to review this information. Internal/Faculty: Program faculty review the data from program evaluations to determine areas of strength and needs for improvement; faculty make revisions to course content and/or curricula -as neededvia the counseling program s curriculum committee. Faculty convene two meetings per year (one in the spring and one in fall) to specifically discuss assessment findings and to make recommendations for

11 2013 Academic Assessment REPORT Template 11 program development/ planning/ improvement as well as to evaluate prior action steps. Internal/Students: All student learning outcomes are available to all graduate students on the departmental website External/College/ Community: The outcome of our program evaluations, SLOs, alumni, student, and site-supervisor surveys are now available on the departmental website Therefore, other college members, students, alumni and community members may review this information. Annually, an summary of our program evaluation is sent out to all counseling stakeholders. 7. In the past several years (e.g., ) you have analyzed data and identified action steps for learning outcomes. Considering action steps from previous years, what has been an impact on student learning as a result of (one of) those action steps? Only one example is needed; you can reference past reports ( ) for this information. The Counseling Program maintains a culture of continuous improvement. The following are a few examples of the evidence of modifications at the overall counseling program level, counseling track level, and course level during the last three years. Program Level: The counseling faculty wished to enhance and strengthen the counselor identity of students in all tracks, while concurrently ensuring that students have a basic level of mastery of the eight core CACREP standards. As such, faculty decided to move towards using the Comprehensive Examination (i.e., Counselor Preparation Comprehensive Exams) for all tracks (addictions, clinical mental health, school, student affairs and college counseling). This became a program policy in fall of Over the past two years, our Student Affairs students have performed well on the CPCE (89 % passing rate) slightly exceeding our expectation of an 85% passing rate. During a review of our SLOs #3 (i.e., Students will demonstrate knowledge

12 8. Optional: Is there anything else you would like to share regarding your assessment report and/or is there any particular area on which you would like assistance or feedback? 2013 Academic Assessment REPORT Template 12 of, and the ability to apply ethical standards and engage in ethical counseling and professional practices), faculty became aware that four students had ethical violations (e.g., plagiarism, failed drug test) within a one-year period. This led to the formation of a sub-committee to review and revise the program s Academic Integrity Policy and the program s review and retention policy. In the past students were only systematically reviewed after 21 hours within the program - now students are systematically reviewed at two critical points: (1) after 9 credit hours, and (2) before entering into their practicum experience. Additionally, a new review and retention form was created that allowed faculty to evaluate a student s level of academic performance, communication (i.e., written and oral), academic integrity, wellness, and professional competency. NA