M.S. & Ed.S. in School Psychology Assessment in the Major Report By Dr. Christine Peterson, Program Director Submitted: October 2015

Transcription

1 M.S. & Ed.S. in School Psychology Assessment in the Major Report By Dr. Christine Peterson, Program Director Submitted: October 2015

2 Table of Contents Description of Methods... 2 Program Disposition Review... 3 Practicum and Internship Student Evaluations... 5 PRAXIS II: Content Test Summary or National School Psychologist Exam Results PRAXIS II: Content Test Summary Praxis II First Time Test Takers Other Information Plan for Communicating Assessment Results Conclusions and Action Plan to Improve Teaching and/or Advisement in the School Psychology Program Goals for School Psychology AIM Report Page 1

3 Description of Methods The following assessment tools and strategies were utilized to determine student performance, progress, and levels of satisfaction in the major: Benchmark I Faculty review/selection of program applicants Transcripts review Acceptance into the School Psychology Program Benchmark II Initial portfolio review Student disposition reviews Review of practicum I evaluations Faculty reviews of student performance, including academic progress, experiential evaluations, and professional conduct Benchmark III Second portfolio review Student disposition reviews Faculty reviews of student performance, including academic progress, experiential evaluations, and professional conduct Review of practicum II evaluations Student scores on the National Certification in School Psychology Exam (NCSP) or Praxis II Benchmark IV Survey of interns regarding their perceptions of program competencies and coursework Final portfolio review Exit Student Survey Enrollment Data Year Enrolled Completed School Psychology AIM Report Page 2

4 Program Disposition Review Disposition reviews are conducted at Benchmark II (end of first year in the program) and III (mid-term during second year in the program) for all school psychology students. Domain scores rated as: 1 (unsatisfactory), 2 (minimal), 3 (satisfactory) to 4 (proficient). Total composite scores range from 15 (unsatisfactory) to 32 (proficient). Figure 1 Comparative Benchmark II Means Commented [PC1]: Can we cut both Disposition Tables to 3 most recent years comparison and put most recent year at the top (just to be consistent across all tables)? School Psychology AIM Report Page 3

5 Figure 2 Comparative Benchmark III Means School Psychology AIM Report Page 4

6 Outcome of the Disposition Review Results As a whole, the school psychology students displayed satisfactory to proficient attitudes and behaviors within the program (see Fig. 1 and 2). Scores from Benchmark II data exceeded an average of 3.6 (out of 4.0) or higher across all domains. Benchmark III data averaged 3.7 (out of 4.0) or higher across all domains. Conclusion(s): Benchmark II and III Dispositions allow faculty and students to engage in reflective dialogue regarding each student s progress and development toward the mastery of critical skills necessary for success as a grad student and as a professional. Data from suggests that students are progressing appropriately in these important skills domains. Practicum and Internship Student Evaluations Graduate students in the school psychology program complete two (2) supervised field practica totaling 600 hours during their second year in the program, and a culminating 1200 hour internship that is their third and final year in the program. Graduate students completing both Practicum I and Practicum II are evaluated through the use of The Field Practicum Evaluation tool. This newly revised tool utilizes the same items, along a developmental scale, in order to demonstrate and review growth as students advance through each experience. Scoring metrics for each tool are as follows: Unestablished- 1 Observer- 2 Novice-3 Advanced Novice-4 Practicum I Practicum II Observer- 2 Novice-3 Advanced Novice-4 Emerging Practitioner-5 Internship Advanced Novice-4 Emerging Practitioner-5 Established Practitioner-6 Advanced Practitioner-7 School Psychology AIM Report Page 5

7 Outcomes of the Practicum and Internship Student Evaluations a) Practicum Evaluations Results from the Fall 2014 evaluations indicate Practicum I students area scores (means) ranged from 2.9 (Preventive and Responsive Services) to 3.6 (Data-Based Decision Making and Accountability) on the 4-point developmental scale Evaluation Guide for School Psychology. Scores 3.0 and above are rated in the Novice to Advanced Novice range, which is considered in the accepted standard for Practicum I students on the developmental scale. Practicum I students obtained an average score of 2.9 on the 3.0 Professional Work Characteristics scales. Results from the Spring 2015 evaluations indicate Practicum II students rating scores ranged from 4.6 (Intervention and Support Academic Learning) to 5.0 (Research and Program Evaluation) on the 5.0 point developmental scale. Results indicate that practicum students earned above satisfactory scores (4.0 or higher) in all NASP Domains. Professional Work Characteristics ranged 2.9 out of a 3.0 point scale. Figure 3 Practicum I Evaluation Means School Psychology AIM Report Page 6 Commented [PC2]: For Practicum I, II and Internship tables, can we separate out the

8 Figure 4 - Practicum II Evaluation Means School Psychology AIM Report Page 7

9 Evaluation Guides - Practicum II Domain Means: Scale 2-5 (Prof. Work Characteristics: Scale 1-3) Professional Work Characteristics NASP Domain 10: Legal, Ethical, and Professional Practice NASP Domain 9: Research and Program Evaluation NASP Domain 8: Diversity in Development and Learning NASP Domain 7: Family and School Collaborations NASA Domain 6: Preventative and Responsive Services NASP Domain 5: School-Wide Practices to Support Learning NASP Domain 4: Interventions and Mental Health Services to NASPA Domain 3: Interventions and Instructional Support to NASP Domain 2: Consultation and Collaboration NASP Domain 1: Data-Based Decision Making and Accountability N= N= N= School Psychology AIM Report Page 8

10 b) Internship Evaluations Results indicate that internship area mean scores ranged from 6.3 (Interventions and Instructional Support to Develop Academic Skills) to 6.7 (Legal, Ethical and Professional Practice) on the 7-point Evaluation Guide for School Psychology for the spring 2015 interns. Professional Work Characteristics ratings averaged 2.9 out of a 3.0 point scale. Results indicate that interns earned above satisfactory to highly satisfactory scores (scores of 6.0 and above) in all Evaluation Guide areas (see Figure 5). The revised Intern Evaluation scale is designed to demonstrate growth from the Emerging Practitioner during Practicum II, to the Established and Advanced Practitioner during Internship. Figure 5 Internship Evaluation Means Commented [PC3]: Can we cut this to most recent 3 years comparison? School Psychology AIM Report Page 9

11 Results indicate that interns scored in the Established Practitioner or above range on all Evaluation Guide areas (Data-based Decision Making; Consultation and Collaboration; Effective Instruction and Development of Cognitive/Academic Skills; Socialization and Development of Life Skills; Student Diversity in Development and Learning; School and Systems Organization, Policy Development, and Climate; Prevention, Crisis Intervention, and Mental Health; Home/School/Community Collaboration; Research and Program Evaluation; School Psychology Practice and Development; Professional Work Characteristics; and Communication Skills). Conclusion(s): Reviews of the Evaluation Guide for School Psychology suggest that UW-Stout graduate students are adequately prepared to enter the field of practice with professional skills for success as practicing school psychologists. We are pleased with the accuracy of the data that our recently revised evaluation tools have provided. PRAXIS II: Content Test Summary or National School Psychologist Exam Results It is a program requirement to take the National Certification in School Psychology (NCSP) exam (PRAXIS II) used by the National Association of School Psychologists (NASP) as requirement for national certification. The passing score established by NASP and recognized by the Wisconsin Department of Public Instruction (DPI) is 147. This passing score reflects a major revision to the Praxis metric, implemented in Fall of 2014 (previous to that, passing score was 165). In addition to the total score received by examinees, the PRAXIS II exam also report scores addressing the following content areas: Professional Practices Direct and Indirect Services Systems Level Services Foundations of School Psychological Service Delivery Outcomes of the NCSP Results In , 100% of the students obtained scores of 147 or higher on their first attempt at the Praxis. As such, every student who took the PRAXIS Exam met the passing criteria of NASP and DPI. School Psychology AIM Report Page 10

12 PRAXIS II: Content Test Summary School Psychology Praxis Test Code Content Test from ETS (0402) 14/15 Number of Examinees: 8 Highest Observed Score: 179 Lowest Observed Score: 151 WI Score Needed to Pass: 147 Number with WI Passing Score: Percent with WI Passing Score: 8/8 100% Percent Correct (percentage of items answered correctly) by Area UW- Points National School Psychology Test Category Stout Available 14/15 14/15 Professional Practices % 73% Direct and Indirect Services % 82% Systems Level Services % 77% Foundations of School Psychological Service Delivery % 72% School Psychology AIM Report Page 11

13 Figure 7 Praxis II Domain Percentages Commented [PC4]: Can we add the national average data to this table, for comparison? School Psychology AIM Report Page 12

14 Praxis II First Time Test Takers Beginning in the school year, all teacher preparation program in the state of Wisconsin are required to report on the passage rate on the first attempt of students and graduates of the program on examinations administered for licensure (Act 166, section 17). SPSY Content Test # first time test takers Spring # (and %) passed # first time test takers # (and %) passed # first time test takers # (and %) passed 10 9 (90%) 8 8 (100%) 7 7 (100%) Conclusions Overall results indicate UW-Stout s School Psychology program is preparing students to achieve the foundational knowledge necessary to pass the NCSP exam. Analysis of Stout performance trends across Praxis domains is unavailable at this time, due to recent revisions to the test. First time test takers of the newly revised exam performed best on the Systems and Preventive/Responsive Services domains. Lowest scores were observed on the Legal/Ethics domain. Information from PRAXIS II data review will be used to inform on-going program review with focus on continuous improvement in identified areas, as well to inform program development in general. School Psychology AIM Report Page 13

15 Other Information a) Portfolio Assessment (Benchmark II and Benchmark III) Students initiate and complete components of a portfolio over the course of the program. The completed portfolio includes evidences or artifacts demonstrating professional competence in all ten (10) NASP s Domains of School Psychology training and practice (same as the Department of Public Instruction content guidelines). A partially completed portfolio (Benchmark II) demonstrating competency in 8 out the 10 domains is submitted to the program faculty for review after completion of the M.S.Ed. and prior to admittance to the Ed.S. in School Psychology Program. At Benchmark III, students re-submit their portfolio to demonstrate competency in all ten (10) domains. Faculty members rate the portfolio materials. Based on a review of the portfolio ratings, academic performance, and professional conduct, a student is either recommended for, or denied admittance to, the Ed.S. degree in School Psychology (for Benchmark II) or Internship (Benchmark III). Outcomes of the Portfolio Assessment A review of the portfolio ratings in 2015 revealed all students produced satisfactory portfolios at Benchmarks II and III. Based on faculty consensus, there is however, a need to further highlight the relevancy of the portfolio for candidates, particularly during the internship year. Faculty has discussed a number of ways to motivate students to sustain their efforts to update and enhance their professional portfolio during internship. b) Survey of Intern Result In , as in previous years, school psychology interns were surveyed to determine their perceptions regarding their previous coursework. Each intern was asked to respond to questions on a 5-point Likert-style survey (1 = No knowledge or Skill gained to 5 = A lot of knowledge or Excellent skills gained) to assess how much knowledge or skill the student gained in each course. Further, interns are asked to identify the five most important courses in terms of how helpful they were in preparing them for their internship year. They also are asked to identify which courses were not helpful or had overlapping content. Outcomes of the Intern Survey Results from the spring of 2015 survey indicate students believed that they gained above average to a lot of knowledge and skill (x = 4.29) in most program courses. However, students continue to indicate that they gained below average knowledge and skills in the Research Foundations (EDUC 740, x = 2.29). Variable opinions were expressed about the courses related to counseling skills and theory, with some students considering it a value add to the program (x=4.10), while others questioned the relevance of this coursework to their jobs in the field (x=2.6). Additional comments indicated that most students reported they believe they were well prepared for the field of school psychology after taking all program requirements. Students did have additional feedback related to the nature of research expectations and timelines in the program. Revisions based in part on this feedback, are underway. School Psychology AIM Report Page 14

16 c) Program Advisory Committee (PAC) The program faculty members meet annually with the School Psychology Program s Advisory Committee (PAC). Colleagues from related programs on campus, as well as practitioners and stakeholders in the community meet to discuss updates in the program and solicit information from the committee about program needs and future goals. PAC membership consists of colleagues and related stakeholders, including practitioners in the field, representatives from higher education, and the state department (DPI). The PAC has been consulted as we have continued to revise and update the program in order to align with the new NASP standards. Their input continues to be valuable as we further develop and evolve our program. Outcome(s) of the Program Advisory Committee Results: Most recent PAC focus continued to relate to revisions to the program. Focus of recent meetings has been on NASP accreditation efforts. New discussions will include program revisions and recommendations from the NASP report. Plan for Communicating Assessment Results The School Psychology Program faculty meets on a bi-weekly basis to discuss program issues and plan for the future. This Assessment in the Major report will be disseminated to all members of the program faculty, and the results will be discussed by the program faculty at an upcoming meeting. This guide is a valuable resource for faculty to use in guiding program revisions. Furthermore, this report will be disseminated to the Program Advisory Committee (PAC) in the fall of the academic year. School Psychology AIM Report Page 15

17 Conclusions and Action Plan to Improve Teaching and/or Advisement in the School Psychology Program Summary and review of all available data related to the School Psychology programs at UW-Stout, the following goals are targeted for the year: Targeted Goals for Based a variety of sources (NAPS review, program advisory, student feedback), program faculty will work to implement a program revision that aims for continuous improvement and efficiency. a. Revisions were developed and courses revised throughout the Spring and Summer of Approval process will begin Fall of In coordination with Goal #1, program faculty will work in collaboration to comprehensively review research expectations, timelines and associated coursework for School Psychology, and revise accordingly. a. This goal is on-going, in conjunction with a comprehensive program revision. 3. Faculty will work with Grad School and Stout Marketing to develop a more concrete and tangible marketing plan for the School Psychology program. a. Program PD met with both Grad School and Marketing reps in spring of 2015 to develop a strategic marketing plan, including revisions to our web presence. 4. Program faculty will work with SOE support members to transition to an E-portfolio process for students. a. This goal is on-going. School Psychology AIM Report Page 16

18 Goals for A comprehensive program revision (including revised and new coursework), will be approved for a Fall 2016 implementation. 2. Marketing and Recruitment efforts will help us achieve a higher number of quality applicants to the program. 3. Program faculty will work with SOE support members to transition to an E-portfolio process for students. 4. Program will review and renew efforts to embed applied activities across relevant coursework, with a focus on Year One. Respectfully submitted, Christine R. Peterson, Ph.D. Associate Professor Program Director for School Psychology University of Wisconsin-Stout School Psychology AIM Report Page 17