2. Satisfactory or higher by the Chair on the Faculty Annual Report/ Evaluation.

Transcription

1 1 FLORIDA INTERNATIONAL UNIVERSITY Program Outcome Assessment Academic Unit: College of Education Degree Program: Ed.S. School Psychology Link to Unit s Mission: To prepare professionals who the knowledge, abilities, and habits of mind to facilitate and enhance learning and development within diverse settings; promote and facilitate the discovery, development, documentation, assessment, and dissemination of knowledge related to teaching and learning; and develop professional partnerships in the larger community. Program Outcome (1 per page) Stated in Measurable Terms Assessment Method Results (Data Summary and Interpretation) Faculty will demonstrate excellent teaching Procedures: How will it be measured? Faculty/Course Evaluations (Fall and Spring semesters) Chair s Evaluation on the Faculty Annual Report/Evaluation Adjunct faculty are evaluated once a year by full time core faculty What is the desired level of achievement? 1. Program faculty will be rated overall in the Excellent to Very Good range by candidates completing faculty/course evaluations and 2. Satisfactory or higher by the Chair on the Faculty Annual Report/ Evaluation. 3. Expected level of satisfactory teaching; feedback session with adjunct and program faculty to review the observation 1. Here is the data related to the course evaluations taught by the program faculty and the core adjunct faculty for the academic year. Summer 2008 SPS 6193 Excellent 55% Very good 27% Total 82 % at desired level SPS 6193 P Excellent 63.6% Very good 18.2% Total 81.8% at desired level SPS 6190 Excellent 67.7% Very good 33.3% Total 100% at desired level

2 2 Fall 2008 SPS 6191 Excellent 85.7% Very good 14.3% Total 100% at desired level SPS 7407 Excellent 55 Very good 10% Total 65% at desired level SPS 6191L Excellent 100% Total 100% at desired level Spring 2009 SPS 6192 Excellent 50% Very good 50% Total 100% at desired level SPS 6192 L Excellent 50% Very good 50% Total 100% at desired level SPS 7176 Excellent 82.2%

3 Very good 11.8% Total 100% at desired level 3 EDF % 41.7% Total 100% at desired level SPS % 37.5% Total 93.8% at desired level 1. All courses taught had a majority of ratings in the very good to excellent category. Seven of the 10 course listed had 90% of the students rating the teaching as either excellent or very good on a five point scale. 2. On the faculty Annual Report/Evaluation the Chairperson of the Department of EPS Dr. Marilyn Montgomery wrote, Based on these sets of considerations, your performance in Teaching, Advising and Related Activities is considered above satisfactory. 3. The Program Director visited all the classes taught by the core school psychology adjunct faculty as part of the responsibility to monitor their teaching and level of professionalism. All the adjunct professors were rated as doing an excellent job in teaching. 4. We also evaluated how all the courses in the program were delivered in terms of content, materials, syllabi and the quality of the teaching. However, not all courses were taught by school psychology faculty. The overall rating on a four point scale for all the courses taught by all faculty was 3.73 which means that we were highly effective in the delivery of our courses. Out of the 25 courses evaluated only one course was rated below a 3.0 level and this was not a school psychology core course (that is, has an SPS prefix). The feedback I have received related to this course was that it was oftentimes taught by an adjunct professor or a doctoral student. Also both the graduate section and undergraduate section were combined into one course and students felt the course was either too rudimentary or related to material that they already knew. This concern will be addressed during the next academic year. When the course was taught by a faculty member and when it was not combined, this was not a concern and the course was rated highly.

4 4 Program Outcome (1 per page) Stated in Measurable Terms Employment of school psychology candidates appropriate to their training if positions are available Assessment Method Procedures: How will it be measured? Program leader will keep records of graduate hires as of graduation each year. What is the desired level of achievement? Candidates will be hired as school psychologists in district schools. Results (Data Summary and Interpretation) In all students have been hired and a number have received multiple offers. All but one was hired as a school psychologists and the other one was hired as a behavior specialist because no openings were available for school psychologists. In all students were hired as a school psychologist except for one who was hired as a school counselor. In all students were hired as a school psychologist except for two students. One went to work in the police department and the other was hired as an associate director of a private school. However, both students who took another position were offered jobs as school psychologists. Use of Results for Improving Student Learning We are all awaiting for the hiring of our recent graduates. Due to the budget cutbacks this is a significant concern. All of our 2009 graduates were highly recommended to be hired. We are now telling students when they enter the program that they need to be available to move out of the area or out of state as necessary to receive a job offer if the finances in Florida continue on the present course.

5 5

6 Program Outcome (1 per page) Stated in Measurable Terms As stewards of the discipline, reflective inquirers, and mindful educators school psychology candidates are well prepared by the end of their internship experience Assessment Method Procedures: How will it be measured? School-based supervisors assess candidates on a standard internship evaluation form 4 times during the year-long internship. What is the desired level of achievement? Candidates will be rated competent or highly competent by school based supervisor on their final assessments. The dimensions include: 1. Professionalism 2. Rapport 3. Multiculturalism 4. Evaluation/Assessment 5. Report writing 6. Staffing conferences 7. Interventions in classroom 8. Consultation 9. Counseling 10. Teaching workshop/presentation 11. Crisis intervention 12. Supervision Results (Data Summary and Interpretation) We will use any data where students scored below on a 5.0 scale as an area that needs to be further addressed during the student s program of studies. In looking at the data, students this year did remarkably well. The lowest area assessed was a 4.75 which was in crisis intervention and this should be expected as the students received limited opportunities to engage in this activity. Nonetheless, when they did crisis intervention, they did it well. See Appendix 1 6 Faculty consult with site supervisors to recommend that candidates get more exposure in those areas in which they need more experience (e.g., crisis interventions, group counseling). In some areas (e.g., professionalism, establishing rapport, evaluation/assessment) it is expected that all our graduates would be rated as highly competent and have enough experience to demonstrate their competence. In contrast, in other areas (e.g., crisis intervention, group counseling) our graduates would only get limited experience throughout their internship so it would be

7 expected that their level of competency would be less developed. For example, school psychologists do not have to engage in crisis intervention (such as a death of a child or teacher) on a weekly or even monthly basis. Consequently, the ratings of our graduates should reflect this reality. Nonetheless, our graduates should have the opportunity to engage in all practices expected by the profession and exhibit the requisite knowledge, skills, and dispositions to be a highly competent practitioner. Use of Results for Improving Student Learning 7 We looked at the data and no changes are recommended based on field-based supervisor s feedback. Program Outcome (1 per page) Stated in Measurable Terms Assessment Method Results (Data Summary and Interpretation) There will be a review of the entire school psychology curriculum to determine if there are any gaps in training necessary for students to be ready to enter the profession after the completion of their program. A phone interview was conducted with all the graduates of the school psychology program. The interview attempted to address three questions. 1. How relevant was each course in preparing students to enter the profession of school psychology? 2. How effective was the delivery of each course in preparing students to enter the profession. 3. How well does the student feel prepared to enter the profession after successfully completing the program? See Appendix 2 for the answers to questions 1 and 2 See Appendix 3 for the answer to question 3 Use of Results for Improving Student Learning In looking at the comments made by students related to the school psychology curriculum, the students rated all the courses in terms of their relevance very highly. The overall relevance rating for the entire curriculum was 3.81 on a scale of. Students did not suggest either adding any courses or deleting any courses from the curriculum. However they made a number of suggestions. The first suggestion was that we need more testing materials for the school psychology laboratory. Due to financial restraints we have not been able to keep up with the purchases of new test instruments that are used in the field. The second major suggestion was that we need more core faculty so

8 8 that we can attain National Association of School Psychologists accreditation. Other comments were that students would like courses offered more frequently. A few students suggested that we should place a bit more emphasis on RtI. Students commented: Please include more materials on psychopharmacology in the course Neuropsychological Issues in the Schools. The crisis counseling course was not geared toward school psychologists and was more tailored toward mental health counselors. More academic interventions should be stressed in the academic assessment course. More emphasis should be placed on bilingual assessment in SPS These suggestions will be reviewed by faculty and evaluated for their appropriateness in suggesting minor course revisions. Future Assessments To Use for Improving Student Learning Any competencies that average below the passing threshold for our graduates over a 3 year period will be used as an indicator of program revisions that may be needed. We intend to look at scores on the FTCE in school psychology starting in 2010 when the new exam comes out. We will look at the scores over a three year period to determine if our students need further training in any area. The scores on the domains of practice can be used to look at gaps in knowledge of students in specific areas and to restructure courses, components of courses, teaching or emphasis. We will be looking at any gaps in knowledge of student s competencies but the system has been inconsistent in how the results are communicated. We intend to look at results after the new school psychology subject exam has been administered which should occur in The program director is on the test development team which is now being administered by Pearson. Presently all our students who completed the program have passed the subject area exam. Summarize use of results for continuous improvement of the educational program: Our graduates seem most satisfied with the relevance of the curriculum and the delivery of the courses. The field-based supervisors rated our interns very highly and our program graduates felt most confident that they are ready to start the job of a school psychologist on day one. Some suggestions for program improvement were made but all students wanted the curriculum to remain the same. A few issues came up in regard to doctoral students teaching a few courses but students were most satisfied with the quality of instruction of the school psychology faculty and core adjunct faculty members. The two major issues were that we need more tenure earning school psychology professors and that we need to upgrade our school psychology assessment laboratory. Both of these issues require financial resources.

9 9 Appendix 1 Below is the average of ratings of all the school psychology graduates for These ratings are based on the student s primary supervisor. Each student had a minimum of two supervisors and others had up to five different supervisors related to the types of rotations each student completed and the way the internship was designed in each county. Intern Evaluation (Circle): (Final) Date of Evaluation: Supervisor: Intern: Direction: The ratings of the Intern should be based upon the actual observation and/or reports from school personnel and families. In the space provided, write the number of the scale that best describes the intern s competence in relation to a beginning, first year, school psychologist. A description of the scale points is provided below. 1 - Competence is considered to be in need of further training. Constant supervision is required. 2 - Competence is emerging. Close supervision is required. 3 - Competence is at an average level for functioning with moderate supervision. 4 - Competence is assessed to be above average suggesting only a minimal need for supervision. 5 - Competence is very well developed and reflects a capacity for independent functioning with little or no supervision. NO No Opportunity Competencies 1. Professionalism (Overall) 4.9 Follows Ethical Guidelines 4.94

10 Follows Legal Guidelines 5.00 Follows County and State Procedures 4.7 Maintains Professional Appearance/ Attitude/ Responsibility 5.0 Is Prompt and Dependable Rapport (Overall) 5.0 Establishes Rapport With Clients (Children/Adolescents) 5.0 Establishes Rapport with School Personnel 5.0 Establishes Rapport with Parents Multiculturalism 4.99 Respects Individuals from Different Cultures and Backgrounds 5.0 Understands Individuals who have Different World Views 5.0 Possesses Knowledge about Research regarding Multicultural Issues 5.0 Follows Guidelines regarding Working with Culturally and Linguistically Diverse 5.0 Individuals Is Proactive in Learning about Individuals from Different Cultures and 5.0 Backgrounds Advocates for Individuals who come from Different Cultures and Backgrounds 5.0 Is Sensitive to Cultural and Linguistic Diversity in Conducting Psychological Assessment Is sensitive to Cultural and Linguistic Diversity i n Consultation and Collaboration 5.0 Designs Interventions that are Culturally Sensitive and Appropriate 4.9 Conducts Counseling that is Culturally Sensitive and Appropriate Evaluation/ Assessment (Overall Administration) 4.93 Demonstrates Appropriate Interviewing Skills 4.94 Accurately Administers Intellectual/ Cognitive Instrument(s) 4.9 Accurately Administers Academic Instrument (s) 5.0 Accurately Administers Process Instrument(s) 5.0 Accurately Administers Social-Emotional Instrument(s) 4.89

11 Accurately Executes a Functional Assessment of Behavior Report Writing 4.93 Report is Well Organized 4.9 Data is Integrated in Report 4.9 Uses Clear and Concise Language 4.9 Uses Appropriate Grammar and Correct Spelling 5.0 Functional Information is Revealed as Necessary (Strengths and Weaknesses) 4.9 Significant Points are Emphasized in the Body of the Report 4.9 Referral Questions are Clarified and Answered 4.9 Recommendations are Appropriate and Practical Staffing Conference 4.88 Makes the Parent Feel Comfortable 5.0 Explains Assessment Results in an Understandable Fashion 4.89 Problem Solves in a Win/Win Manner Whenever Possible 4.88 Maintains Poise Under Pressure (e.g. Handling Volatile Situations) 4.75 Shares Difficult Information in an Empathetic Manner 5.0 Responds to Spontaneous Questions 4.75 Makes Appropriate Referrals as Necessary Interventions in Classroom (Overall) 4.9 Establishes a Collaborative Relationship 4.9 Identifies the Problem 4.9 Makes Specific Recommendations Related to Problem 4.9 Designs a Practical Intervention 4.9 Monitors Implementation of Intervention 4.9 Provides Follow Up 4.9 Evaluates and Revises Intervention as Necessary 4.9

12 12 8. Consultation 4.89 Parent Establishes a Consultative Relationship 4.89 Helps Clarify the Problem 4.89 Makes Appropriate and Practical Suggestions 4.89 Provides Appropriate Resources, Information, and Referrals as Necessary 4.89 Consultation 4.89 Staff Establishes a Consultative Relationship 4.89 Helps Clarify the Problem 4.89 Makes Appropriate and Practical Suggestions 4.89 Provides Appropriate Resources, Information, and Referrals as Necessary Counseling 4.96 Individual Establishes Rapport 5 Helps Identify Counseling Goals and Objectives 5 Encourages Self-Awareness 5 Uses Effective Counseling Skills (e.g. empathy, active listening, eye contact) 4.75 Uses Appropriate Counseling Techniques, Materials, and Strategies 5 Facilitates Generalization of Positive Behaviors to Settings Outside of the Counseling 5 Relationship Group 5.0 Establishes Rapport 5 Helps Identify Counseling Goals and Objectives 5

13 Encourages Self-Awareness 5 Uses Effective Counseling Skills (e.g. empathy, active listening, eye contact) 5 Uses Appropriate Counseling Techniques, Materials, and Strategies 5 Facilitates Generalization of Positive Behaviors to Settings Outside of the Counseling 5 Relationship Understands Group Dynamics 5 Manages and Maintains Focus of the Group 5 Uses Group Dynamics to Help Group Accomplish Their Goals and Objectives 5 13 Program Director s comment. This was an optional competency and only two of the 10 students participated in group counseling. Also, another two students were already school counselors and had a number of years of experience running groups. 10. Teaching/Workshop Presentation (Overall) 5.0 Plans and Structures Workshop or Presentation 5 Provides Necessary Materials and Equipment for Workshop or Presentation 5 Conducts Workshop or Presentation Professionally 5 Uses Participant and Supervisor Feedback to Evaluate Workshop Crisis Intervention 4.75 Uses understanding of Crisis Intervention Theory and Practice 4.67 Follows Appropriate Policies and Procedures Regarding Responding to Crisis 4.67 Works as a Part of a Collaborative Team Effort 4.83 Uses Appropriate Techniques and Strategies to Provide Support and Assistance to 4.83 Other Personnel 12. Supervision 5.0 Seeks Supervisory 5.0 Uses Supervisory Input 5.0

14 14 Please summarize any intern strengths or weaknesses not mentioned on the above rating scale. Note any training experiences that should be planned for this intern. I have read and understand this evaluation. Intern s signature.

15 Comments from supervisors 15 Student 1 E.A E.A. is a great intern. Fantastic presentation to high school special education teachers on the administration of the WJ-III. Excellent reports. E. looks like a natural when explaining results in staffings. E. A is always asking questions. She consistently asks for and follows suggestions. She generalizes knowledge to future situations. E. A. s strength is certainly her report writing and presentation of data at staffing. She does well when given time to prepare ahead of time. Student 2. D.L. D.L continues to demonstrate superior characteristics and skills with regards to school psychology and relating to others. He is very professional with his work ethic, timeliness and interpersonal interventions with peers, school principals, parents and students. In all areas he is extremely competent, and he always goes above and beyond. He has responded to several school discussion opportunities, without hesitation, and has been praised by school personal for his efforts. He is well-liked by all, is a good team player, and handles constructive criticism well. Also, he is extremely competent with technology and is always willing to try something new or to tackle new ideas. Student 3 P. O. Excellent intern! It was a pleasure working and supervising such an excellent candidate. She will be and asset to the field of School Psychology. Student 4. J. M. I believe that Student J. M. will apply the same degree of passion and desire for excellence that she has shown during her internship to her new career in school psychology. It is with the upmost confidence that I recommend J. M. for a position as a school psychologist. She is very personable. J. is a respectful individual who always goes the extra mile. It has been a pleasure working with J! Student 5. C. F. I have said this since the first evaluation she is he strongest intern I have ever worked with! She is more then ready to do the job, and do it well. Has worked in English and Spanish, and does an outstanding job. Outstanding in all respects. C. F. has done an incredible job this year. She has been a huge help to my school and has brought much to offer the schools I work with and their students. Her years of teaching children with special needs

16 have shaped her into a strong evaluator who understands how to assess and diagnose students levels of functioning. I have said this many times and will say it again: she is the strongest intern I have ever worked with! 16 Student 6. D. P.: Outstanding! Outstanding level of professionalism Outstanding relational skills Outstanding report writing skills! Outstanding consultation skills! One of the best interns I ve had in 35 years! In my 35 years as a school psychologist and having trained a large number of students. D. P. is one of my top student interns. Her previous teaching experiences (more than 10 years), her intellect, sensitivity, relational skills and emotional maturity makes her a unique school psychologist that will be an asset in our field. Since the beginning of her internship, she is aware of her purpose and passion in her life. Student 7. M.G. M. G. s assessment skills are excellent. She has had a wide variety of cases. Student M writes a wonderful psycho-educational expert. Her recommendations are appropriate and well thought out. Even with an increased caseload these past few months, her reports are completed in a timely manner. (Consultation) This is a real strength for M. G. Her schools love her and want her back after this year. She is always available, offers good suggestions and has established positive relationships. (Supervision) I have learned much from M.G. this year! She has used supervision effectively. I look forward to continuing to mentor her as she practices independently. I asked for final feedback from school personnel M. G. works with and every comment was positive. We are so pleased she is returning and her schools want her back. M. G. is ready to practice independently. She will seek help when she needs it as she encounters new cases. She has become an integral part of our team. I look forward to continuing to see her grow as a School Psychologist. Student 8. C.M. C. M. is very positive and professional. She is very knowledgeable with regard to working with culturally and linguistically diverse populations. She conveys warmth and has approachable interpersonal skills. C. M. is well spoken and conveys technical information in a manner that is meaningful to parents and teachers. She always incorporates constructive feedback into her practice. C.M. possesses many professional and interpersonal qualities which can only enhance her practice as a school psychologist.

17 17 Student 9. W.P. W.P does a very nice job in establishing rapport. W.P. is extremely professional. She is wonderful in being sensitive to multicultural issues. W.P does a great job with assessment. W.P has come a long way when integrating and writing reports. Her reports are now well written requiring minimal editing. When talking and sharing information, W.P does a thorough job in covering information in a manner that is understandable. W. P. does a great job in consultation. W.P presented information to our faculty and did a wonderful job. She is willing to do presentations and enjoys preparing for them. It has been a pleasure and a privilege to supervise W. P. this year. W. P. has made many growths in all areas. She always takes initiative and is eager to learn new assessments. She actively participates in meetings and is not hesitant to speak up and provide answers as well as ask critical questions. She works well with her colleagues and treats all people she encounters with respect and professionalism. Student 10. A. T. AT. is a wonderful, bright and sensitive individual. Her counseling skills come in handy when she works with teachers, students and families. She will be a great school psychologist.

18 Appendix 2 18 Students were asked to rate the relevance and the effectiveness of all the required courses in school psychology on a scale from 1 to 4. A score of 1 meant the course was not important A score of 2 meant the course was slightly important A score of 3 meant the course was important A score of 4 meant the course was extremely important A score of 1 meant the course was delivered in an ineffective way A score of 2 meant the course was delivered in an effective or average way A score of 3 meant the course was delivered in a very effective way A score of 4 meant the course was delivered in an excellent way AREA COURSE PREFIX/ NUMBER COURSE TITLE R E L E V E N C E E F F E C T I V E N E S S PSYCHOLOGICAL FOUNDATIONS (12) SPS 7195 Child Psychopathology: Assessment and Intervention in the Schools

19 SPS 7993 Neuropsychological Issues in the Schools EDF 6211 Educational Psychology: Applied Foundations EDP 6505 EDP 6506 Human Development: Childhood and Adolescence OR Human Development: Across the Life span EDUCATIONAL FOUNDATIONS (9) EEX 5259 Literacy in Special Education EEX 6051* Educational Needs of Students with Exceptionalities 3.88 SPS 6805 Professional Problems and Issues in School Psychology ASSESSMENT (16) SPS 6190 Academic Assessment and Intervention in the Schools 3.77 SPS 6191 Psycho-educational Assessment I

20 SPS 6191L Psycho-educational Assessment I: Lab 20 SPS 6192 Psycho-educational Assessment II SPS 6192L Psycho-educational Assessment II: Lab 3.88 SPS 6193 Psycho-educational Assessment III INTERVENTION (15) SPS 7407 Behavioral Interventions in the Schools 3.88 MHS 5400 Counseling Skills & Techniques SDS 5460 Crisis Counseling and Intervention SPS 6199 Family-School Consultation and Collaboration SPS 7176 Consultation and Assessment with CLD Populations

21 21 RESEARCH AND MEASUREMENT METHODOLOGY (9) EDF 5432 EDF 5481 Measurement and Evaluation in Education Foundations of Educational Research EDP 7058 Behavioral Intervention Research & Evaluation in Education SUPERVISED FIELD EXPERIENCE (13) SPS 6193 P Psycho-educational Assessment III: Practicum SPS 6678 Supervised Field Experience in School Psychology (1200 clock hours) ELECTIVES (6) (At least two) Other electives may be approved by advisor MHS 6511 SDS 6411 Group Counseling with Children and Adolescents Counseling Children and Adolescents *Students with a background in special education should substitute an elective for EEX 6051 Students who do not have a background in teaching need to take

22 one course in classroom management and another course in teaching methodology. 22 Appendix 3 Students were asked how well prepared they believed they were to start the job of a school psychologist by the conclusion of their internship on a scale from 1 to 4. A score of 1 meant that the student felt ill prepared to start the job. A score of 2 meant that the student felt moderately prepared (at an average level) to start the job. A score of 3 meant that the student felt very well prepared to start the job. A score of 4 meant the student felt exceptionally well prepared to start the job. The mean score for all the students was 3.97.

23 23