Trainer: Brett Zyromski, Ph.D.

Transcription

1 S I M P L I F Y I N G E V I D E N C E - BA S E D S C H O O L C O U N S E L I N G : U S I N G A L O G I C M O D E L T O D R I V E DATA - D R I V E N, C O M P R E H E N S I V E S C H O O L C O U N S E L I N G Trainer: Brett Zyromski, Ph.D.

2 Goal 1: Define Evidence-Based, Data-Driven Comprehensive School Counseling Goal 2: Apply a Logic Model to prioritize student needs, drill-down to student populations, and organize intervention and evaluation. Goal 3: Create Goals Goal 4: Learn and Apply Evaluation Strategies Goal 5: Practice Creating Pre-Post Tests Goal 6: Learn to Produce Flashlights

3 Dimmitt, Carey and Hatch (2007) define evidence-based school counseling as the intentional use of the best available evidence in planning, implementing, and evaluating school counseling interventions and programs (p. ix). EVIDENCE-BASED SCHOOL COUNSELING It is inferred, therefore, that we (a) know what to prioritize what really needs to be addressed, (b) can research what is likely to work to meet that need and (c) have the skills to evaluate if the intervention made a difference.

4 THE STEPS: A) Assess the needs of the students and the barriers blocking their success (Data-Driven Decision Making). B) Either continue using something that is being successful or choose an intervention that the research suggests works with a similar population to yours to remove the barrier to success (Evidence-Based Intervention). C) Evaluate your intervention and evolve (Evaluation).

5 THE FOLLOWING MUST BE COMPLETED: (a) The needs of the students must be accurately assessed and be heard from their voices. (b) Interventions must be driven by student needs, not adult preferences. (c) Evaluation is vital.

6 NATURAL CONSEQUENCES (a) Prioritization of SC time occurs (b) Equity gaps and groups are served (c) Program goals are set (d) Job roles change and evolve to meet student needs (e) Interventions evolve based on results.

7

8 ASSESS Needs Assessments School Improvement Plan Report Card The purpose is to understand equity gaps, issues related to access, the focus of the school, and the issues identified by students.

9 ACTIVITY YOU ASSESS 1. Needs Assessments use data from Needs Assessment Results 2. School Improvement Plan (See Tichenor School Improvement Plan Summary) Usually have academic goals listed e.g. Improve Math Scores or Improve Reading Scores 3. Report Card use to identify equity gaps

10 Grade Total # % # % 1 Coping with stress Fear of making mistakes Difficulty controlling anger Skipping school Receiving one or more failing grades Test anxiety and test taking on a report card skills 4 Fights Not getting along with teachers 5 Student use of alcohol or drugs Domestic violence A divorce or family separation Impairment or disability Bullying or harassment Communication problems Loss of a close friend or relationship Physical or sexual abuse Exploring career options Neglect Different values between generations Attention in class or on school work 11 Dealing with people of a different ethnicity, race, or religion Homelessness Drug or alcohol abuse in the family Rumors and gossip Learning about college opportunities Major health concerns in family 14 Thinking or talking about suicide Low self-regard Feeling sad or depressed a lot Trouble getting along with friends or others 16 Poverty Death of a loved one Dropping out of school Hurting or cutting oneself Problems with eating or body image Adjusting to a new place or culture 19 Gender or sexual orientation issues Setting and carrying out goals Time management and organization Teenage pregnancy Fights (47.8% of those polled list this as a concern) 2 Rumors and gossip (36.2% of those polled list this as a concern) 3 Difficulty controlling anger (30.4% of those polled list this as a concern) 4 Bullying or harassment (29.0% of those polled list this as a concern) 5 Receiving one or more failing grades (26.1% of those polled list this as a concern) Most Common Student Frustrations: Gossip -- Bullying -- Transition from Elementary -- Friends (making them and kee Counselor Can Help By: Stop bullying -- Talk to students about their problems -- Help with organization/time managem

11 Grade Boys # % # % 1 Coping with stress Fear of making mistakes Difficulty controlling anger Skipping school Receiving one or more failing grades Test anxiety and test taking on a report card skills 4 Fights Not getting along with teachers 5 Student use of alcohol or drugs Domestic violence A divorce or family separation Impairment or disability Bullying or harassment Communication problems Loss of a close friend or relationship Physical or sexual abuse Exploring career options Neglect Different values between Attention in class or on school generations work 11 Dealing with people of a different ethnicity, race, or religion Homelessness Drug or alcohol abuse in the family Rumors and gossip Learning about college opportunities Major health concerns in family 14 Thinking or talking about suicide Low self-regard Feeling sad or depressed a lot Trouble getting along with friends or others 16 Poverty Death of a loved one Dropping out of school Hurting or cutting oneself Problems with eating or body image Adjusting to a new place or culture 19 Gender or sexual orientation issues Setting and carrying out goals Time management and organization Teenage pregnancy Fights (65.5% of those polled list this as a concern) 2 Difficulty controlling anger (31.0% of those polled list this as a concern) 3 Bullying or harassment (27.6% of those polled list this as a concern) 4 Divorce or family separation (27% of those polled list this as a concern) 5 Failing/Self-harm/Teacher conflict (24.1% list this as a concern) ds (making them and keeping them) anization/time management -- "I don't know"

12 Grade Girls # % # % 1 Coping with stress Fear of making mistakes Difficulty controlling anger Skipping school Receiving one or more failing grades Test anxiety and test taking skills on a report card 4 Fights Not getting along with teachers Student use of alcohol or drugs Domestic violence A divorce or family separation Impairment or disability Bullying or harassment Communication problems Loss of a close friend or relationship Physical or sexual abuse Exploring career options Neglect Different values between generations Attention in class or on school work 11 Dealing with people of a different ethnicity, race, or religion Homelessness Drug or alcohol abuse in the family Rumors and gossip Learning about college opportunities Major health concerns in family Thinking or talking about suicide Low self-regard Feeling sad or depressed a lot Trouble getting along with friends or others 16 Poverty Death of a loved one Dropping out of school Hurting or cutting oneself Problems with eating or body image Adjusting to a new place or culture 19 Gender or sexual orientation issues Setting and carrying out goals Time management and organization Teenage pregnancy Rumors and gossip (36.2% of those polled list this as a concern) 2 Fights (65.5% of those polled list this as a concern) 3 Difficulty controlling anger (31.0% of those polled list this as a concern) 4 Bullying or harassment (27.6% of those polled list this as a concern) 5 Receiving one or more failing grades (26.1% of those polled list this as a concern)

13 SCHOOL IMPROVEMENT PLAN SUMMARY

14 Tichenor School Improvement Plan Summary Goal 1: Continuous Instructional Improvement through Professional Development Goal 2: Proficiency in Next Generation Learners Overall Score (Improve Achievement Scores: Reading, Math) Goal 3: Proficiency in Next Generation Instructional Programs and Support What strategies are being used to achieve goals?

15 FORM GOALS Goals need to be SMART: Specific this is vital drill-down to identify specific populations. Measurable Achievable Realistic Time-Bound

16 CHOOSING DATA POINTS See Menu of Data Elements handout.

17 Menu of Data Elements Data elements that can be assessed are anything that reflects behavior change. The following are examples: ACT Scores Attendance Rates Classroom Behavioral Referrals Drop-Out Rates Early Warning Indicators Eighth Grade Graduation Rates Enrollment in AP Courses Enrollment in IB Courses Gallup Poll Hope, Engagement, and Well-Being GPA High School Readiness Rates ILP Completion Rates Kindergarten Readiness Rates K-PREP Scores MAP Scores Number of Scholarship Post-Secondary Enrollment Rate Promotion Rates PSAT/SAT PtGt Indicators

18 ASSESS DATA INDICATORS drill down - disaggregate DATA GOALS INTERVENTIONS LOGISTICS EVALUATION EVOLUTION Adapted from: Dimmitt, C., Carey, J.C., & Hatch, T. (2007). Evidence-based school counseling: Making a difference with data-driven practices. Thousand Oaks, CA: Corwin Press.

19 Visual for Goal Development The Who (Specific Population You Are Targeting) Grade level, age, gender, any other subgroup identifiers My Who The What (Baseline Data be specific) Critical Data Elements Include cut-offs (dates, scores, % answer by when? by how much?) The How (Interventions and Logistics) Which intervention? Delivered when? At what level (Tier 1, 2, or 3)? For how long? My What My How The Other What (Outcome/Target Increases) What will you know if the intervention is successful? How would baseline data change? What growth to what outcomes? My Other What My Goal

20 Lindeman Elementary School SCHOOL COUNSELING PROGRAM TRACKING SCHOOL: Lindeman Elementary School GOAL PROGRAM/ACTIVITY AUDIENCE (Delivery; Management) 1 Differentiated Intervention Differentiated Intervention (Tier 2 or 3) Tier 1 (Classroom guidance as Special) Tier 2 (Small Groups) 6 students with two or more office referrals 3 rd grade students scoring novice or apprentice in MAP assessment DELIVERY PROCESS (I, S, L, W)* All I, S 1 st grade L PARTNERS (Management) School counselors; Academic Team (administration, teachers and specialists) School counselors, Speech Therapist School counselors, School Counseling Intern 1 st grade S School counselors FUNCTION/PURPOSE To reduce office referrals by 20% Increase MAP scores by 10% in Spring 2015 MAP To increase MAP scores from Fall to Spring compared to statewide growth To increase MAP scores from Fall to Spring compared to statewide growth * I = Individual S = Small Group L = Large Group W = Whole School TIMELINE (Management) End of Quarter 1 to Quarter school year Fall 2014 Spring 2015 Fall 2014 Spring 2015 Fall 2014 Spring 2015 MEASURABLE OUTCOMES (Accountability; CSIP) Perception Data (P) Anecdotal; Teacher observation noted on office referral Needs Assessment Needs Assessment Needs Assessment Results Data (R) Office Referrals; MAP Scores MAP Scores MAP Scores MAP Scores Goal 1: The 6 students with two or more behavior referrals at the end of quarter 1 will receive Tier 2 or Tier 3 School Counseling Interventions, which will reduce the number of behavior referrals by 20% for this group. Students actively involved in the court system or removed from school for medical reasons will not be included in the analysis. Goal 2: Third grade students scoring at Novice or Apprentice on the Fall MAP Assessment from Fall 2014 will receive Tier 2 School Counseling Interventions to increase MAP scores by 10% more than students not receiving Tier 2 School counseling interventions by Spring 2015 MAP. Goal 3: First grade students that receive school counseling guidance as a special class will increase MAP scores from Fall to Spring by 10% more than the average growth reported on the instrument. Results will be compared with growth of first graders to previous year.

21

22 LOGIC MODEL PRACTICE What might the logic model look like if: You have 7 th grade boys struggling with social skills and self management. 7 th Grade Boys are scoring low on Reading Scores. Or, what might the logic model look like if: You have 11 th grade girls absent, struggling with post-secondary options, struggling with social skills, and unengaged? 11grade girls are preparing for the ACT, but scoring low on Math evaluations.

23 LOGIC MODEL PRACTICE Reverse complete the logic model: The school counselor implemented three mentoring small groups impacting 22 students. A comparison group of 54 other students was used to analyze the impact of the mentoring intervention. Fifth Grade Students receiving the mentoring intervention increased their reading scores across Fall, Winter, and Spring assessments by points. The group that did not receive the intervention increased by

24

25 ??????? Reading Scores 5-pt increase Mentoring Groups Small groups MAP Scores - comparison

26 CREATING EXCEL SPREADSHEETS Only three categories of data: 1. Demographics 2. Interventions 3. Critical Data Elements (Types of Data Elements?) Examples of Excel Documents from Schools

27 EZANALYZE Using EZAnalyze an Add On to Excel Can also use Excel, Access, or PowerPoint See handouts for instructions Often Central Office will analyze data if you collect it properly CSCORE will analyze data if you collect it properly

28 CREATING PRE-POSTS Dr. Tim Poynton always says the first step in survey development is to Avoid Making a Survey. In other words know which surveys or pre-posts are going to benefit the students and your program. How is the pre-post going to help you with evaluation? How does it fit into your data analysis of process, perception, and Outcome (Results) Data?

29 TYPES OF DATA Process What Who When Perception Knowledge gained Attitudes or Beliefs held Competencies achieved Results Attendance Records Discipline Referrals Graduation Rates Improved Test Scores

30 TYPES OF QUESTIONS Hatch (2014) helps organize pre/post survey questions into questions assessing students: Attitudes Knowledge Skills

31 ATTITUDE QUESTIONS Measure opinions or beliefs of students Create using a Likert Scale usually 1-5 (see below for example): 1. Strongly Agree 2. Agree 3. Neither Agree nor Disagree 4. Disagree 5. Strongly Disagree Can replace agree/disagree with Important/Unimportant, Excellent/Average/Poor, Always/Never, or Frequent/Infrequently

32 EXAMPLES OF ATTITUDE QUESTIONS I believe that raising my hand will result in the teacher calling on me. I believe applying self-management skills will help me earn better grades in school. I believe when I apply my anger management skills it is more difficult to make me angry.

33 Knowledge questions measure what students know prior to a lesson and then after a lesson. These types of questions are often used as part of an evaluation of a school counseling intervention. Likert Scales are often used with multiple options for answers not necessarily a scale. The primary strategy for increasing self-management skills is through rapid, deep breathing through the nose and mouth. Name the two classes which must be taken to gain access to honors courses. EXAMPLES OF KNOWLEDGE QUESTIONS

34 SKILLS QUESTIONS Skills questions are more advanced questions. They often require a scenario be created for the student to analyze. Hatch (2014) suggests student might do some of the following to illustrate skill mastery: Complete a career assessment Fill out a financial aid form Fill out a four-year plan Complete a job application

35 AVOID Essay, short answer, or fill in-the-blank questions. Focusing on content you don t really care about. Asking more than one question at a time. Using five dollar words when a ten cent word will suffice. Asking general questions be specific!

36 YOUR TURN Let s practice creating pre/post surveys. Label questions with an (A) to indicate if the question is assessing Attitudes, a (K) if it is assessing Knowledge, and a (S) if it is assessing Skills. Get to Hatching Results and access free pre/post assessments: (Ch. 7)

37 PRESENTING RESULTS Flashlight Approach see Hatching Results examples: Data/online-appendix.cfm

38 Academic Achievement Academic Achievement Future Planning Transitions Leadership Student Success Skills Career Cruising Campus Visits Student Success Skills Ready to Learn K-1 Ready for Success 2-3 Student Success Skills 4-5 Student Success Skills B.E.S.T. Partnership 5: Middle School Pre-K: Latonia Elementary Readi-fest Girls On The Run Student Technology Leadership Program Parent Success Skills Kindergarten Jump Start

39 Personal/Social Umbrella Bullying/Conflict Resolution Social Skills Transitions Olweus SSS 6th and 8th Grade Orientation Peer Helping Peer Helping Peer Helping Life Skills