Anytown Area School District:

Transcription

1 Anytown Area School District: LifeSkills Training Program Student Survey Results School Year Report Date: September 2014 Authors: Sebrina Doyle, M.S., Kristine Bova and Stephanie Bradley, Ph.D. The EPISCenter represents a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), and the Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by PCCD and the Department of Public Welfare. This resource was developed by the EPISCenter through PCCD grant VP-ST

2 Table of Contents Executive Summary 3-4 Youth Reported Substance Use in Anytown Area School District 3 Summary of Program Outcomes in Anytown Area School District 4 Summary of LST Program 5 Evaluation Method 6 Program Implementation 6 Sample 6 Measures 6 Program Outcomes 7-14 Understanding the Bar Charts and Tables 8-9 Outcome Charts Outcomes Tables Appendix A: Scales, items, response options

3 LifeSkills Training Program Student Survey Results Executive Summary LifeSkills Training (LST) is an evidence-based, universal substance abuse prevention program shown to reduce the risks of alcohol, tobacco, drug abuse, and violence by targeting the major social and psychological factors that promote the initiation of substance use and other risky behaviors. The Center for the Study and Prevention of Violence (CSPV) received funding to disseminate the LifeSkills Training drug and alcohol prevention program in several states. The grant enables CSPV to provide three years of training and technical assistance and program materials to interested schools or districts. Participating sites also receive sustainability training and technical assistance for developing and implementing a plan to sustain the LST program. In order to evaluate the effectiveness of the program, the Pennsylvania Commission on Crime and Delinquency (PCCD) provided support for schools to collect pre and post-test surveys from students participating in the LST program. This data collection process was voluntary for schools participating in the CSPV initiative. All together, we had over 4,700 students complete pre- and post-tests in 54 different schools around the state. These schools were located in 34 school districts across 22 counties. PCCD coordinated with the Survey Research Center and the Evidence-based Prevention and Intervention Support Center (EPISCenter) at The Pennsylvania State University for processing and reporting data at the school district and statewide level. In this executive summary you will find highlights from the data collected in the Anytown Area School District followed by additional detail on findings in the sections that follow. Comparison data from the Pennsylvania Youth Survey (PAYS) and from the overall sample have been provided whenever possible. Please note that both of these data sources have limitations and are not necessarily representative of all PA youth or school districts. Youth and school districts that participated were not randomly selected; those that did select to be a part of the process may be fundamentally different than those that did not. Additionally, neither the PAYS nor the LST data sample include data from youth in every county throughout Pennsylvania (e.g.,- the PAYS sample does not include students from the Philadelphia School District); not having these students may artificially lower or raise the overall average for the sample. It is important to recognize these limitations when reviewing the results. Youth Reported Substance Use in Anytown Area School District In order to understand the context of the following results of the surveys, it is first important to understand the level of drug use within your school district. LST is a prevention program, so it is not meant to bring about a reduction in drug District students Statewide 6 th Grade Statewide 8 h Grade District students* Statewide 6 th Grade^ Substance % Reported Lifetime Use % Reported 30-Day or Monthly Use Statewide 8 h Grade^ Cigarettes 2.8% 2.4% 10.2% 0% 0.5% 3.9% Alcohol 12.9% 13.3% 35.1% 0% 3.0% 9.6% Marijuana 0% 0.8% 6.4% 0% 0.4% 3.3% Inhalants 4.2% 5.3% 6.9% 0% 2.2% 2.5% * District- students who reported using monthly or more. ^ State-students who reported using in the last 30 days. use, but rather prevent it from taking place. You can use the chart to compare students in your school district with the state percentages taken from the Pennsylvania Youth Survey (PAYS) 2013 State Report at The PAYS is administered in 6 th, 8 th, 10 th and 12 th grade so 6 th and 8 th grade data is provided for comparison. Seventh grade students in your school district reported low levels of lifetime use of substances at approximately the same level as 6 th graders throughout Pennsylvania. 3

4 Summary of Program Outcomes in Anytown Area School District Knowledge From pre- to post-test, student knowledge of the effects of drugs and life skills knowledge improved significantly; Overall, students showed a 16% increase in knowledge after receiving the curriculum (average of 67% of questions correctly answered at pre and 83% correct at post); Specifically, there were statistically significant positive changes in the ALL knowledge areas: knowledge of drug effects, life skills, positive self-image, good decision making, negative influence of media, and anxiety reduction and effective communication. Impressively, students in the Anytown School District were above the overall LST sample average at post-test in all knowledge areas. These results indicate that this LST program implementation successfully improved student knowledge in key areas shown to affect long-term substance use outcomes. Skills Students reported significant increases in their relaxation skills (Mean=3.43 at pre, 3.82 at post on a 5 point scale) after receiving the LST Program. Additionally, although it did not reach statistical significance, there was a small positive effect on students assertiveness skills at the post-test (Mean=3.53 at pre, 3.72 at post). Finally, Anytown Area students reported higher drug refusal skills at pre-test than the overall sample at post-test, indicating that students had higher than average skills at the start of the program; this possibly accounted for the lack of statistically significant change in this area. These skills are paramount for the long-term goals of substance use reduction. Risks From pre- to post-test, students reported a significant decrease in perceived adult use of alcohol, tobacco, and other drugs (Mean=2.78 at pre-test and 2.18 at post-test). Additionally, although the change did not reach statistical significance, there was also a small positive effect on perceptions of peer use (Mean=2.04 at pre-test and 1.89 at post-test). These changes indicate that students, at the end of the LST program, felt that fewer people (adults and peers their age) used substances than they did at the pre-test. Long-term, students are less likely to use substances if they believe that the majority of people do not use them. In sum, the program was most successful in impacting the most direct targets of intervention (e.g., knowledge of the effects of drugs, life skills knowledge, relaxation skills, and perceptions of substance use). Where improvement in the intended direction was not achieved, in some cases it was explained by a ceiling or floor effect: pre-test scores that leave little room for improvement (e.g., peer refusal skills, substance use). However, the program also did not achieve its intended effects on favorable attitudes toward substance use or task persistence skills. It may be worth exploring whether the lessons targeting these constructs could be improved. Charts and tables with more detail on these outcomes begin on page 10.

5 LifeSkills Training Program Student Survey Results Summary of LifeSkills Training Program Life Skills Training (LST) is an evidence-based substance abuse prevention program shown to reduce the risks of alcohol, tobacco, drug abuse, and violence by targeting the major social and psychological factors that promote the initiation of substance use and other risky behaviors. Rather than merely teaching information about the dangers of drug abuse, LST promotes healthy alternatives to risky behavior through activities designed to: Teach students the necessary skills to resist social (peer) pressures to smoke, drink, and use drugs, Help students to develop greater self-esteem and self-confidence, Enable students to effectively cope with anxiety, Increase their knowledge of the immediate consequences of substance abuse and correct misperceptions of peer and adult norms of substance use, Enhance cognitive and behavioral competency to reduce and prevent a variety of health risk behaviors. Instead of just providing information about drugs, LST focuses on three major domains to reduce the likelihood that youth participate in high-risk behaviors. The three domains are: Drug Resistance Skills Enables young people to recognize and challenge common misconceptions about tobacco, alcohol and other drug use. Through coaching and practice, they learn information to counter those misconceptions and practical Alcohol, Tobacco, and Other Drug (ATOD) use resistance skills for dealing with peers and media pressure to engage in ATOD use. Personal Self-Management Skills Students learn how to examine their self-image and its effects on behavior; set goals and keep track of personal progress; identify everyday decisions and how they may be influenced by others; analyze problem situations and consider the consequences of each solution before making decisions; reduce stress and anxiety, and look at personal challenges in a positive light. General Social Skills Students develop the necessary skills to overcome shyness, communicate effectively and avoid misunderstandings, initiate and carry out conversations, handle social requests, utilize both verbal and nonverbal assertiveness skills to make or refuse requests, and recognize that they have choices other than aggression or passivity when faced with tough situations. This program is designed to be taught in sequence over three years in either middle or junior high school. The first year of the program has 15 sessions, the second year contains 10 sessions, and the third year consists of five sessions. 5

6 LifeSkills Training Program Student Survey Results Evaluation Measuring program impact can be difficult without a comparison group (such as a comparable group of students in the same grade levels with similar demographics who did not receive the LST program). This difficulty arises because it is normative for risk-taking behavior to increase during adolescence, therefore increases in ATOD use are expected as youth age. Without a comparison group, we are unable to determine whether the increased rate of use is faster, slower, or at the same rate as youth who do not participate in the program. In addition, prevalence rates of ATOD use at this age are generally very low, making it hard to decrease rates of use most kids report no use at both pre-test and post-test, leaving little room for reduction (i.e., floor effect ). To address these challenges, we assess program impact by focusing our attention on changes that occur in the risk and protective mechanisms targeted by the program that might ultimately decrease (or slow the rate of increase in) substance use in the long run. LST Program Targets Youth Skills, Attitudes, Knowledge, Perceptions Impacts Youth Intentions to Use Changes Long-term Youth ATOD Use Program Implementation The LST program was implemented by school staff during regular class time; staff from CPSV monitored each site for fidelity of implementation and provided feedback to teachers on their delivery of the program. All data was collected by schools at the beginning and end of the LST program and sent to The Pennsylvania State University for processing. Sample Students in the 7 th grade in Anytown Area School District participated in the LST program. They each completed a questionnaire before and after they received the LST curriculum. At pre-test, 71 students completed the questionnaire, at post-test 70 students completed the questionnaire. Of those students, we were able to match the data for 69 students who completed both pre- and post-test surveys; 40.8% of youth were female and 97.2% reported their race as white. Measures Students completed the same 82-item questionnaire at pre- and post-test. The questionnaire includes eight items on demographics and characteristics of the participants (such as the grades they receive in school). The questionnaire has items on Alcohol, Tobacco, and other Drug (ATOD) use, and risk and protective factors that are expected to change as a result of the LST program. Several scales have been created to determine the level of change in ATOD use and attitudes, and knowledge and skill development from pre- to post-test. 6

7 Scales in this survey include: Knowledge 1. Total Knowledge Increase 2. Knowledge of Drug Effects 3. Total LifeSkills Knowledge 4. Knowledge of Positive Self-Image 5. Good Decision Making Knowledge 6. Negative Media Influence Knowledge 7. Anxiety Reduction Knowledge 8. Effective Communication Knowledge Skills 9. Relaxation Skills 10. Task Persistence Skills 11. Assertiveness Skills 12. Drug Refusal Skills Attitudes 13. Favorable Attitudes Toward Alcohol and Tobacco 14. Favorable Attitudes Toward Smoking 15. Favorable Attitudes Toward Drinking Perceptions 16. Peer Use Perceptions 17. Adult Use Perceptions Intentions 18. Intent to Use Alcohol, Tobacco and Drugs Behavior 19. Alcohol, Tobacco and Drug Use Note: Subscales #4-8 are used to create the larger LifeSkills (#3) subscale, subscales #2 and #3 are combined to compute the overall Total Knowledge Increase (#1) scale. Subscales #14-15 are used to create the Overall Favorable attitudes toward Alcohol & Tobacco (#13) scale. Appendix A shows which items belong to each of the scales, as well as the response options provided for each question. You can also find a copy of the survey itself by visiting the EPISCenter website at Need More Information? If you have questions about this report or would like additional information please contact Sebrina Doyle, Research Coordinator for the Evidence-based Prevention and Intervention Support Center (EPISCenter), at sld40@psu.edu or by phone at Kris Glunt, LST Prevention Coordinator at the EPISCenter, is responsible for coordination of data collection and communication efforts with CSPV and with school districts. If you have questions about this data collection project, to reach Kris, please call or kglunt@episcenter.org. For general questions, you may also visit the EPISCenter website at call us at or EPISCenter@psu.edu. 7

8 LifeSkills Training Program Student Survey Results Program Outcomes Understanding the Bar Charts and Tables: Pre-test and post-test means Means are the average of youth responses, and were calculated by adding up all the scores for each of the youth and dividing by the total number of youth who responded. They show, on average, how youth respond to certain items. The dark blue bars on the charts (pages 10-12) represent your school district s pre-test mean on each scale. The light grey shaded bars, located behind the dark blue bars, are provided for comparison; these bars represent the whole-sample pre-test means for all students (N~4700) in all school districts throughout Pennsylvania who provided pre- and post-data as a part of the LST CPSV project. The orange bars on the charts represent your school district s post-test mean on each scale. The dark grey shaded bars, located behind the dark orange bars, represent the whole sample (N~4700) post-test means. Your school district s pre and post-test means are also listed in the tables on pages Standard deviations (SD) A standard deviation shows how spread out the scores are from the mean. A large standard deviation indicates that there is substantial variability across students scores, whereas a small standard deviation indicates that students scores are closely grouped around the mean. Standard deviations show the extent to which youth answer similarly (smaller SD) or dissimilarly (larger SD) to items. Standard deviations are reported for each scale in the tables on pages N The number of youth surveys for whom we have adequate data (75% or more) at pre- and post-test on each scale. T-test A t-test assesses whether there is a difference between the group means at pre-test and at post-test. A key goal of universal prevention programs is to improve the group level mean on key risk and protective factors. p-value The p-value represents the statistical significance of the difference between pre-test and post-test means (evaluated by the t-test). It indicates the likelihood that the difference between pre- and post-test occurred by chance. Using a p-value of less than.05 indicates that a significant finding has less than a 5% chance of having been found in error or by chance alone. Small sample sizes can affect the significance of results, so we have also made notations when changes are trending in a certain direction. Trend level effects are indicated when there is a 10% or less chance (between ) that the results were found by chance or error. Although these results are not as reliable, they are worth noting. Results- yellow stars and red stop signs Scales on the bar charts and in the tables marked with a yellow star indicate statistically significant differences (p<0.05) that are in the intended direction; scales marked with a stop sign indicate changes that are significant in the unintended direction. 8

9 Results- pluses (+) and minuses (-) Trend level p-values (p<.10) have also been indicated in the tables only (+ or ). Small sample sizes can impact statistical tests, while trend level changes are not statistically significant, they do demonstrate an overall trend in the data that may continue if measured in the future with a larger sample. Effect size, strength Cohen s d effect size provides a measure of practical significance, or magnitude, of changes identified from pre-test to post-test. Here it can be thought of how different the post-test mean is from the pre-test mean when taking into account the standard deviation (or spread) of the scores. The further the change is from 0, the stronger the effect. Effect sizes between.20 and.49 are considered small, between =medium,.80 and over=large. For ease, the interpretation of the effect size (strength) is provided as well as the actual number. 9

10 Anytown Area Chart % Knowledge 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Overall Knowledge ATOD Knowledge LifeSkills Knowledge Positive Self- Image Good Decision Making Negative Media Influence Anxiety Reduction Effective Communication Whole Sample Pre 65% 58% 69% 83% 85% 64% 66% 65% Whole Sample Post 75% 71% 78% 87% 89% 73% 78% 74% District Pre 67% 58% 74% 88% 91% 67% 68% 71% District Post 83% 82% 84% 96% 97% 77% 83% 80% Legend indicate statistically significant differences (p < 0.05) that are in the intended direction indicate those that are significant (p <0.05) in the unintended direction 10

11 Anytown Area Chart 2. Better 5 Life Skills Worse 1 Peer Refusal Assertiveness Relaxation Task Persistance Whole Sample Pre Whole Sample Post District Pre District Post Legend indicates statistically significant differences (p < 0.05) that are in the intended direction indicates those that are significant (p <0.05) in the unintended direction 11

12 Anytown Area Chart 3. Risks Worse Better 1 Favorable Attitudes-A&T Favorable Attitudes-Smoking Favorable Attitudes-Drinking Peer Use Perceptions Adult Use Perceptions Intent to Use ATOD Whole Sample Pre Whole Sample Post District Pre District Post ATOD Use Legend indicates statistically significant differences (p < 0.05) that are in the intended direction indicates those that are significant (p <0.05) in the unintended direction 12

13 Anytown Area Table 1. Results Scale Program Outcomes Expected to Increase or Remain High Knowledge Pre-Test Mean (0%- 100%) SD Post-Test Mean (0%- 100%) Number of matched surveys T-test p-value (likelihood that a significant difference between pre and posttests was found by chance) Cohen s d value (difference between the pre- and post-test means, accounting for the spread of the scores) Size of effect (practical significance) 1. Knowledge Total 67% % Large 2. Knowledge of Drugs 58% % Large 3. Life Skills Knowledge 74% % Large 4. Positive Self-Image Knowledge 88% % Small 5. Good Decision Making Knowledge 91% % Small 6. Negative Media Influence Knowledge 67% % Small SD 7. Anxiety Reduction Knowledge 68% % Medium + 8. Effective Communication Knowledge 71% % Medium Result Legend Scale Mean (1-5) SD Skills Mean (1-5) SD Number of matched surveys T-test p-value (likelihood that a significant difference between pre and posttests was found by chance) Cohen s d value (difference between the pre- and post-test means, accounting for the spread of the scores) Size of effect (practical significance) 9. Refusal Skills N/A Assertiveness Skills Small Relaxation Skills Small 12. Task Persistence Skills N/A indicates statistically significant differences (p < 0.05) that are in the intended direction indicates those that are significant (p <0.05) in the unintended direction + indicates trend level significance (p= ) in the intended direction -- indicates trend level significance (p= ) in the unintended direction 13

14 Anytown Area Table 2. Result Scale Program Outcomes Expected to Decrease or Remain Low Pre-Test Mean (1-5) SD Risk Factors Post-Test Mean (1-5) SD Number of matched surveys T-test p-value (likelihood that a significant difference between pre and posttests was found by chance) Cohen s d value (difference between the pre- and post-test means, accounting for the spread of the scores) Size of effect (practical significance) 13. Favorable Attitudes Toward ATOD N/A 14. Favorable Attitudes Toward SMOKING N/A 15. Favorable Attitudes Toward DRINKING N/A 16. Peer Use Perceptions Small 17. Adult Use Perceptions Large 18. Intent to Use ATOD N/A 19. Substance Use N/A Legend indicates statistically significant differences (p < 0.05) that are in the intended direction indicates those that are significant (p <0.05) in the unintended direction + indicates trend level significance (p= ) in the intended direction -- indicates trend level significance (p= ) in the unintended direction 14

15 Appendix A: Subscales This Appendix provides which items belong to each of the subscales, as well as the response scale provided for each question. If you are interested in seeing the actual survey as it was formatted for students to fill out, it is available at Subscale Survey Section Item # Question Response Scale Knowledge 1. Total Knowledge Increase B 1-32 Combines Knowledge of Drug Effects and Life Skills Knowledge 2. Knowledge of Drugs Effects B 1R 2R 3 4R 5 6R 7R 12 13R 14 15R 16 17R Most adults smoke cigarettes. Smoking a cigarette causes your heart to beat slower. Few adults drink wine, beer, or liquor every day. Most people my age smoke marijuana. Smoking marijuana causes your heart to beat faster. Most adults use cocaine or other hard drugs. Cocaine and other hard drugs always make you feel good. Smoking can affect the steadiness of your hands. A stimulant is a chemical that slows down the body. Smoking reduces a person s endurance for physical activity. A serving of beer or wine contains less alcohol than a serving of hard liquor. Alcohol is a depressant. Smoking marijuana can improve your eyesight. 3. LifeSkills Knowledge B 8-11, Combination of Self-Image, Decision Making, Media Influences, Anxiety Reduction, and Effective Communication 4. Positive Self-Image B 8 What we think about ourselves affects the way we act or behave. 9R It is not possible to change you self-image. 5. Good Decision Making B 10 It is important to measure how far you have come in reaching your goals. 11R It is a good idea to make a decision and think about the consequences later. 6. Negative Media Influence B 18 19R 20 Some advertisers are deliberately deceptive. Companies advertise only because they want you to have all the facts about their product. It s a good idea to get all information about a product from ads. 1=True 0=False 1=True 0=False 1=True 0=False 1=True 0=False 15

16 7. Anxiety Reduction B 21R R 8. Effective Communication B 25R 26 27R R 32R R indicates that the item was reverse coded (i.e.- 0=1, 1=0) There is very little you can do when you feel anxious. Most people experience anxiety. Deep breathing is one way to lessen anxiety. Mental rehearsal is a poor relaxation technique. You can avoid misunderstandings by assuming the other person knows what you mean. Effective communication is when both sender and receiver interpret a message in the same way. Relaxation techniques are of no use when meeting people. A compliment is more effective when it is said sincerely. A nice way to end a conversation is to tell the person you enjoyed talking with the. Sense of humor is an example of a non-physical attribute. It s better to be polite and lead someone on, even if you don t want to go out with them. Almost all people who are assertive are either rude of hostile. 1=True 0=False 1=True 0=False Subscale Skills 9. Drug Refusal Skills Scale was recoded so higher means on this scale indicate greater confidence in saying no. D Survey Section Item # Question Response Scale 1R 2R 3R 4R 5R Say no if someone tries to get you to smoke a cigarette Say no if someone tries to get you to drink beer, wine, or liquor Say no if someone tries to get you to smoke marijuana Say no if someone tries to get you to use cocaine or other hard drugs Say no if someone tries to get you to sniff glue, paint, gas or other things to get high 1=Definitely would 2=Probably would 3=Not sure 4=Probably would not 5=Definitely would not 16

17 10. Assertiveness Skills Scale was recoded so higher means on this scale indicate greater willingness to be assertive. 11. Relaxation Skills Scale was recoded so higher means indicate more likeliness to use relaxation skills. 12. Task Persistence Skills D 11 D D 6R 7R 8R 9R 10R R indicates that the item was reverse coded (i.e.- 5=1, 4=2, 2=4, 5=1) 12 Tell someone if they give you less change than you re supposed to get back after you pay for something. Say no to someone who asks to borrow money from you. Tell someone to go to the end of the line if they try to cut in front of you. Try to relax all the muscles in your body Breathe in and out slowly, while counting If you find that something is really difficult, you get frustrated and quit. You stick to what you are doing until you're finished with it. 1=Definitely would 2=Probably would 3=Not sure 4=Probably would not 5=Definitely would not 5=Definitely would 4=Probably would 3=Not sure 2=Probably would not 5=Definitely would not 1=Strongly Disagree 2=Disagree 3=Neither agree nor disagree 4=Agree 5=Strongly Agree Subscale Attitudes towards ATOD Survey Section Item # Question Response Scale 13. Overall Attitudes toward Alcohol C 1-8 Alcohol and Smoking attitudes combined and Tobacco 14. Smoking Attitudes C Smoking cigarettes makes you look cool. Kids who smoke have more friends. Smoking cigarettes lets you have more fun. Kids who smoke cigarettes are more grown up. 15. Alcohol Attitudes C Kids who drink alcohol are more grown up. Kids who drink alcohol have more friends. Drinking alcohol makes you look cool. Drinking alcohol lets you have more fun. 1=Strongly disagree 2=Disagree 3=Neither agree nor disagree 4=Agree 5=Strongly agree 1=Strongly disagree 2=Disagree 3=Neither agree nor disagree 4=Agree 5=Strongly agree 17

18 Subscale Perceptions 16. Peer Use Perceptions How many people your age do you think 17. Adult use perceptions How many adults do you think Intent to use ATOD 18. Intent to use ATOD Do you think you will use any of these within the next year? ATOD Use 19. ATOD Use About how often (if ever) do you Survey Section E E Item # E E Question Smoke cigarettes Drink alcohol not counting religious ceremonies Smoke marijuana Use cocaine or other hard drugs other than marijuana Sniff glue, paint, gas or other things to get high Smoke cigarettes Drink alcohol not counting religious ceremonies Smoke marijuana Use cocaine or other hard drugs other than marijuana Sniff glue, paint, gas or other things to get high Cigarettes Drink alcohol not counting religious ceremonies Marijuana Cocaine or other hard drugs Sniff glue, paint, gas or other things to get high Smoke cigarettes Drink alcohol not counting religious ceremonies Drink until you get drunk Smoke marijuana or hashish Smoke marijuana or hashish until you get stoned Sniff glue, paint, gas or other things to get high 1=None 2=Less than 50% 3=About 50% 4=More than 50% 5=All or almost all 1=None 2=Less than 50% 3=About 50% 4=More than 50% 5=All or almost all 1=Definitely not 2=Probably not 3=Maybe 4=Probably will 5=Definitely will Response Scale 1=Never 2=A few times but NOT in past year 3=A few times a year 4=Once a month 5=A few times a month 6=Once a week 7=A few times a week 8=Once a day 9=More than once a day 18