What Do Our Data Tell Us: Two Reports Examining Correla;ons in Utah Data

Transcription

1 What Do Our Data Tell Us: Two Reports Examining Correla;ons in Utah Data SUSAN LOVING, TRANSITION SPECIALIST UTAH STATE OFFICE OF EDUCATION 1

2 Disclaimer This presenta-on is designed to share models for: Looking for correla-on data across Indicators 1, 2, 13, and 14, and; Looking at mobility, dropout, and gradua-on rates across different disability categories These data are preliminary and should not be used in isola-on to reach defini-ve conclusions. The report was prepared under Contract ED- IES- 12- C from the U.S. Department of Educa-on, Ins-tute of Educa-on Sciences, by the Regional Educa-on Laboratory West, administered by West Ed. 2

3 In the beginning January, 2012, was the official start of the Utah Alliance for Dropout Preven-on for Students with Disabili-es Mee-ng outcomes included the decision to: Examine a dataset of a sample of 200 transi-on plans for high school students with disabili-es and inves-gate by exit status their postsecondary outcomes based upon Indicator 13; and Study Utah s special popula-on, grades 6/7-12 with a focus on final high school outcomes 3

4 Naviga;ng the Data Collec;on Waters Utah State Office of Educa-on (USOE) needed to agree to the concept MOUs needed to be developed and signed IRB approval from West Ed needed to be obtained Required for studies involving human subjects USOE data requests needed to be submi`ed and clarified and clarified New USOE data staff needed to be updated on the project Data needed to be sorted and cleaned and cleaned and cleaned 4

5 Transi;on Plans, Exit Status, and Postsecondary Outcomes 5

6 Research Ques;ons Study 1: What is the single- year dropout rate of SWD whose IEP transi-on plans were monitored, and does it vary by compliance status of their transi-on plans? Does the compliance rate of the monitored IEP transi-on plans vary by student and school characteris-cs? What are the gradua-on outcomes of SWD whose IEP transi-on plans were monitored, and do they vary by compliance status of their transi-on plans? What are the postsecondary outcomes one year aeer leaving school of SWD whose IEP transi-on plans were monitored, and do they vary by compliance status of their transi-on plans? 6

7 Data Collec;on IEP Transi;on Plans The review process that was part of the Utah Program Improvement Planning System (UPIPS) at the -me of study, was developed to ensure that each LEA is included in the formal monitoring process. The USOE used a stra-fied sampling strategy and the state s LEAs (districts and charter schools) were randomly assigned to one of five groups to be monitored during a five- year monitoring cycle. Based on selec-on criteria, LEAs were assigned to one of five years in the monitoring cycle. Data were collected from sample sets of IEP files during on- site visits in the second year of the monitoring cycle. Trained USOE staff or contract monitors reviewed files using a web- based compliance monitoring applica-on, using review ques-ons based on the NSTTAC Indicator 13 checklist. 7

8 USOE- SES provided two IEP file review data bases containing all of the items monitored to determine IEP file compliance with Indicator IEP files monitored in IEP files monitored in State Student Iden-fica-on numbers (SSIDs) were iden-fied for 335 of the 356 students whose IEP files were monitored. Other than the three LEAs with more than 50,000 students, different LEAs were sampled in each year. The smaller number of files reviewed in was due to the large number of newly- established elementary level charter schools included in that cohort. 8

9 Data Collec;on Gradua;on and Dropout Defini-ons: Graduate = any student who received a regular high school diploma Dropout = any student who dropped out or was expected to return the following school year, but did not. The USOE provide two student- level data extracts from the statewide longitudinal data system to track enrollment informa-on for and These data extracts included: student demographic characteris-cs, grade level, educa-onal environment, all enrollment records, exit codes, and high school final status codes. The USOE also provided a separate data file containing the SSIDs and student names. 9

10 Data Collec;on Postsecondary Outcomes The USOE (through its contractor) conducts a telephone survey between June and September in the year following the year of the students exit from school, a`emp-ng to contact each former student up to six -mes. Former students were those who exited school due to gradua-ng with a high school diploma, receiving a cer-ficate of comple-on, reaching maximum age for eligibility (aging out), or dropping out. Contact informa-on for former students is submi`ed by LEAs to the USOE data warehouse and obtained for the survey, from the data warehouse. The survey is designed to allow individuals other than the former student, e.g., other family members, to answer the survey if the former student is not available or can not provide the informa-on. 10

11 Study Limita;ons The analysis of compliance rates for Indicator 13 is limited by the small number of files monitored (358 files) during the and school years. No compliance rates are reported for popula-ons of less than 10 students. The post high survey had a low response rate (21% and 32% for and respec-vely), and only 23 respondents from both years were also students whose IEPs were monitored. 11

12 Does the compliance rate of monitored IEP files vary by student and school characteris;c? Student characteris-cs: Gender Missing for 18 students Race/ethnicity Missing for 38 students Demographic characteris-cs Asian, American Indian, and Pacific Islander excluded from analysis due to sample size of less than 10 Age Grade Missing for 1 student English learner Missing for 27 students Low socioeconomic status Missing for 26 students 12

13 13

14 Disability Classifica;on Disability classifica-on of students whose files were reviewed in during the study years: Specific Learning Disability 213 students Other Health Impairments 35 students Intellectual Disability 34 students Au-sm 21 students Mul-ple Disabili-es 18 students Emo-onal Disturbance 15 students All other disability classifica-ons were excluded from the analysis due to sample size of less than 10 14

15 15

16 School Characteris;cs School type were available for 353 of the 356 IEP files reviewed 316 students a`ended regular schools 37 students a`ended other school types School size Small schools Less than 600 students 144 files reviewed Medium size schools students 115 files reviewed Large schools More than 1200 students 94 files reviewed 16

17 17

18 18

19 What is the single- year dropout rate of students whose files were reviewed and does it vary by compliance status of their plans? The exit codes of students whose IEP files were reviewed. The USOE allocates an exit code for each student who leaves school before gradua-ng. The single- year dropout rate (the propor-on of students who dropout in a single year without comple-ng high school) was calculated by the number of students with an exit code in the dropout category ( dropout or unknown ) divided by the total number of students in the analysis. Exit code informa-on was available for: 114 students whose IEP files were compliant 216 students whose IEP files were non- compliant Exit code informa-on was not available for 28 students 19

20 20

21 What are the gradua;on outcomes of students whose plans were monitored and do they vary by compliance status of their plans? The USOE allocates a high school final status code for each grade 12 student at the end of the school year. Final high school comple-on status was available for: 36 students whose IEP files were compliant 86 students whose IEP files were non- compliant Grade 12 students who did not exit were considered con-nuously enrolled were considered con-nuously enrolled and grouped with retained seniors. 21

22 22

23 What are the postsecondary outcomes and do they vary by compliance status of their transi;on plans? During the two years of this study, 1,327 of 5,102 students with disabili-es completed the telephone survey for a 26% response rate. Among the 356 students with disabili-es whose IEPs were examined for Indicator 13 compliance in this analysis, 23 of these former students completed the survey. Given the small number of these former students whose IEPs were monitored and who completed the survey, any es-mate derived from that sample would have very low reliability and may not apply to the remainder of this sample, or to students with IEPs statewide. 23

24 Compliant Plans Among the 23 former students whose IEPs were monitored for Indicator 13 compliance and who responded to the survey, most (16 of 23 or 70%) did not have compliant transi-on plans. Only 7 (30%) of 23 former students had compliant transi-on plans. Nearly all (21 of 23 or 91%) former students whose IEPs were monitored for Indicator 13 compliance and who completed the post high school survey graduated. Among these graduates, 7 of 21 (33%) had compliant transi-on plans. Neither of the 2 students who did not graduate (1 dropped out, 1 par-cipated in the alternate assessment) had IEPs compliant with Indicator

25 Higher Educa;on About half (11 of 23) of the former students a`ended some kind of higher educa-on or postsecondary training. Of the 11 former students enrolled in postsecondary educa-on, 5 (45%) had compliant transi-on plans. Twelve of 23 former students were not a`ending some kind of higher educa-on or postsecondary training. Only 2 (17%) of these 12 former students had compliant transi-on plans 25

26 Employment More than half (14 of 23 or 61%) of the former students were employed; of these employed former students, 5 of 14 (36%) had compliant transi-on plans. Nine, or 39% of the former students were not working. Out of the 9 unemployed former students, 2 (22%) had a compliant transi-on plan. 26

27 Independent Living Some (6 of 23) of the former students whose IEPs were monitored for Indicator 13 compliance and who responded to the survey reported living independently from parents/guardians, with 5 former students living with a spouse or roommate and 1 residing in military housing. More than half (14 of 23) were living with parents/guardians, and 1 addi-onal former student lived with another family member. Two other former students had other unreported type of living arrangements. Among former students living 16 with parents/guardians, 4 of 14 (29%) had compliant transi-on plans, while 3 of the 6 (50%) former students living independently had compliant transi-on plans. 27

28 Support Services About half (11 of 23) of the former students received support services from agencies. Of the former students who received support services, 3 of 11 (27%) had compliant transi-on plans. Twelve of 23 (52%) former students gave no reports of having received support services during the year since exi-ng high school. Of these 12 former students not receiving support services, 4 (33%) had compliant transi-on plans. 28

29 School Mobility, Dropout, and Gradua;on Rates 29

30 Research Ques;on Study 2: What were the student and school characteris-cs of SWD in grades 6-12 during school year ? What were the number and types of school exits for SWD during school year ? Did they vary by disability category? What were the four- year cohort dropout and gradua-on rates for SWD in the 2011 gradua-ng class? Did the rates vary by disability category? 30

31 Data Collec;on This study used two sets of data from the Utah State Office of Educa-on: Enrollment data for 2010/11 students: Students and school characteris-cs, enrollment records, exit dates, and reasons for exi-ng for the full popula-on of more than 250,000 general educa-on students and nearly 35,000 students with disabili-es enrolled in Utah public schools in grades 6 12 for school year 2010/11. High school comple-on outcomes of the 2011 cohort Disability status, student IDEA disability categories, and the four- year dropout and gradua-on outcomes for the more than 41,000 students statewide that cons-tuted the 2011 cohort, including nearly 4,000 students with disabili-es. The 2011 cohort is defined as all students who started grade 9 for the first -me in 2007/08, adding students who subsequently transferred into the cohort, and subtrac-ng students who subsequently transferred out. The researchers used a unique Student State Iden-fica-on number to link all data sources. 31

32 Defini;ons Nonpromo-onal school exits School changes within the USOE (mobility rate), transfers out of the USOE system, and dropping out (single- year dropout rate for ) Cohort gradua-on and dropout rates Students who had started grade 9 in , plus students who had transferred to Utah schools through and were expected to graduate in 2011, minus any students who transferred out of USOE schools. Graduates: earned the state s basic high school diploma or adult educa-on secondary diploma within 4 years Other completers: GED cer-ficate; Cer-ficate of Comple-on; par-cipated in the Utah Alternate Assessment Con-nuing students: remained enrolled aeer their cohort had graduated; transferred to higher educa-on or Utah College of Applied Technology (UCAT) without receiving a high school diploma Dropouts: officially dropped out of school; withdrawn but not requested a transcript for reenrollment somewhere else; unknown; expelled without receiving a high school diploma 32

33 Defini;ons Students with disabili/es were grades 6 12 students who had an IEP on record and were eligible for special educa-on services in the Utah public schools. General educa/on students were grades 6 12 students who neither had an IEP on record nor were eligible for special educa-on services in the Utah public schools. Disability categories: Students with disabili-es were classified by one of 13 federally defined Individuals with Disabili-es Educa-on Act (IDEA) disability categories. In Utah and for this analysis, the categories hearing impairment and deafness are reported as one group. 33

34 Other demographic characteris;cs Student demographic characteris-cs: The source for the student demographic characteris-cs age, race/ethnicity, English language learner status, and low socioeconomic status was the USOE enrollment extract for 2010/11, and those variables represent student status as of October 1, Specifically: Age was computed based on the date of birth and corresponds to age as of October 1, 2010, English language learner status was determined by USOE s English language proficiency test, and Low socioeconomic status was determined by eligibility for the free and reduced- price lunch program. School characteris-cs: Grade level, school characteris-cs, and school and county of enrollment correspond to the school of enrollment as of October 1, In cases of concurrent enrollment as of October 1, 2010, the school used for the analysis was the one in which the student was enrolled in the longest during the school year. School type and educa/onal environment informa/on were obtained from the SCRAM extract for the school of enrollment as of October 1,

35 The majority of students with disabili;es were enrolled in schools in the coun;es nearest Salt Lake City. 35

36 Twelve percent of Utah students in grades 6 12 were classified with a disability. For school year 2010/11, 291,465 students were enrolled in grades 6 12 in Utah public schools. Of these students, 34,793 (11.9 percent) were classified as having disabili-es and were eligible to receive special educa-on services (figure 1). 36

37 Utah students with disabili;es had higher percentages of racial/ ethnic minority, male, English learner, and low- income students than their general educa;on classmates. English learners Race/ethnicity Gender Low socio- economi c status Male Female White Hispanic Asian Pacific Islander Black American Indian Mul-- race Yes No Yes No Percentage of students 100 Students with disabili-es General educa-on students 37

38 About 95% of students with disabili;es were enrolled in regular schools. Students with disabili-es 94.8 Speech or language impairment Specific learning disability Other health impairment Visual impairment/blindness Orthopedic impairment Trauma-c brain injury Emo-onal disturbance Au-sm Hearing impairment/deafness Intellectual disability Mul-ple disabili-es Deaf and blindness Percentage of students 38

39 Percentage of students by type of school enrollment Percentage of students Regular school Homebound/ hospitalized/ residen-al sesng Separate school Correc-onal facility Students with disabili5es Specific learning disability Other health impairment Intellectual disability Speech or language impairment Au5sm Emo5onal disturbance Mul5ple disabili5es Hearing impairment/deafness Trauma5c brain injury Visual impairment/blindness Orthopedic impairment Deaf- blindness

40 While the propor;on of general educa;on students decreased by grade level from grades 6 to 12, the propor;on of students with disabili;es in grade 12 was higher than in grade 11. General educa-on students Students with disabili-es Speech or language Visual impairment/ blindness Specific learning disability Orthopedic impairment Hearing impairment/ Other health impairment Au-sm Emo-onal disturbance Intellectual disability Mul-ple disabili-es Trauma-c brain injury Percentage of students Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 Grade 11 Grade 12 40

41 Percentage of ;me spent in general educa;on classes, by disability category, grades 6-12 Percentage of students by 5me spent in general educa5on classes At least 80% of the day 40% to 79% of the day Less than 40% of the day Students with disabili5es Specific learning disability Other health impairment Intellectual disability Speech or language impairment Au5sm Emo5onal disturbance Mul5ple disabili5es Hearing impairment/deafness Trauma5c brain injury Visual impairment/blindness Orthopedic impairment

42 Percentage of ;me spent in general educa;on classes, by disability category, grades 6-8 Percentage of students in grades 6-8 by 5me spent in general educa5on classes At least 80% of the day 40% to 79% of the day Less than 40% of the day Students with disabili5es Specific learning disability Other health impairment Intellectual disability Speech or language impairment Au5sm Emo5onal disturbance Mul5ple disabili5es Hearing impairment/deafness Trauma5c brain injury Visual impairment/blindness Orthopedic impairment

43 Percentage of ;me spent in general educa;on classes, by disability category, grades 9-12 Percentage of students in grades 9-12 by 5me spent in general educa5on classes At least 80% of the day 40% to 79% of the day Less than 40% of the day Students with disabili5es Specific learning disability Other health impairment Intellectual disability Speech or language impairment Au5sm Emo5onal disturbance Mul5ple disabili5es Hearing impairment/deafness Trauma5c brain injury Visual impairment/blindness Orthopedic impairment

44 Students with disabili;es had higher rates of reten;on, especially in grade 12, than general educa;on students Grade Percentage of students Students with disabili-es General educa-on students 44

45 The percentage of over- age in grade 12 varied by disability category. Students with disabili-es 49.9 Mul-ple disabili-es Intellectual disability Au-sm 64.2 Trauma-c brain injury 54.4 Other health impairment Speech or language impairment Emo-onal disturbance Hearing impairment/deafness Specific learning disability Percentage of students 45

46 Students with disabili;es exited school during the year at a higher rate than general educa;on students. General educa-on students Students with disabili-es Emo-onal disturbance Other health impairment Visual impairment/ blindness Specific learning disability Trauma-c brain injury Hearing impairment/ deafness Intellectual disability Au-sm Mul-ple disabili-es Speech or language Percentage of students with one or more exits 46

47 Students with disabili;es changed schools and dropped out at higher rates than their general educa;on classmates. Transfer out of the Utah public school system Dropout/ withdrawn School change within USOE General education students Students with disabilities Emotional disturbance Other health impairment Specific learning disability Traumatic brain injury Hearing impairment/deafness Intellectual disability Autism Speech or language impairment Multiple disabilities General education students Students with disabilities Emotional disturbance Other health impairment Specific learning disability Hearing impairment/deafness Intellectual disability Autism Speech or language impairment Multiple disabilities General education students Students with disabilities Emotional disturbance Other health impairment Specific learning disability Intellectual disability Autism Speech or language impairment Multiple disabilities <1.6* < Percentage of students

48 Students with disabili;es had a lower four- year gradua;on rate and a higher four- year dropout rate than general educa;on students. General education students Students with disabilities Hearing impairment/ deafness * Speech or language impairment * 20.6 Specific learning disability Other health impairment Autism Traumatic brain injury * 31.7 Emotional disturbance Intellectual disability Multiple disabilities Percentage of students Graduate Other completers Continuing students Dropout Note 4: (*) indicates that the disaggregated percent among non-graduates are not provided because of small cell sizes (N<10) in one 48 or several non-graduate categories

49 Summary This study documents, for the first -me, the single- year mobility and dropout rates, as well as the four- year cohort dropout and gradua-on rates, for students in Utah by disability categories. The study found that, as a group, students with disabili-es had poorer dropout and gradua-on outcomes than their general educa-on classmates. Specifically, they had: Higher dropout rates, Higher mobility rates during the school year, Higher rates of reten-on in grade 12, and Lower gradua-on rates during the standard four- year -me frame. The findings suggest that students with emo/onal disturbance, mul/ple disabili/es, intellectual disability, trauma/c brain injury, or au/sm had the lowest gradua-on rates and may benefit from interven-ons that address their specific learning needs.. 49

50 Students classified with a specific learning disability, by far the largest group of students with disabili-es in the state, graduated at a lower rate than their general educa-on classmates. Students classified with emo/onal disturbance, mul/ple disabili/es, intellectual disability, trauma/c brain injury, or au/sm were at greatest risk of failing to graduate during the standard four- year high school -me frame, with gradua-on rates lower than 50 percent. Students classified with emo/onal disturbance generally had poor outcomes. During the 2010/11 school year, they had the highest rate of exi-ng school by changing schools within USOE, transferring out of the USOE system, or dropping out. Aeer four years of high school, the most frequent final high school outcome for students in the 2011 cohort who were classified with emo/onal disturbance was dropping out. 50

51 Students classified with au/sm, mul/ple disabili/es, or intellectual disability had dropout rates lower than those of general educa-on students and students with disabili-es as a group. However, they also had low gradua-on rates and the highest reten-on rates aeer four years. Among students classified with an intellectual disability, the percentage of those who were retained aeer four years was nearly as high as the percentage of those who graduated. Students classified with mul/ple disabili/es had the smallest percentage of graduates among the disability categories, with a four- year gradua-on rate of 16.3 percent. The most common four- year outcome for this group was to remain in school as con-nuing students. Conversely, students classified with hearing impairment/deafness or speech or language impairment had four- year gradua-on rates roughly on par with the gradua-on rate for general educa-on students. In fact, students classified with speech or language impairment, a disability more oeen seen during the middle grades than in high school, had lower rates of changing schools or dropping out in 2010/11 than general educa-on students. 51

52 Further research is needed: Why do students with disabili-es drop out at a higher rate than their general educa-on classmates? What are specific reasons why students from some disability categories drop out at much higher rates than students in other categories? Does gender and racial/ethnic dispropor-onality in gradua-on and dropout rates for the general popula-on hold for the same extent for students with disabili-es in general or for specific disability categories? What happens to students with disabili-es, as a group and students in some disability categories in par-cular, who remain in school beyond the conven-onal four years? New data study is in place for to determine why students dropped out of school and then reenrolled ( serial dropouts ). 52

53 Low Points of Data Collec;on Varying levels of understanding of: How data were reported to the USOE Data needed Data defini-ons, e.g., exit codes Program knowledge vs sta-s-cal knowledge Change in the USOE Data and Accountability staff Sharing confiden-al data Monitoring data (Indicator 13) collected and housed on a separate server 53

54 High Points of Data Collec;on USOE administra-on approved the addi-onal work required by staff to prepare the reports Utah u-lizes a Statewide Student Iden-fier (SSID), making it easier to track individual students Student- level data are housed in one system, making it easier to generate reports USOE Special Educa-on Services and the Data and Accountability sec-ons work closely on a variety of projects and ac-vi-es 54

55 Ques-ons? Comments? Concerns? Susan Loving Transi-on Specialist Utah State Office of Educa-on 55