Table of Contents. 1. Background and Scope 1 1.1 What is Disproportionality? 2

On Measures of Racial/Ethnic Disproportionality in Special Education: An Analysis of Selected Measures, A Joint Measures Approach, And Significant Disproportionality Lalit Roy California Department of Education Sacramento, California June 2012

On Measures of Racial/Ethnic Disproportionality in Special Education: An Analysis of Selected Measures, A Joint Measures Approach, And Significant Disproportionality Lalit Roy [This paper was prepared for the California Department of Education, Special Education Division. Any opinions expressed in the paper are those of the author and they do not reflect any opinion, position or policy of the California Department of Education and no endorsement is implied.]

On Measures of Racial/Ethnic Disproportionality in Special Education: An Analysis of Selected Measures, A Joint Measures Approach, And Significant Disproportionality Executive Summary Federal law requires each state to examine racial/ethnic disproportionality in special education in all districts within the state and for the state as a whole on an annual basis. According to the State Performance Plan (SPP), disproportionality shall be examined for seven racial/ethnic groups of students: (1) Native American, (2) Asian, (3) Pacific Islander, (4) African-American, (5) Hispanic, (6) White, and (7) Multiple racial/ethnic groups. For each of these groups, disproportionality shall be examined: (1) in the overall special education program, (2) in six major disability categories, (3) in eight special education service delivery environments, and (4) in suspension and expulsion of students in special education. Selected results from disproportionality calculations are reported to the federal Office of Special Education Programs (OSEP) through the Annual Performance Report (APR) and are also released to the public. Seven commonly used disproportionality measures were analyzed to determine their strengths and weaknesses using enrollment of African-American students in the Intellectual Disability (ID) category in an actual district in California. The measures are: (1) Composition, (2) Relative Difference in Composition, (3) Risk, (4) Risk Ratio, (5) Weighted Risk Ratio, (6) Alternate Risk Ratio, and (7) the E-formula. Each measure was applied to the same district data for determining overrepresentation or underrepresentation of the seven racial/ethnic groups in the ID category. The results were quite different from one measure to another. The measures were tested to determine how well they address the following situations: (1) Effect on districts with different enrollment size; (2) Effect on small enrollments and their fluctuations; (3) Exclusion of groups from disproportionality calculations due to small cell size; (4) Region of Tolerance for Disproportionality; (5) Effect on districts that are racially/ethnically homogeneous and almost homogeneous ; and (6) Effect of the state incidence rate on districts. Again, the measures addressed these issues differently from each other. Finally, nine essential elements were identified to characterize a disproportionality measure. Each measure was judged against the others on the basis of how well the measures incorporate these elements. For each measure, these elements were rated on a five-point scale: five being the best and one the worst. The elements are: (1) Definition of the measure, (2) The calculation process, (3) Interpretation and usefulness of results, (4) Comparability of results among districts, (5) Effect on small enrollments and their fluctuations, (6) Exclusion of groups due to small cell size, (7) Differentiated Region of Tolerance for Disproportionality, (8) Effect on homogeneous and almost homogeneous districts, and (9) Effect of the state incidence rate on districts. Once again, the ratings of the elements were quite different from one measure to another, reflecting their relative strengths and weaknesses with regard to integrating these elements. The results of the rating process put the E-formula on top of the list with 3.4 points, followed by Alternate Risk Ratio (3.2), Risk Ratio (2.7), and Weighted Risk Ratio (2.1). Based on the rating results, the top two measures were applied individually to all districts in California to get an idea about the number of districts of various sizes that are likely to be disproportionate under various thresholds. The same two measures were also applied jointly to the same districts to preview the number and types of districts that are likely to be selected under various combinations of thresholds in the two measures. The results indicate that a joint measures approach, using the two top-rated measures, to determine disproportionality has some distinct advantages over using a single measure. A joint measures approach brings together the strengths of the individual measures and compensates each other for their weaknesses. Several definitions of significant disproportionality were examined using the concepts of frequency, severity, and persistency of disproportionality in a district. The strengths and weaknesses of these definitions were analyzed using various scenarios of disproportionality over a 15-year period. Three definitions of significant disproportionality under persistency appear to be more promising than the rest.

Table of Contents Executive Summary Preface Chapter Page 1. Background and Scope 1 1.1 What is Disproportionality? 2 2. Measures of Disproportionality 4 Composition 6 Relative Difference in Composition 7 Risk 8 Risk Ratio 9 Weighted Risk Ratio 10 Alternate Risk Ratio 12 The E-formula 13 Overrepresentation 14 Underrepresentation 16 2.1. Effect of the Measures on District Size 18 2.2. Effect on Small Enrollments and Their Fluctuations 21 2.3. Exclusion of Groups from Disproportionality Calculations 23 2.4. Region of Tolerance for Disproportionality 25 2.5. Effect of the Measures on Homogeneous and Almost Homogeneous Districts 27 2.6. Effect of the State Incidence Rate on Districts 30 2.7. Summary of Findings 33 Risk Ratio 33 Weighted Risk Ratio 34 Alternate Risk Ratio 34 The E-formula 35 2.8. Discussion of Results 36 3. Rating the Disproportionality Measures 39 4. Effect of the Top Two Measures on Districts 44 5. The Case for a Joint Measures Approach 50 5.1. Four Case Studies under the Joint Measures Approach 52 District A 54 District B 55 District C 55 District D 55 5.2. Recommendation 56 6. Significant Disproportionality 58 6.1. Frequency 58

6.2. Severity 59 6.3 Persistency 60 Definition P-1 62 Definition P-2 63 Definition P-3 64 Definition P-4 66 Definition P-5 66 Definition P-6 67 Definition P-7 70 Definition P-8 71 6.4. Discussion of Results from Various Definitions 73 6.5. Recommendation 74 7. A Final Note 76 Notes 78 Appendix A. Mathematical Expression of Risk Ratio 83 B. Mathematical Expression of Weighted Risk Ratio 84 C. Mathematical Expression of Alternate Risk Ratio 85 D. Alternate Table 18 86 References 87 Attachments A. Actual data from a School District in California: General Education (GE) Enrollment = 16,115; Intellectual Disability (ID) Enrollment = 171 88 B. Hypothetical Small School District: General Education (GE) Enrollment = 1,000; Intellectual Disability (ID) Enrollment = 10 91 C. Hypothetical Medium Sized School District: General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 94 D. Hypothetical Large School District: General Education (GE) Enrollment = 50,000; Intellectual Disability (ID) Enrollment = 500 97 E. Hypothetical Small School District Effect of One New Native American Student: General Education (GE) Enrollment = 1,001; Intellectual Disability (ID) Enrollment = 11 100 F. Hypothetical Medium Sized School District Effect of One New Native American Student: General Education (GE) Enrollment = 10,001; Intellectual Disability (ID) Enrollment = 101 103 G. Hypothetical Medium Sized School District Perfectly Homogeneous (100% Hispanic): General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 106 H. Hypothetical Medium Sized School District Almost Homogeneous (90% Hispanic): General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 109 I. Hypothetical Medium Sized School District Effect on High Incidence Rate in District: General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 200 112 About the Author 115

Preface This project started out as a brief position paper on the effectiveness of various measures of racial/ethnic disproportionality in special education. As the work progressed, several related issues surfaced that needed to be addressed in order to make the outcome of the project meaningful. In the end, the results of this effort turned out to be more like an investigative report than a simple position paper. Needless to say, the project took much longer to complete than was originally planned. The paper is intended primarily for the policy makers at the state and federal levels who often struggle with how best to define and measure racial/ethnic disproportionality in special education. The analysis of various measures in the paper would, hopefully, provide helpful information in making appropriate policy decisions in this respect. The program administrators at the local level, who will ultimately be responsible for implementing such policies, would also benefit from the analysis provided in the paper. Education professionals in the academic circles might find the analysis informative. The paper is comprised of three components: an analysis of selected disproportionality measures is presented in Chapters 1-3; a Joint Measures Approach to disproportionality is introduced in Chapters 4-5; and various definitions of significant disproportionality and their implications are discussed in Chapter 6. Several colleagues at the California Department of Education helped me define the focus and scope of the project. It was Chris Drouin who brought this topic to my attention and asked me to take an in-depth look into the various disproportionality measures used by the states. Along with Chris Drouin, Ben Traverso and Bruce Little reviewed earlier drafts of the paper and made good suggestions. Bernie Yaklin and Alexa Slater assisted me with developing the graphics. The paper benefited from input from some of my professional colleagues in other states and agencies: Sandra McQuain of West Virginia Department of Education, Inni Barone of New York Department of Education, Jean Taylor of Idaho Department of Education, and Cesar D Agord of the Western Regional Resource Center (WRRC) at the University of Oregon were kind enough to review an earlier draft of the paper and gave me their feedback. I am particularly grateful to Dr. Tom Munk of the Data Accountability Center (DAC) at Westat Corporation, Rockville, Maryland, for a thorough technical review of the document. He brought to my attention several errors and omissions in an earlier version of the document and made valuable suggestions clarifying a number of issues and statements in the paper. DAC (2007-2012) was funded by the U.S. Department of Education, Office of Special Education Programs (OSEP). My special thanks are due to Anita Salvo and Lisa Stie at the California Department of Education who took the time thoroughly reviewing the document. They did an excellent job editing the document and pointing out many errors and inconsistencies that had escaped my senses completely. I sincerely apologize to those friends and colleagues who provided feedback and assistance in various forms throughout the development of the paper, whose names I may have failed to mention here. The document reflects my synthesis of all input that I have received; any remaining errors and omissions are entirely mine. Finally, I must recognize my family for lending me their support, putting up with my preoccupation with the paper during family times, and for participating in sporadic discussions on the paper during many evenings and weekends. Lalit Roy Sacramento, California June 2012

On Measures of Racial/Ethnic Disproportionality in Special Education: An Analysis of Selected Measures, A Joint Measures Approach, And Significant Disproportionality

1. Background and Scope Racial/ethnic disproportionality in special education has become a national issue since the 1997 amendments of the Individuals with Disabilities Education Act (IDEA). In subsequent years, it became an indicator in the State Performance Plan (SPP), which requires the states to annually examine and monitor racial/ethnic disproportionality in special education for each district within the state as well as for the state as a whole (as a single entity). The monitoring process includes, among others, determination of racial/ethnic disproportionality and significant disproportionality in the following areas of special education: 1. Overall special education program 2. Major disability categories in special education 3. Special education service delivery environments (placement settings) 4. Suspension and expulsion of students in special education Selected results from disproportionality calculations are reported annually to the federal Office of Special Education Programs (OSEP) at the U.S. Department of Education through the Annual Performance Report (APR) and are released to the public. If a district has significant racial/ethnic disproportionality in any of the above areas of special education, the state must direct the district to use 15 percent of its IDEA funds to address the disproportionality issues. The magnitude of this requirement is enormous. The volume of calculations alone to determine racial/ethnic disproportionality for all districts in a state like California is incredibly large. To get an idea of the sheer size of this process, consider the following facts. Each year, a state collects, among others, enrollment data from all school districts in the state and reports them to the U.S. Department of Education in order to comply with the federal reporting requirements and to obtain federal funds. According to the federal regulations, each district must identify and report all students in general education and special education (students who receive special education and/or related services with an Individualized Education Program or IEP) in one of the following seven racial/ethnic groups: 1. Native American (American Indian or Alaskan Native) 2. Asian 3. Pacific Islander 4. Black or African-American (not Hispanic; the term Black is used by the Office for Civil Rights and the U.S. Department of Education for all data collections and reports at the federal level; both terms, Black and African-American, are used in the text interchangeably) 5. Hispanic 6. White (not Hispanic) 7. Multiple racial/ethnic group (more than one racial/ethnic background) For each of these racial/ethnic groups, disproportionality must be examined in the special education program as a whole (overall program) and in each of the following six major disability categories: 1. Autism (AUT) 2. Emotional Disturbance (ED) 1

3. Intellectual Disability (ID), previously known as Mental Retardation (MR) 4. Other Health Impairment (OHI) 5. Specific Learning Disability (SLD) 6. Speech and Language Impairment (SLI) Again, for each of the seven racial/ethnic groups, disproportionality calculations shall be carried out in each of the following eight special education service delivery environments (also known as placement settings or placement categories): 1. Correctional Facility 2. Homebound / Hospital Program 3. Parentally Placed Private School 4. Regular Education Class, 0-39 percent of school day 5. Regular Education Class, 40-79 percent of school day 6. Regular Education Class, 80-100 percent of school day 7. Residential Facility 8. Separate School Once again, for each of the seven racial/ethnic groups, disproportionality shall be examined for students in special education (students with an IEP), who are disciplined (suspended and/or expelled) for ten or more days during the school year. All of the above analyses are to be conducted for all school districts in the state (there are about 1,000 school districts in California) and for the state as a whole (as a single entity). The purpose of this paper is twofold: (1) to examine various measures of racial/ethnic disproportionality in special education that many states currently use in order to comply with the federal requirements, review their strengths and weaknesses, and provide necessary information so the user can make the best possible decision in selecting one or more measures to determine disproportionality; and (2) to explore different ways to define significant disproportionality in special education, evaluate their effectiveness in addressing various disproportionality scenarios, and recommend one or more such definitions for possible use. 1.1. What is Disproportionality? In general, disproportionality may be defined as a situation when two or more proportions are not the same or are not within an agreed upon range of values. If two proportions are the same or are within an agreed upon range of values, then it is implied that there is no disproportionality between the two proportions. If, on the other hand, the two proportions are not the same or are outside the agreed upon range of values, then the proportions are considered disproportionate. There are two broad categories of definitions of racial/ethnic disproportionality in special education, commonly known as Composition and Risk. These two categories are different from each other as to how the proportion of a racial/ethnic group in special education (the statistic) is calculated and how it is compared against the proportion of the comparison group. In Composition, the proportion of all special education students (or all students in a subcategory of special education, such as a disability category or a placement setting) in a particular racial/ethnic group in a district is compared against the proportion of all general education students in the same 2

racial/ethnic group in the district. The underlying assumption in this category of definitions is that the proportion of all general education students in a racial/ethnic group in the district is the benchmark, norm, standard or socially acceptable proportion. 1 Under the broad category of Risk, the proportion of all general education students in a racial/ethnic group who are enrolled in special education (or in a subcategory of special education, such as a disability category or placement setting) in a district is compared against the proportion of general education students in all other racial/ethnic groups combined who are enrolled in special education (or in the same subcategory of special education) in the district. 2 This definition assumes that the proportion of the comparison group (all other racial/ethnic groups combined) is the benchmark, norm, standard or socially acceptable proportion. Since the comparison group is composed of all other racial/ethnic groups, except the racial/ethnic group in question, the composition of the comparison group is not the same for each racial/ethnic group. Therefore, the benchmark, norm, standard or social acceptability of the proportion of the comparison group is always different for each racial/ethnic group. 3 3

2. Measures of Disproportionality Under the two broad categories, Composition and Risk, there are several measures to determine racial/ethnic disproportionality in special education that are currently used by the states or have been used in the past. These measures are intended to produce two types of results: overrepresentation and underrepresentation. 4 These terms are defined below. Note that, unless otherwise specified, the term special education in this paper includes any subpopulation in special education, such as students in a disability category or in a service delivery environment (placement setting). As stated in the last chapter, racial/ethnic disproportionality under the broad category of Composition is defined by the difference between the proportion of all special education (SE) students in a racial/ethnic group and the proportion of all general education (GE) students in the same racial/ethnic group. In this category of measures, overrepresentation is defined as when the proportion of the racial/ethnic group is more in special education than in general education. Underrepresentation occurs when the proportion is less in special education than in general education. Three commonly used measures of disproportionality fall under Composition: (1) Composition, by itself, (2) Relative Difference in Composition, and (3) the E-formula. Also stated in the last chapter, racial/ethnic disproportionality under the broad category of Risk is determined by comparing the risk of one racial/ethnic group to be in special education against the corresponding risk of all other racial/ethnic groups combined. Overrepresentation occurs when the risk of a racial/ethnic group is higher than that of the comparison group. Underrepresentation happens when the risk of a racial/ethnic group is lower than that of the comparison group. Four commonly used measures of disproportionality fall under Risk: (1) Risk, by itself, (2) Risk Ratio, (3) Weighted Risk Ratio, and (4) Alternate Risk Ratio. (Actually, Weighted Risk Ratio is a hybrid measure; it combines district risk with statewide composition, discussed later in the paper.) In all measures, any disproportionality or discrepancy would be considered significant when the disproportionality (overrepresentation or underrepresentation) crosses a threshold set by state policy or if it meets the definition of significant disproportionality adopted by the state, discussed later in the paper in detail. To assist states in monitoring racial/ethnic disproportionality in special education, the OSEP and Westat Corporation (a private consulting firm under contract with OSEP) convened a task force to address this issue. The OSEP/Westat Task Force developed a document, Methods for Assessing Racial/Ethnic Disproportionality in Special Education: A Technical Assistance Guide, which lists a number of measures to calculate racial/ethnic disproportionality in special education and discusses their strengths and limitations. The document, however, does not include all disproportionality measures that are currently used by the states, such as the E-formula that has been in use in California since the 1970 s and subsequently in other states. This paper analyzes some of the most commonly used measures of racial/ethnic disproportionality that are currently used by the states. Each measure is discussed individually, illustrated with actual data from a school district, and is followed by a discussion of its strengths and limitations. Following individual presentations, the measures are compared against each other using a set of hypothetical district data of various sizes (small, medium, and large) to examine how the measures affect 4

different sized districts. Next, the measures are tested for their impact on a number of situations such as, their effect on enrollment fluctuations, effect on districts that are racially/ethnically homogeneous and almost homogeneous, and so forth. Following that, the measures are tested for their sensitivity to high or low incidence rates in the district, compared to the state incidence rates. Finally, the measures are rated on a rating scale using a set of criteria that are critical to any procedure to determine disproportionality. The measures presented in this paper are: Composition Relative Difference in Composition Risk Risk Ratio Weighted Risk Ratio Alternate Risk Ratio The E-formula There may be other measures used by the states that are not included in the above list. One such measure that we decided not to include in this analysis was recommended by OSEP several years ago. According to this measure, which falls under the broad category of Composition, a state is allowed to set a percentage threshold above and below a district s general education enrollment percentage of a racial/ethnic group. If the actual percentage of the same racial/ethnic group in special education or in a disability category or in a special education service delivery environment is beyond that threshold, then the group is considered overrepresented or underrepresented, depending on the direction of the threshold. California Department of Education (CDE) used this approach in the past with a 20 percent threshold for overrepresentation and a 40 percent threshold for underrepresentation. Soon it became evident that the results of this measure were rather flat and they did not provide the necessary flexibility to address varying enrollments or impacts of enrollment fluctuations in different sized districts. Eventually, the measure lost its appeal to the states and its support at the federal level. Several new and more sophisticated measures emerged in recent years, although not without limitations of their own, which are currently supported by OSEP and included in this paper. 5 Because of the volume of data and the large number of calculations involved in determining disproportionality in all possible combinations of racial/ethnic groups, disability categories, and special education service delivery environments, this paper will limit the analysis to only one racial/ethnic group in one disability category. The paper does not address disproportionality in special education programs as a whole (overall program) or in special education service delivery environments (placement categories) or in suspension and expulsion (discipline) in special education. Any issues arising from disproportionality in a disability category should give the reader some idea about similar issues in the other three situations as well. The focus of the analysis is kept at the district level only; it does not address state-level disproportionality issues. Also, the reporting requirements, monitoring of district policies and procedures, and fiscal implications are beyond the scope of this paper. To the extent appropriate, each measure is described below in the format of a question it attempts to answer. This is followed by a definition of the measure resulting in a statistic that answers the question. For the sake of simplicity and unless otherwise indicated, we have used African-American or Black students as the racial/ethnic group and Intellectual Disability (ID) as the disability category 5

in special education in describing each measure. However, data on all racial/ethnic groups are shown in the attachments at the end of the paper. 6 Composition Composition is a simple way to look into the racial/ethnic background of students in special education. It is the percentage distribution of all racial/ethnic groups who are enrolled in special education or identified in a disability category or who receive services in a special education service delivery environment. Composition attempts to answer a question like this: Question: Measure: What percentage of all students in a district receiving special education and related services under the identification of the ID category is Black or African-American? [(Number of Black or African-American students in the ID category) / (Total number of students in all racial/ethnic groups in the ID category)] * 100 Actual data from a district are shown in Table 1, illustrating the racial/ethnic composition of students in the ID category. The detailed calculations are shown in Attachment A. Table 1. Racial/Ethnic Composition of Enrollment in the ID Category Native Asian Pacific Black Hispanic White Multiple Total ID Enrollment (N) 0 26 1 81 34 25 4 171 Composition (%) 0.00 15.20 0.58 47.37 19.88 14.62 2.34 100.00 Source: Attachment A. Table 1 shows that the Composition of Black or African-American students in the ID category is 47.37 percent and the Composition of White students, for example, is 14.62 percent. These numbers, by themselves, do not provide any information on racial/ethnic disproportionality, and therefore, are not very useful. However, when they are compared against the composition of African-American and White students in general education (GE) enrollment (or total enrollment 7 ) in the same district, they reveal a discrepancy (see Table 2). Table 2. Racial/Ethnic Composition of Enrollment in General Education (GE) and in the ID Category Native Asian Pacific Black Hispanic White Multiple Total GE Enrollment (N) 106 3,404 324 5,013 5,210 1,674 384 16,115 GE Composition (%) 0.66 21.12 2.01 31.11 32.33 10.39 2.38 100.00 ID Enrollment (N) 0 26 1 81 34 25 4 171 ID Composition (%) 0.00 15.20 0.58 47.37 19.88 14.62 2.34 100.00 Source: Attachment A. 6

The composition of Black or African-American students in general education in the same district is 31.11 percent, compared to 47.37 percent in the ID category, which shows that there are proportionately more African-American students identified in the ID category than are in general education in the district. If the percentage is higher in special education than in general education, then the racial/ethnic group is overrepresented and if the percentage is lower then it is underrepresented. In this case, African-American students are overrepresented in the ID category. By comparison, the composition of white students in the ID category is 14.62 percent, compared to 10.39 percent in general education still overrepresented, but by a smaller discrepancy (4.23 percentage points) than African-American students (16.26 percentage points). Relative Difference in Composition Relative Difference in Composition for a racial/ethnic group is the difference between its special education composition and general education composition, expressed as a percentage of its general education composition. This measure allows comparing disproportionality of various racial/ethnic groups against each other. This is an improvement over using Composition by itself or even the difference of compositions between special education and general education, but still not very useful, as we shall see later. Relative Difference in Composition attempts to answer a question like this: Question: Measure: What is the difference between the compositions of African-American students in the ID category and in general education (GE) in terms of percentage of the composition of African-American students in general education? [((Composition (%) of African-American students in the ID category) (Composition (%) of African-American students in general education)) / (Composition (%) of African-American students in general education)] * 100 The results of the Relative Difference in Composition for various racial/ethnic groups are shown in Table 3. Table 3. Relative Difference in Composition between General Education (GE) and the ID Category Native Asian Pacific Black Hispanic White Multiple Total GE Enrollment (N) 106 3,404 324 5,013 5,210 1,674 384 16,115 GE Composition (%) 0.66 21.12 2.01 31.11 32.33 10.39 2.38 100.00 ID Enrollment (N) 0 26 1 81 34 25 4 171 ID Composition (%) 0.00 15.20 0.58 47.37 19.88 14.62 2.34 100.00 Difference (%) -0.66-5.92-1.43 16.26-12.45 4.23-0.04 NA Relative Difference (%) -100.00-28.02-70.91 52.27-38.50 40.74-1.83 NA Source: Attachment A. NA = Not Applicable; Bold = Overrepresentation; Bold and Italics = Underrepresentation 7

The Relative Difference in Composition for Black or African-American students in the ID category is 52.27 percent, which means that the composition of African-American students identified in the ID category is 52.27 percent more than the composition of African-American students in general education. By comparison, the composition of White students identified in the ID category is 40.74 percent more than the composition of White students in general education. On the other hand, the composition of Asian students identified in the ID category is 28.02 percent less than the composition of Asian students in general education. Note that a positive Relative Difference in Composition means overrepresentation and a negative Relative Difference in Composition means underrepresentation. The interpretation of Relative Difference in Composition is not quite intuitive. The statistic, being the percentage of a percentage, makes it a little difficult to comprehend easily. Also, one needs to exercise caution in interpreting the statistic resulting from small numbers in the numerator, such as in Native American and Pacific Islander groups. For example, the Relative Difference in Composition for Native American students is -100.00 percent because there are no Native American students in the ID category at all, no matter how large the Native American enrollment in general education is in the district. Pacific Islanders have only one student identified in the ID category. One additional Native American or Pacific Islander student in the ID category could alter the statistic considerably (this is discussed in detail in a later section in the paper). Risk Like Composition, Risk is also a relatively simple approach to examine ethnic disproportionality in special education. As defined before, it is the percentage of students in a racial/ethnic group who are enrolled in special education or in any subcategory of special education. It is also described as the risk of a racial/ethnic group of being (or to be) in special education. 8 The relative risk can be determined by comparing the risk of one racial/ethnic group against that of another. In special education, Risk refers to the percentage of all general education students in a racial/ethnic group who are enrolled in special education and related services or in a disability category or in a special education service delivery environment. Risk attempts to answer a question like this: Question: Measure: What percentage of all African-American students in a district is receiving special education and related services under the identification of the ID category? [(Number of African-American students in the ID category) / (Total number of African- American students in general education)] * 100 A generalized expression of this measure is given by: R ed = (DSE ed / DGE e ) (100.00) Where: R ed = Risk of racial/ethnic group e in disability category d in a district DSE ed = District special education enrollment of racial/ethnic group e in disability category d DGE e = District general education enrollment of racial/ethnic group e 8

The Risk for various racial/ethnic groups to be identified in the ID category is shown in Table 4. Table 4. Risk for Various Racial/Ethnic Groups in General Education (GE) to be in the ID Category Native Asian Pacific Black Hispanic White Multiple Total GE Enrollment (N) 106 3,404 324 5,013 5,210 1,674 384 16,115 ID Enrollment (N) 0 26 1 81 34 25 4 171 ID Risk (%) 0.00 0.76 0.31 1.62 0.65 1.49 1.04 1.06 Source: Attachment A. In the district in Table 4, the Risk for Black or African-American students identified in the ID category is 1.62 percent, which is higher than 1.49 percent for White students and 1.06 percent for all students. These values provide some idea about the relative risk of various racial/ethnic groups to be identified in the ID category in relation to another group. 9 As stated in the last section, one needs to exercise caution in interpreting Risk as well when the numerator is small or zero. In this case, the Risk for Native American students is 0.00 percent or none. This is because the Native American enrollment in the ID category in the district is zero, no matter how large the Native American enrollment in general education is in the district. Risk Ratio Risk for a racial/ethnic group, by itself, does not provide sufficient information about racial/ethnic disproportionality unless it is compared against the Risk of a comparison group, when it is known as the Risk Ratio. Most often, the comparison group is comprised of all other racial/ethnic groups combined, not including the racial/ethnic group in question. Risk Ratio attempts to answer a question like this: Question: Measure: What is the Risk for African-American students receiving special education and related services in the ID category in a district, compared to the Risk for all other students receiving special education and related services in the ID category in the same district? [(Risk (%) of African-American students in the ID category) / (Risk (%) of all other students combined in the ID category)] A generalized mathematical version of this definition is shown in Appendix A. The results of the Risk Ratio for various racial/ethnic groups are shown in Table 5. The data in Table 5 show that, African-American students are 1.99 times at risk (or as likely) to be identified in the ID category as all other racial/ethnic groups combined to be identified in the ID category. Asian students are 0.67 times as likely to be identified in the ID category as all other groups combined to be identified in the ID category. The value of the Risk Ratio more than 1.00 for a racial/ethnic group means higher risk (overrepresentation) and less than 1.00 means lower risk 9

(underrepresentation) than the comparison group. In this case, African-American and White students are overrepresented; Asian, Pacific Islander, Hispanic, and Multiple racial/ethnic groups are underrepresented in the ID category. Table 5. Risk Ratio for Various Racial/Ethnic Groups in the ID Category Native Asian Pacific Black Hispanic White Multiple Total GE Enrollment (N) 106 3,404 324 5,013 5,210 1,674 384 16,115 ID Enrollment (N) 0 26 1 81 34 25 4 171 ID Risk (%) 0.00 0.76 0.31 1.62 0.65 1.49 1.04 1.06 Others ID Risk (%)* 1.07 1.14 1.08 0.81 1.26 1.01 1.06 NA ID Risk Ratio 0.00 0.67 0.29 1.99 0.52 1.48 0.98 NA Source: Attachment A. GE = General Education; NA = Not Applicable; Bold = Overrepresentation; Bold and Italics = Underrepresentation * Others ID Risk (%) refers to the risk for all other racial/ethnic groups in the district combined to be in the ID category. The interpretation of Risk Ratio is rather straight forward, and therefore, the usefulness of the results is also quite clear. Because the Risk Ratio results for a district are based on data from that district only, they can be used for making any changes in policy, program or practices in that district. For example, if the district has a procedure in place that results in inappropriate identification of a large number of African-American students in the ID category, compared to other racial/ethnic groups in the ID category, then a change in such procedure would be warranted by the Risk Ratio results. Any such changes in policy, program or procedure can also be implemented in the district because the district board has the full authority in implementing these changes, which, hopefully, could yield desired results. Like the preceding measures, one should exercise caution in the interpretation and use of Risk Ratio resulting from small numbers. Weighted Risk Ratio While Risk and Risk Ratio calculations use district level data, Weighted Risk Ratio takes into account the racial/ethnic composition of all students in the state in the calculations. Since Risk and Risk Ratio measures for a district use data from that district only, their results are not comparable with other districts results. To enable the results comparable across all districts in the state, the OSEP/Westat Task Force recommends Weighted Risk Ratio as another approach to examine racial/ethnic disproportionality. In this approach, the risk for each racial/ethnic group in a district is weighted by the racial/ethnic composition of the state. 10 The computational process of the Weighted Risk Ratio is far from simple, as described in the Task Force Report, and is stated below. Weighted Risk Ratio attempts to answer a question like this: Question: What is the risk for African-American students receiving special education and related services in the ID category in a district, compared to the risk for all other students in the district in the ID category when both risks (of African-American 10

students and of all other racial/ethnic groups combined) are weighted according to the racial/ethnic composition of the state? 11 Measure: [(District-level risk for African-American students in the ID category, weighted by the composition of all other racial/ethnic groups combined in the state) / (Sum of risks of all other racial/ethnic groups in the ID category in the district, each individually weighted by the composition of the same racial/ethnic group in the state)] A generalized mathematical version of this definition is shown in Appendix B. In operational terms, the measure is defined as: [{District African-American ID Risk * (1 State African-American Composition)} / {(District Native American ID Risk * State Native American Composition) + (District Asian ID Risk * State Asian Composition) + (District Pacific Islander ID Risk * State Pacific Islander Composition) + (District Hispanic ID Risk * State Hispanic Composition) + (District White ID Risk * State White Composition) + (District Multiple Racial/Ethnic Group ID Risk * State Multiple Racial/Ethnic Group Composition)}] [Note: All terms (Risk and Composition) in the above measure are fractions not percentages.] The results of the Weighted Risk Ratio for various racial/ethnic groups are shown in Table 6. Table 6. Weighted Risk Ratio of Various Racial/Ethnic Groups in the ID Category Native Asian Pacific Black Hispanic White Multiple Total District GE Enrollment (N) 106 3,404 324 5,013 5,210 1,674 384 16,115 District ID Enrollment (N) 0 26 1 81 34 25 4 171 District ID Risk* 0.0000 0.0076 0.0031 0.0162 0.0065 0.0149 0.0104 0.0106 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 State GE Composition* 0.0074 0.1124 0.0061 0.0698 0.5131 0.2753 0.0159 1.0000 Weighted ID Risk* 0.0000 0.0068 0.0031 0.0150 0.0032 0.0108 0.0103 NA Weighted Others ID Risk** 0.0096 0.0088 0.0096 0.0085 0.0063 0.0055 0.0095 NA ID Weighted Risk Ratio 0.00 0.77 0.32 1.77 0.51 1.96 1.08 NA Source: Attachment A. GE = General Education; NA = Not Applicable; Bold = Overrepresentation; Bold and Italics = Underrepresentation * Risk and Composition in this table are fractions not percentages. ** Weighted Others ID Risk refers to the sum of the risks of all other racial/ethnic groups in the district, each individually weighted by the statewide composition of the same racial/ethnic group. The interpretation of Weighted Risk Ratio is neither intuitive nor clear. In the example in Table 6, Black or African-American students are 1.77 times as likely to be identified in the ID category (overrepresentation) as all other racial/ethnic groups combined to be identified in the ID category when both risks are weighted by the racial/ethnic composition of the state. Similarly, Asian students are 0.77 times as likely to be identified in the ID category (underrepresentation) as all other racial/ethnic groups combined to be identified in the ID category when both risks are weighted by the racial/ethnic composition of the state. What does this mean? 11

The Weighted Risk Ratio imposes statewide racial/ethnic composition onto the district risk in order to make the ratios comparable across districts within the state. Although one could debate about the appropriateness of inter-district comparisons because of variations in local district policies and practices, the same reason could also be an impetus for comparing racial/ethnic disproportionality across the districts in a state. Results from the Weighted Risk Ratio can be quite different from the Risk Ratio results. By weighing local district risk with the statewide racial/ethnic composition, the district risk is affected by the relative magnitudes of the composition of various racial/ethnic groups in the state. Because of the complexity of the Weighted Risk Ratio calculations, it is not easy to understand how and in what direction the district risk is affected by the statewide composition. In the case of African-American students, the Weighted Risk Ratio (1.77) is less than their Risk Ratio (1.99, Table 5); whereas, for White students the Weighted Risk Ratio (1.96) is higher than their Risk Ratio result (1.48, Table 5). It might be possible that a particular racial/ethnic group is overrepresented in one measure and underrepresented in another. Because the interpretation of Weighted Risk Ratio is not clear, it is difficult to figure out how the results from Weighted Risk Ratio can be used by a district to make any program or policy changes. Since the district results are influenced by the statewide demographics and the district does not have any authority over the other districts that comprise the state or over the state itself, the usefulness of the results of Weighted Risk Ratio is somewhat limited. Like Risk and Risk Ratio, one should use caution in the interpretation and use of the Weighted Risk Ratio results from small numbers. Small variations in the number of students in a racial/ethnic group or the comparison group can produce dramatic changes in the Weighted Risk Ratio results as well, discussed later in the paper. Alternate Risk Ratio In situations when a district is racially/ethnically homogeneous or almost homogeneous and/or a comparison group is not available, Alternate Risk Ratio offers an option for examining racial/ethnic disproportionality. 12 In Alternate Risk Ratio, the district risk for a racial/ethnic group is compared against the risk for all other racial/ethnic groups in the state. This approach is similar to Weighted Risk Ratio in concept, but the calculation methodology is far simpler than the Weighted Risk Ratio and the interpretation of the statistic is also relatively clear. Like Weighted Risk Ratio, the results of the Alternate Risk Ratio for districts can also be compared against each other within the state. Alternate Risk Ratio attempts to answer a question like this: Question: Measure: What is the risk for African-American students receiving special education and related services in the ID category in a district, compared to the risk for all other students receiving special education and related services in the ID category in the state? [(District-level Risk (%) of African-American students in the ID category) / (State-level Risk (%) of all other students combined in the ID category)] A generalized mathematical expression of this definition is shown in Appendix C. The results of the Alternate Risk Ratio calculations for various racial/ethnic groups are shown in Table 7. 12

Table 7. Alternate Risk Ratio of Various Racial/Ethnic Groups in the ID Category Native Asian Pacific Black Hispanic White Multiple Total District GE Enrollment (N) 106 3,404 324 5,013 5,210 1,674 384 16,115 District ID Enrollment (N) 0 26 1 81 34 25 4 171 District ID Risk (%) 0.00 0.76 0.31 1.62 0.65 1.49 1.04 1.06 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 State ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 State ID Risk (%) 0.60 0.46 0.58 1.02 0.69 0.55 0.46 0.64 State Others ID Risk (%)* 0.64 0.67 0.64 0.62 0.59 0.68 0.65 NA ID Alternate Risk Ratio 0.00 1.14 0.48 2.62 1.10 2.19 1.61 NA Source: Attachment A. GE = General Education; NA = Not Applicable; Bold = Overrepresentation; Bold and Italics = Underrepresentation * State Others ID Risk (%) refers to the risk of all other racial/ethnic groups in the state combined to be in the ID category. Interpretation of results of the Alternate Risk Ratio is relatively simple, compared to the Weighted Risk Ratio. Black or African-American students in this district are 2.62 times as likely to be identified in the ID category (overrepresentation) as all other students in the state combined to be identified in the ID category. By contrast, Pacific Islander students are 0.48 times as likely to be identified in the ID category (underrepresentation) as all other students in the state combined to be identified in the ID category. Although both Weighted Risk Ratio and Alternate Risk Ratio use statewide demographics in the comparison groups, the methodological differences in the two measures are evident in the differences in their results. In the same district, Asian and Hispanic students are underrepresented in Risk Ratio (0.67 and 0.52, Table 5) and in Weighted Risk Ratio (0.77 and 0.51, Table 6), but are overrepresented in Alternate Risk Ratio (1.14 and 1.10, Table 7). The interpretation of Alternate Risk Ratio is relatively simple. However, it does not hold promise that any changes made in district policies and procedures, based on the results of the Alternate Risk Ratio, without making the necessary changes in the rest of the districts in the state would produce the desired outcome; any such effort can produce, at best, only partial results. Therefore, the usefulness of the results of Alternate Risk Ratio, like the Weighted Risk Ratio, is also somewhat limited. Alternate Risk Ratio is also subject to the properties and limitations of small numbers, as mentioned in the previous risk measures. The E-formula The E-formula first appeared in 1974 in a court order resulting from a class action lawsuit (Diana, et al. v. State Board of Education, et al.) in the U.S. District Court in Northern California. The plaintiffs in the case successfully argued that there were proportionately more Chicano students in EMR (Educable Mentally Retarded) classes than in general education in the districts in a county in Northern California. After listening to lengthy hearings, arguments, and counter-arguments from both parties and opinions from experts in the field, the presiding judge ordered the state (California 13

Department of Education) to monitor districts showing significant variance between the percentage of Chicano children in EMR classes and the percentage of Chicano children in the school population by one standard deviation, which subsequently became known as the E-formula. The E-formula also became the measure to determine disproportionate placement of Black children in EMR classes in the settlement agreement under the Larry P. vs. Riles lawsuit in California, which started in the 1970 s and continued through the 1980 s and early 1990 s. Similar to the Diana lawsuit, the plaintiffs in the Larry P. case also alleged, among others, that the number of young African-American students identified as Educable Mentally Retarded (EMR) and placed in Special Day Class (SDC) setting for special education services was disproportionately higher than in the general education program in the district. As part of the settlement agreement of the Larry P. lawsuit, the same presiding judge ordered the California Department of Education (CDE) to monitor disproportionate placement of African- American students identified as EMR in SDC placement setting, using the E-formula. 13 Following that order, California Department of Education has monitored for many years overrepresentation of African-American students who were identified as EMR and placed in an SDC setting. Overrepresentation Neither the EMR disability category nor the SDC placement setting exists today in California. However, the E-formula has been found to be an effective measure to determine racial/ethnic disproportionality in special education. This is because the underlying statistical properties of the E- formula make the measure robust, allowing flexibility for different sized districts. The intent of the original E-formula was to determine overrepresentation only. The E-formula is defined as: E = A + ((A (100-A)) / N) Where: E = A = N = Maximum percentage of the total special education enrollment (or special education enrollment in a disability category or service delivery environment) in a district allowed for a specific racial/ethnic group Percentage of the same racial/ethnic group in general education in the district The total special education enrollment (or special education enrollment in the same disability category or service delivery environment) in the district, as defined in E. In the E-formula, special education enrollment can be viewed as a sample drawn from a population of general education enrollment. Programmatically (and statistically), this is a valid assumption because all special education students are also general education students in the same district. In statistical terms, the second component in the E-formula, ((A (100-A)) / N), is comparable to standard error of the sampling distribution of the proportion of a racial/ethnic group in special education (the estimate). The formula represents a sampling event or a draw when the proportion of a racial/ethnic group in the sample (special education) is identical to the proportion of the same group in the population (general education). The original E-formula allows or adds one standard error to the general education proportion of a racial/ethnic group as a threshold to determine 14

overrepresentation, which establishes an upper bound of tolerance or maximum allowance, beyond which the proportion of the racial/ethnic group in special education (or special education enrollment in a disability category or service delivery environment) is considered disproportionate (overrepresented). Since we are no longer restricted to use the E-formula solely in its original context and/or prohibited from modifying it for use in other situations, we are introducing three variations of the E-formula by adding two, three and four standard errors to the general education proportion that will allow us to examine the effect of the formula with different thresholds while retaining the statistical properties of the original E-formula. 14 The original E-formula and the three variations are: The E-formula: E 1 = A + 1 ( ((A (100-A)) / N)) (one standard error) Variation: E 2 = A + 2 ( ((A (100-A)) / N)) (two standard errors) Variation: E 3 = A + 3 ( ((A (100-A)) / N)) (three standard errors) Variation: E 4 = A + 4 ( ((A (100-A)) / N)) (four standard errors) Table 8 shows the results of the original E-formula and the three variations in overrepresentation of various racial/ethnic groups identified in the ID category. Table 8. Overrepresentation of Various Racial/Ethnic Groups in the ID Category under the E-formula Native Asian Pacific Black Hispanic White Multiple Total District GE Enrollment (N) 106 3,404 324 5,013 5,210 1,674 384 16,115 A = District GE Composition (%) 0.66 21.12 2.01 31.11 32.33 10.39 2.38 100.00 N = District ID Enrollment (N)* 0 26 1 81 34 25 4 171 Standard Error (%) 0.62 3.12 1.07 3.54 3.58 2.33 1.17 NA District ID Composition (%) 0.00 15.20 0.58 47.37 19.88 14.62 2.34 100.00 Maximum Percent of ID Allowed at: E 1 = A + One Standard Error (%) 1.28 24.24 3.08 34.65 35.91 12.72 3.55 NA E 2 = A + Two Standard Errors (%) 1.89 27.37 4.16 38.19 39.48 15.05 4.72 NA E 3 = A + Three Standard Errors (%) 2.51 30.49 5.23 41.73 43.06 17.39 5.88 NA E 4 = A + Four Standard Errors (%) 3.13 33.61 6.30 45.27 46.64 19.72 7.05 NA Source: Attachment A. GE = General Education; NA = Not Applicable; Bold = Overrepresentation * The E-formula calculations use the total district enrollment in the ID category - not the enrollments in the individual racial/ethnic groups in the ID category, which are shaded for differentiation. In this example, African-American students constitute 31.11 percent of general education enrollment in the district but 47.37 percent in the ID category. Under the original E-formula (at one standard error threshold) the allowed maximum for them not to be overrepresented in the ID category is 34.65 percent. Since the actual percentage of African-American students identified in the ID category is 47.37 percent, which is higher than the allowed maximum, they are overrepresented. 15

They are also overrepresented at the two, three, and four standard errors thresholds. However, they are not overrepresented at five standard errors and beyond, because the threshold at five standard errors is 48.81 percent, which is higher than the actual percentage of African-American students in the ID category in the district (data not shown in table but can be derived from Attachment A). By comparison, White students are also overrepresented at the one standard error threshold because the actual percentage of White students identified in the ID category (14.62) is higher than the maximum percentage allowed (12.72). But unlike African-American students, White students are not overrepresented at two standard errors or higher thresholds. Asian students, on the other hand, at 15.20 percent of the total number of students identified in the ID category in the district, are well below their threshold of overrepresentation under the original E- formula (24.24 percent maximum), and therefore, are not overrepresented. In fact, none of the other ethnic groups, except African-American and White, are overrepresented at any of the four thresholds. Note that lack of overrepresentation does not imply underrepresentation. The interpretation of E-formula results for a district is relatively simple and the results can be used to make program or policy changes in that district. Any such changes can also be implemented in the district because the district board has the full authority in implementing these changes, which, hopefully, could yield desired results. Underrepresentation The mathematical expression for underrepresentation in the E-formula is quite similar to the original formula for overrepresentation, except that the connector between the first and the second component is a minus (-) sign, instead of a plus (+) sign. This establishes a lower bound of tolerance or minimum needs from the percentage of a racial/ethnic group in general education below which the proportion of the racial/ethnic group in special education (or special education enrollment in a disability category or service delivery environment) is considered disproportionate (underrepresented). The E-formula for underrepresentation can be shown as: E = A - ((A (100-A)) / N) Where: E = A = N = Minimum percentage of the total special education enrollment (or special education enrollment in a disability category or service delivery environment) in a district needed for a specific racial/ethnic group Percentage of the same racial/ethnic group in general education in the district The total special education enrollment (or special education enrollment in the same disability category or service delivery environment) in the district, as defined in E. Like overrepresentation, we are also introducing three variations in the E-formula by adding two, three and four standard errors to the general education proportion to examine the effects of various thresholds for underrepresentation. Because the original E-formula was intended for overrepresentation only, all versions of the E-formula for underrepresentation are considered variations of the original E-formula. The variations in underrepresentation are: 16

Variation: E 1 = A - 1 ( ((A (100-A)) / N)) (one standard error) Variation: E 2 = A - 2 ( ((A (100-A)) / N)) (two standard errors) Variation: E 3 = A - 3 ( ((A (100-A)) / N)) (three standard errors) Variation: E 4 = A - 4 ( ((A (100-A)) / N)) (four standard errors) Table 9 shows the results of the E-formula variations for underrepresentation of various racial/ethnic groups in the ID category. Table 9. Underrepresentation of Various Racial/Ethnic Groups in the ID Category Under the E-formula Native Asian Pacific Black Hispanic White Multiple Total District GE Enrollment (N) 106 3,404 324 5,013 5,210 1,674 384 16,115 A = District GE Composition (%) 0.66 21.12 2.01 31.11 32.33 10.39 2.38 100.00 N = District ID Enrollment (N)* 0 26 1 81 34 25 4 171 Standard Error (%) 0.62 3.12 1.07 3.54 3.58 2.33 1.17 NA District ID Composition (%) 0.00 15.20 0.58 47.37 19.88 14.62 2.34 100.00 Minimum Percent of ID Needed at: E -1 = A - One Standard Error (%) 0.04 18.00 0.94 27.57 28.75 8.05 1.22 NA E -2 = A - Two Standard Errors (%) -0.58 14.88-0.14 24.03 25.18 5.72 0.05 NA E -3 = A - Three Standard Errors (%) -1.20 11.76-1.21 20.49 21.60 3.39-1.12 NA E -4 = A - Four Standard Errors (%) -1.81 8.64-2.28 16.95 18.02 1.06-2.28 NA Source: Attachment A. GE = General Education; NA = Not Applicable; Bold and Italics = Underrepresentation * The E-formula calculations use the total district enrollment in the ID category - not the enrollments in the individual racial/ethnic groups in the ID category, which are shaded for differentiation. The table shows that Asian students constitute 21.12 percent of general education enrollment in the district but 15.20 percent in the ID category. At one standard error threshold the needed minimum for Asian students not to be underrepresented in the ID category is 18.00 percent. Since the actual percentage of Asian students in the ID category is 15.20, which is less than the needed minimum, they are underrepresented. However, the Asian students are not underrepresented at two standard errors or higher thresholds. Black or African-American students, on the other hand, at 47.37 percent of the total number of students identified in the ID category in the district, are well above the threshold for underrepresentation at one standard error (27.57 percent minimum), and therefore, are not underrepresented. Except for African-American, White, and the Multiple racial/ethnic group, all other racial/ethnic groups are underrepresented at the one standard error threshold. As the threshold increases to higher standard errors, these groups gradually move away from the underrepresentation domain in 17

varying degrees, depending on their enrollment size. Note that lack of underrepresentation does not imply overrepresentation. Unlike the risk measures, the E-formula and its variations do not appear to be affected by small numbers, at least not to the same degree. For Native American students, whose enrollment in the ID category is zero in the district, all E-formula values in Table 8 and Table 9, including the negative values 15 in Table 9, show reasonable limits for overrepresentation and underrepresentation. Effects of small numbers in all measures are discussed later in the paper in further details. 2.1. Effect of the Measures on District Size Having an understanding of how each disproportionality measure works, we would like to examine how the measures affect districts of different enrollment sizes. To investigate this, we have created three hypothetical districts with general education enrollments of 1,000 (small), 10,000 (medium), and 50,000 (large), and applied all disproportionality measures to these districts. So we can determine the effect of the measures on district size only and nothing else, all affecting variables are held constant across these districts for each racial/ethnic group. The composition of African- American students is held at 10 percent in general education (GE) and at 20.00 percent in the ID category in all three districts. The risk in the ID category is held at 1.00 percent for all students and at 2.00 percent for African-American students for each district. These constants are different for different racial/ethnic groups, but are the same for each group across the districts. The detailed calculations for this exercise are shown in Attachments B, C, and D for small, medium, and large districts, respectively. Table 10 summarizes the results of this comparison for African-American students in the ID category. As we have seen from the discussion of individual measures in the preceding pages, three of the measures, Composition, Relative Difference in Composition, and Risk, do not provide sufficient information to determine racial/ethnic disproportionality, compared to the other four measures. 16 Therefore, we will refrain from discussing these three measures any further, although the results of the comparison of all seven measures are shown in the attachments. The remaining four measures, Risk Ratio, Weighted Risk Ratio, Alternate Risk Ratio, and the E-formula, offer sufficient promise in determining ethnic disproportionality, and they are also used by various states. The subsequent analysis in the paper will mainly focus on these four measures in comparing their effects on district size and in other situations. To provide a comparative picture of the three hypothetical districts against an actual district, Table 10 includes the actual district data that have been used throughout the paper in describing the measures. The data for this district are drawn from Attachment A, and are logically placed in the table according to the district size and shaded for differentiation from the hypothetical districts. Table 10 shows that the results of all risk ratio measures (Risk Ratio, Weighted Risk Ratio or Alternate Risk Ratio) are the same for all three hypothetical districts regardless of their size differences. The Risk Ratio, for example, is 2.25 for each district: small, medium, and large. This means that under any of the risk ratio measures, African-American students have the same risk of being identified in the ID category in a small district as in a medium or large district. This situation is somewhat like a flat tax rate, when all individuals are taxed at the same rate regardless of their levels of income. As we know, statistically as well as from our experience in the field, a small district is more vulnerable to overrepresentation or underrepresentation resulting from small enrollment fluctuations than a large district. 18

Table 10. Effect of Various Disproportionality Measures on Different Sized Districts District Size Small Medium Actual District Large Data Source Attachment B Attachment C Attachment A Attachment D Total GE Enrollment (N) 1,000 10,000 16,115 50,000 Total ID Enrollment (N) 10 100 171 500 African-American Students GE Composition (%) 10.00 10.00 31.11 10.00 ID Composition (%) 20.00 20.00 47.37 20.00 ID Risk (%) 2.00 2.00 1.62 2.00 Risk Ratio Measures: Risk Ratio 2.25 2.25 1.99 2.25 Weighted Risk Ratio 2.60 2.60 1.77 2.60 Alternate Risk Ratio 3.25 3.25 2.62 3.25 E-formula (Overrepresentation : Maximum Percent of ID Allowed at): One Standard Error (%) 19.49 13.00 34.65 11.34 Two Standard Errors (%) 28.97 16.00 38.19 12.68 Three Standard Errors (%) 38.46 19.00 41.73 14.02 Four Standard Errors (%) 47.95 22.00 45.27 15.37 E-formula (Underrepresentation : Minimum Percent of ID Needed at): Asian Students One Standard Error (%) 0.51 7.00 27.57 8.66 Two Standard Errors (%) -8.97 4.00 24.03 7.32 Three Standard Errors (%) -18.46 1.00 20.49 5.98 Four Standard Errors (%) -27.95-2.00 16.95 4.63 GE Composition (%) 15.00 15.00 21.12 15.00 ID Composition (%) 10.00 10.00 15.20 10.00 E-formula (Underrepresentation : Minimum Percent of ID Needed at): One Standard Error (%) 3.71 11.43 18.00 13.40 Two Standard Errors (%) -7.58 7.86 14.88 11.81 Three Standard Errors (%) -18.87 4.29 11.76 10.21 Four Standard Errors (%) -30.17 0.72 8.64 8.61 GE = General Education; ID = Intellectual Disability Category Bold = Overrepresentation; Bold and Italics = Underrepresentation The E-formula values, on the other hand, show a contrasting picture. The results are different for different sized districts. In the overrepresentation calculation, the E-formula value at one standard error threshold is the highest for the small district (19.49 percent), and less for the medium sized 19

district (13.00 percent), and less further for the large district (11.34 percent), even though the proportions of African-American students in general education (10.00 percent) and in the ID category (20.00 percent) in all three districts are the same. The same property holds true for all thresholds across the districts of various sizes, which demonstrates that the E-formula allows proportionately more flexibility to smaller districts than larger districts. Allowing proportionately different flexibility for different sized districts in determining racial/ethnic disproportionality is a unique feature of the E-formula, which sets itself distinctively apart from the risk ratio measures. The denominator N (the total number of students in the ID category in the district) in the second component of the E-formula, ((A (100-A)) / N), is critical in this feature. As we know, in divisions with positive numbers, the smaller the denominator, the larger is the quotient and the larger the denominator, the smaller is the quotient. 17 The value of N, the denominator, is generally small for smaller districts, thus produces a relatively large E-formula value; whereas, the relatively large value of N for larger districts results in a smaller E-formula value. As the district size increases, the value of N generally increases and the proportionate value of the E-formula gradually decreases, resulting in lesser and lesser flexibility or proportionately less tolerance for disproportionality for larger districts. Similarly, when the district size decreases, the value of N generally decreases as well and the proportionate value of the E-formula gradually increases, resulting in more and more flexibility or proportionately more tolerance for disproportionality for smaller districts. The gradation of change in the E-formula values for different sized districts is almost continuous, i.e., the E-formula values for districts would be different from one another even if the enrollment size of one district is different from another by a single student. All E-formula values for underrepresentation also show similar properties of proportionately more flexibility for smaller district and proportionately less flexibility for larger districts. For example, the needed minimum percentage of the total ID enrollment for African-American students at one standard error threshold is the lowest for the small district (0.51 percent), but higher for the medium sized district (7.00 percent) and higher further for the large district (8.66 percent), which demonstrates proportionately less restriction (or more flexibility) for small districts and more restriction (or less flexibility) for large districts. The same property holds true for the other E-formula thresholds as well. It is important to note that in overrepresentation under the E-formula the maximum percentage of the total ID enrollment allowed is higher for small districts and lower for large districts, but in underrepresentation the minimum percentage of the total ID enrollment needed is lower for small districts and higher for large districts. Although it is obvious, none of the three hypothetical districts show underrepresentation of African-American students in the ID category under the E-formula and its variations because a racial/ethnic group cannot be both overrepresented and underrepresented at the same time in the same program category. To illustrate underrepresentation in the E-formula, we have added data for Asian students for all four districts in the table. Similar to the principle for African-American students, the composition of Asian students in general education (GE) is held constant at 15.00 percent and in the ID category at 10.00 percent in all three hypothetical districts. The risk for all students in the ID category remains the same (1.00 percent) and for Asian students in the ID category is 0.67 percent in all three districts (see Attachments B, C, and D). Table 10 shows that at one standard error threshold, Asian students are not underrepresented in the small district, but they are underrepresented in the medium and large districts. When the threshold is lowered (or extended) to two standard errors, Asian students are no longer 20

underrepresented in the medium district, but they are still underrepresented in the large district. The large district continues to remain underrepresented at three standard errors but finally escapes underrepresentation at the four standard errors threshold. The various thresholds reflect different flexibility by varying percentage points depending on the size of the district. 2.2. Effect on Small Enrollments and Their Fluctuations Small numbers of students in a disability category and/or in a racial/ethnic group do not appear to have any differential effect under any of the risk ratio measures, compared to large numbers. According to the data in Table 10, the number of African-American students in the ID category is only 10 for the small district, but not so small for the medium district (100) and quite large for the large district (500); yet, the results of all risk ratio measures are exactly the same for all districts. By contrast, the same districts under the same conditions, show differential effects in the E-formula. Small enrollment fluctuations in special education can produce dramatic changes in the results of disproportionality calculations. These changes are also affected by the size of the numerator and the denominator in the calculations. To examine the effects of small enrollment fluctuations in various measures of racial/ethnic disproportionality, we have created the following scenario: a Native American family with one disabled student diagnosed with Intellectual Disability moves into our hypothetical small district from another district within the state; another Native American family, also with one disabled student with Intellectual Disability, moves into our hypothetical medium sized district from another district within the state. The enrollments in general education and in the ID category go up by one student in these two districts but the state enrollments in general education and in the ID category remain the same. Table 11 shows the results of this exercise. For the small district, the results show dramatic changes in all risk ratio measures when the Native American enrollment in the ID category increased from one to two. The Risk Ratio, for example, went up from 2.67 to 5.20 and the Weighted Risk Ratio increased from 2.96 to 5.78, almost double the original value. Similar changes occurred in the results of the Alternate Risk Ratio as well. The E-formula values do not show such dramatic changes, however. For the small district, the E- formula value under one standard error threshold changed from 10.20 to 10.07, considerably small compared to the changes in the risk ratio measures; the higher thresholds also show a similar picture. The changes in all E-formula values appear to be more reflective of the marginal change in the ID enrollment (by one student) than those in the risk ratio measures. For the medium sized district, the changes in the results of all risk ratio measures are not as dramatic as in the small district. This is due to the difference in the size of the ID enrollment (the numerator) in the two districts (one student in the small district but 10 in the medium sized district) as well as the difference in the size of their general education enrollments, the denominator (40 and 400, respectively). 18 The smaller the numbers, the bigger is the change; the bigger the numbers, the smaller is the change. When the two numbers (the numerator and the denominator) are considerably large, the result is far less dramatic than in the examples in the table, and the interpretation of the results is meaningful. The changes in the E-formula values for the medium sized district barely show any effect of the change in the ID enrollment by one student. This is because of the inherent statistical properties of the E-formula and the effect of the district size in the calculations. 21

Table 11. Effect of Small Changes in the Native American ID Enrollment in Small and Medium Districts Racial/Ethnic Group Native American Native American District Size Small District Medium District Data Source Attachment B Attachment E Attachment C Attachment F Change From To Difference From To Difference GE Enrollment (N) 40 41 1 400 401 1 ID Enrollment (N) 1 2 1 10 11 1 GE Composition (%) 4.00 4.10 0.10 4.00 4.01 0.01 ID Composition (%) 10.00 18.18 8.18 10.00 10.89 0.89 ID Risk (%) 2.50 4.88 2.38 2.50 2.74 0.24 Risk Ratio Measures: Risk Ratio 2.67 5.20 2.53 2.67 2.93 0.26 Weighted Risk Ratio 2.96 5.78 2.82 2.96 3.25 0.29 Alternate Risk Ratio 3.88 7.57 3.69 3.88 4.26 0.38 E-formula (Overrepresentation Maximum Percent of ID Allowed at): One Standard Error (%) 10.20 10.07-0.13 5.96 5.96 0.00 Two Standard Errors (%) 16.39 16.05-0.34 7.92 7.91-0.01 Three Standard Errors (%) 22.59 22.02-0.57 9.88 9.87-0.01 Four Standard Errors (%) 28.79 28.00-0.79 11.84 11.82-0.02 E-formula (Underrepresentation Minimum Percent of ID Needed at): One Standard Error (%) -2.20-1.88 0.32 2.04 2.06 0.02 Two Standard Errors (%) -8.39-7.86 0.53 0.08 0.11 0.03 Three Standard Errors (%) -14.59-13.83 0.76-1.88-1.85 0.03 Four Standard Errors (%) -20.79-19.81 0.98-3.84-3.80 0.04 Bold = Overrepresentation; Bold and Italics = Underrepresentation GE = General Education; ID = Intellectual Disability category If Risk Ratio was the disproportionality measure of choice and the threshold for overrepresentation was set at 3.00 or 4.00 or 5.00, then one new Native American student would make the small district overrepresented but not the medium sized district. A similar situation would occur in the Weighted Risk Ratio with overrepresentation threshold set at 4.00 or 5.00 and in the Alternate Risk Ratio with overrepresentation threshold set at 5.00, 6.00 or 7.00. In the E-formula, one new Native American student would make the small district overrepresented at one and two standard errors thresholds, but not at three standard errors or higher. The disproportionality status of the medium sized district under the E-formula, however, does not appear to be noticeably affected by one new Native American student. Note that the thresholds in the risk ratio measures and in the E-formula do not equate with each other, which is discussed later in the paper. 22

It is important to note that one additional Native American student in the ID category changed the racial/ethnic composition of the two districts, and as a result, all risk ratio values for both districts went up, indicating a higher risk for Native American students in the ID category in the newly reconfigured districts. The E-formula values, however, went down for overrepresentation and went up for underrepresentation in the reconfigured districts. Because the newly reconfigured districts became slightly larger than before, the maximum percentage of Native American students allowed in the ID category (tolerance for overrepresentation) was reduced by a small amount, as reflected in the lower E-formula values. Similarly, for underrepresentation, the minimum percentage of Native American students needed in the ID category (tolerance for underrepresentation) was increased by a small amount, as reflected in the higher E-formula values under various thresholds. The increase in enrollment by one student made the two districts slightly larger than their previous demographic configurations, and the E-formula automatically adjusted the flexibility for these two districts accordingly. Regardless of the magnitudes of changes in the results within each measure, any disproportionality calculations with small numbers and any changes in the small numbers (numerator and/or denominator) should be treated with caution. A small change in enrollment for a racial/ethnic group can not only shift the composition and risk for that particular group, as shown in Table 11, it can also alter the demographic landscape for the entire district, resulting in changes in the disproportionality status of other racial/ethnic groups in the district, even though the student populations in other racial/ethnic groups remain unchanged (see Attachment B and Attachment E). In addition, interpretation of results based on small numbers still remains problematic in any disproportionality measure, especially for all risk ratio measures. 2.3. Exclusion of Groups from Disproportionality Calculations Because of the difficulties associated with small enrollments and their fluctuations, many states exclude disproportionality calculations for a racial/ethnic group when enrollment of that group in general education and/or in special education or in a program subcategory (a cell) is small, such as 10 or 20. States vary considerably from one another in their practices of excluding disproportionality calculations based on cell size. The OSEP/Westat Task Force Report recommends excluding disproportionality calculations when the district general education enrollment in a racial/ethnic group (the denominator) is less than 10. The Report does not address the enrollment size in special education or in a program subcategory for a racial/ethnic group (the numerator). In other words, the Task Force Report defines a cell as the denominator only. But as we have observed, the numerator is equally important as the denominator in any disproportionality calculations, particularly when both the numerator and the denominator are small. Another reason for exclusion due to small cell size, although not often cited, is that if a decision is to be made based on the probability distribution of a statistic (Risk Ratio, for example), a small cell size might not be sufficient to approximate the actual probability distribution of that statistic. Since the nature of the probability distribution of the statistic from individual measures has not been explored, the results from various disproportionality measures are often treated as stand-alone, outside their probabilistic realm. None of the risk ratio measures or the E-formula attaches itself to any probability distribution, although the E-formula comes close in terms of its statistical properties but does not label itself as such. 19 If we exclude disproportionality calculations for cells (numerator or denominator) with numbers less than 10, then for many districts it will amount to a large number of exclusions, and for some districts 23

perhaps entirely, in any of the risk ratio measures. Such exclusions would be considerably fewer under the E-formula because the E-formula does not use the same input variables as the risk ratio measures. Consider the district in Table 12 showing enrollments in general education and in the six major disability categories in special education for each racial/ethnic group. Table 12. Input Variables in the Risk Ratio and the E-formula Measures in a District Native Asian Pacific Black Hispanic White Multiple Total General Education Enrollment (N) 106 3,404 324 5,013 5,210 1,674 384 16,115 Autism (AUT) 0 42 0 26 17 14 9 108 Emotional Disturbance (ED) 1 3 1 86 23 25 6 145 Intellectual Disability (ID) 0 26 1 81 34 25 4 171 Other Health Impairment (OHI) 0 7 1 32 17 10 4 71 Specific Learning Disability (SLD) 6 46 6 368 213 81 39 759 Speech and Language Impair. (SLI) 2 53 4 127 105 44 19 354 Using the data from Table 12, the following input variables are needed to calculate the risk ratio measures for African-American students in the ID category. 1. Number of Black or African-American students in general education in the district (5,013) 2. Total number of other students (not African-American) in general education in the district (16,115 5,013 = 11,102) 3. Number of African-American students in the ID category in the district (81) 4. Total number of other students (not African-American) in the ID category in the district (171-81 = 90) In addition, Weighted Risk Ratio uses statewide racial/ethnic composition and Alternate Risk Ratio uses statewide enrollment data for the comparison group, both of which are generally derived from large numbers. In the E-formula calculations for African-American students in the ID category, the following numbers are used: 1. Number of African-American students in general education in the district (5,013) 2. Total number of (general education) students in the district (16,115) 3. Total number of students in the ID category in the district (171) Note that the number of African-American students in the ID category (81) is used in the risk ratio calculations but not in the E-formula calculations; instead, the total number of students in the ID category in the district (171) is used in the E-formula. Because the total number of students (in all racial/ethnic groups) in a disability category in a district is almost always higher than the individual number of students in any single racial/ethnic group in that disability category, the number of exclusions of cases from disproportionality calculations would almost always be fewer in the E- formula than in the risk ratio measures. 20 24

In examining racial/ethnic disproportionality in the six major disability categories in the above district, calculations based on cell size (numerator and denominator) 10 or more would mean excluding 18 cells (out of 42) from disproportionality calculations in all three risk ratio measures. Calculations based on cell size 20 or more would exclude 23 out of 42 cells. Native American and Pacific Islander students would be excluded entirely in all disability categories under both cell size criteria (10 or 20). In the E-formula calculations, none of the racial/ethnic groups in any disability categories would be excluded under either exclusion criterion because none of the input variables (general education enrollments in each racial/ethnic group and the district total in each disability category) are less than 10 or 20. If a district of this size (16,115 students), which is more than 2.5 times larger than the average school district in California (the average district size is about 6,000 students), ends up with such a large number of cells for exclusions, imagine what would happen to a small district in a similar situation. In states such as California, where there are a large number of small districts, crosstabulation of racial/ethnic group by disability category would result in many cells with enrollments less than 10 or 20. Depending upon the exclusion rule, many of these cells would be excluded from disproportionality calculations under all risk ratio measures but would not be under the E-formula. Not only could this happen to most small districts and many medium sized school districts, but to some large districts as well. Exclusion from disproportionality calculations, for one reason or another, provides the district an escape option from state oversight on this issue, which is clearly not the intent of the federal law. The number of exclusions due to small cell size will be fewer when the enrollment threshold is small (such as less than 10) than when the threshold is large (such as less than 20). Although it will be impossible to eliminate exclusion of cells from disproportionality calculations entirely on the basis of small cell size, they can certainly be minimized under the E-formula. 2.4. Region of Tolerance for Disproportionality In disproportionality calculations a Region of Tolerance for Disproportionality can be defined as a range or space around the point of non-disproportionality within which the value of a measure is considered not disproportionate (a neutral region). This is a necessary element in disproportionality calculations which allows districts to operate their programs within a so-called wiggle room. Without a Region of Tolerance for Disproportionality, non-disproportionality would be simply a point of existence. It would be like standing on a point or walking on a very thin line of nondisproportionality without having any room even for chance variations. In its purest form, any deviation, however small, from the point or the line of non-disproportionality would put a district in the territory of overrepresentation or underrepresentation. Any disproportionality measure without a Region of Tolerance for Disproportionality can be compared to driving a car on a road without any shoulder, or driving in a lane that is exactly as wide as the car itself. As one can imagine, it would be virtually impossible to run a special education program where all racial/ethnic groups are perfectly proportionate between general education and special education and/or among each other. This means that the results of all risk ratio measures for each racial/ethnic group in any special education program category would have to be a perfect 1.00 and the percentages of all racial/ethnic groups in special education and in various program categories must be exactly the same as the percentages of the same racial/ethnic groups in general education. 25

The size or area of the Region of Tolerance for Disproportionality is critical to districts of various enrollment sizes. Small districts that are vulnerable to small fluctuations in enrollments would need proportionately a large Region of Tolerance for Disproportionality, so they do not cross the threshold of overrepresentation or underrepresentation when a family with one or two disabled children moves into or out of the district. Large districts probably would not need the same degree of flexibility as the small districts because they can absorb the effect of such fluctuations in enrollment without showing any noticeable difference. A conceptual depiction of this feature is shown in Chart 1. Region of Tolerance for Disproportionality Percent Below Percent Above < - - - - - - - - - - - - - - - - - - - - > Overrepresentation Points/Line of Non-disproportionality Underrepresentation < - - Smaller - - - - - - - - District Size - - - - - - - - Larger - - > Chart 1. Region of Tolerance for Disproportionality in Special Education As we have seen, any changes in enrollment in one racial/ethnic group also affect the results of disproportionality calculations in other groups. The magnitude of such effects depends on individual measures and the size of the districts. For example, one new Native American student in the medium sized district affected the Risk Ratio for students in the Multiple racial/ethnic group, which is reduced from 1.00 (Attachment C) to 0.99 (Attachment F), even though the number of students in the Multiple racial/ethnic group remained unchanged. Without a Region of Tolerance for Disproportionality, the district would be underrepresented for students in the Multiple racial/ethnic group, resulting from changes in one Native American student. For reasons such as this, and possibly others, Region of Tolerance for Disproportionality is a critical element in disproportionality calculations. None of the risk ratio measures have a built-in Region of Tolerance for Disproportionality in their calculations. If the value of a risk ratio measure is exactly 1.00, the district is not disproportionate. But if the value is more than 1.00 by any amount, the district is overrepresented, and if it is less than 1.00 it is underrepresented. No district can run its programs in such a rigid condition. Because of this reason, thresholds are rarely set at 1.00 in any risk ratio measures; they are generally set at higher levels such as 2.00, 3.00, 4.00, and so forth. But a higher threshold does not necessarily establish a varying region of tolerance for different sized districts; they are simply a cut-off point or bar beyond which a district, regardless of its size, is considered disproportionate, and therefore, they are still flat. 26

To allow a reasonable operating environment for special education programs, all risk ratio measures would require a Region of Tolerance for Disproportionality to be added externally through policy decisions or based on empirical data, and they would have to be different for different sized districts as well. The thresholds would also have to be different for different types of risk ratio measures because the marginal changes in the results of the risk ratio measures are different from each other (see Table 11). The E-formula has a built-in Region of Tolerance for Disproportionality, and therefore, it is transparent to the user. The size of this Region is proportionately larger for smaller districts and proportionately smaller for larger districts. It does not need to be set externally as for the risk ratio measures. In the original E-formula, the Region of Tolerance for Disproportionality for overrepresentation was set by the court order at one standard error above the percentage of a racial/ethnic group in general education in the district. But as we have discussed earlier, this Region of Tolerance can be set at other levels of standard errors as well, such as 2.0, 3.0, 3.5, and so forth. Since the value of the second component of the E-formula (the standard error) varies inversely as the size of a district s special education enrollment, the Region of Tolerance for Disproportionality proportionally decreases as the district size increases and vice versa. 2.5. Effect of the Measures on Homogeneous and Almost Homogeneous Districts In the preceding pages, the focus of all disproportionality measures has been on districts that have a racially/ethnically heterogeneous student population. In a large state, such as California, and perhaps in many other states as well, there are many districts that are not racially/ethnically heterogeneous, even though California is probably the most diverse state in the nation. Therefore, it is important to examine how a measure affects a district that is not racially/ethnically heterogeneous so we do not need to create another rule to avoid any unwanted outcome. When a district is comprised entirely of a single racial/ethnic group, it is known as a homogeneous district. In such a district, for this racial/ethnic group, a comparison racial/ethnic group simply would not exist for calculating the Risk Ratio. If a district is almost homogeneous, the comparison group for some racial/ethnic groups may be too small to calculate the Risk Ratio. The OSEP/Westat Task Force Report recommends using Alternate Risk Ratio when a comparison group either does not exist or the size of the group is too small. It is not clear in the Report if the recommendation is to use one measure for districts that are racially/ethnically heterogeneous and another measure for districts that are homogeneous or almost homogeneous. Using different measures of disproportionality for different districts in the state would not be appropriate or fair because the results would be different under different measures for the same district. To examine this issue, we have created two variations of racial/ethnic homogeneity in our hypothetical medium sized district with the following enrollment compositions. Homogeneous: Hispanic 100 percent Almost homogeneous: Hispanic 90 percent African-American 3 percent White 7 percent 27

We then applied all disproportionality measures to these two districts and summarized the results of this analysis in Tables 13 and 14. The detailed calculations are shown in Attachments G and H. Table 13. Effect of Various Disproportionality Measures on Racially/Ethnically Homogeneous District Native Asian Pacific Black Hispanic White Multiple Total GE Enrollment (N) 0 0 0 0 10,000 0 0 10,000 GE Composition (%) 0.00 0.00 0.00 0.00 100.00 0.00 0.00 100.00 ID Enrollment (N) 0 0 0 0 100 0 0 100 ID Composition (%) 0.00 0.00 0.00 0.00 100.00 0.00 0.00 100.00 ID Risk (%) ** ** ** ** 1.00 ** ** 1.00 Risk Ratio Measures: Risk Ratio ** ** ** ** ** ** ** NA Weighted Risk Ratio ** ** ** ** ** ** ** NA Alternate Risk Ratio ** ** ** ** 1.69 ** ** NA E-Formula (Overrepresentation Maximum Percent of ID Allowed at): One Standard Error (%) 0.00 0.00 0.00 0.00 100.00 0.00 0.00 NA Two Standard Errors (%) 0.00 0.00 0.00 0.00 100.00 0.00 0.00 NA Three Standard Errors (%) 0.00 0.00 0.00 0.00 100.00 0.00 0.00 NA Four Standard Errors (%) 0.00 0.00 0.00 0.00 100.00 0.00 0.00 NA E-Formula (Underrepresentation Minimum Percent of ID Needed at): One Standard Error (%) 0.00 0.00 0.00 0.00 100.00 0.00 0.00 NA Two Standard Errors (%) 0.00 0.00 0.00 0.00 100.00 0.00 0.00 NA Three Standard Errors (%) 0.00 0.00 0.00 0.00 100.00 0.00 0.00 NA Four Standard Errors (%) 0.00 0.00 0.00 0.00 100.00 0.00 0.00 NA Source: Attachment G. GE = General Education; ID = Intellectual Disability category; NA = Not Applicable Bold = Overrepresentation; Bold and Italics = Underrepresentation ** Not Useable (division by zero) Table 13 shows that for a homogeneous district, none of the risk ratio measures are useable for disproportionality calculations for racial/ethnic groups with zero enrollments because they run into division by zero (also known as zero divide). For Hispanic students, who are 100 percent of all students, the Risk Ratio and the Weighted Risk Ratio calculations also run into the same situation (zero divide), and therefore, they are not useable either. Alternate Risk Ratio is the only risk ratio measure that produces a useable value (1.69). To avoid zero divides, which are a nuisance in any computational process, the racial/ethnic groups with no enrollments in general education (and therefore, none in the ID category) would need to be excluded from disproportionality calculations under all risk ratio measures. Note that they can also be excluded from any risk ratio measures under the small cell size rule (less than 10 or 20). 28

The E-formula calculations do not run into the zero divide problem. The E-formula values set the logical or appropriate limits for overrepresentation and underrepresentation for each racial/ethnic group, even though there are no students in the group. For example, both upper and lower limits for all racial/ethnic groups with zero enrollments are zero because there is no student in any of these groups in general education. Strictly from computational point of view, none of these racial/ethnic groups would need to be excluded from the E-formula calculations due to small cell size because the total district enrollment in the ID category is 100, which is greater than 10 or 20. Recall from earlier discussion in the paper, the E-formula uses the total district enrollment in the ID category not the ID enrollments in the individual racial/ethnic groups. The limits for Hispanic students are 100 percent for both overrepresentation and underrepresentation because all students in the ID category are Hispanic and there are no more students in the ID category left in the district. Table 14. Effect of Various Disproportionality Measures on Racially/Ethnically Almost Homogeneous District Native Asian Pacific Black Hispanic White Multiple Total GE Enrollment (N) 0 0 0 300 9,000 700 0 10,000 GE Composition (%) 0.00 0.00 0.00 3.00 90.00 7.00 0.00 100.00 ID Enrollment (N) 0 0 0 9 85 6 0 100 ID Composition (%) 0.00 0.00 0.00 9.00 85.00 6.00 0.00 100.00 ID Risk (%) ** ** ** 3.00 0.94 0.86 ** 1.00 Risk Ratio Measures: Risk Ratio ** ** ** 3.20 0.63 0.85 ** NA Weighted Risk Ratio ** ** ** 3.87 1.03 0.89 ** NA Alternate Risk Ratio ** ** ** 4.87 1.59 1.26 ** NA E-Formula (Overrepresentation Maximum Percent of ID Allowed at): One Standard Error (%) 0.00 0.00 0.00 4.71 93.00 9.55 0.00 NA Two Standard Errors (%) 0.00 0.00 0.00 6.41 96.00 12.10 0.00 NA Three Standard Errors (%) 0.00 0.00 0.00 8.12 99.00 14.65 0.00 NA Four Standard Errors (%) 0.00 0.00 0.00 9.82 102.00 17.21 0.00 NA E-Formula (Underrepresentation Minimum Percent of ID Needed at): One Standard Error (%) 0.00 0.00 0.00 1.29 87.00 4.45 0.00 NA Two Standard Errors (%) 0.00 0.00 0.00-0.41 84.00 1.90 0.00 NA Three Standard Errors (%) 0.00 0.00 0.00-2.12 81.00-0.65 0.00 NA Four Standard Errors (%) 0.00 0.00 0.00-3.82 78.00-3.21 0.00 NA Source: Attachment H. GE = General Education; ID = Intellectual Disability Category; NA = Not Applicable Bold = Overrepresentation; Bold and Italics = Underrepresentation ** Not Useable (division by zero) Table 14 summarizes the results of all disproportionality measures for an almost homogeneous district. Like the homogeneous district in Table 13, all risk ratio calculations for racial/ethnic groups with zero enrollments in an almost homogeneous district are also not useable because they run 29

into the same computational problem (zero divides), and therefore, would need to be excluded. The calculations for African-American and White students are based on small numbers (nine and six, respectively), and therefore, may be excluded from all risk ratio calculations under the small cell size rule. 21 When disproportionality calculations are excluded for African-American and White students due to small cell size, even though the corresponding denominators are fairly large (300 and 700, respectively), the district would practically become homogeneous and the results of the disproportionality calculations would be somewhat similar to the ones in Table 13. If the ID enrollments for African-American and White students were not subject to exclusion due to the small cell size rule, then the values of the risk ratio measures for them would remain as they are. It is important to note that exclusion of a racial/ethnic group from disproportionality calculations due to small cell size does not automatically eliminate it from disproportionality calculations for other racial/ethnic groups. The Risk Ratio for Hispanic students in the ID category is 0.63 (Table 14), which is clearly an underrepresentation. But the Hispanic students are overrepresented in both Weighted Risk Ratio (1.03) and Alternate Risk Ratio (1.59). This is, in part, due to the fact that the Risk for Hispanic students in the ID category in the district (0.94 percent) is considerably higher than the statewide Risk for Hispanic students in the ID category (0.69 percent, see Attachment H). It also reflects, perhaps more so in Alternate Risk Ratio than in Weighted Risk Ratio, that the district has a higher identification rate of Hispanic students in the ID category than non-hispanic students in the state. Regardless of the size or composition of the Hispanic student population in the district (90 percent) or its Risk in the ID category at the district level (0.94 percent), the statewide demographics pushed the district into the overrepresentation territory. The results of the E-formula calculations in Table 14 for an almost homogeneous district appear to be reflective of the district demographics, as for the homogeneous district in Table 13. The computational process also appears to be less problematic for the E-formula than for the risk ratio measures. However, one should exercise caution in interpreting the results of disproportionality calculations for a racially/ethnically homogeneous or almost homogeneous district, which could result in a so-called tunnel vision that may be far removed from the expected range of disproportionality for that group. Disproportionality results of a homogeneous and almost homogeneous district should, therefore, be compared against the statewide statistics for the same racial/ethnic group or against other districts with similar demographics or against districts with reasonably large same racial/ethnic group. 2.6. Effect of the State Incidence Rate on Districts So far, we have observed that all measures trigger varying degrees of disproportionality for a racial/ethnic group if enrollment for that group in special education (or in a program subcategory) crosses certain thresholds or is considerably different from the other racial/ethnic groups in a district. But what if a district does not appear to have any enrollment discrepancy among various racial/ethnic groups but the overall district incidence rate is considerably high or low, compared to other districts or the state? 22 For such a district, an overall very high or very low incidence rate (and therefore, for individual racial/ethnic groups as well) should be a matter of concern in examining racial/ethnic disproportionality in special education, even if the district does not show any noticeable difference in risks or incidence rates among various racial/ethnic groups. This issue does not seem to draw sufficient attention since the primary focus of racial/ethnic disproportionality in special education is to examine disproportionality for individual racial/ethnic groups within a district, and 30

rarely for the district as a whole in relation to other districts or the state, although two measures (Weighted Risk Ratio and Alternate Risk Ratio) use statewide data in their calculation process. To examine how the measures affect districts with high (or low) incidence rates, we have modified the data in our hypothetical medium sized school district to reflect high overall incidence rate in the ID category and then applied all disproportionality measures. For the sake of simplicity, we have decided to focus on high incidence rate only. Any issues arising from the effect of high incidence rate would give the readers some idea about the effect of low incidence rate as well. For this exercise, we have used 2.00 percent as the district s overall ID incidence rate, which is more than three times the state s overall incidence rate (0.64 percent) in the ID category. Such differences for individual racial/ethnic groups range from about two times for Black or African- American students (2.00 percent compared to 1.02 percent) to more than four times for students in Asian and Multiple racial/ethnic groups (2.00 percent compared to 0.46 percent). So we can examine the effect of the overall high incidence rate only, and no other factors, we hold the composition of each racial/ethnic group in the district the same in general education (GE) and in the ID category, and the risk for all racial/ethnic groups in the ID category the same as the overall district risk. The state-level data and the district general education data remain the same. Table 15 summarizes the results of this analysis. The detailed calculations are shown in Attachment I. Note that the compositions of each racial/ethnic group in general education (GE) and in the ID category are the same: 4.00 percent for Native American, 15.00 percent for Asian, 3.00 percent for Pacific Islander, and so forth. Also note that the risk or incidence rate for all racial/ethnic groups in the ID category is held constant at 2.00 percent, the same as the district s overall risk or incidence rate in the ID category. Table 15 shows that the Risk Ratio results are the same (1.00, perfectly proportional) for all racial/ethnic groups, indicating that the overall high incidence rate in the ID category in the district has no impact in determining disproportionality among any of the racial/ethnic groups. In other words, all racial/ethnic groups are perfectly proportional to their respective comparison groups, even though the district and the individual racial/ethnic groups have a very high overall incidence rate in the ID category, compared to the state. The reason that the Risk Ratio results are not influenced by the differences between the district risk and the state risk is that the calculations in Risk Ratio use only the district-level data from the same district (not the state-level data) and the risks in the ID category for all racial/ethnic groups are the same. This also indicates that, under Risk Ratio, the individual risks of various racial/ethnic groups are more critical in determining racial/ethnic disproportionality than the district s overall risk or incidence rate. The results of the Weighted Risk Ratio, which uses state-level data, in addition to the district-level data, also show a value of 1.00 for all racial/ethnic groups, indicating that the high incidence rate in the ID category in the district has no impact on the individual groups. If you will recall, in the Weighted Risk Ratio the district risks of all racial/ethnic groups are weighted with statewide composition. The weighting process equalizes the risk of a racial/ethnic group in question (the numerator) with the weighted risk of its comparison group (the denominator), which results in exactly the same numerator and denominator for each racial/ethnic group (see Attachment I). Like Risk Ratio, this also indicates that the individual risks of various racial/ethnic groups are more critical in determining racial/ethnic disproportionality than the district s overall risk or incidence rate, even though the Weighted Risk Ratio uses state-level data in the calculations. Also, note that the Weighted Risk Ratio uses statewide composition data, which are fractions and can be derived from 31

small numbers as well, and therefore, the results are not influenced by the magnitude of the actual state-level data. Table 15. Effect of Various Measures on Overall High Incidence Rate in the ID Category in District State Enrollment Data: Native Asian Pacific Black Hispanic White Multiple Total GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 GE Composition (%) 0.74 11.24 0.61 6.98 51.31 27.53 1.59 100.00 ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 ID Composition (%) 0.69 7.99 0.54 11.03 55.20 23.41 1.14 100.00 ID Risk (%) 0.60 0.46 0.58 1.02 0.69 0.55 0.46 0.64 District Enrollment Data: GE Enrollment (N) 400 1,500 300 1,000 3,200 2,600 1,000 10,000 GE Composition (%) 4.00 15.00 3.00 10.00 32.00 26.00 10.00 100.00 ID Enrollment (N) 8 30 6 20 64 52 20 200 ID Composition (%) 4.00 15.00 3.00 10.00 32.00 26.00 10.00 100.00 ID Risk (%) 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Risk Ratio Measures: Risk Ratio 1.00 1.00 1.00 1.00 1.00 1.00 1.00 NA Weighted Risk Ratio 1.00 1.00 1.00 1.00 1.00 1.00 1.00 NA Alternate Risk Ratio 3.10 3.00 3.10 3.25 3.37 2.94 3.09 NA E-Formula (Overrepresentation Maximum Percent of ID Allowed at): One Standard Error (%) 5.39 17.52 4.21 12.12 35.30 29.10 12.12 NA Two Standard Errors (%) 6.77 20.05 5.41 14.24 38.60 32.20 14.24 NA Three Standard Errors (%) 8.16 22.57 6.62 16.36 41.90 35.30 16.36 NA Four Standard Errors (%) 9.54 25.10 7.82 18.49 45.19 38.41 18.49 NA Source: Attachment I. GE = General Education; ID = Intellectual Disability category; NA = Not Applicable Bold = Overrepresentation Alternate Risk Ratio is the only measure that shows overrepresentation of all racial/ethnic groups in the ID category under the same conditions. This is because Alternate Risk Ratio compares district risks of individual racial/ethnic groups directly against the statewide risks or incidence rates of all other racial/ethnic groups, and therefore, the differences between the district incidence rates and the state incidence rates are reflected in the results. Even though the risk or incidence rate in the ID category is the same for each racial/ethnic group in the district, the effect of the high overall and individual incidence rates in the district in relation to the statewide incidence rates (risk) is evident in the overrepresentation of all racial/ethnic groups in the ID category. 32

The E-formula results also do not show overrepresentation for any of the racial/ethnic groups. This is again due to the fact that, like the Risk Ratio, the E-formula calculations are based on the same district-level data, not the state-level data, and therefore, they are not affected by the relatively low statewide incidence rates. In addition, the proportions of each racial/ethnic group in general education (GE) and in the ID category are the same in the district, and therefore, the E-formula results do not show any overrepresentation, no matter how large the district s overall incidence rate is in the ID category. The E-formula results, however, show the necessary flexibility for various racial/ethnic groups of different sizes under the various standard error thresholds. Based on our analysis of the four measures, Alternate Risk Ratio is the only measure that can address disproportionality issues arising from the differences in the incidence rates between a district and the state. Whether or not the overall high or low district incidence rate in special education is the primary focus of disproportionality calculations, any unusually high or low incidence rate in a district should raise alarm to the district and the state. As we have seen from this exercise and depending upon which measure a district or state uses, a high incidence rate in a district with little or no difference among various racial/ethnic groups could put the disproportionality issues under the radar, when there may be disproportionality among all or some of the racial/ethnic groups. It is, therefore, important that while examining disproportionality for various racial/ethnic groups in a district, the overall incidence rate in special education or in a program category should also be examined against the state or other comparable districts. 2.7. Summary of Findings The results from all seven disproportionality measures show that three measures are relatively simple to use, but they do not provide sufficient information to determine racial/ethnic disproportionality in special education. They are: Composition, Relative Difference in Composition, and Risk. Therefore, we will not report any findings from these three measures. Findings from the remaining four measures, Risk Ratio, Weighted Risk Ratio, Alternate Risk Ratio, and the E-formula are summarized below. Risk Ratio The definition of Risk Ratio is fairly simple, the calculation is relatively straightforward, and its interpretation is intuitive and clear. Risk Ratio calculations use district level data from individual districts only; therefore the results are not comparable against other districts in the state. The results of Risk Ratio are the same for different sized districts. Risk Ratio does not provide, proportionately, the necessary flexibility to small districts for addressing small fluctuations in special education enrollment, compared to large districts. Small numbers are problematic in using the Risk Ratio measure. Any small fluctuations in enrollment can produce dramatic changes in the Risk Ratio results, which make interpretation of results difficult. The number of exclusions from disproportionality calculations due to small cell size (such as less than 10 or 20) can be large because the measure uses special education enrollment in each racial/ethnic group individually, which can result in a relatively large number of cells with a small number of students. The measure does not have a Region of Tolerance for Disproportionality around the point or line of non-disproportionality, where a district can operate special education programs 33

without being disproportionate. This Region would have to be established through external policy decisions, analysis of empirical data or other influencing factors. For racially/ethnically homogeneous and almost homogeneous districts, the measure runs into division by zero (zero divides) for groups with zero enrollments, requiring exclusions from the calculation process. The measure is not useable for homogeneous districts but is useable for almost homogeneous districts. Risk Ratio is not sensitive to a district s overall high (or low) risk or incidence rate in relation to that of the state. Its focus is on the differences in risks or incidence rates among various racial/ethnic groups within the district. Weighted Risk Ratio The definition of Weighted Risk Ratio is far from simple, the calculation is relatively complex, and its interpretation is neither intuitive nor clear. Weighted Risk Ratio calculations combine district level risk with statewide composition of racial/ethnic groups; therefore, the Weighted Risk Ratios for different districts are comparable across the state. The results of Weighted Risk Ratio are the same for different sized districts. Weighted Risk Ratio does not provide, proportionately, the necessary flexibility for small districts for addressing small fluctuations in special education enrollment, compared to large districts. Weighted Risk Ratio incorporates statewide demographics in the calculations, which can affect a district by pushing it in the direction of overrepresentation or underrepresentation, when the district may not be disproportionate under Risk Ratio. Small numbers are problematic in using the Weighted Risk Ratio measure. Any small fluctuations in enrollment can produce dramatic changes in the Weighted Risk Ratio results, which make interpretation of results difficult. The number of exclusions from disproportionality calculations due to small cell size (such as less than 10 or 20) can be large because the measure uses special education enrollment in each racial/ethnic group individually, which can result in a relatively large number of cells with a small number of students. The measure does not have a Region of Tolerance for Disproportionality around the point or line of non-disproportionality, where a district can operate special education programs without being disproportionate. This Region would have to be established through external policy decisions, analysis of empirical data or other influencing factors. For racially/ethnically homogeneous and almost homogeneous districts, the measure runs into division by zero (zero divides) for groups with zero enrollments, requiring exclusions from the calculation process. The measure is not useable for homogeneous districts but is useable for almost homogeneous districts. Weighted Risk Ratio is not sensitive to a district s overall high (or low) risk or incidence rate in relation to that of the state. Its focus is on the differences in risks or incidence rates among various racial/ethnic groups within the district. Alternate Risk Ratio The definition of Alternate Risk Ratio is relatively simple, the calculation is also fairly simple, and its interpretation is relatively clear and intuitive. 34

In Alternate Risk Ratio calculation, the district risk is compared against the statewide comparison group; therefore, the Alternate Risk Ratios for different districts are comparable across the state. The results of Alternate Risk Ratio are the same for different sized districts. Alternate Risk Ratio does not provide, proportionately, the necessary flexibility for small districts for addressing small fluctuations in special education enrollment, compared to large districts. Alternate Risk Ratio incorporates statewide demographics in the calculations, which can affect a district by pushing it in the direction of overrepresentation or underrepresentation, when the district may not be disproportionate under Risk Ratio. Small numbers are problematic in using the Alternate Risk Ratio measure. Any small fluctuations in enrollment can produce dramatic changes in the Alternate Risk Ratio results, which make interpretation of results difficult. The number of exclusions from disproportionality calculations due to small cell size (such as less than 10 or 20) can be large because the measure uses special education enrollment in each racial/ethnic group individually, which can result in a relatively large number of cells with a small number of students. The measure does not have a Region of Tolerance for Disproportionality around the point or line of non-disproportionality, where a district can operate special education programs without being disproportionate. This Region would have to be established through external policy decisions, analysis of empirical data or other influencing factors. For racially/ethnically homogeneous and almost homogeneous districts, the measure runs into division by zero (zero divides) for groups with zero enrollments, requiring exclusions from the calculation process. The measure is useable for both homogeneous and almost homogeneous districts. Alternate Risk Ratio is sensitive to a district s overall high (or low) risk or incidence rate in relation to that of the state. It takes into consideration the statewide risk or incidence rate and the results are reflected in the disproportionality status for various racial/ethnic groups in the district. The E-formula The definition of the E-formula is based on statistical principles, the calculation is relatively simple, and its interpretation is intuitive and clear. The E-formula calculations use district level data from individual districts only; therefore the results are not comparable against other districts in the state. The E-formula calculation produces different results for different sized districts. It provides, proportionately, more flexibility to small districts for addressing small fluctuations in special education enrollment, than to large districts. Small numbers are not as much a problem in the E-formula as in the risk ratio measures. Small changes in enrollments produce reasonable changes in the E-formula results, which make interpretation of results sensible. The number of exclusions from disproportionality calculations due to small cell size (such as less than 10 or 20) is relatively small because the E-formula uses the total special education enrollment in all racial/ethnic groups, instead of special education enrollments in each racial/ethnic group. The E-formula has a built-in Region of Tolerance for Disproportionality around the point or line of non-disproportionality, which provides the district a program operating environment without being disproportionate. No external rule is necessary to create a Region of Tolerance for Disproportionality. 35

The computation process for racially/ethnically homogeneous and almost homogeneous districts does not run into division by zero (zero divides) for groups with zero enrollments. It produces standard limits for overrepresentation and underrepresentation for all racial/ethnic groups in a district regardless of its racial/ethnic composition. The measure is useable for both homogeneous and almost homogeneous districts. The E-formula is not sensitive to a district s overall high (or low) risk or incidence rate in relation to that of the state. Its focus is on the differences in compositions between general education (GE) and special education (SE) for each racial/ethnic group within the district. 2.8. Discussion of Results The findings from various measures of racial/ethnic disproportionality show that some measures are more suitable for specific purposes than others and all of them have strengths and weaknesses. Selecting one measure over another must be based on the purpose of determining racial/ethnic disproportionality in special education and whether the results from the selected measure will serve that purpose. If comparing districts in the state against each other is the main purpose of determining racial/ethnic disproportionality, then Weighted Risk Ratio and Alternate Risk Ratio are the only two choices. None of the other measures are designed to compare one district against another. If a district s overall incidence rate in relation to the state s incidence rate is a critical issue, then Alternate Risk Ratio is the only option. If, however, no single purpose dominates the disproportionality calculations, then all measures deserve to be examined based on their relative strengths and weaknesses against each other. All disproportionality measures, except the E-formula, produce the same results for different sized districts. Proportionately, smaller districts would need more room to accommodate enrollment fluctuations, which larger districts probably would not need to the same degree. A family with 1-2 disabled students moving into a small district can push the district over the limit; whereas, a relatively large district could probably accommodate several such families without showing any sign of strain. For all risk ratio measures, this will require setting different thresholds for different sized districts, which would mean creating new rules or adding exceptions to existing rules. The E-formula has a built-in property to address the flexibility issue. In the E-formula, the standard error for a small district is larger than the standard error for a large district, which allows proportionately more flexibility to small districts than to large districts. Therefore, the same rule or threshold would work for different sized districts without having to create any new rules. In the E- formula, large districts are held accountable under stricter (percentage) limits compared to small districts, but this is done without explicitly labeling different flexibility or creating different rules for different sized districts. In calculating disproportionality for a particular racial/ethnic group, all risk ratio measures use a comparison group that includes all other racial/ethnic groups, except the one for which the disproportionality is calculated. Thus the comparison group is always different for each racial/ethnic group. In other words, under the risk ratio measures, each racial/ethnic group is judged against a different standard. One could argue if such comparison leaves room for possible unfairness. If, however, the comparison group consisted of all racial/ethnic groups, including the one for which disproportionality is calculated, then all racial/ethnic groups would have the same comparison group, and the issue could be avoided. 36

Most measures of racial/ethnic disproportionality are unfavorably affected by small enrollments in varying degrees because interpretation of results based on small numbers is problematic. In practice, whenever the numerator or the denominator (cell size) is small, such as 10 or 20, for a disability category or a racial/ethnic group or any combination thereof, that category or group is excluded from all risk ratio calculations. The number of exclusions based on small cell size can be large because special education represents a small percentage (about 10 percent) of general education enrollment, and when these enrollments are broken down into several racial/ethnic groups and/or disability categories, many cells end up with numbers less than 20 or 10 or even zero. Not to mention small districts, which there are many in a large state like California, even large districts are not immune from having cells with small numbers when enrollments are broken down by ethnicity and disability. In selecting a measure for racial/ethnic disproportionality in special education, we must ensure that the measure has the fewest exclusion rules or it produces the fewest cases of exclusions from disproportionality calculations, compared to other measures. If a measure has more exclusion rules than another measure or if the measures excludes a large number of cases from disproportionality calculations for one reason or another (small cell size, for example), then it is probably not a very desirable measure. As we have seen, different measures are affected differently by these rules. The threshold for determining disproportionality in a measure must provide a reasonable allowance for disproportionality or a Region of Tolerance for Disproportionality around the point or line of nondisproportionality so a district can run its programs within an acceptable operational range rather than in an air-tight environment. This allowance or Region of Tolerance for Disproportionality must be set in such a way that it does not fail to identify districts that are grossly disproportionate and yet allows sufficient room to districts that may experience marginal disproportionality from time to time due to enrollment fluctuations, student mobility, and other demographic factors that are beyond the control of the district. An analogy of this principle would be the ability to identify automobiles that are driving at an excessively high or excessively low speed compared to the posted speed limit on a highway (such as, more than 85 miles or less than 45 miles per hour with a 65 miles per hour limit) and to be not too strict on automobiles that are driving within a reasonable range around the same speed limit (60-70 miles per hour, for example). In disproportionality measures, this would translate to maximizing the identification of gross overrepresentation and gross underrepresentation and minimizing the identification of marginal overrepresentation and marginal underrepresentation. Other factors, such as resources and timeline, will also play a role in setting the threshold in such a way so we do not identify so many districts that we cannot monitor, or identify too few districts that are obvious, which may render the measure meaningless. For all risk ratio measures, the value 1.00 represents non-disproportionality. Any value more than 1.00 is overrepresentation and less than 1.00 is underrepresentation. Overrepresentation can take a value from more than 1.00 (>1.00) to any large number, but underrepresentation extends from less than 1.00 (<1.00) to greater than zero (>0.00) only. Because the two ranges of values are different between overrepresentation and underrepresentation under risk ratio measures, one will need to exercise caution in setting any Region of Tolerance for Disproportionality with an expectation that the allowance for non-disproportionality in both directions would be similar or comparable. The E- formula values for overrepresentation and underrepresentation are on the same scale, and therefore, no such caution is necessary. The usefulness of the results from various disproportionality calculations vary from measure to measure. The results of the Risk Ratio and the E-formula that use data only from the district under 37

consideration can be useful in making changes in program or policy in the district, hoping for desired outcome. Other measures, such as the Weighted Risk Ratio and the Alternate Risk Ratio that use data from the district and the state, may not have the same degree of usefulness as the Risk Ratio and the E-formula. The results from these two measures (Weighted Risk Ratio and Alternate Risk Ratio) are primarily for information purposes; they are of limited use because any changes in program or policy in the district based on the results of these two measures, without making any necessary changes in other districts or the state might not yield the desired outcome. 38

3. Rating the Disproportionality Measures With all the information on various disproportionality measures presented so far, along with their strengths and weaknesses, it may still be quite confusing to the user to select a specific measure over another to examine racial/ethnic disproportionality in a district or the state. If the measures were compared against each other using a set of criteria and/or a rating scale, it would provide the user with another tool to evaluate them prior to selecting one. Absence any such criteria or rating scale in the literature to evaluate the measures, it is somewhat presumptuous to propose or develop a set of criteria for such an endeavor. However, this venture promises sufficient benefit to the user; therefore, we have decided to develop a set of criteria and rate the measures, even with the anticipation that there will be differences of opinions among professionals regarding the choice of the criteria as well as the rating process. It must be kept in mind that these ratings are only suggestive, reflecting only one point of view, and the reader should feel free to make his or her own set of criteria or ratings. In our effort to provide a comparative picture of the various disproportionality measures and their relative effectiveness, we have identified several elements which, we believe, are critical to any measure of racial/ethnic disproportionality. These elements are derived from our analysis of the measures discussed in the paper, review of related literature, and experience of education professionals who have worked on disproportionality issues over the years. In addition, the list includes elements that are unique to a particular measure (statewide comparability of districts, for example, is unique to Weighted Risk Ratio and Alternate Risk Ratio). Once again, these elements are only suggestive and they are not exhaustive or exclusive by any means. The elements are described below: 1. Definition of the measure. The definition of a measure is the most fundamental element in examining racial/ethnic disproportionality in special education. It must capture the concept of disproportionality in the most appropriate way possible and translate it into operational terms. The definition must also be simple for practitioners to understand. The simpler the definition, the more desirable is the measure. 2. The calculation process. The calculations in a disproportionality measure follow its definition literally. It translates the definition into mathematical operations. The calculation process should be simple enough for users to understand and carry out. It should include as few variables and mathematical operations as possible. A measure with fewer variables and/or operations would be more desirable than one with a large number of variables and/or complex operations. 3. Interpretation and usefulness of results. The results of a disproportionality measure should be relatively easy to comprehend and interpret in operational terms, and be useful to all education professionals. They should be intuitive to the practitioners in the field with varying degrees of expertise in special education programs as well as their level of comfort with numbers. Implications of the results should be obvious to the user as to what they mean for the district or the state, and how the results may be used to make necessary changes in program, policy, and practice. A measure that produces more intuitive and useful results would be more preferable than another that does not. 39

4. Comparability of results among districts. This is a characteristic of some disproportionality measures that allows the results of one district to be compared against those of another in a state. This is a desirable property of a measure if one of the purposes of disproportionality calculations is to compare the districts against each other. If the results of a measure are comparable (or more comparable) across the districts in a state, it would be preferable to other measures of which the results are not comparable (or less comparable) across the districts in a state. 5. Effects on small enrollments and their fluctuations. Effects on small enrollments, including the effect on fluctuations of small enrollments, both in the numerator and in the denominator, are an important factor in determining the quality of a disproportionality measure. Interpretations of results based on small numbers should be as reasonable as possible and they should not challenge the conventional wisdom or common sense. Measures showing dramatic changes in the results from fluctuations in small numbers would be less preferable over others where the results are not so dramatic or the impacts of such fluctuations are relatively small. 6. Exclusion of groups due to small cell size. Exclusion of groups from disproportionality calculations due to small cell size (numerator or denominator) is another important element in any disproportionality measure. Because the purpose of examining racial/ethnic disproportionality is to investigate every possible combination of racial/ethnic group, disability category, and service delivery environment, it is in the best interest of the society as well as the intent of the law, that disproportionality measures should minimize the number of exclusions due to small cell size. The smaller the number of exclusions, the more preferable is the measure. 7. Differentiated Region of Tolerance for Disproportionality. This is an important element in a measure that allows districts to operate their special education programs within a range of non-disproportionality based on the size of the district. It encompasses two properties of a measure that are somewhat related: (1) the ability to differentiate the impact of the district size in its results, i.e., the results of the calculations in a measure are different for different sized districts and (2) allowing proportionately more flexibility or Region of Tolerance for Disproportionality to small districts than large districts in operating their special education programs. This is an important property of a measure to address situations resulting from enrollment fluctuations and other demographic changes in districts. A measure that offers such flexibility would be more favorable than another that does not have such flexibility. It is also desirable if the Region of Tolerance for Disproportionality is transparent to the user. 8. Effect on homogeneous and almost homogeneous districts. Ideally, the effect of a disproportionality measure, with respect to its calculation process and interpretation of results, should be the same for all districts with varying degrees of racial/ethnic composition. It should be able to address districts that are racially or ethnically homogeneous or almost homogeneous the same way as those that are heterogeneous. If the calculations in a measure cannot be carried out for a district due to its degree of racial/ethnic composition, then it is probably a weakness of the measure. Such situations would call for using different measures for districts with different racial/ethnic composition, which would be unfair and problematic. A measure that does not require any exception in the computation process and if the interpretation of its results for districts with varying degrees of racial/ethnic composition is intuitive, then it would be a desirable measure; otherwise, the measure would be less desirable. 40

9. Effect of the state incidence rate on districts. Although the main focus of all disproportionality measures is to determine disproportionality across various racial/ethnic groups within a district, and for that district only, it is also important to examine if the overall incidence rate in the district is too high or too low compared to the state. As we have seen, disproportionality measures show the impact of the differences between the district and the state incidence rates in varying degrees in their results. A measure that is more sensitive to this issue would be more desirable than another that is not. The measures incorporate these elements differently from each other and their results reflect these differences. To evaluate the measures with respect to these elements, each element was rated on a five-point scale: five (5) points for the best incorporation of an element in a measure, or if the element is least problematic in the measure, or if it produces the most desirable results in the measure; and one (1) point for the worst incorporation of the element in a measure, or if the element is most problematic in the measure, or if it produces the least desirable results in the measure, or if the element is non-existent in the measure. The results of the rating process are shown in Table 15. Table 15. Rating of Critical Elements in Various Measures of Ethnic Disproportionality Elements of Disproportionality Measures and Rating Scale <- Worse Better -> 1 - - - - - 2 - - - - - 3 - - - - - 4 - - - - - 5 Risk Ratio Weighted. Risk Ratio Alternate Risk Ratio E-formula 1. Definition of the Measure 2. The Calculation Process Least Simple (1) Most Simple (5) 5 2 4 3 Least Simple (1) Most Simple (5) 5 2 5 4 3. Interpretation and Usefulness of Results Least Clear (1) Most Clear (5) 5 2 3 4 4. Comparability of Results Among Districts Least Comparable (1) Most Comparable (5) 1 5 5 1 5. Effects on Small Enrollments and Their Fluctuations Most Problematic (1) Least Problematic (5) 2 2 2 4 6. Exclusion of Groups Due to Small Cell Size Most Exclusions (1) Fewest Exclusions (5) 2 2 2 5 7. Differentiated Region of Tolerance for Disproportionality Least Transparent (1) Most Transparent (5) 1 1 1 5 8. Effect on Homogeneous and Almost Homogeneous Districts Most Problematic (1) Least Problematic (5) 2 2 3 4 9. Effect of the State Incidence Rate on Districts Least Sensitive (1) Most Sensitive (5) 1 1 4 1 Total Points 24 19 29 31 Unweighted Average (Overall Rating) 2.7 2.1 3.2 3.4 41

The rating points for an individual element in the measures reflect how well the element is addressed in each measure on its own merit and in relation to how well it is addressed in the other measures. This process is illustrated below by using the ratings of two elements in the measures. In Element #1 (Definition of the Measure), Risk Ratio received a rating of five points because we believe Risk Ratio has the simplest (or most simple) definition of all measures. At the other end of the rating scale, we gave Weighted Risk Ratio only two points because it has the most complex (or least simple) definition among the four measures. We did not give it the lowest rating of one point because there might be, just might be, a measure (perhaps not yet surfaced in the literature) that has more complex definition than the Weighted Risk Ratio. We gave Alternate Risk Ratio a rating of four points, not five, even though the definition of Alternate Risk Ratio is also quite simple. However, in our judgment the definition of Alternate Risk Ratio is a little less simple than that of the Risk Ratio, because the definition of Alternate Risk Ratio includes data beyond the district (state-level). The E- formula received a rating of three points because it includes mathematical symbols and operations such as (square root) and some statistical explanations that go beyond the four basic arithmetic operations. But, in our opinion, the E-formula definition is still simpler than the Weighted Risk Ratio, and therefore, we did not give it a rating of two points. In Element #5 (Effects of Small Enrollments and their Fluctuations), all three risk ratio measures received a rating of two points because the effect of small numbers (numerator or denominator) and their fluctuations are quite dramatic in all three risk ratio measures. We did not give them a rating of one point because there might be a measure that shows even more dramatic results than the three risk ratio measures. The E-formula received a rating of four points because the E-formula results are far less dramatic than the results of the three risk ratio measures and they are more reflective of the actual numbers and the marginal changes in the numbers. We did not give the E-formula a rating of five points because there might be a measure that is even better than the E-formula with respect to small numbers and their fluctuations. In general, we followed a centrist approach in rating the elements. In some cases an element received the highest (5) or the lowest (1) rating. The highest rating was given to an element in a measure only when that element was best addressed in the measure and that no other measure could possibly have any better outcome. For example, in Element #4 (Comparability of Results Among Districts), we gave Alternate Risk Ratio a rating of five points because the Alternate Risk Ratio calculations for a racial/ethnic group in a district uses state level data for the comparison group, thus allowing inter-district comparison of results. Any other measure, at best, could be as good as the Alternate Risk Ratio on this element, such as the Weighted Risk Ratio which also got five points, but not any better. The lowest rating was given to an element in a measure only when that element was not addressed at all or addressed most poorly and that no other measure could possibly have any worse outcome. For example, in Element #7 (Differentiated Region of Tolerance for Disproportionality), we gave one point to all three risk ratio measures because this element was not addressed at all in any of the risk ratio measures. No other measure could be any worse than the risk ratio measures on this element. As one can see, the rating process was quite conservative, and hopefully, fair. However, in any rating situation there are bound to be differences of opinion among readers, and one may end up with a completely different set of elements and outcomes. As mentioned before, the list of elements presented here is by no means exclusive or exhaustive. For example, since small enrollments pose an interpretation problem and there are a large number of districts with small enrollments in a state 42

like California, should the Effect on Small Enrollments be an element of its own? Would it make sense to split Element #5 (Effects on Small Enrollments and Their Fluctuations) into two elements: (5a) Effect on Small Enrollments and (5b) Effect on Fluctuations of Small Enrollments? If we did, all risk ratio measures would rate relatively low compared to the E-formula, as we have seen in Tables 10 and 12. We decided to keep them together as a single element because their effects are somewhat similar. One could also argue that because cells with small enrollments would be excluded from disproportionality calculations, interpretation of results from small enrollments would not be an issue any longer. If that is the case, then Element #6 (Exclusion of Groups Due to Small Cell Size) would be a very important element in a measure, and perhaps, should carry more weight than the other measures. If Element #6 was weighted more than the other elements, then all risk ratio measures would be rated relatively low and the E-formula would be rated relatively high because of the fewer exclusions in the E-formula, compared to the risk ratio measures. There may be other elements that are equally important but not listed here. Nevertheless, these elements capture the essential components of a good measure and the associated ratings should provide some helpful information in selecting a measure that is best suitable for a state and the districts. To minimize any risk of bias and to maintain objectivity in the rating process, none of the elements are weighted against each other in terms of their relative importance. This also underscores our belief that all of these elements are equally important to any disproportionality calculations. The unweighted average puts the E-formula on top of the list with 3.4 points, followed by Alternate Risk Ratio (3.2 points) and Risk Ratio (2.7 points). The Weighted Risk Ratio finished at the bottom with 2.1 points. A measure with an overall rating of less than 2.5 would be considered relatively weak and would be weaker for lesser ratings. An overall rating of more than 2.5 points would be considered relatively strong and stronger for higher values. According to the results of the ratings, the E-formula appears to be the best overall measure, followed by Alternate Risk Ratio, Risk Ratio, and Weighted Risk Ratio. 43

4. Effect of the Top Two Measures on Districts Before selecting a specific measure or measures to determine racial/ethnic disproportionality, it would be helpful for the user to find out how the two top-ranking measures from this analysis affect the districts in a state. Of particular interest is to learn if one measure identifies a certain type of district more often than another type (such as, large vs. small), compared to the other measure. A second reason for such a study is to examine if there is any difference in the types of districts identified under the two broad categories of disproportionality measures, Risk and Composition, which address two different types of disproportionality questions. Since the E-formula and the Alternate Risk Ratio, the top two measures based on the rating results, represent the two broad categories of disproportionality measures, we will apply these two measures to all districts in a state for this preview. The 2009-10 general education and special education enrollment data from California Department of Education (CDE) have been used for this analysis. The general education enrollment data are derived from the Fall 2009 data reported by school districts through the California Longitudinal Pupil Achievement Data System (CALPADS). The special education enrollment data are the December 2009 special education student level data reported by Special Education Local Plan Areas (SELPA) through the California Special Education Management Information System (CASEMIS). These are the same data that are also reported to the U.S. Department of Education to meet the federal reporting requirements. For this analysis, we will focus on racial/ethnic disproportionality in the six major disability categories only, and only on overrepresentation. A district will be identified as disproportionate if any one of the seven racial/ethnic groups is found to be overrepresented in any one of the six major disability categories. Note that all districts are the district of residence. For both measures, the E-formula and the Alternate Risk Ratio, the following students, cases, schools, districts, cells or situations were excluded: Students less than five years old Students who are enrolled in a Licensed Children s Institution (LCI) State Special Schools Districts with total general education enrollment less than 20 Districts with total special education enrollment less than 20 Districts reporting a higher number of students of any racial/ethnic group in special education than in general education 23 Racial/ethnic groups with general education enrollment less than 20 for which disproportionality is calculated 24 Disability category with total district enrollment less than 20 in that category 25 In addition, for the Alternate Risk Ratio calculations, any cell with enrollment of a racial/ethic group in a disability category less than 20 is also excluded when the focus of the disproportionality calculations is on that particular cell. 26 44

Calculations for this analysis were carried out using a computer program developed as a parallel project to this paper. 27 All data and the results presented in this section are derived from the output of this application. The districts are classified into eight enrollment size groups, ranging from less than 1,000 students in general education to 50,000 or more, reflecting a reasonable spread of the number of districts across the enrollment groups, and to identify if any noticeable characteristics emerge along the district size groupings. The results of this analysis are shown in Table 16. Table 16. Number of Eligible Districts and Valid Cells for the E-formula and the Alternate Risk Ratio Measures State Total Less than 1,000 Number of Districts with General Education Enrollments 1,000 to 2,999 3,000 to 4,999 5,000 to 9,999 10,000 to 19,999 20,000 to 29,999 30,000 to 49,999 50,000 or More No. of Eligible Districts* 614 103 162 99 104 79 39 20 8 Percent (%) 100.00 16.78 26.38 16.12 16.94 12.87 6.35 3.26 1.30 Maximum Possible Cells** 25,788 4,326 6,804 4,158 4,368 3,318 1,638 840 336 Potentially Usable Cells*** 13,781 421 1,736 2,162 3,540 3,127 1,625 840 330 Percent of Maximum (%) 53.44 9.73 25.51 52.00 81.04 94.24 99.21 100.00 98.21 The E-formula: Potentially Usable Cells*** 13,781 421 1,736 2,162 3,540 3,127 1,625 840 330 Actual No. of Cells Used 13,781 421 1,736 2,162 3,540 3,127 1,625 840 330 Percent Used (%) 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 The Alternate Risk Ratio: Potentially Usable Cells*** 13,781 421 1,736 2,162 3,540 3,127 1,625 840 330 Actual No. of Cells Used**** 3,815 84 397 443 798 950 574 377 192 Percent Used (%) 27.68 19.95 22.87 20.49 22.54 30.38 35.32 44.88 58.18 * Number of Eligible Districts is the number of districts in the state after excluding county-operated programs, state special schools, districts with total enrollment in general education less than 20, and districts with total enrollment in special education less than 20. This number also excludes districts that reported a higher number of students of any racial/ethnic group in special education than in general education (95 districts, including a few large ones, reported such discrepancies). ** Maximum Possible Cells is the Number of Eligible Districts multiplied by the number of disability categories (6) multiplied by the number of racial/ethnic groups (7). Example: State Total, 25,788 = (614 x 6 x 7). *** Potentially Usable Cells includes cells for all racial/ethnic groups in each disability category with total district enrollment of 20 or more in that disability category if the cell size for corresponding racial/ethnic groups in general education is 20 or more. If the total district enrollment in a disability category is 20 or more but the general education enrollment for a racial/ethnic group is less than 20, then the cell for that racial/ethnic group is excluded. In other words, selection of a Potentially Usable Cell for a racial/ethnic group within a disability category must meet the following conditions: (i) the total district enrollment (for all racial/ethnic groups) in that disability category must be 20 or more and (ii) the district general education enrollment for that racial/ethnic group must also be 20 or more. **** Actual Number of Cells Used under the Alternate Risk Ratio calculations is the number of cells after excluding individual cells (of racial/ethnic groups) of size less than 20 from disability categories with total district enrollment of 20 or more in those categories. Table 16 shows the number of districts that are eligible for this analysis, broken down into the various enrollment size groups and the corresponding number of cells used for these calculations. To refresh our memory, a cell is defined as a specific racial/ethnic group (Hispanic, for example) in a specific program category in special education (the ID category, for example). For this analysis, a district can have up to six major disability categories in special education, and each disability category can have up to seven cells, one for each racial/ethnic group. Therefore, a district can have 45

up to 42 cells (seven racial/ethnic groups multiplied by six disability categories) in special education. In general education, the number of cells is seven for a district, one for each of the seven racial/ethnic groups (general education does not have any disability category). The cell size is the number of students in a particular cell. Out of over 1000 school districts and county programs in the state, 614 school districts are eligible for disproportionality calculations and about 43 percent of them have fewer than 3,000 students. At the other end, only eight or 1.30 percent of the eligible districts have 50,000 or more students. The total number of maximum possible cells in the eligible districts is 25,788, broken down according to their enrollment size groups. However, only about 53 percent of these cells are potentially useable for disproportionality calculations. This is because when the total special education enrollment in a district is disaggregated by disability categories, many of these individual categories end up with enrollments less than 20, and therefore, are excluded from the calculations. This is more prevalent in smaller districts than in larger districts. For districts with fewer than 1,000 students, only about 10 percent of the cells are potentially usable. This situation improves as the district size increases, reaching 52 percent for districts between 3,000 and 4,999. For districts with enrollments 10,000 or higher, more than 90 percent of the cells are potentially usable. The table also shows that all potentially usable cells are actually used in the E-formula calculations. There are no further exclusions due to individual cell sizes because, as stated earlier, the E-formula calculations use the total district enrollment in a disability category not the enrollment in individual cells for each racial/ethnic group within a disability category. For the Alternate Risk Ratio calculations, overall, only about 28 percent of the potentially usable cells are actually used, which means about 72 percent of the cells are excluded from calculations due to their cell size less than 20. In other words, for every 100 cells that are used in the E-formula calculations, only 28 are used in the Alternate Risk Ratio calculations, which indicates that there are far more situations in which districts can be tested for disproportionality under the E-formula than under the Alternate Risk Ratio. This confirms our earlier observation that districts have more ways to avoid the issue of disproportionality under the Alternate Risk Ratio (and for the other risk ratio measures) than under the E-formula. Small districts are more susceptible to exclusions due to small cell size than large districts. In districts with less than 1,000 students, only about 20 percent of the potentially usable cells are actually used, which means 80 percent are excluded. The situation improves gradually but slowly, as the district size gets larger, reaching up to a little over 58 percent for districts with enrollments of 50,000 or more still far below 100 percent under the E-formula. As one can see, even very large districts are not immune from such exclusionary situations under the risk ratio measures. Any disproportionality measure would identify a large number of districts if the threshold for disproportionality is set relatively low and fewer districts if the threshold is set high. From the state s point of view, the number of districts identified as disproportionate is an important factor in disproportionality calculations because many of these districts would probably require monitoring, technical assistance, and other support from the state. To make the disproportionality determination process meaningful, the number of districts identified as disproportionate should not be too high for the state to monitor the districts, given the limited resources available at the state to do so, or too low that hardly requires a sophisticated measure for making such determination. To take a look at the number and size of districts that are likely to be identified as disproportionate in the top two measures at different thresholds, each measure was applied to the eligible districts in the state and the results are shown in Table 17. 46

Table 17. Number of Districts Overrepresented in the E-formula and the Alternate Risk Ratio at Various Thresholds State Total Less than 1,000 Number of Districts with General Education Enrollments 1,000 to 2,999 3,000 to 4,999 5,000 to 9,999 10,000 to 19,999 20,000 to 29,999 30,000 to 49,999 50,000 or More No. of Eligible Districts* 614 103 162 99 104 79 39 20 8 Percent (%) 100.00 16.78 26.38 16.12 16.94 12.87 6.35 3.26 1.30 The E-formula: One Standard Error 573 75 150 98 104 79 39 20 8 Two Standard Errors 459 20 100 89 104 79 39 20 8 Three Standard Errors 379 9 61 68 95 79 39 20 8 Four Standard Errors 292 1 29 47 71 78 39 19 8 Five Standard Errors 228 0 12 24 53 73 39 19 8 Six Standard Errors 176 0 5 12 37 56 39 19 8 Seven Standard Errors 134 0 2 6 22 45 32 19 8 Eight Standard Errors 100 0 1 5 18 29 22 18 7 Nine Standard Errors 82 0 0 2 14 25 18 16 7 Ten Standard Errors 58 0 0 2 8 17 11 13 7 The Alternate Risk Ratio: More than 1.00 542 61 140 93 102 79 39 20 8 2.00 or More 382 34 74 58 83 70 36 19 8 3.00 or More 176 11 27 33 42 36 20 12 5 4.00 or More 95 6 14 11 18 20 11 10 5 5.00 or More 44 3 4 5 10 10 6 5 1 * Number of Eligible Districts is the number of districts in the state after excluding county-operated programs, state special schools, districts with total enrollment in general education less than 20, and districts with total enrollment in special education less than 20. This number also excludes districts that reported a higher number of students of any racial/ethnic group in special education than in general education (95 districts, including a few large ones, reported such discrepancies). The table shows the number of eligible districts in various general education enrollment size groups that are likely to be overrepresented under different thresholds in each measure. Note that a district is counted only if it is overrepresented in at least one cell (one racial/ethnic group in one disability category) and is counted only once, even if it is overrepresented in more than one cell. It is important to keep in mind that the thresholds in the two measures are not comparable or equitable. Not only do the measures answer two different questions of disproportionality and produce two different kinds of results, they are also on two different scales and the range of values of the measures are different as well. The E-formula values are equidistant from the point of nondisproportionality (the percentage of a racial/ethnic group in general education) in both directions, for overrepresentation and underrepresentation. They can take any value, including negative values, as we have seen earlier. If you will also recall, for all risk ratio measures, including the Alternate Risk Ratio, the range of values for overrepresentation is different from 47

underrepresentation. Overrepresentation can be any number greater than one (>1.00) but underrepresentation lies between less than one and more than zero (<1.00 and >0.00). Because the two measures produce two different types of indices, the thresholds in the two measures should not be compared against each other. One must not be tempted to equate, for example, two standard errors threshold in the E-formula with 2.00 in the Alternate Risk Ratio. As expected, the total number of districts overrepresented is large when the E-formula threshold is low and this number gets smaller as the threshold gets higher. This phenomenon is more noticeable for smaller districts, but less so for larger districts. For districts under 1,000 students, nine districts out of the total of 103 districts (about nine percent) remain overrepresented at the three standard errors threshold; this number drops to one district at four standard errors threshold and none at five standard errors or higher thresholds. Under the same thresholds, the number of overrepresented districts goes down at a relatively slower pace for districts in the successively higher enrollment groupings. The disproportionality status of districts with enrollments over 20,000 hardly changes until the threshold is considerably high and it gets harder for larger districts to move away from the territory of overrepresentation even at higher thresholds. This is because the E-formula allows proportionately lesser flexibility or smaller Region of Tolerance for Disproportionality to larger districts, compared to smaller districts; but for very large districts, the higher thresholds do not always offset the lesser flexibility imposed by the E-formula. Like the E-formula results, the number of districts overrepresented under the Alternate Risk Ratio is also large when the threshold is low and this number gets smaller when the threshold gets higher. But the similarity ends there. All districts, regardless of their size, seem to follow the same movement pattern when the threshold is increased under the Alternate Risk Ratio. This is unlike the E-formula, where smaller districts tend to move out of the overrepresentation territory sooner than the larger districts, as the threshold increases. One reason for this difference is, as we have discussed before, the calculations under the Alternate Risk Ratio (and for all risk ratio measures) are based on ratios or fractions, which can be derived from large as well as small numbers, and therefore, the district size does not have any effect on the increasing thresholds. Also, recall that Alternate Risk Ratio (and all other risk ratio measures) does not allow any Region of Tolerance for Disproportionality, which also contributes to the lack of differential movement patterns of different sized districts under a range of thresholds. Table 17 shows the number of districts one would expect to find overrepresented under various thresholds in the two measures. If one were to select the E-formula as the measure of choice and three standard errors as the threshold for overrepresentation, then 379 districts would likely be overrepresented. If the threshold was raised to higher standard errors such as four or five, then the total number of districts overrepresented would go down to 292 or 228, respectively. However, one needs to exercise caution in using higher thresholds just to reduce the number of districts because there is a high probability that the districts that are overrepresented at lower thresholds but would be excluded at higher thresholds, may, in fact, have true 28 disproportionality. 29 Also note that, under the E-formula, proportionately more and more of the large districts and proportionately fewer and fewer of the small districts (perhaps, none) would likely be selected as the threshold gets higher and higher. If one were to select Alternate Risk Ratio as the measure of choice and 3.00 as the threshold for overrepresentation, then 176 districts would likely be overrepresented. If the threshold was increased to 4.00, then the number of districts overrepresented would go down to 95 and further 48

increment of the threshold would result in even fewer districts. Again, like the E-formula thresholds, caution should be exercised in using higher thresholds in the Alternate Risk Ratio just to reduce the number of districts, because districts that are overrepresented at lower thresholds but would be excluded at higher thresholds, may, in fact, have true disproportionality. Unlike the E-formula, however, large districts are equally likely to be selected as small districts in the Alternate Risk Ratio at all thresholds. 49

5. The Case for a Joint Measures Approach Now that we have some idea about the number of districts of different enrollment sizes that would likely be selected under various thresholds in the E-formula and in the Alternate Risk Ratio, some questions still remain. Does one measure capture the essence of racial/ethnic disproportionality in special education more appropriately than another? How critical are the loss of elements that are unique in one measure when another measure is selected for disproportionality calculations? Does the E-formula affect large districts unfavorably? Is it possible to combine the best elements of both measures? Before we can answer these questions we need to acknowledge that each disproportionality measure has its strengths and weaknesses. Despite being the top-rated measure, the E-formula is not sensitive to the differences between the district and the state incidence rates; the results of the E-formula are not comparable among districts; and large districts seem to be more susceptible to disproportionality than small districts. These limitations can be compensated if the results of the E- formula are combined with the results of the next highest-rated measure, the Alternate Risk Ratio, which does not have the same limitations. Alternate Risk Ratio, however, has its own weaknesses such as, the results of the Alternate Risk Ratio do not make any difference among different sized districts; the measure does not allow any Region of Tolerance for Disproportionality; and the number of exclusions of cells from disproportionality calculations are considerably high, which can be compensated by the E-formula because the E-formula does not have these limitations. If a district is disproportionate in one measure (Measure A, for example) but not in another measure (Measure B, for example), then one might argue that Measure A treats the district unfavorably. Similarly, if another district is disproportionate in Measure B but not in Measure A, then this district can make the same argument about Measure B. Situations like this produce a dilemma for policymakers in the state in selecting a single measure that best serves the needs of the districts as well as of the state. Not only would it be unfair to use different measures for different districts, it would also be improper to even consider such an approach. If two measures of disproportionality compensate each other for their weaknesses and bring together their individual strengths, then a joint measures approach appears to be a viable option. In this approach if a district is disproportionate in both measures, then any arguments of unfavorable effect under one measure or the other would be put to rest. If a district is found to be disproportionate in both measures, it is highly likely that the district is truly disproportionate, regardless of the differences in the definitions of the two measures and the limitations of the individual measures. To examine the effect of two measures in determining disproportionality we are proposing to combine the two top-rated measures and apply them jointly to all districts in California using the same data. In this approach, if a district is found overrepresented in both measures, then the district is considered overrepresented. The overrepresentation must occur in the same cell in both measures (for example, White students in Autism disability category must be overrepresented in the E-formula and in the Alternate Risk Ratio). If a district is overrepresented in one cell in the E-formula but in another cell in the Alternate Risk Ratio, then the district is not considered overrepresented in the joint measures approach. For example, if a district is overrepresented for African-American students in the ID disability category in the E-formula but not in the Alternate Risk Ratio, even if the district is overrepresented for another racial/ethnic group in the same or another disability category in the Alternate Risk Ratio, then the district is not considered overrepresented. The results of the 50

joint application of the two top-rated measures, displaying the number of districts under various combinations of thresholds are shown in Table 18. 30 Of the 573 districts overrepresented at one standard error threshold in the E-formula and of the 542 districts overrepresented at threshold level more than 1.00 in the Alternate Risk Ratio, 436 are the same districts in both measures and they are overrepresented in the same cell. If the E-formula threshold is raised to three standard errors and the Alternate Risk Ratio threshold is raised to 4.00 or more, then the number of districts overrepresented at these thresholds becomes 77, which is far fewer than 379 districts at three standard errors alone in the E-formula or 95 districts at 4.00 or more alone in the Alternate Risk Ratio. At the other end of the spectrum, 18 districts are overrepresented at eight standard errors threshold in the E-formula as well as at 5.00 or more in the Alternate Risk Ratio. It is obvious that in a single measure, the E-formula or the Alternate Risk Ratio, the number of districts overrepresented at any individual threshold is going to be much higher than when the thresholds are combined under the joint measures approach. Note that these districts are overrepresented in the same cells (for the same racial/ethnic group in the same disability category) in both measures, despite the differences in the definitions of the two measures. In other words, any district found to be disproportionate in the joint measures approach, meets the requirements of disproportionality as defined in each measure. Table 18. Number of Districts Overrepresented under Both E-formula and Alternate Risk Ratio Measures* The Alternate Risk Ratio Thresholds The E-formula Thresholds More than 1.00 2.00 or More 3.00 or More 4.00 or More 5.00 or More 542 382 176 95 44 One Standard Error 573 436 340 166 92 43 Two Standard Errors 459 349 285 146 83 39 Three Standard Errors 379 295 237 128 77 35 Four Standard Errors 292 232 186 107 68 34 Five Standard Errors 228 194 148 94 62 31 Six Standard Errors 176 152 115 74 47 25 Seven Standard Errors 134 120 89 59 39 22 Eight Standard Errors 100 92 66 42 31 18 * Districts in this table do not include the results of calculations for the 95 districts that reported a higher number of students of any racial/ethnic group in special education than in general education. See Appendix D showing all districts that are likely to be overrepresented if none of the districts were excluded. There are several reasons why a joint measures approach is an attractive option over using a single measure. First, a joint measures approach incorporates the best elements of both measures. In this approach, the measures combine their individual strengths in the disproportionality determination process and compensate each other for their weaknesses. Second, the two measures in this analysis are the top two measures based on the ratings, and they also represent the two broad categories of disproportionality measures (Composition and Risk). Since each category defines racial/ethnic disproportionality differently, a joint measures approach would bring both definitions together. Third, if a district is disproportionate in both measures, not just in one, then there is a high 51

probability that the district has true disproportionality. Finally, a joint measures approach allows the user to examine disproportionality of districts at reasonable levels of thresholds in both measures to select a desirable number of districts. The same desired outcome would require setting a much higher threshold in a single-measure approach, as discussed earlier. Statistically, high level thresholds would let some districts escape disproportionality issues altogether when, in fact, these districts may have true disproportionality at lower thresholds. An analysis of the relationship between the two measures revealed that the correlation coefficient (Pearson s r) between the E-formula and the Alternate Risk Ratio is 0.20, which means that only about four percent (r 2 ) of the total variation in one measure is accounted for by the relationship with the other measure. This indicates that the two measures are virtually independent, which puts the Joint Measures Approach on a stronger footing than any single measure by itself. The results from the joint measures approach can be interpreted in relatively simple language. Using the example of the 77 districts that are overrepresented at 3.00 standard errors threshold in the E-formula and at 4.00 or more in the Alternate Risk Ratio: for those racial/ethnic groups that are overrepresented in a specific disability category (or categories), not only is there a very high probability 31 that the differences between the proportions of individual racial/ethnic groups in their specific disability category (or categories) of overrepresentation and their corresponding general education proportions in the districts are true or real differences, the Risk for each of the same racial/ethnic groups in the same disability category (or categories) in these districts is more than 4.00 times the Risk of all other racial/ethnic groups combined in the state in the same disability category (or categories). As one can see, in the joint measures approach, the determination of disproportionality satisfies two sets of criteria, which makes the approach far more rigorous than either measure by itself. 5.1. Four Case Studies under the Joint Measures Approach What are the characteristics of a district when it is disproportionate in one measure but not in another? Under what circumstances does a district become disproportionate in both measures? To address these questions and to get an insight into the demographic characteristics of districts, we have identified four districts showing four different outcomes under the joint measures approach. Again, to keep the analysis simple, we will focus on overrepresentation in the six major disability categories, using 3.00 standard errors threshold in the E-formula and 4.00 in the Alternate Risk Ratio. The characteristics of the four districts are: District A, overrepresented in the E-formula, but not in the Alternate Risk Ratio District B, overrepresented in the Alternate Risk Ratio, but not in the E-formula District C, overrepresented in the E-formula and in the Alternate Risk Ratio, but not in the same cell; and District D, overrepresented in both the E-formula and the Alternate Risk Ratio in the same cell, and therefore, is overrepresented under the joint measures approach. All necessary data used in the calculations for the E-formula and the Alternate Risk Ratio for these districts are shown in Table 19. These data include enrollments in general education and in the six major disability categories in special education for the state and the four districts listed above, broken down by seven racial/ethnic groups. The results of the calculations are highlighted in the table and are discussed below. 52

Table 19. Enrollments and Results of the E-formula and the Alternate Risk Ratio Calculations in Four Districts State Enrollments: Native Asian Pacific Black Hispanic White Multiple Total Total GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 Autism (N) 270 7,270 204 4,066 17,890 21,140 1,113 51,953 Emotional Disturbance (N) 315 712 102 5,102 7,370 11,438 500 25,539 Intellectual Disability (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 Other Health Impairment (N) 431 1,843 215 6,513 17,606 23,342 749 50,699 Spec. Learning Disability (N) 2,301 8,782 1,238 34,908 166,458 67,520 2,668 283,875 Speech and Language (N) 990 12,049 743 8,808 71,283 44,360 2,371 140,604 District A: Total GE Enrollment (N) 265 905 67 504 2,494 8,460 36 12,731 Autism (N) 0 4 0 4 11 91 3 113 Emotional Disturbance (N) 0 2 0 3 8 64 4 81 Intellectual Disability (N) 2 10 0 5 21 74 1 113 Other Health Impairment (N) 0 3 2 5 19 76 1 106 Spec. Learning Disability (N) 13 26 2 31 141 351 14 578 Speech and Language (N) 2 16 2 12 80 184 10 306 District B: Total GE Enrollment (N) 45 217 26 123 974 2,391 230 4,006 Autism (N) 0 3 0 0 9 26 1 39 Emotional Disturbance (N) 1 0 0 2 6 42 4 55 Intellectual Disability (N) 0 0 0 0 6 8 3 17 Other Health Impairment (N) 0 0 0 1 6 29 1 37 Spec. Learning Disability (N) 0 0 3 9 41 122 4 179 Speech and Language (N) 1 6 1 4 34 49 4 99 District C: Total GE Enrollment (N) 64 563 70 786 1,018 2,640 43 5,184 Autism (N) 0 1 0 4 6 23 2 36 Emotional Disturbance (N) 1 0 0 12 7 37 1 58 Intellectual Disability (N) 0 1 0 4 4 31 1 41 Other Health Impairment (N) 2 1 0 14 4 43 2 66 Spec. Learning Disability (N) 4 15 1 77 76 147 5 325 Speech and Language (N) 0 9 0 20 16 54 5 104 District D: Total GE Enrollment (N) 29 3,932 66 90 3,993 1,420 189 9,719 Autism (N) 0 104 4 3 59 40 3 213 Emotional Disturbance (N) 0 2 0 2 7 12 1 24 Intellectual Disability (N) 0 22 0 1 17 5 0 45 Other Health Impairment (N) 0 11 1 11 42 46 3 114 Spec. Learning Disability (N) 3 29 1 3 152 29 5 222 Speech and Language (N) 0 107 1 4 125 49 10 296 GE = General Education Bold = Overrepresentation in the E-formula at 3.00 standard errors threshold Bold and Italics = Overrepresentation in the Alternate Risk Ratio at 4.00 threshold Bold, Italics, and Shaded = Overrepresentation in both the E-formula and the Alternate Risk Ratio 53

District A This is a district with 12,731 students in general education and 1,388 students or about 10.90 percent in special education (data not shown in table), with 1,297 students in the six major disability categories. In the E-formula, the district shows overrepresentation of White students in Autism disability category. It also shows overrepresentation of students with Multiple racial/ethnic backgrounds in Autism, Emotional Disturbance, Specific Learning Disability, and Speech and Language Impairment in the E-formula. In this district, none of the racial/ethnic groups are overrepresented in the Alternate Risk Ratio. Note that all 42 cells (six disability categories multiplied by seven racial/ethnic groups) for this district are used in the E-formula calculations, but only 11 cells (with cell size 20 or more) are used in the Alternate Risk Ratio calculations. Let us take a closer look at the students with Multiple racial/ethnic backgrounds in District A. They constitute 0.28 percent in general education but 2.65 percent in Autism, which is 9.46 times higher than their general education percentage (data not shown in table). Even though the number of students in Autism is very small (only three), still at 3.00 standard errors threshold in the E-formula, the group shows overrepresentation, which however, disappears if the threshold is raised to 5.00 standard errors (data not shown in table). This group is not subject to the calculations in the Alternate Risk Ratio due to the small cell size. (For the sake of academic interest, the Alternate Risk Ratio for the Multiple racial/ethnic group in Autism would be 9.81, if the group was not excluded from calculations. Note that the discrepancies in both measures for this group are extremely high.) White students in Autism disability category in the same district portray a slightly different picture. Because the cell size in this case is 91, they are subject to both the E-formula and the Alternate Risk Ratio calculations. White students constitute 66.45 percent in general education but 80.53 percent in Autism, resulting in overrepresentation at 3.00 standard errors threshold in the E-formula. However, they no longer remain overrepresented at 4.00 standard errors or beyond (data not shown in table). The Alternate Risk Ratio for White students in Autism is 1.54 (data not shown in table), which is far less than the 4.00 threshold, and therefore, they are not overrepresented in this measure. Results from the joint measures approach should be interpreted in simple language so it is easily understood by the users of this approach. The results for White students in Autism can be stated as: there is a very high probability, perhaps more than 99 percent, that the difference between the proportion of White students in Autism and the proportion of White students in general education in the district is true difference (or, there is perhaps less than one percent probability that the difference between the two proportions is due to chance) but the risk of White students in Autism in the district is only 1.54 times (far less than 4.00 times) as the statewide risk of students in all other racial/ethnic groups combined in Autism. Based on the results of the two measures, as applicable for individual cells, District A is not overrepresented under the joint measures approach. If we used only one measure, the E-formula, the district would be overrepresented for more than one reason. But we must also take note that enrollments of the Multiple racial/ethnic group in Autism and Emotional Disturbance disability categories are quite small. A single family with two or three disabled children could easily create such a situation, raising an issue if a district should be identified as overrepresented because of such a small enrollment size for a group. Under the joint measures approach, situations such as this are avoided by combining the E-formula results with those of the Alternate Risk Ratio, which excludes such cells from calculations. The only large cell with 91 White students in Autism that shows overrepresentation in the E-formula does not show overrepresentation in the Alternate Risk 54

Ratio at our selected thresholds. Since none of the cells in the district are overrepresented in both measures, the district is not overrepresented under the joint measures approach. District B This district has 4,006 students in general education and 452 students or about 11.28 percent in special education (data not shown in table), with 426 students in the six major disability categories. The district is overrepresented in only one cell in the Alternate Risk Ratio for White students in the Emotional Disturbance (ED) disability category. None of the cells are overrepresented in the E- formula measure at our selected threshold (3.00 standard errors), and therefore, the district is not considered overrepresented under the joint measures approach. The results for this district can be stated as: although the risk of White students in the ED disability category in the district is 4.00 or more times as the statewide risk of students in all other racial/ethnic groups combined in the ED disability category, but the probability is not sufficiently high that the difference between the proportion of White students in the ED disability category and the proportion of White students in general education in the district is a true difference. Therefore, the district is considered not overrepresented under the joint measures approach. District C This is a district with 5,184 students in general education and 654 students or about 12.62 percent in special education (data not shown in table), with 630 students in the six major disability categories. In this district, several groups of students are overrepresented in the E-formula calculations: (1) Black or African-American students in Specific Learning Disability, (2) White students in Intellectual Disability, and (3) students of Multiple racial/ethnic group in Autism and in Speech and Language Impairment. In the Alternate Risk Ratio calculations, only White students are overrepresented in Emotional Disturbance. Since none of the same racial/ethnic groups of students are overrepresented in the same disability category in both measures, the district is considered not overrepresented in the joint measures approach. District D This district has 9,719 students in general education and 961 students or about 9.89 percent in special education (data not shown in table), with 914 students in the six major disability categories. In this district, five groups of students are overrepresented in the E-formula: (1) Black or African- American students in Emotional Disturbance; (2) Black or African-American students in Other Health Impairment, (3) Hispanic students in Specific Learning Disability, (4) White students in Emotional Disturbance, and (5) White students in Other Health Impairment. In the Alternate Risk Ratio calculations, three groups of students are overrepresented: (1) Asian students in Autism, (2) White students in Autism, and (3) White students in Other Health Impairment. Only White students in the Other Health Impairment (OHI) disability category are overrepresented in both measures. Let us take a closer look at White students in the OHI disability category. First of all, the cell size for this group is 46, a reasonably large number that is considerably higher than the minimum 20, and therefore, is subject to the calculations in both measures. Second, in the E-formula measure, the discrepancy between special education and general education percentages for this group in the OHI 55

disability category is so large that the group remains overrepresented until the threshold is raised to eight standard errors (data not shown in table)! Third, in the Alternate Risk Ratio measure, the discrepancy between this group (White students) and the comparison group (all non-white students combined) is also so large that the group remains overrepresented even if the threshold is raised from 4.00 to 5.00 (data not shown in table). In other words, not only is there a very high probability that the difference between the proportion of White students in the Other Health Impairment disability category and the proportion of White students in general education in the district is a true difference, but the risk of White students to be in the Other Health Impairment disability category in the district is also much higher than the statewide risk of students in all other racial/ethnic groups combined to be in the same disability category. There would be little debate that this group is truly overrepresented in the district. 5.2. Recommendation In selecting a measure or measures for examining disproportionality, one needs to keep in mind that each measure answers a different question on disproportionality and each has its strengths and weaknesses. The user must decide beforehand if a particular question, and therefore, a particular measure, addresses the needs of the districts and the state and incorporates the intent of the law. If a single question (and therefore, a single measure) does not capture all or most of the elements of disproportionality that one would like to see addressed, then using more than one measure to examine disproportionality should not be ruled out. The analysis in this paper provides helpful information to policy makers in the state in selecting a measure or measures. Based on the analysis of various measures in this paper, we believe that a joint measures approach provides much better information on racial/ethnic disproportionality in special education than any single measure. Therefore, we strongly recommend a joint measures approach to examine racial/ethnic disproportionality in special education. We also recommend that the E-formula and the Alternate Risk Ratio should be the two measures of choice for reasons discussed in the preceding pages. The strengths of the E-formula (Differentiated Range of Tolerance for Disproportionality for Different Sized Districts, for example) compensate for the lack of them in the Alternate Risk Ratio, and the strengths of the Alternate Risk Ratio (Comparability of Results across Districts, for example) compensate for the lack of them in the E-formula. The determination of disproportionality should be based on when a racial/ethnic group in a program category (a cell) is disproportionate in both measures not just in only one of the measures. The thresholds in the two measures can be set based on empirical analysis of data, as shown in Table 18, and based on consensus among the stakeholders in the state (one such example is 3.00 standard errors in the E-formula and 4.00 in the Alternate Risk Ratio). If for some reason one were to select only one measure for disproportionality calculations, then based on the data and analysis presented in this paper, the E-formula offers the most promising approach in determining racial/ethnic disproportionality in special education. It has the necessary strengths and fewest weaknesses among all measures of disproportionality that we have examined. The Alternate Risk Ratio would be a good second choice. If a joint measures approach is used to examine disproportionality, it is also possible to weight the results of each measure in determining overrepresentation or underrepresentation for a district. The weighting factor will depend upon the relative importance of the questions associated with the 56

measures selected. In the example using the Alternate Risk Ratio and the E-formula, if comparability of districts is more important to the user than the differences in the racial/ethnic composition between general education and special education, then the results of the Alternate Risk Ratio should be weighted more than the results of the E-formula. If, on the other hand, the discrepancy between general education and special education among the various racial/ethnic groups is the dominant issue, then the E-formula would carry more weight than the Alternate Risk Ratio. Prior to selecting measures (or a measure) for examining disproportionality and setting thresholds, one would need to review the results from the measures (or measure) in light of their implications for any monitoring and follow-up activities. If the districts showing overrepresentation or underrepresentation are subject to on-site review by the state, then factors such as available resources, personnel, and the time necessary for monitoring should be taken into consideration in the selection process. However, these external factors should not drive the process of selecting measures or setting thresholds in examining disproportionality; these factors are secondary to the process that will identify districts that are truly disproportionate, based on the definitions of the individual measures. One must exercise caution so that the notion of a pre-determined number of districts to be selected is not viewed as a quota. Also, any pre-determined number of districts to be selected has the risk of excluding districts that may be truly disproportionate. 57

6. Significant Disproportionality While the underlying purpose of examining disproportionality is to correct any racial/ethnic imbalance in special education in relation to general education and/or any discrepancy among various racial/ethnic groups, significant disproportionality takes this effort a step further. 32 When a district is identified to have significant disproportionality (overrepresentation) for a racial/ethnic group in a program or disability category, it is required to spend 15 percent of its federal funds under IDEA during each year it remains significantly disproportionate (overrepresented) to redress the disproportionality issues. It is challenging enough for a district to address any disproportionality issues in general, but it is probably far less desirable to be in a situation of significant disproportionality because it leads to difficult fiscal consequences plus additional monitoring and oversight from the state. Therefore, the definition of significant disproportionality could become a contentious item for districts that would be adversely affected by it. There are no specific directions from OSEP on how to define significant disproportionality - only general guidelines. It is up to the states to define significant disproportionality in relation to disproportionality in general. However, any definition of significant disproportionality must be approved by OSEP before a state can use it with districts. There are at least three aspects of racial/ethnic disproportionality based on which one can define significant disproportionality. They are: frequency, severity, and persistency. Before we go further into the details, it would be helpful to set the context of disproportionality in which the definitions of these terms and the ways to define significant disproportionality would be meaningful. Let us assume that a state adopts the joint measures approach to determine racial/ethnic disproportionality in special education and the measures of choice are the E-formula and the Alternate Risk Ratio. Also assume that the thresholds for disproportionality (overrepresentation) are set at 3.0 standard errors for the E-formula and at 4.0 for the Alternate Risk Ratio. A district would be considered disproportionate for a racial/ethnic group in a program or disability category (cell) if both the E-formula and the Alternate Risk Ratio results for this cell (the same cell) cross these two thresholds. For the sake of clarity, we will call this situation simple disproportionality and distinguish it from significant disproportionality which we will define in this chapter. Within this framework and based on the concepts of frequency, severity, and persistency, as stated above, significant disproportionality can be defined in a number of ways, as described below. 6.1. Frequency Frequency is the number of cells that are disproportionate (or have simple disproportionality) among all possible cells in a program category in a district in a given year. In disability categories, for example, a district may be disproportionate in, say, three different cells (White students in Emotional Disturbance, African-American students in Intellectual Disability, and Hispanic students in Specific Learning Disability), out of 42 possible cells (six disability categories times seven racial/ethnic groups). It may also be expressed as the percentage of the cells that are disproportionate out of the total number of possible cells in a district. In the above district, 7.1 percent ((3/42)*100) of the cells are disproportionate. 58

Under frequency, significant disproportionality may be defined as a number or percentage of cells with simple disproportionality as a threshold beyond which a district would be considered significantly disproportionate. For example, if the threshold is set at six cells (or 14.3 percent) for disability categories, then a district would be significantly disproportionate if it has more than six cells that have (simple) disproportionality or if more than 14.3 percent of the cells are disproportionate. Using the previous example, where the district has only three cells that are disproportionate (7.1 percent), it is not significantly disproportionate. Frequency offers a relatively simple way to define significant disproportionality for a district. It is based on a snapshot of one year s data on (simple) disproportionality. It does not provide any information on the nature of disproportionality for a particular racial/ethnic group in a particular disability category that may be severe or persistent (defined later). One of the weaknesses of this definition is that a district may have a number of cells that are disproportionate year after year or the degree of disproportionality in any of these cells may be severe, but they may be so few that the district remains below the threshold and can escape significant disproportionality altogether. For this reason, and perhaps others, frequency is rarely used as a definition of significant disproportionality by the states. 6.2. Severity Severity may be defined as a situation when the level or degree of disproportionality for a racial/ethnic group in a program or disability category (cell) is very high or more severe than (simple) disproportionality. In this definition, significant disproportionality can be determined by setting the thresholds in the two measures (the E-formula and the Alternate Risk Ratio) at higher levels than those for (simple) disproportionality, as shown below. Under severity, the E-formula threshold could be raised from 3.0 standard errors for (simple) disproportionality to a higher standard error value (such as, 3.5, 4.0, etc.) for significant disproportionality and the Alternate Risk Ratio threshold could be raised from 4.0 for (simple) disproportionality to a higher value (such as 4.5, 5.0, etc.) for significant disproportionality or at any other combination. Note that a higher threshold in either measure alone would also create significant disproportionality. If in a particular year a district is found to be disproportionate for a racial/ethnic group in any program or disability category (cell) at these higher thresholds, then the district would be considered significantly disproportionate for that racial/ethnic group in that program or disability category in that year. This definition is fairly simple and should be intuitive to the user. A higher level of thresholds in either or both measures would identify a racial/ethnic group in a program or disability category in districts that are very highly disproportionate beyond any reasonable doubt (or at an extremely high probability), indicating that the level of disproportionality is more severe or significant than (simple) disproportionality at lower levels of thresholds. The lower right quadrant of Table 18 would give an idea about the number of districts that are likely to be identified as significantly disproportionate (overrepresented in disability categories) in California at various combinations of thresholds during the 2009-10 school year. Note that these districts are also disproportionate at the thresholds set for (simple) disproportionality. Like frequency, severity is also based on a snapshot of one year s data. But unlike frequency, which provides an overall picture of a district, severity focuses on specific cells in the district. Neither frequency nor severity addresses continuing disproportionality of specific racial/ethnic groups in 59

specific program categories year after year which could have adverse impact on these groups. Like frequency, severity is also rarely used by the states to define significant disproportionality. 6.3. Persistency Persistency may be defined as a situation when a district is disproportionate (or has simple disproportionality) for a particular racial/ethnic group in a particular program or disability category in both the E-formula and the Alternate Risk Ratio for a number of years during a period of years. The number of years of disproportionality may be continuous such as two or three years in a row or noncontinuous such as any three years during a four-year period or any three years during a five-year period or in some other combination. If a district is persistently disproportionate for the same racial/ethnic group in the same program or disability category (the same cell) then the district would be significantly disproportionate for that cell in the current or the final year. There are several ways to characterize persistency in disproportionality, and each results in individual definitions of significant disproportionality. Persistency is probably the most common aspect of disproportionality based on which many states define significant disproportionality; the definitions vary from state to state, however. To understand the differences among various definitions of significant disproportionality under persistency, the effects of these definitions are illustrated using 12 scenarios of disproportionality for a specific cell (a specific racial/ethnic group in a specific disability category) during a 15-year period, as shown in Table 20. From the state s perspective, each scenario may also be viewed as an individual district displaying its disproportionality status during the 15-year period. Keep in mind that these scenarios are purely hypothetical and they may not reflect the actual disproportionality status of districts in a state. The scenarios are used for three reasons: (1) to study the effects of each definition on a variety of disproportionate situations; (2) to compare the impact of each definition on these scenarios individually and collectively; and (3) to evaluate the relative effectiveness of the definitions by comparing their results. The scenarios in Table 20 are arranged in increasing order of the number of disproportionate years in a particular cell during the 15-year period (Column b). If you will notice, they are also complementary to each other from the end scenarios to the middle scenarios, with respect to disproportionate and non-disproportionate years. For example, Scenario A is complementary to Scenario Z, Scenario B is complementary to Scenario K, and so forth. The choice of complementary scenarios for this analysis is to ensure that there is no bias in assessing the effects of a particular definition of significant disproportionality on a particular scenario without taking into account its effect on the opposite (complementary) scenario. The overall effect of a definition can be judged (and compared against the effects of other definitions) in a simple (although not very precise) way by summing its effect on all scenarios in the table. It is important to note that the 12 scenarios in Table 20 are not a representative sample of all possible scenarios that may exist in the districts in a state; they are only a subset of the all possible scenarios. Also, the choice of the specific scenarios in Table 20, along with their cyclic properties (repeating every few years) and even distribution (only one of each scenario in the table), is only to test the effects of various definitions on identical disproportionate situations. In reality, the actual scenarios would probably reflect random distributions of disproportionate cells during any 15-year period, and also the overall scenarios are less likely to be distributed evenly throughout the state. Although one could always add other disproportionality scenarios to the existing ones in Table 20, for the purposes of comparing the effects of the definitions of significant disproportionality on 60

various and diverse disproportionality situations, the choice of the scenarios in Table 20 seems to be adequate. Table 20. Various Scenarios of Disproportionality in a Cell during a 15-year Period in a District Disproportionality Scenario Year Total Number of Years / Cells Number Disproportionate Percent Disproportionate of Total (%) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (a) (b) (b/a) A 15 0 0.0 B D D D 15 3 20.0 C D D D D D 15 5 33.3 D D D D D D D 15 6 40.0 E D D D D D D D 15 7 46.7 F D D D D D D D 15 7 46.7 G D D D D D D D D 15 8 53.3 H D D D D D D D D 15 8 53.3 I D D D D D D D D D 15 9 60.0 J D D D D D D D D D D 15 10 66.7 K D D D D D D D D D D D D 15 12 80.0 Z D D D D D D D D D D D D D D D 15 15 100.0 Total 180 90 50.0 D = Disproportionate at specified thresholds in the same cell in the joint measures approach Since there are so many ways one can define significant disproportionality under persistency that we cannot possibly discuss all of them in this paper, we have elected to present only a few that we believe would reflect some of the most commonly understood notions of significant disproportionality. These definitions and their results will give the reader an idea about how to formulate additional definitions, if necessary, and how they might impact the districts in a state. The definitions under persistency ensure that if a district is identified (or is a candidate to be identified) as significantly disproportionate for a cell in a particular year then it must also be disproportionate in the same cell in that year. In other words, a district would not be identified as significantly disproportionate in a particular year (Year Y) if it is not also disproportionate in the same year (Year Y). It would seem rather odd if a district is identified as significantly disproportionate in a particular year when it is not disproportionate in that year, even though it has been disproportionate in the preceding years which propelled the district into significant disproportionality. 33 61

Another underlying principle in the proposed definitions under severity is that a well formulated definition should not identify too many or too few instances of significant disproportionality in relation to the instances of (simple) disproportionality. The definitions that we are about to explore differentiate themselves from each other by their ability to address various disproportionality scenarios in Table 20 and by identifying significant disproportionality in these scenarios in varying degrees. If a definition produces significant disproportionality in all or most of the 90 instances of (simple) disproportionality in Table 20, then it is probably an extremely liberal definition and would be practically useless; and if it detects none or very few instances of disproportionality as significant, then it would be considered a very restrictive or limiting definition and would also be of no value. Let us approach these definitions systematically from simple to some of the more complex ones and examine their effects on these scenarios. Definition P-1 Perhaps the simplest definition under persistency is: if a district is disproportionate at thresholds for (simple) disproportionality in the same cell for two years in a row, including the current year, then the district would be considered to have significant disproportionality in that cell in the current or the final year. In other words, if a district is disproportionate in the current year (Year Y) for a cell and is also disproportionate in the same cell in the preceding year (Year Y-1) then the district would be considered significantly disproportionate in the current year (Year Y). Table 21 shows the effects of this definition on the same disproportionality scenarios as in Table 20. Definition P-1 addresses seven of the 12 scenarios, and overall, it identifies 43 instances of significant disproportionality out of 90 instances of (simple) disproportionality (about 48 percent) in all scenarios combined during the 15-year period. Individual scenarios provide a detailed picture of how well this definition addresses the various scenarios of disproportionality. The district would be significantly disproportionate in the second year of every two-year period of disproportionality (Scenarios D, E, H, and J). In Scenarios I and K, the district would be significantly disproportionate in the second year and the third year during each three-year period of continuous disproportionality. In Scenarios F and G, the district can stay out of being significantly disproportionate by moving in and out of a disproportionate situation every year. And in Scenario Z, except for the first year, the district would have significant disproportionality every year. Note that the district is disproportionate in seven years during the 15-year period in each of the Scenarios E and F. However, it is not significantly disproportionate in any of those seven years in Scenario F but is significantly disproportionate in three of those seven years in Scenario E. One might wonder if Definition P-1 is unfair to Scenario E, when both scenarios (E and F) result in comparable instances of (simple) disproportionality over the same period, but yield different outcomes of significant disproportionality. The critical difference between these two scenarios is the number of consecutive years of disproportionality during the 15-year period; in the case of Scenario E, any benefit resulting from of two consecutive years of non-disproportionality becomes inconsequential to the two consecutive years of disproportionality. A similar situation exists between Scenarios G and H under Definition P-1 during the same period. Despite having the same number (eight) of disproportionate years in these two scenarios, the district is significantly disproportionate for four of these eight years in Scenario H but not in Scenario G at all. Once again, the critical difference between these two scenarios is the number of consecutive years of disproportionality; two years of disproportionality in a row in Scenario H makes 62

the district significantly disproportionate, whereas the lack of it in Scenario G spares the district from being significantly disproportionate. Table 21. Scenarios of Disproportionality and Significant Disproportionality during the 15-year Period Under Definition P-1* Disproportionality Scenario Year Total Number of Years / Cells Number Disproportionate Number Significant Percent Disproportionate of Total (%) Percent Significant of Total (%) Percent Significant of Disproportionate (%) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (a) (b) (c) (b/a) (c/a) (c/b) A 15 0 0 0.0 0.0 NA B D D D 15 3 0 20.0 0.0 0.0 C D D D D D 15 5 0 33.3 0.0 0.0 D D D D D D D 15 6 3 40.0 20.0 50.0 E D D D D D D D 15 7 3 46.7 20.0 42.9 F D D D D D D D 15 7 0 46.7 0.0 0.0 G D D D D D D D D 15 8 0 53.3 0.0 0.0 H D D D D D D D D 15 8 4 53.3 26.7 50.0 I D D D D D D D D D 15 9 6 60.0 40.0 66.7 J D D D D D D D D D D 15 10 5 66.7 33.3 50.0 K D D D D D D D D D D D D 15 12 8 80.0 53.3 66.7 Z D D D D D D D D D D D D D D D 15 15 14 100.0 93.3 93.3 Total 180 90 43 50.0 23.9 47.8 * District is significantly disproportionate in the second year if it is disproportionate in the same cell for two years in a row D = Disproportionate in the same cell at specified thresholds in the joint measures approach; NA = Not Applicable Shaded = Significantly Disproportionate under Definition P-1 Definition P-2 In this definition, if a district is disproportionate in the same cell for any two years during a three-year period, including the current or the final year, then the district would be considered significantly disproportionate in the final year. In other words, if a district is disproportionate in the current year (Year Y) in a cell and is also disproportionate in the same cell for any one of the two preceding years (Year Y-1 or Year Y-2) then the district would be considered significantly disproportionate in the current year (Year Y). The effect of this definition on the various scenarios of disproportionality is shown in Table 22. 63

Table 22. Scenarios of Disproportionality and Significant Disproportionality during the 15-year Period Under Definition P-2* Disproportionality Scenario Year Total Number of Years / Cells Number Disproportionate Number Significant Percent Disproportionate of Total (%) Percent Significant of Total (%) Percent Significant of Disproportionate (%) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (a) (b) (c) (b/a) (c/a) (c/b) A 15 0 0 0.0 0.0 NA B D D D 15 3 0 20.0 0.0 0.0 C D D D D D 15 5 0 33.3 0.0 0.0 D D D D D D D 15 6 3 40.0 20.0 0.0 E D D D D D D D 15 7 3 46.7 20.0 0.0 F D D D D D D D 15 7 6 46.7 40.0 0.0 G D D D D D D D D 15 8 7 53.3 0.0 0.0 H D D D D D D D D 15 8 4 53.3 0.0 0.0 I D D D D D D D D D 15 9 6 60.0 20.0 33.3 J D D D D D D D D D D 15 10 9 66.7 26.7 40.0 K D D D D D D D D D D D D 15 12 11 80.0 46.7 46.7 Z D D D D D D D D D D D D D D D 15 15 14 100.0 86.7 86.7 Total 180 90 63 50.0 35.0 70.0 * District is significantly disproportionate in the third year if it is disproportionate in the same cell in any two years during a three-year period. Note that the district must be disproportionate in the same cell in the third year. D = Disproportionate in the same cell at specified thresholds in the joint measures approach; NA = Not Applicable Shaded = Significantly Disproportionate under Definition P-2 Definition P-2 addresses a relatively large number of the disproportionality scenarios (9 out of 12), but one might also find it quite liberal. The total number of instances of significant disproportionality is 63 in this definition (70 percent), which is quite high. Note that Scenarios F and G, in which the district goes in and out of disproportionality situation every year, would now have significantly disproportionate years under this definition, which was not the case under Definition P-1. Definition P-3 According to this definition, if a district is disproportionate in the same cell at thresholds for (simple) disproportionality for the past three years in a row, including the current year, then the district would be considered significantly disproportionate in the current or the final year. This may also be stated as, if a district is disproportionate in the current year (Year Y) in a cell and is also disproportionate in the same cell for the two preceding years (Year Y-1 and Year Y-2), then the district would be 64

considered significantly disproportionate in the current year (Year Y). The effect of this definition on the various scenarios of disproportionality is shown in Table 23. Table 23. Scenarios of Disproportionality and Significant Disproportionality during the 15-year Period Under Definition P-3* Disproportionality Scenario Year Total Number of Years / Cells Number Disproportionate Number Significant Percent Disproportionate of Total (%) Percent Significant of Total (%) Percent Significant of Disproportionate (%) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (a) (b) (c) (b/a) (c/a) (c/b) A 15 0 0 0.0 0.0 NA B D D D 15 3 0 20.0 0.0 0.0 C D D D D D 15 5 0 33.3 0.0 0.0 D D D D D D D 15 6 0 40.0 0.0 0.0 E D D D D D D D 15 7 0 46.7 0.0 0.0 F D D D D D D D 15 7 0 46.7 0.0 0.0 G D D D D D D D D 15 8 0 53.3 0.0 0.0 H D D D D D D D D 15 8 0 53.3 0.0 0.0 I D D D D D D D D D 15 9 3 60.0 20.0 33.3 J D D D D D D D D D D 15 10 0 66.7 0.0 0.0 K D D D D D D D D D D D D 15 12 4 80.0 26.7 33.3 Z D D D D D D D D D D D D D D D 15 15 13 100.0 86.7 86.7 Total 180 90 20 50.0 11.1 22.2 * District is significantly disproportionate in the third year if it is disproportionate in the same cell for three years in a row D = Disproportionate in the same cell at specified thresholds in the joint measures approach; NA = Not Applicable Shaded = Significantly Disproportionate under Definition P-3 Table 23 shows that, overall, Definition P-3 results in 20 instances of significant disproportionality out of 90 instances of (simple) disproportionality (about 22 percent) in all scenarios combined during the 15-year period, which is considerably less than in Definition P-1 (about 48 percent) and Definition P-2 (70 percent). Also, Definition P-3 addresses only three of the 12 disproportionality scenarios, far less than Definitions P-1 and P-2. Definition P-3 has no impact on scenarios with less than three consecutive years of disproportionality. Of particular interest are Scenarios D, E, H, and J, which have two consecutive years of disproportionality each but are not affected by this definition. This is at its extreme in Scenario J, when the district, after being disproportionate for two years in a row, gets out of the disproportionate situation for only one year in every three-year cycle. Although the district is 65

disproportionate in 10 out of 15 years (about 67 percent) during the 15-year period in Scenario J, it escapes being identified as significantly disproportionate entirely during that period. Compare this situation against Scenario I, when the district is disproportionate in nine out of 15 years (one less than in Scenario J) and yet would be identified with three years of significant disproportionality. One might wonder if Scenario J could be attributed to chance alone. Even in Scenario K, when the district is disproportionate in 12 out of 15 years (80 percent), it is identified as significantly disproportionate in only four out of those 15 years (about 27 percent) or about 33 percent of the time it is identified as disproportionate. By comparison, Scenario Z, which records only three more disproportionate years than Scenario K, is significantly disproportionate in 13 out of 15 years (about 87 percent). Three years of non-disproportionality appearing in the Years 4, 8, and 12 in Scenario K result in only four significantly disproportionate years for the district. It appears that, under Definition P-3, if a district encounters continuous disproportionality in a cell every year or almost every year, such as in Scenario Z or Scenario K, it can actually reduce its years of significant disproportionality considerably by working on having the minimum number of non-disproportionate years at strategic intervals. Should the district be identified as significantly disproportionate in more than four years in Scenario K? Definition P-4 If a district is disproportionate in the same cell for any three years during a four-year period, including the current or the final year, then the district would be considered significantly disproportionate in the final year. Stated in another way, if a district is disproportionate in the current year (Year Y) in a cell and is also disproportionate in the same cell for any two of the preceding three years (Years Y-1, Y-2 or Y-3), then the district would be considered significantly disproportionate in the current year (Year Y). The effect of this definition on the various scenarios of disproportionality is shown in Table 24. The language of this definition is fairly simple, but it addresses only four disproportionality scenarios (out of 12). The total number of instances of significant disproportionality during the 15-year period under this definition is 34 out of 90 or about 38 percent, which is around the middle of the range of results from the definitions that we have discussed so far. Definition P-4 addresses some of the loopholes in Definition P-3 as illustrated in Scenarios J and K. Scenario J with 10 instances of (simple) disproportionality is now vulnerable to significant disproportionality when it was not at all under Definition P-3. In Scenario K with 12 instances of (simple) disproportionality, the district now would have 10 of instances of significant disproportionality, compared to only four under Definition P-3. Definition P-5 According to this definition, if a district is disproportionate in the same cell for any three years during a five-year period, including the current or the final year, then the district would be considered significantly disproportionate in the final year. In other words, if a district is disproportionate in the current year (Year Y) in a cell and is also disproportionate in the same cell for any two of the preceding four years (Years Y-1, Y-2, Y-3 or Y-4), then the district would be considered significantly disproportionate in the current year (Year Y). The effect of this definition on the various scenarios of disproportionality is shown in Table 25. 66

Table 24. Scenarios of Disproportionality and Significant Disproportionality during the 15-year Period Under Definition P-4* Disproportionality Scenario Year Total Number of Years / Cells Number Disproportionate Number Significant Percent Disproportionate of Total (%) Percent Significant of Total (%) Percent Significant of Disproportionate (%) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (a) (b) (c) (b/a) (c/a) (c/b) A 15 0 0 0.0 0.0 NA B D D D 15 3 0 20.0 0.0 0.0 C D D D D D 15 5 0 33.3 0.0 0.0 D D D D D D D 15 6 0 40.0 0.0 0.0 E D D D D D D D 15 7 0 46.7 0.0 0.0 F D D D D D D D 15 7 0 46.7 0.0 0.0 G D D D D D D D D 15 8 0 53.3 0.0 0.0 H D D D D D D D D 15 8 0 53.3 0.0 0.0 I D D D D D D D D D 15 9 3 60.0 20.0 33.3 J D D D D D D D D D D 15 10 8 66.7 53.3 80.0 K D D D D D D D D D D D D 15 12 10 80.0 66.7 83.3 Z D D D D D D D D D D D D D D D 15 15 13 100.0 86.7 86.7 Total 180 90 34 50.0 18.9 37.8 * District is significantly disproportionate in the fourth year if it is disproportionate in the same cell in any three years during a four-year period. Note that the district must be disproportionate in the same cell in the fourth year. D = Disproportionate in the same cell at specified thresholds in the joint measures approach; NA = Not Applicable Shaded = Significantly Disproportionate under Definition P-4 Definition P-5 is fairly simple to understand and use, but it produces one of the highest instances of significant disproportionality, 60 out of 90 (about 67 percent), by addressing a relatively large number of disproportionality scenarios (8 out of 12). In Scenarios F and G, the district cannot escape being significantly disproportionate by moving in and out of disproportionate situations every year. Note that Definition P-5 requires monitoring five years of disproportionality data, compared to two, three or four years of disproportionality data in the previous definitions. Definition P-6 In this definition, if a district is disproportionate in the same cell for any four years during a five-year period, including the current or the final year, then the district would be considered significantly disproportionate in the final year. Stated in another way, if a district is disproportionate in the current year (Year Y) in a cell and is also disproportionate in the same cell for any three of the preceding four years (Years Y-1, Y-2, Y-3 or Y-4), then the district would be considered significantly 67

disproportionate in the current year (Year Y). The effect of this definition on the various scenarios of disproportionality is shown in Table 26. Table 25. Scenarios of Disproportionality and Significant Disproportionality during the 15-year Period Under Definition P-5* Disproportionality Scenario Year Total Number of Years / Cells Number Disproportionate Number Significant Percent Disproportionate of Total (%) Percent Significant of Total (%) Percent Significant of Disproportionate (%) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (a) (b) (c) (b/a) (c/a) (c/b) A 15 0 0 0.0 0.0 NA B D D D 15 3 0 20.0 0.0 0.0 C D D D D D 15 5 0 33.3 0.0 0.0 D D D D D D D 15 6 0 40.0 0.0 0.0 E D D D D D D D 15 7 5 46.7 33.3 71.4 F D D D D D D D 15 7 5 46.7 33.3 71.4 G D D D D D D D D 15 8 6 53.3 40.0 75.0 H D D D D D D D D 15 8 6 53.3 40.0 75.0 I D D D D D D D D D 15 9 7 60.0 46.7 77.8 J D D D D D D D D D D 15 10 8 66.7 53.3 80.0 K D D D D D D D D D D D D 15 12 10 80.0 66.7 83.3 Z D D D D D D D D D D D D D D D 15 15 13 100.0 86.7 86.7 Total 180 90 60 50.0 33.3 66.7 * District is significantly disproportionate in the fifth year if it is disproportionate in the same cell in any three years during a five-year period. Note that the district must be disproportionate in the same cell in fifth year. D = Disproportionate in the same cell at specified thresholds in the joint measures approach; NA = Not Applicable Shaded = Significantly Disproportionate under Definition P-5 The language of Definition P-6 is only slightly different from that of Definition P-5, but the scope of their definitions is very different from each other, reflected in the dramatic differences in their results. Definition P-6 produces 25 instances (about 28 percent) of significant disproportionality by addressing only three scenarios (the lowest among all previous measures), compared to Definition P-5 which identifies 60 instances (about 67 percent) of significant disproportionality by addressing eight scenarios. Definition P-6 also generates the second fewest instances of significant disproportionality, behind Definition P-3 which produces the fewest (about 22 percent). Although Definition P-6 scans five years of disproportionality data, it misses several disproportionality scenarios altogether, including Scenarios F and G, the so-called revolving door 68

scenarios and Scenario I with three years of continuous disproportionality in a row, separated by two consecutive non-disproportionate years. Table 26. Scenarios of Disproportionality and Significant Disproportionality during the 15-year Period Under Definition P-6* Disproportionality Scenario Year Total Number of Years / Cells Number Disproportionate Number Significant Percent Disproportionate of Total (%) Percent Significant of Total (%) Percent Significant of Disproportionate (%) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (a) (b) (c) (b/a) (c/a) (c/b) A 15 0 0 0.0 0.0 NA B D D D 15 3 0 20.0 0.0 0.0 C D D D D D 15 5 0 33.3 0.0 0.0 D D D D D D D 15 6 0 40.0 0.0 0.0 E D D D D D D D 15 7 0 46.7 0.0 0.0 F D D D D D D D 15 7 0 46.7 0.0 0.0 G D D D D D D D D 15 8 0 53.3 0.0 0.0 H D D D D D D D D 15 8 0 53.3 0.0 0.0 I D D D D D D D D D 15 9 0 60.0 0.0 0.0 J D D D D D D D D D D 15 10 4 66.7 26.7 40.0 K D D D D D D D D D D D D 15 12 9 80.0 60.0 75.0 Z D D D D D D D D D D D D D D D 15 15 12 100.0 80.0 80.0 Total 180 90 25 50.0 13.9 27.8 * District is significantly disproportionate in the fifth year if it is disproportionate in the same cell in any four years during a five-year period. Note that the district must be disproportionate in the same cell in the fifth year. D = Disproportionate in the same cell at specified thresholds in the joint measures approach; NA = Not Applicable Shaded = Significantly Disproportionate under Definition P-6 In the preceding pages we have explored a number of definitions of significant disproportionality by varying the characteristics of persistency and examined their impacts on various disproportionality scenarios. All of the definitions are relatively simple but they also displayed a number of weaknesses in their inability to address certain disproportionality scenarios which are hard to ignore. To address some of the weaknesses in these definitions we would like to take a different approach by combining more than one condition in persistency, as illustrated in the following two definitions. 69

Definition P-7 This definition, which essentially combines two definitions (Definition P-3 and a new variation), and is perhaps a bit complicated, addresses some of the issues that surfaced in the results of the previous definitions. According to this definition, a district would be significantly disproportionate in the current year, if (a) it is disproportionate in the same cell for three years in a row, including the current year (Definition P-3) or (b) it is disproportionate in the same cell for two years in a row, including the current year and in any two years during the preceding three-year period. In other words, a district would be significantly disproportionate in Year Y, if (a) it is disproportionate in the same cell in Years Y, Y-1, and Y-2 or (b) it is disproportionate in Years Y and Y-1 and is also disproportionate in the same cell in any two of the Years Y-2, Y-3, and Y-4. Table 27 shows the results of this definition on the same scenarios as in the previous tables. Table 27. Scenarios of Disproportionality and Significant Disproportionality during the 15-year Period Under Definition P-7* Disproportionality Scenario Year Total Number of Years / Cells Number Disproportionate Number Significant Percent Disproportionate of Total (%) Percent Significant of Total (%) Percent Significant of Disproportionate (%) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (a) (b) (c) (b/a) (c/a) (c/b) A 15 0 0 0.0 0.0 NA B D D D 15 3 0 20.0 0.0 0.0 C D D D D D 15 5 0 33.3 0.0 0.0 D D D D D D D 15 6 0 40.0 0.0 0.0 E D D D D D D D 15 7 0 46.7 0.0 0.0 F D D D D D D D 15 7 0 46.7 0.0 0.0 G D D D D D D D D 15 8 0 53.3 0.0 0.0 H D D D D D D D D 15 8 0 53.3 0.0 0.0 I D D D D D D D D D 15 9 3 60.0 20.0 33.3 J D D D D D D D D D D 15 10 4 66.7 26.7 40.0 K D D D D D D D D D D D D 15 12 7 80.0 46.7 46.7 Z D D D D D D D D D D D D D D D 15 15 13 100.0 86.7 86.7 Total 180 90 27 50.0 15.0 30.0 * District is significantly disproportionate in the third or current year if it is disproportionate in the same cell for three years in a row or in the second or current year if it is disproportionate in the same cell in two years in a row and in any two of the preceding three years. Note that the district must be disproportionate in the same cell in the current year. D = Disproportionate in the same cell at specified thresholds in the joint measures approach; NA = Not Applicable Shaded = Significantly Disproportionate under Definition P-7 70

Definition P-7 results in 27 instances of significant disproportionality which is the third lowest among the definitions discussed so far. But it addresses only four disproportionality scenarios. In Definition P-7, the district is no longer significantly disproportionate in Scenarios D, E or H (two years of continuous disproportionality separated by two or more non-disproportionate years) as it is in Definition P-1 and Definition P-2. This may be viewed as a reward to the district for staying nondisproportionate for two or more years in a row during every five-year period. Scenario I also benefits from having two consecutive years of non-disproportionality after going through three disproportionate years in a row in each five-year cycle, which is not the case in Definitions P-1, P-2, and P-5. Recall the results of Definition P-3 in Scenario K, when the district is significantly disproportionate only in four out of 12 years of disproportionality; now under Definition P-7, the district is significantly disproportionate in seven of these years, which appears to be more reasonable when compared against Scenario Z in all the definitions. Two issues associated with Definition P-7 need to be pointed out, however; (1) the language in Definition P-7 is a little bit complicated compared to the other definitions and (2) Definition P-7 requires monitoring disproportionality for a period of five years in a row for each district, which is perhaps not a major issue because two other definitions (P-5 and P-6) also require monitoring disproportionality for five years. One may still find that some scenarios can avoid the wrath of significant disproportionality altogether under several definitions. Scenarios F and G in the above tables are two cases in point. Neither of these two scenarios shows any sign of significant disproportionality in several definitions, except in Definitions P-3 and P-5. By moving in and out of disproportionality every year, popularly known as the revolving door scenario, a district can manage to stay under the radar of significant disproportionality. Situations like this can be addressed by making appropriate changes in the definition, but one needs to keep in mind that the language of such a definition could be more complicated than the ones that we have discussed so far, and that any solution to an existing problem could also raise other issues. Definition P-8 This definition combines three features of persistency but its language is quite complex compared to any of the previous definitions. The definition, however, addresses many nagging problems that we have encountered in the previous definitions. In this definition, a district would be significantly disproportionate in the current year, if (a) it is disproportionate in the same cell for three years in a row, including the current year; or (b) it is disproportionate in the same cell for two years in a row, including the current year and in any two years during the preceding three-year period; or (c) it is disproportionate in the same cell in any three out of five years, including the current year and not identified as significantly disproportionate in the same cell in any year during the preceding four-year period. In other words, a district would be significantly disproportionate in Year Y, if (a) it is disproportionate in the same cell in Years Y, Y-1, and Y-2; or (b) it is disproportionate in the same cell in Years Y and Y-1 and is also disproportionate in the same cell in any two of the Years Y-2, Y-3, and Y-4; or (c) it is disproportionate in the same cell in Year Y and in any two of the Years Y-1, Y-2, Y-3, and Y-4 71

and not identified as significantly disproportionate in the same cell in any of the Years Y-1, Y-2, Y-3, and Y-4. The effects of Definition P-8 are shown in Table 28. Table 28. Scenarios of Disproportionality and Significant Disproportionality during the 15-year Period Under Definition P-8* Disproportionality Scenario Year Total Number of Years / Cells Number Disproportionate Number Significant Percent Disproportionate of Total (%) Percent Significant of Total (%) Percent Significant of Disproportionate (%) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (a) (b) (c) (b/a) (c/a) (c/b) A 15 0 0 0.0 0.0 NA B D D D 15 3 0 20.0 0.0 0.0 C D D D D D 15 5 0 33.3 0.0 0.0 D D D D D D D 15 6 0 40.0 0.0 0.0 E D D D D D D D 15 7 2 46.7 13.3 28.6 F D D D D D D D 15 7 2 46.7 13.3 28.6 G D D D D D D D D 15 8 2 53.3 13.3 25.0 H D D D D D D D D 15 8 2 53.3 13.3 25.0 I D D D D D D D D D 15 9 3 60.0 20.0 33.3 J D D D D D D D D D D 15 10 4 66.7 26.7 40.0 K D D D D D D D D D D D D 15 12 7 80.0 46.7 46.7 Z D D D D D D D D D D D D D D D 15 15 13 100.0 86.7 86.7 Total 180 90 35 50.0 19.4 38.9 * District is significantly disproportionate in the third or current year if it is disproportionate in the same cell for three years in a row or in the second or current year if it is disproportionate in the same cell in two years in a row and in any two of the preceding three years or in the current year if it is disproportionate in the same cell in any three out of five years, including the current year, and not identified as significantly disproportionate in the same cell in the preceding four years. Note that the district must be disproportionate in the same cell in the current year. D = Disproportionate in the same cell at specified thresholds in the joint measures approach; NA = Not Applicable Shaded = Significantly Disproportionate under Definition P-8 Note that Definition P-8 is an extension of Definition P-7. The first two components of this definition are the same as Definition P-7. The third component in Definition P-8 is intended to address the weaknesses in Definition P-7. Definition P-8 appears to display a good balance between the number of instances identified as significant disproportionality and the number of various disproportionality scenarios addressed. It addresses eight of the twelve disproportionality scenarios generating a total of 35 instances of 72

significant disproportionality. The definition also addresses the so-called revolving door scenarios in F and G that several other definitions fail to address. The main drawback of this definition is its language; it is rather complex, and therefore, not as intuitive as any of the previous definitions. This definition provides an example of versatility of many possible definitions to address the shortcomings of other definitions. Many other variations of persistency or combinations of more than one definition, as exemplified in Definitions P-7 and P-8, can lead to additional definitions of significant disproportionality. Some definitions may include combinations of severity and persistency such as, being disproportionate in a cell at higher thresholds than for (simple) disproportionality and remaining disproportionate in the same cell at these higher thresholds for a specified number of years. Another option could be using the three aspects of disproportionality (frequency, severity, and persistency) in any combination to come up with a definition that captures the intended purpose of significant disproportionality. The possibility of ways to define significant disproportionality is almost limitless. 6.4. Discussion of Results from Various Definitions The results of the above definitions show that most of them are quite efficient in identifying significant disproportionality when a district is disproportionate year after year continuously or almost continuously. But their effectiveness varies considerably from each other when a district is in and out of disproportionate situations frequently. A good definition should identify reasonable instances of significant disproportionality within individual disproportionality scenarios as well as address various scenarios of disproportionality over the years. However, it is not easy to incorporate both of these properties in a definition that is also simple to understand and use. The definitions that can address a relatively large number of scenarios (P-1, P-2, and P-5), including the so-called revolving door scenarios, also produce some of the highest instances of significant disproportionality. The ones that produce reasonable numbers of instances of significant disproportionality (P-3, P-4, and P-6) can address no more than four scenarios (out of 12) and they are not able to touch the so-called revolving door situations in Scenarios F and G. Definition P-7 shows a good balance but its language is not as simple as the other definitions (except for Definition P-8) and it also is helpless in addressing Scenarios F and G. Definition P-8 addresses the weaknesses of Definition P-7 by addressing eight scenarios, including Scenarios F and G, but its language is far more complex than any of the other definitions. As we can see, except for Definition P-8, none of the other definitions are able to address most of the issues that have surfaced in the discussions nor can they address a wide range of disproportionality patterns and still produce reasonable instances of significant disproportionality. What constitutes a reasonable number or an acceptable percentage of instances of significant disproportionality, out of the total number of instances of (simple) disproportionality? There is no simple answer to this question, because it depends not only on the number of disproportionate years over a period of time, but also on the pattern of those years of disproportionality during that period. To get a comparative picture of the definitions under persistency and to judge their relative effectiveness, a summary of their results is shown in Table 29. Table 29 shows that Definitions P-1 through P-6 are fairly simple to understand and use. Definitions P-7 and P-8 are successively more complex in relation to the other definitions. Each definition is 73

also quite unique in its own ways in identifying significant disproportionality from the same scenarios. No noticeable pattern seems to emerge from the length (in years) of monitoring disproportionality data. Table 29. Summary of Results from Various Definitions of Significant Disproportionality in 12 Scenarios During the 15-year Period Definition of Significant Disproportionality Year of Disproportionality Total Number of Cells No. of Cells Disproportionate No. of Cells Significant No. of Scenarios Addressed Percent of Cells Disproportionate of Total (%) Percent of Cells Significant of Total (%) Percent of Cells Significant of Disproportionate (%) Y-4 Y-3 Y-2 Y-1 Y* (a) (b) (c) (d) (b/a) (c/a) (c/b) P-1 D D 180 90 43 7 50.0 23.9 47.8 P-2 Disprop. in any one year D 180 90 63 9 50.0 35.0 70.0 P-3 D D D 180 90 20 3 50.0 11.1 22.2 P-4 Disproportionate in any two years D 180 90 34 4 50.0 18.9 37.8 P-5 Disproportionate in any two years D 180 90 60 8 50.0 33.3 66.7 P-6 Disproportionate in any three years D 180 90 25 3 50.0 13.9 27.8 P-7 P-8 D D D Disproportionate in any two years D D D D D Disproportionate in any two years D D Disproportionate in any two years but not significant D 180 90 27 4 50.0 15.0 30.0 180 90 35 8 50.0 19.4 38.9 * Year of Significant Disproportionality determination, the current year (Year Y); district must be disproportionate in Year Y D = Disproportionate in the same cell at specified thresholds in the joint measures approach Because Definitions P-1, P-2, and P-5 generate some of the largest numbers of instances of significant disproportionality they are probably not strong candidates for further consideration, even though they address a relatively large number of disproportionality scenarios. At the other end, Definitions P-3 and P-6 address only three disproportionality scenarios, and therefore, are probably not very desirable options either. That leaves Definitions P-4, P-7, and P-8 as serious contenders in selecting a suitable definition. 6.5. Recommendation It appears that some definitions of significant disproportionality are better than others and none of them are perfect. They all have strengths and weaknesses. By selecting one definition over another, one would have to sacrifice one or more desirable properties of a definition not selected in favor of 74

the one that is selected. An ideal definition would be the one that can address as many diverse disproportionality scenarios as possible without identifying too many or too few instances of significant disproportionality. Because of the uniqueness of each definition, we have decided not to single out any particular definition as a recommendation. We hope there is sufficient information on the strengths and limitations of the individual definitions that we have presented in the paper so the user can make the most appropriate decision in adopting a definition of significant disproportionality or even venture into developing a new one. If complexity of the language was not a factor, then Definition P-8 would be the most promising among all the definitions presented in the paper. It addresses eight of the 12 disproportionality scenarios by identifying 35 instances of significant disproportionality, which is perhaps the best combination of the two components of a good definition as mentioned before. Definitions P-4 and P-7 address the same four scenarios but they generate 34 and 27 instances of significant disproportionality respectively. The effects of Definition P-4 on Scenarios J and K (particularly K) are probably a little overkill; these two scenarios do not seem to get a break, even after remaining non-disproportionate intermittently. In identifying districts with significant disproportionality, one must ensure that any definition of significant disproportionality is fair and unbiased, and that it produces a true or reasonable number of cases of significant disproportionality in a consistent manner. A state should not adopt a particular definition just to avoid or minimize adverse fiscal impact or monitoring burden on districts and the state. No matter which definition a state adopts to identify significant disproportionality, there may be situations in the future when a definition might not seem to be serving its purpose any longer and the appropriateness of that definition could be put to question. If or when this happens, possibilities of other definitions or variations of the existing definition should be explored, their effects should be examined using empirical data, and the results should be discussed with all stakeholders. Hopefully, a new workable definition of significant disproportionality would emerge from consensus among the districts and the state. 75

7. A Final Note Adopting a joint measures approach or selecting a single measure to examine disproportionality and using it for all districts in the state is to ensure that districts with racial/ethnic disproportionality (and significant disproportionality) in special education are identified in a consistent manner. The results would probably trigger some obvious questions: Why are some districts overrepresented or underrepresented and others not? What are the reasons for such disproportionality in these districts? What can be done to prevent districts from being overrepresented or underrepresented? While identifying the reasons for disproportionality or recommending any corrective actions is not within the scope of this paper (research is plentiful in this area), such triggers should initiate followup activities such as, monitoring and review of district policies and procedures to determine reasons for disproportionality, making program or policy changes to ameliorate such discrepancies, and preventing further deterioration of any racial/ethnic imbalance between general education and special education as well as among all racial/ethnic groups. Using the best possible approach or selecting the most appropriate measure to identify districts with racial/ethnic disproportionality and significant disproportionality is a critical first step in addressing the issues around disproportionality in special education. Regardless of which approach (joint measures or a single measure) a state adopts to determine racial/ethnic disproportionality in special education and where the thresholds are set for determining (simple) disproportionality and which definition is adopted for significant disproportionality, some districts will probably show overrepresentation or underrepresentation for one or more racial/ethnic groups in one or more disability categories. An overrepresentation does not automatically imply that the district has failed to provide appropriate education services to these students in general education, and thus has identified them for special education services. There may be valid reasons for overrepresentation in that district. For example, the district may be the home of an institution that provides specialized services to students with a specific disability in the county or the state, and all neighboring districts send their students who are in need of these services to this facility. Similarly, underrepresentation does not always mean that the district has failed to provide the needed special education services to students of certain racial/ethnic groups. There may be good reasons for underrepresentation as well. The district may have done an excellent job of providing education services to all students in general education, and their academic performance does not warrant the need for any special education services. On the other hand, a district not showing overrepresentation or underrepresentation does not always guarantee that all students in the district are receiving appropriate education services in their rightful service delivery environments. There may be situations when students in a district are not receiving appropriate education services, but they may not trigger overrepresentation or underrepresentation under any measure due to a small number of cases or for other reasons. Regulatory procedures such as, Due Process, Complaint Procedures, and Program Monitoring are available to investigate these situations. Small districts, if excluded from disproportionality calculations due to small enrollments, should not be left on their own without any state oversight. The very same program issues of providing 76

appropriate education services to all students, however small their numbers may be, still remain as important in small districts as in the medium and large districts. 77

Notes 1 Under this assumption, enrollment in special education (or in a subcategory of special education, such as a disability category or a placement setting) is considered a subset of all students enrolled in general education (grades Kindergarten through Twelve) in a district. Since special education students are part of all general education students in a district, statistically, the district general education enrollment may also be viewed as the population from which all special education students (or students in any subcategory of special education) are drawn as a sample. As the enrollment in special education increases, the distribution of the proportions of various racial/ethnic groups in special education approaches the corresponding distribution in general education. This distribution, at most, will be the same as that in general education, if or when all general education students in the district constitute the entire special education enrollment. 2 In the context of racial/ethnic disproportionality, the term risk is defined as the percentage of all general education students in a racial/ethnic group who are enrolled or receive services in special education (or in a subcategory of special education, such as a disability category or placement setting) according to their IEP (Individualized Education Program). It is also described as the risk of that racial/ethnic group to be in special education. For example, if 12 percent of all Hispanic students in a district receive special education services, then the risk of Hispanic students to be in special education in that district is 12 percent. It may also be stated as that the special education incidence rate of Hispanic students in the district is 12 percent. Note that some measures under the broad category of Risk include state-level data. 3 For example, if the racial/ethnic group in question is African-American students, then the comparison group would consist of Native American, Asian, Pacific Islander, Hispanic, White, and students in Multiple racial/ethnic group; for Hispanic students, the comparison group will include Native American, Asian, Pacific Islander, African- American, White, and students in Multiple racial/ethnic group. This assumption raises at least two issues: (1) there is no empirical evidence or convincing argument to support the notion that the risk of the comparison group is ideal or socially desirable, compared to the risk of the racial/ethnic group in question; and (2) because the comparison group is not the same for each racial/ethnic group, whether the risk measures yield comparable results across various racial/ethnic groups in special education. 4 Actually, non-disproportionality (neither overrepresentation nor underrepresentation) is a third possible outcome. This means that both proportions (of the reference racial/ethnic group and the comparison group) are the same or within an agreed upon range of values. 5 There is one other measure, known as Disparity Index, which has been used by the California Department of Education in the past but is not included in this paper. The Disparity Index is defined as the difference between the highest and the lowest risks or incidence rates among all racial/ethnic groups in a district. It does not take into account the risks for other racial/ethnic groups in between. For example, among the risks for each racial/ethnic group in special education in a district, if the highest risk is 15.50 percent for African-American students and the lowest risk is 5.25 percent for Asian students, then the Disparity Index for the district is 10.25 (15.50-5.25). Note that Disparity Index is an overall index for a district; it does not generate an index for a particular racial/ethnic group. For example, questions such as Are African-American students overrepresented in the Intellectual Disability category in a district? can not be answered using the Disparity Index. Another concern is that a district may have a small Disparity Index but the risks or incidence rates of all racial/ethnic groups in the district may be far above the state level incidence rates. Similarly, another district may have a relatively large Disparity Index but the overall district incidence rates may be well below the state level rates. It is important to make the distinction that Disparity Index focuses on the overall disparity across all racial/ethnic groups not on disproportionality of individual racial/ethnic groups in a district. One might question if Disparity Index appropriately addresses racial/ethnic disproportionality, according to the intent of the federal law. 6 The attachments A through I, developed in Microsoft Excel, show detailed calculations for all seven measures for all racial/ethnic groups in a single special education program category (Intellectual Disability category or ID in this case) for one district. The input variables for these calculations are: (1) the state total enrollment in general education, broken down into seven racial/ethnic groups; (2) the state total enrollment in the ID category, broken down into seven racial/ethnic groups; (3) the total district enrollment in general education, broken down by each racial/ethnic group; and (4) the total ID enrollment in the district, also broken down by each racial/ethnic group. The input variables are shown in four rows (in white or unshaded) at the beginning of the spreadsheet under appropriate labels. The 78

remaining rows, generated by the worksheet and shaded in grey for easy differentiation, show the detailed step-bystep calculations including the results (output). 7 The term total enrollment is used in the OSEP/WESTAT Report to represent the total district enrollment, which could include enrollments in adult and infant programs as well. We believe general education (GE) enrollment is perhaps a more appropriate term which includes enrollments from kindergarten through twelfth grade, and is comparable with special education students in age group 5-21. 8 The term risk has many definitions in the literature in various contexts. In this paper, Risk is a stand-alone measure of disproportionality and is defined as a deterministic event. However, the term risk generally implies a probability event with certain degree of uncertainty, and therefore, it is often described in such languages as, outcome x has n percent risk or result x is n times as likely to be out of m attempts, and so forth. The language used in the paper includes both deterministic and probabilistic natures of the term, as appropriately as possible, depending upon the context in which the term is used. 9 In addition to relative risk, other measures of disproportionality may be developed around risk, such as risk difference, relative risk difference, and so forth. These measures, depending upon how they are defined, are most likely to be localized at the individual district levels, and therefore, are unlikely to add any new properties to the analysis of measures included in the paper. 10 There are two definitions of Weighted Risk Ratio in the OSEP/WESTAT Report, expressed in two equations (page 16). The first equation defines the Weighted Risk Ratio, weighing the denominator only. There are no definitions of these weights in the report, however. The second equation shows weights or multipliers in both the numerator and the denominator. If both equations are correct, then one has to reconstruct the weights from the two equations. Because the weights are not clearly defined, nor are they explained sufficiently in the report, the second equation appears to be a workable definition of the Weighted Risk Ratio which weights (using multipliers) both the numerator and the denominator. 11 In a subsequent conversation, a staff of the OSEP/WESTAT Report acknowledged the missing definition of the weights, described in Note 10, as an oversight. Because the working definition of the Weighted Risk Ratio in the OSEP/WESTAT Report shows weights (multipliers) in both the numerator and the denominator, the paper describes the definition as such, not just the weights in the denominator, as stated in the text of the OSEP/WESTAT Report. 12 The OSEP/WESTAT Task Force Report recommends using Alternate Risk Ratio for a racial/ethnic group when there are at least 10 students in the group in the district and there are fewer than 10 students or none in the comparison group enrolled in the district (page 22). It is, however, unclear if the report implies using different measures for different districts in a state, depending on the size of the racial/ethnic composition of individual districts. Not only would it be improper to do so because different measures yield different results, it would also be unfair because they could also alter the disproportionality status of a district. 13 In the original order (1979) under the Larry P. lawsuit, disproportionate placement was defined, if the rate of Black EMR pupil enrollment (is) one standard deviation above the district rate of White EMR pupil enrollment. During the litigation process both plaintiffs and defendants in Larry P. agreed that the remedy should at least be similar to the one in the Diana lawsuit and the measure of disproportionate placement should be the same. Even though this reason is not explicitly stated in the court order, the language in the 1986 modified judgment under the Larry P. and all subsequent official documents mention E-formula as the measure to determine significant variance or disproportionate placement, and not the original 1979 language. 14 The choice of the three variations (two, three, and four standard errors) of the E-formula is somewhat arbitrary and is for illustrative purposes. A state may choose other variations of the E-formula to examine how a particular variation (or threshold) affects the districts in that state. An analysis of data from a wide range of school districts revealed, as expected, that more districts show disproportionality at lower thresholds and fewer districts at higher thresholds. As the threshold increases, the number of districts showing disproportionality decreases. Smaller districts tend to go under the radar of disproportionality first, followed by the medium and large districts as the threshold is raised. Some large districts, however, show disproportionality even at considerably high thresholds such as 10 or even15 standard errors. 15 The negative E-formula values in Table 9 are an artifact of the computational process. These values would make sense when viewed along a number line. For all practical purposes, they should be treated as zero. 16 One could argue that Relative Difference in Composition provides sufficient information to determine racial/ethnic disproportionality in special education. To refresh our memory, it is the difference between the 79

proportions of special education enrollment and general education enrollment for a particular racial/ethnic group, expressed as a percentage of the group s general education proportion. It does not take into consideration the proportions of other racial/ethnic groups in the district neither in special education nor in general education. In other words, it is a measure of discrepancy exclusively within a single racial/ethnic group. Therefore, in our judgment it is of limited value, compared to the four measures (Risk Ratio, Weighted Risk Ratio, Alternate Risk Ratio, and the E- formula) that we have selected for further analysis. 17 The numerator and the denominator in the second component of the E-formula, ((A (100-A)) / N) are generally positive, with the following exceptions. The numerator in the second component in the E-formula, (A(100- A)) is almost always positive, except when the district is composed of only one racial/ethnic group (perfectly homogenous) or if there are no students in a racial/ethnic group in question, when the value of A or (100-A) is zero. When the numerator is zero, the value of the second component becomes zero and the E-formula value coincides with A in general education. The denominator N is always positive if the district has any student in special education program or in a subcategory (ID, for example) for which the disproportionality is calculated; otherwise, the issue of disproportionality for that program or subcategory would not exist (not applicable). 18 In any division of two positive integers, any changes in the numerator and/or the denominator would result in a larger change when the numbers in the division (the numerator and the denominator) are relatively small, such as 3/10 or 5/20, than when the same changes are applied to the numerator and/or the denominator in a division with large numbers, such as 30/100 or 50/200. 19 In the E-formula, the proportion of special education (or general education) students in a racial/ethnic group has the properties of a binomial distribution. If the proportion is around 50 percent and the sample size is large, the binomial distribution approximates the normal distribution. However, when the proportions depart considerably from 50 percent and/or when the sample size is relatively small, the normal approximation to the binomial distribution is quite inaccurate (Arkin and Colton, 1970; Snedecor and Cochran, 1989). For a large number of districts in California, the proportion of White and Hispanic students may be around 50 percent and the number of students in special education or in a disability category (the sample size) may also be large; but for many districts, especially smaller districts, the proportion of other racial/ethnic groups, such as the Native American or Pacific Islander students, may be considerably less than 50 percent and the number of students in special education or in a disability category (the sample size) may also be critically small. 20 The only exception is that if the total district enrollment in a disability category consists of only one racial/ethnic group (homogeneous), then the total district enrollment in that disability category would be the same as the enrollment of that racial/ethnic group in that disability category; for such districts the number of exclusions in the risk ratio measures would be the same as in the E-formula. 21 The OSEP/WESTAT Task Force Report defines cell as the denominator in a division or ratio; it does not address the numerator. But for reasons stated earlier in the paper, both numerator and the denominator are critical in the result of a division or ratio, particularly when they are small numbers. Any reference to a cell in the paper implies both the numerator and the denominator. 22 One could possibly extend this argument to propose questions such as: What if the district or state incidence rate is too high or too low compared to the nation? What if the district or the state or the nation s incidence rate is too high or too low compared to...? And so forth. 23 During 2009-10, out of about 1,000 school districts in California, 95 districts, including some large ones, reported discrepancies between general education and special education enrollments. These districts reported higher numbers of students in special education than in general education for some racial/ethnic groups and are excluded from this analysis. Note that 2009-10 was the first year of general education data collected through the California Longitudinal Pupil Achievement Data System (CALPADS). The data discrepancies between general education and special education in these districts were being investigated by the California Department of Education for resolution while the paper was in progress. However, these discrepancies were not resolved by the time the paper was finalized. 24 If a racial/ethnic group has less than 20 students in general education (GE) in a district then the group is excluded from disproportionality calculations when the focus of disproportionality calculations is on that group only. The group is not excluded from disproportionality calculations for other racial/ethnic groups in the district. 25 If the total enrollment in a disability category in a district is less than 20, which implies that enrollments in all seven racial/ethnic groups in that disability category are also less than 20, then all racial/ethnic groups are excluded from disproportionality calculations in that disability category in that district. 80

26 This number represents the minimum cell size (20) for disproportionality calculations. As stated in the paper, a cell is defined as either the numerator or the denominator in any division of enrollments in the calculations (the OSEP/WESTAT Task Force defines a cell as the denominator only). Also note that, in the calculations for Alternate Risk Ratio (as well as for other risk ratio measures), a cell is excluded from disproportionality calculations only when (1) the cell is the focus of disproportionality calculations and (2) if the cell size is less than 20. It is not excluded from disproportionality calculations for other racial/ethnic groups in the districts. 27 The computer program carries out disproportionality calculations for all seven racial/ethnic groups in all districts in a state in any or all of the following four measures: 1. The E-formula and its variations of up to 18 standard errors thresholds 2. Risk Ratio, with five thresholds (see below) 3. Weighted Risk Ratio, with five thresholds (see below) 4. Alternate Risk Ratio, with five thresholds (see below) The five thresholds in the three risk ratio measures are: For Overrepresentation: >1.00, >=2.00, >=3.00, >=4.00, and >=5.00 For Underrepresentation: <1.00, <=0.80, <= 0.60, <= 0.40, and <=0.20 Each of the four measures can show racial/ethnic disproportionality, if any, under the above thresholds in various combinations of the following items, as applicable: Level of aggregation (district or state as a single entity) Type of district (district of service, district of residence or district of accountability) Type of disproportionality (overrepresentation, underrepresentation or over/under) Overall special education program Six major disability categories Eight special education placement categories Discipline (suspension and/or expulsion for 10 or more days) Students in Licensed Children s Institutions (include or exclude) Each measure generates the following three tables/reports: 1. A detailed full report showing for each district in the state (or the state as a single entity as selected by the user), for each selected program item (overall, disability, placement, discipline or in any combination thereof) within the district, the general education and special education enrollments in each racial/ethnic group, the results of disproportionality calculations for each racial/ethnic group, the disproportionality status of each racial/ethnic group in each selected program item under each threshold, and counts of overrepresentation and/or underrepresentation for each racial/ethnic group in each program item under each threshold. 2. A district-level summary report for each district showing the general education and special education enrollments for each racial/ethnic group and a total count of overrepresentation and/or underrepresentation for all thresholds. 3. A state-level summary report showing the total of all district results for each selected program item and for each racial/ethnic group under all thresholds. It also shows the total count of overrepresentation and/or underrepresentation and the total number of districts in the counts for all thresholds. In addition, the program also generates the following two reports: 1. A list of districts that are excluded from disproportionality calculations for various reasons such as, errors in reported data, small cell size, county-operated programs, state special schools, and so forth. 2. A separate list of districts showing discrepancies in reported data between general education and special education. 28 The term true is used here in a probabilistic sense. (Example: it is probably true that Roger Federar will beat me in a tennis match.) It does not imply the notion of absolute truth (as opposed to absolute false, such as the earth is flat ) as used in other contexts. 29 In normal distribution, at three standard errors, the probability is more than 99 percent that the difference between the population proportion (of a racial/ethnic group in general education) and the sample proportion (of the same racial/ethnic group in special education) is true or real difference. In other words, the probability is less than one 81

percent that such difference is due to chance. At higher standard errors, such as four or five, the probability is even higher that the disproportionality is not due to chance. 30 The calculations for the joint measures approach were carried out as an option in the same computer program (see Note 27), which can analyze data for all districts in the state for any combination of the following thresholds in the top two measures. The E-formula thresholds are: For Overrepresentation: 1.0 through 9.9 standard errors, with increments of 0.1 For Underrepresentation: 1.0 through 9.9 standard errors, with increments of 0.1 The Alternate Risk Ratio thresholds are: For Overrepresentation: 1.0 through 9.9, with increments of 0.1 For Underrepresentation: 1.0 through 0.1, with decrements of 0.1 Each combination of thresholds generates three tables/reports similar to the ones described in Note 27, with the following modifications: 1. The detailed full report shows separately the E-formula results under the selected E-formula threshold, the Alternate Risk Ratio results under the selected Alternate Risk Ratio threshold, and the results of the two thresholds combined together. 2. The district-level summary report shows a count of overrepresentation and/or underrepresentation for the E- formula and the Alternate Risk Ratio separately under the individual thresholds, and then jointly under the combined thresholds. 3. The state-level summary report shows the total of all district results for the E-formula and the Alternate Risk Ratio separately under the individual thresholds, and then jointly under the combined thresholds. 31 The actual probability under the E-formula results will vary from district to district and from one racial/ethnic group to another depending on the proportion of each racial/ethnic group in general education (the population) in the district and on the number of students in special education or in a disability category (the sample). At 3.00 standard errors threshold under the E-formula, the probability could be higher than 99 percent if the proportion of a racial/ethnic group in general education is around 50 percent and the sample size is large (assuming normal approximation of binomial distribution). The probability would be less than 99 percent if the proportion is not around 50 percent (away from 50 percent in either direction) and/or if the sample size is considerably small. 32 The term significant disproportionality is used in the federal language to put additional emphasis on situations when the disproportionality is severe, persistent or very critical. The term is used in a social context and does not imply statistical significance. 33 Another reason for this condition is to prevent districts from being identified more than once as significantly disproportionate for the same reason. Consider a district with two disproportionate years in a row in the same cell, preceded by a non-disproportionate year and followed by another non-disproportionate year. In other words, during a four-year period (Years Y-3, Y-2, Y-1, and Y) the district is disproportionate in the Years Y-2 and Y-1 in the same cell (note that Year Y is the current year). Let s say that the definition of significant disproportionality is: a district would be considered significantly disproportionate in the third year if it is disproportionate for any two years during a three-year period. Using this definition, as stated, the district would be significantly disproportionate in Years Y-1 and Y, twice for the same reason. By imposing the condition of (simple) disproportionality in the year of determination of significant disproportionality, the district will now be significantly disproportionate only in Year Y-1 and not in Year Y. 82

Appendix A Mathematical Expression of Risk Ratio The Risk Ratio (RR) of a racial/ethnic group e in a disability category d for a district can be expressed as: (DSE ed / DGE e ) (100.00) RR ed = p e p e (( DSE pd ) / ( DGE p )) (100.00) p p Where: RR ed = Risk Ratio of racial/ethnic group e in disability category d in a district DSE ed = District special education enrollment of racial/ethnic group e in disability category d DGE e = District general education enrollment of racial/ethnic group e e, p = 1, 2, 3,..., 7 (seven racial/ethnic groups) d = 1, 2, 3,..., 6 (six major disability categories) 83

Appendix B Mathematical Expression of Weighted Risk Ratio The Weighted Risk Ratio (WRR) of a racial/ethnic group e in a disability category d for a district can be expressed as: DR ed (1 SGC e ) WRR ed = p e DR pd SGC p p Where: WRR ed = Weighted Risk Ratio of racial/ethnic group e in disability category d in a district DR ed = District risk of racial/ethnic group e in disability category d, and is defined as, DR ed = DSE ed / DGE e DSE ed = District special education enrollment of racial/ethnic group e in disability category d DGE e = District general education enrollment of racial/ethnic group e SGC e = State general education composition of racial/ethnic group e, and is defined as, SGC e = SGE e / SGE e e SGE e = State general education enrollment of racial/ethnic group e DR pd = District risk of racial/ethnic group p in disability category d SGC p = State general education composition of racial/ethnic group p e, p = 1, 2, 3,..., 7 (seven racial/ethnic groups) d = 1, 2, 3,..., 6 (six major disability categories) 84

Appendix C Mathematical Expression of Alternate Risk Ratio The Alternate Risk Ratio (ARR) of a racial/ethnic group e in a disability category d for a district can be expressed as: (DSE ed / DGE e ) (100.00) ARR ed = p e p e (( SSE pd ) / ( SGE p )) (100.00) p p Where: ARR ed = Alternate Risk Ratio of racial/ethnic group e in disability category d in a district DSE ed = District special education enrollment of racial/ethnic group e in disability category d DGE e = District general education enrollment of racial/ethnic group e SSE pd = State special education enrollment of racial/ethnic group p in disability category d SGE p = State general education enrollment of racial/ethnic group p e, p = 1, 2, 3,..., 7 (seven racial/ethnic groups) d = 1, 2, 3,..., 6 (six major disability categories) 85

Appendix D Alternate Table 18 Note: Table 18 in the paper does not include the results of calculations for 95 districts that reported data discrepancies between general education and special education enrollments. If these districts were not excluded, then Table 18 would be as shown below. Table 18. Number of Districts Overrepresented under Both E-formula and Alternate Risk Ratio Measures* The Alternate Risk Ratio Thresholds The E-formula Thresholds More than 1.00 2.00 or More 3.00 or More 4.00 or More 5.00 or More 627 435 209 115 57 One Standard Error 650 493 383 195 109 56 Two Standard Errors 513 394 323 172 98 52 Three Standard Errors 415 325 265 146 89 46 Four Standard Errors 321 257 210 122 79 44 Five Standard Errors 252 215 166 107 72 40 Six Standard Errors 197 172 132 86 56 33 Seven Standard Errors 155 139 105 69 47 29 Eight Standard Errors 118 109 80 51 39 25 * All districts are included in calculations, including those that reported general education enrollment in certain racial/ethnic groups less than in special education. 86

References Arkin, Herbert and Colton, Raymond R. Statistical Methods. (1970). New York. Barnes and Noble Books. Diana, et al. vs. State Board of Education, et al. (January 7, 1970). C-70-37 LHT Complaint for Injunction and Declaratory Relief (Civil Rights). (U.S. District Court, Northern District of California. January 7, 1970). Diana, et al. vs. State Board of Education, et al. (May 22, 1974). C-70-37 RFP Memorandum and Order. (U.S. District Court, Northern District of California. May 24, 1974). Larry P., et al. vs. Wilson Riles, et al. (December 12, 1979). No. C-71-2270 RFP. (U.S. District Court, Northern District of California. December 12, 1979). Larry P., et al. vs. Wilson Riles, et al. (April 25, 1986). No. C-71-2270 RFP Order Modifying Judgment. (U.S. District Court, Northern District of California. September 25, 1986). OSEP / Westat. (No Date). Methods for Assessing Racial/Ethnic Disproportionality in Special Education: A Technical Assistance Guide. (Unpublished Paper). Washington, D.C. Office of Special Education Programs, U.S. Department of Education. Roy, Lalit M. (1997). Overrepresentation of Ethnic Minorities in Special Education: An Analysis of Enrollment in Five School Districts in California. Sacramento, California. California Department of Education, Special Education Division. Snedecor, George W. and Cochran, William G. (1989). Statistical Methods. Ames. Iowa State University Press. 87

Attachment A Page A-1 Actual Data from a School District in California: General Education (GE) Enrollment = 16,115; Intellectual Disability (ID) Enrollment = 171 1 Native Asian Pacific Black Hispanic White Multiple Total/Overall 2 3 State Data 4 5 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 6 State GE Composition (Fraction) 0.00739 0.11240 0.00609 0.06983 0.51308 0.27529 0.01592 1.00000 7 State GE Composition (%) 0.73902 11.24017 0.60886 6.98307 51.30752 27.52918 1.59217 100.00000 8 9 State ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 10 State ID Composition (Fraction) 0.00692 0.07988 0.00544 0.11030 0.55202 0.23408 0.01136 1.00000 11 State ID Composition (%) 0.69207 7.98815 0.54395 11.02967 55.20200 23.40773 1.13642 100.00000 12 State ID Risk (Fraction) 0.00603 0.00458 0.00575 0.01017 0.00693 0.00548 0.00460 0.00644 13 State ID Risk (%) 0.60320 0.45777 0.57546 1.01739 0.69302 0.54769 0.45975 0.64413 14 15 16 District Data 17 18 District GE Enrollment (N) 106 3,404 324 5,013 5,210 1,674 384 16,115 19 District GE Composition (Fraction) 0.00658 0.21123 0.02011 0.31108 0.32330 0.10388 0.02383 1.00000 20 District GE Composition (%) 0.65777 21.12318 2.01055 31.10766 32.33013 10.38784 2.38287 100.00000 21 22 District ID Enrollment (N) 0 26 1 81 34 25 4 171 23 District ID Composition (Fraction) 0.00000 0.15205 0.00585 0.47368 0.19883 0.14620 0.02339 1.00000 24 District ID Composition (%) 0.00000 15.20468 0.58480 47.36842 19.88304 14.61988 2.33918 100.00000 25 District ID Risk (Fraction) 0.00000 0.00764 0.00309 0.01616 0.00653 0.01493 0.01042 0.01061 26 District ID Risk (%) 0.00000 0.76381 0.30864 1.61580 0.65259 1.49343 1.04167 1.06112 27 28 29 Relative Diff. in Composition [Overrepresentation or Underrepresentation] 30 31 Relative Diff. in ID Composition (%) -100.00000-28.01898-70.91365 52.27251-38.49996 40.74039-1.83358 0.00000 32 88

Attachment A Page A-2 Actual Data from a School District in California: General Education (GE) Enrollment = 16,115; Intellectual Disability (ID) Enrollment = 171 33 Native Asian Pacific Black Hispanic White Multiple Total/Overall 34 35 Risk Ratio [Overrepresentation or Underrepresentation] 36 37 District ID Risk (%) 0.00000 0.76381 0.30864 1.61580 0.65259 1.49343 1.04167 1.06112 38 All Other ID in District 171 145 170 90 137 146 167 39 All Other GE in District 16,009 12,711 15,791 11,102 10,905 14,441 15,731 40 ID Risk for All Others (%) 1.06815 1.14074 1.07656 0.81066 1.25630 1.01101 1.06160 41 ID Risk Ratio 0.00000 0.66957 0.28669 1.99318 0.51945 1.47716 0.98123 42 43 44 Weighted Risk Ratio [Overrepresentation or Underrepresentation] 45 46 District ID Risk (Fraction) 0.00000 0.00764 0.00309 0.01616 0.00653 0.01493 0.01042 0.01061 47 (1-State GE Comp)*(Dist ID Risk) 0.00000 0.00678 0.00307 0.01503 0.00318 0.01082 0.01025 48 (State GE Comp)*(Dist ID Risk) 0.00000 0.00086 0.00002 0.00113 0.00335 0.00411 0.00017 0.00963 49 Sum of All Others in the Row Above 0.00963 0.00877 0.00961 0.00850 0.00628 0.00552 0.00947 50 Weighted ID Risk Ratio 0.00000 0.77281 0.31914 1.76762 0.50577 1.96077 1.08300 51 52 53 Alternate Risk Ratio [Overrepresentation or Underrepresentation] 54 55 District ID Risk (%) 0.00000 0.76381 0.30864 1.61580 0.65259 1.49343 1.04167 1.06112 56 State ID (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 57 All Other ID in State 38,887 36,030 38,945 34,839 17,542 29,992 38,713 58 All Other GE in State 6,034,320 5,395,929 6,042,233 5,654,729 2,960,136 4,405,680 5,982,455 59 Statewide All Others' ID Risk (%) 0.64443 0.66773 0.64455 0.61610 0.59261 0.68076 0.64711 60 Alternate ID Risk Ratio 0.00000 1.14389 0.47885 2.62261 1.10122 2.19377 1.60972 61 89

Attachment A Page A-3 Actual Data from a School District in California: General Education (GE) Enrollment = 16,115; Intellectual Disability (ID) Enrollment = 171 62 Native Asian Pacific Black Hispanic White Multiple Total/Overall 63 64 E-formula Data 65 66 District ID Enrollment (N) 0 26 1 81 34 25 4 171 67 A = District GE Composition (%) 0.65777 21.12318 2.01055 31.10766 32.33013 10.38784 2.38287 100.00000 68 Std. Error = [A*(100-A)/N] (%) 0.61817 3.12145 1.07337 3.54015 3.57687 2.33318 1.16631 69 70 71 E-formula [Overrepresentation] 72 73 District ID Composition (%) 0.00000 15.20468 0.58480 47.36842 19.88304 14.61988 2.33918 100.00000 74 Maximum Percent ID Allowed: 75 A + One Standard Error (%) 1.27594 24.24463 3.08392 34.64781 35.90700 12.72102 3.54919 76 A + Two Standard Errors (%) 1.89411 27.36608 4.15729 38.18795 39.48387 15.05419 4.71550 77 A + Three Standard Errors (%) 2.51228 30.48752 5.23066 41.72810 43.06074 17.38737 5.88181 78 A + Four Standard Errors (%) 3.13045 33.60897 6.30402 45.26825 46.63761 19.72055 7.04813 79 80 81 E-formula [Underrepresentation] 82 83 District ID Composition (%) 0.00000 15.20468 0.58480 47.36842 19.88304 14.61988 2.33918 100.00000 84 Minimum Percent ID Needed: 85 A - One Standard Error (%) 0.03960 18.00173 0.93718 27.56752 28.75326 8.05466 1.21656 86 A - Two Standard Errors (%) -0.57856 14.88028-0.13619 24.02737 25.17638 5.72148 0.05025 87 A - Three Standard Errors (%) -1.19673 11.75883-1.20956 20.48723 21.59951 3.38830-1.11607 88 A - Four Standard Errors (%) -1.81490 8.63738-2.28293 16.94708 18.02264 1.05512-2.28238 89 90

Attachment B Page B-1 Hypothetical Small School District: General Education (GE) Enrollment = 1,000; Intellectual Disability (ID) Enrollment = 10 1 Native Asian Pacific Black Hispanic White Multiple Total/Overall 2 3 State Data 4 5 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 6 State GE Composition (Fraction) 0.00739 0.11240 0.00609 0.06983 0.51308 0.27529 0.01592 1.00000 7 State GE Composition (%) 0.73902 11.24017 0.60886 6.98307 51.30752 27.52918 1.59217 100.00000 8 9 State ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 10 State ID Composition (Fraction) 0.00692 0.07988 0.00544 0.11030 0.55202 0.23408 0.01136 1.00000 11 State ID Composition (%) 0.69207 7.98815 0.54395 11.02967 55.20200 23.40773 1.13642 100.00000 12 State ID Risk (Fraction) 0.00603 0.00458 0.00575 0.01017 0.00693 0.00548 0.00460 0.00644 13 State ID Risk (%) 0.60320 0.45777 0.57546 1.01739 0.69302 0.54769 0.45975 0.64413 14 15 16 District Data 17 18 District GE Enrollment (N) 40 150 30 100 320 260 100 1,000 19 District GE Composition (Fraction) 0.04000 0.15000 0.03000 0.10000 0.32000 0.26000 0.10000 1.00000 20 District GE Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 21 22 District ID Enrollment (N) 1 1 1 2 3 1 1 10 23 District ID Composition (Fraction) 0.10000 0.10000 0.10000 0.20000 0.30000 0.10000 0.10000 1.00000 24 District ID Composition (%) 10.00000 10.00000 10.00000 20.00000 30.00000 10.00000 10.00000 100.00000 25 District ID Risk (Fraction) 0.02500 0.00667 0.03333 0.02000 0.00938 0.00385 0.01000 0.01000 26 District ID Risk (%) 2.50000 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00000 27 28 29 Relative Diff. in Composition [Overrepresentation or Underrepresentation] 30 31 Relative Diff. in ID Composition (%) 150.00000-33.33333 233.33333 100.00000-6.25000-61.53846 0.00000 0.00000 32 91

Attachment B Page B-2 Hypothetical Small School District: General Education (GE) Enrollment = 1,000; Intellectual Disability (ID) Enrollment = 10 33 Native Asian Pacific Black Hispanic White Multiple Total/Overall 34 35 Risk Ratio [Overrepresentation or Underrepresentation] 36 37 District ID Risk (%) 2.50000 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00000 38 All Other ID in District 9 9 9 8 7 9 9 39 All Other GE in District 960 850 970 900 680 740 900 40 ID Risk for All Others (%) 0.93750 1.05882 0.92784 0.88889 1.02941 1.21622 1.00000 41 ID Risk Ratio 2.66667 0.62963 3.59259 2.25000 0.91071 0.31624 1.00000 42 43 44 Weighted Risk Ratio [Overrepresentation or Underrepresentation] 45 46 District ID Risk (Fraction) 0.02500 0.00667 0.03333 0.02000 0.00938 0.00385 0.01000 0.01000 47 (1-State GE Comp)*(Dist ID Risk) 0.02482 0.00592 0.03313 0.01860 0.00456 0.00279 0.00984 48 (State GE Comp)*(Dist ID Risk) 0.00018 0.00075 0.00020 0.00140 0.00481 0.00106 0.00016 0.00856 49 Sum of All Others in the Row Above 0.00838 0.00781 0.00836 0.00717 0.00375 0.00750 0.00840 50 Weighted ID Risk Ratio 2.96230 0.75742 3.96352 2.59637 1.21676 0.37150 1.17116 51 52 53 Alternate Risk Ratio [Overrepresentation or Underrepresentation] 54 55 District ID Risk (%) 2.50000 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00000 56 State ID (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 57 All Other ID in State 38,887 36,030 38,945 34,839 17,542 29,992 38,713 58 All Other GE in State 6,034,320 5,395,929 6,042,233 5,654,729 2,960,136 4,405,680 5,982,455 59 Statewide All Others' ID Risk (%) 0.64443 0.66773 0.64455 0.61610 0.59261 0.68076 0.64711 60 Alternate ID Risk Ratio 3.87939 0.99841 5.17159 3.24621 1.58199 0.56498 1.54533 61 92

Attachment B Page B-3 Hypothetical Small School District: General Education (GE) Enrollment = 1,000; Intellectual Disability (ID) Enrollment = 10 62 Native Asian Pacific Black Hispanic White Multiple Total/Overall 63 64 E-formula Data 65 66 District ID Enrollment (N) 1 1 1 2 3 1 1 10 67 A = District GE Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 68 Std. Error = [A*(100-A)/N] (%) 6.19677 11.29159 5.39444 9.48683 14.75127 13.87083 9.48683 69 70 71 E-formula [Overrepresentation] 72 73 District ID Composition (%) 10.00000 10.00000 10.00000 20.00000 30.00000 10.00000 10.00000 100.00000 74 Maximum Percent ID Allowed: 75 A + One Standard Error (%) 10.19677 26.29159 8.39444 19.48683 46.75127 39.87083 19.48683 76 A + Two Standard Errors (%) 16.39355 37.58318 13.78888 28.97367 61.50254 53.74167 28.97367 77 A + Three Standard Errors (%) 22.59032 48.87477 19.18332 38.46050 76.25381 67.61250 38.46050 78 A + Four Standard Errors (%) 28.78709 60.16636 24.57777 47.94733 91.00508 81.48333 47.94733 79 80 81 E-formula [Underrepresentation] 82 83 District ID Composition (%) 10.00000 10.00000 10.00000 20.00000 30.00000 10.00000 10.00000 100.00000 84 Minimum Percent ID Needed: 85 A - One Standard Error (%) -2.19677 3.70841-2.39444 0.51317 17.24873 12.12917 0.51317 86 A - Two Standard Errors (%) -8.39355-7.58318-7.78888-8.97367 2.49746-1.74167-8.97367 87 A - Three Standard Errors (%) -14.59032-18.87477-13.18332-18.46050-12.25381-15.61250-18.46050 88 A - Four Standard Errors (%) -20.78709-30.16636-18.57777-27.94733-27.00508-29.48333-27.94733 89 93

Attachment C Page C-1 Hypothetical Medium Sized School District: General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 1 Native Asian Pacific Black Hispanic White Multiple Total/Overall 2 3 State Data 4 5 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 6 State GE Composition (Fraction) 0.00739 0.11240 0.00609 0.06983 0.51308 0.27529 0.01592 1.00000 7 State GE Composition (%) 0.73902 11.24017 0.60886 6.98307 51.30752 27.52918 1.59217 100.00000 8 9 State ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 10 State ID Composition (Fraction) 0.00692 0.07988 0.00544 0.11030 0.55202 0.23408 0.01136 1.00000 11 State ID Composition (%) 0.69207 7.98815 0.54395 11.02967 55.20200 23.40773 1.13642 100.00000 12 State ID Risk (Fraction) 0.00603 0.00458 0.00575 0.01017 0.00693 0.00548 0.00460 0.00644 13 State ID Risk (%) 0.60320 0.45777 0.57546 1.01739 0.69302 0.54769 0.45975 0.64413 14 15 16 District Data 17 18 District GE Enrollment (N) 400 1,500 300 1,000 3,200 2,600 1,000 10,000 19 District GE Composition (Fraction) 0.04000 0.15000 0.03000 0.10000 0.32000 0.26000 0.10000 1.00000 20 District GE Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 21 22 District ID Enrollment (N) 10 10 10 20 30 10 10 100 23 District ID Composition (Fraction) 0.10000 0.10000 0.10000 0.20000 0.30000 0.10000 0.10000 1.00000 24 District ID Composition (%) 10.00000 10.00000 10.00000 20.00000 30.00000 10.00000 10.00000 100.00000 25 District ID Risk (Fraction) 0.02500 0.00667 0.03333 0.02000 0.00938 0.00385 0.01000 0.01000 26 District ID Risk (%) 2.50000 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00000 27 28 29 Relative Diff. in Composition [Overrepresentation or Underrepresentation] 30 31 Relative Diff. in ID Composition (%) 150.00000-33.33333 233.33333 100.00000-6.25000-61.53846 0.00000 0.00000 32 94

Attachment C Page C-2 Hypothetical Medium Sized School District: General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 33 Native Asian Pacific Black Hispanic White Multiple Total/Overall 34 35 Risk Ratio [Overrepresentation or Underrepresentation] 36 37 District ID Risk (%) 2.50000 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00000 38 All Other ID in District 90 90 90 80 70 90 90 39 All Other GE in District 9,600 8,500 9,700 9,000 6,800 7,400 9,000 40 ID Risk for All Others (%) 0.93750 1.05882 0.92784 0.88889 1.02941 1.21622 1.00000 41 ID Risk Ratio 2.66667 0.62963 3.59259 2.25000 0.91071 0.31624 1.00000 42 43 44 Weighted Risk Ratio [Overrepresentation or Underrepresentation] 45 46 District ID Risk (Fraction) 0.02500 0.00667 0.03333 0.02000 0.00938 0.00385 0.01000 0.01000 47 (1-State GE Comp)*(Dist ID Risk) 0.02482 0.00592 0.03313 0.01860 0.00456 0.00279 0.00984 48 (State GE Comp)*(Dist ID Risk) 0.00018 0.00075 0.00020 0.00140 0.00481 0.00106 0.00016 0.00856 49 Sum of All Others in the Row Above 0.00838 0.00781 0.00836 0.00717 0.00375 0.00750 0.00840 50 Weighted ID Risk Ratio 2.96230 0.75742 3.96352 2.59637 1.21676 0.37150 1.17116 51 52 53 Alternate Risk Ratio [Overrepresentation or Underrepresentation] 54 55 District ID Risk (%) 2.50000 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00000 56 State ID (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 57 All Other ID in State 38,887 36,030 38,945 34,839 17,542 29,992 38,713 58 All Other GE in State 6,034,320 5,395,929 6,042,233 5,654,729 2,960,136 4,405,680 5,982,455 59 Statewide All Others' ID Risk (%) 0.64443 0.66773 0.64455 0.61610 0.59261 0.68076 0.64711 60 Alternate ID Risk Ratio 3.87939 0.99841 5.17159 3.24621 1.58199 0.56498 1.54533 61 95

Attachment C Page C-3 Hypothetical Medium Sized School District: General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 62 Native Asian Pacific Black Hispanic White Multiple Total/Overall 63 64 E-formula Data 65 66 District ID Enrollment (N) 10 10 10 20 30 10 10 100 67 A = District GE Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 68 Std. Error = [A*(100-A)/N] (%) 1.95959 3.57071 1.70587 3.00000 4.66476 4.38634 3.00000 69 70 71 E-formula [Overrepresentation] 72 73 District ID Composition (%) 10.00000 10.00000 10.00000 20.00000 30.00000 10.00000 10.00000 100.00000 74 Maximum Percent ID Allowed: 75 A + One Standard Error (%) 5.95959 18.57071 4.70587 13.00000 36.66476 30.38634 13.00000 76 A + Two Standard Errors (%) 7.91918 22.14143 6.41174 16.00000 41.32952 34.77268 16.00000 77 A + Three Standard Errors (%) 9.87878 25.71214 8.11762 19.00000 45.99428 39.15903 19.00000 78 A + Four Standard Errors (%) 11.83837 29.28286 9.82349 22.00000 50.65905 43.54537 22.00000 79 80 81 E-formula [Underrepresentation] 82 83 District ID Composition (%) 10.00000 10.00000 10.00000 20.00000 30.00000 10.00000 10.00000 100.00000 84 Minimum Percent ID Needed: 85 A - One Standard Error (%) 2.04041 11.42929 1.29413 7.00000 27.33524 21.61366 7.00000 86 A - Two Standard Errors (%) 0.08082 7.85857-0.41174 4.00000 22.67048 17.22732 4.00000 87 A - Three Standard Errors (%) -1.87878 4.28786-2.11762 1.00000 18.00572 12.84097 1.00000 88 A - Four Standard Errors (%) -3.83837 0.71714-3.82349-2.00000 13.34095 8.45463-2.00000 89 96

Attachment D Page D-1 Hypothetical Large School District: General Education (GE) Enrollment = 50,000; Intellectual Disability (ID) Enrollment = 500 1 Native Asian Pacific Black Hispanic White Multiple Total/Overall 2 3 State Data 4 5 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 6 State GE Composition (Fraction) 0.00739 0.11240 0.00609 0.06983 0.51308 0.27529 0.01592 1.00000 7 State GE Composition (%) 0.73902 11.24017 0.60886 6.98307 51.30752 27.52918 1.59217 100.00000 8 9 State ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 10 State ID Composition (Fraction) 0.00692 0.07988 0.00544 0.11030 0.55202 0.23408 0.01136 1.00000 11 State ID Composition (%) 0.69207 7.98815 0.54395 11.02967 55.20200 23.40773 1.13642 100.00000 12 State ID Risk (Fraction) 0.00603 0.00458 0.00575 0.01017 0.00693 0.00548 0.00460 0.00644 13 State ID Risk (%) 0.60320 0.45777 0.57546 1.01739 0.69302 0.54769 0.45975 0.64413 14 15 16 District Data 17 18 District GE Enrollment (N) 2,000 7,500 1,500 5,000 16,000 13,000 5,000 50,000 19 District GE Composition (Fraction) 0.04000 0.15000 0.03000 0.10000 0.32000 0.26000 0.10000 1.00000 20 District GE Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 21 22 District ID Enrollment (N) 50 50 50 100 150 50 50 500 23 District ID Composition (Fraction) 0.10000 0.10000 0.10000 0.20000 0.30000 0.10000 0.10000 1.00000 24 District ID Composition (%) 10.00000 10.00000 10.00000 20.00000 30.00000 10.00000 10.00000 100.00000 25 District ID Risk (Fraction) 0.02500 0.00667 0.03333 0.02000 0.00938 0.00385 0.01000 0.01000 26 District ID Risk (%) 2.50000 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00000 27 28 29 Relative Diff. in Composition [Overrepresentation or Underrepresentation] 30 31 Relative Diff. in ID Composition (%) 150.00000-33.33333 233.33333 100.00000-6.25000-61.53846 0.00000 0.00000 32 97

Attachment D Page D-2 Hypothetical Large School District: General Education (GE) Enrollment = 50,000; Intellectual Disability (ID) Enrollment = 500 33 Native Asian Pacific Black Hispanic White Multiple Total/Overall 34 35 Risk Ratio [Overrepresentation or Underrepresentation] 36 37 District ID Risk (%) 2.50000 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00000 38 All Other ID in District 450 450 450 400 350 450 450 39 All Other GE in District 48,000 42,500 48,500 45,000 34,000 37,000 45,000 40 ID Risk for All Others (%) 0.93750 1.05882 0.92784 0.88889 1.02941 1.21622 1.00000 41 ID Risk Ratio 2.66667 0.62963 3.59259 2.25000 0.91071 0.31624 1.00000 42 43 44 Weighted Risk Ratio [Overrepresentation or Underrepresentation] 45 46 District ID Risk (Fraction) 0.02500 0.00667 0.03333 0.02000 0.00938 0.00385 0.01000 0.01000 47 (1-State GE Comp)*(Dist ID Risk) 0.02482 0.00592 0.03313 0.01860 0.00456 0.00279 0.00984 48 (State GE Comp)*(Dist ID Risk) 0.00018 0.00075 0.00020 0.00140 0.00481 0.00106 0.00016 0.00856 49 Sum of All Others in the Row Above 0.00838 0.00781 0.00836 0.00717 0.00375 0.00750 0.00840 50 Weighted ID Risk Ratio 2.96230 0.75742 3.96352 2.59637 1.21676 0.37150 1.17116 51 52 53 Alternate Risk Ratio [Overrepresentation or Underrepresentation] 54 55 District ID Risk (%) 2.50000 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00000 56 State ID (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 57 All Other ID in State 38,887 36,030 38,945 34,839 17,542 29,992 38,713 58 All Other GE in State 6,034,320 5,395,929 6,042,233 5,654,729 2,960,136 4,405,680 5,982,455 59 Statewide All Others' ID Risk (%) 0.64443 0.66773 0.64455 0.61610 0.59261 0.68076 0.64711 60 Alternate ID Risk Ratio 3.87939 0.99841 5.17159 3.24621 1.58199 0.56498 1.54533 61 98

Attachment D Page D-3 Hypothetical Large School District: General Education (GE) Enrollment = 50,000; Intellectual Disability (ID) Enrollment = 500 62 Native Asian Pacific Black Hispanic White Multiple Total/Overall 63 64 E-formula Data 65 66 District ID Enrollment (N) 50 50 50 100 150 50 50 500 67 A = District GE Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 68 Std. Error = [A*(100-A)/N] (%) 0.87636 1.59687 0.76289 1.34164 2.08614 1.96163 1.34164 69 70 71 E-formula [Overrepresentation] 72 73 District ID Composition (%) 10.00000 10.00000 10.00000 20.00000 30.00000 10.00000 10.00000 100.00000 74 Maximum Percent ID Allowed: 75 A + One Standard Error (%) 4.87636 16.59687 3.76289 11.34164 34.08614 27.96163 11.34164 76 A + Two Standard Errors (%) 5.75271 18.19374 4.52578 12.68328 36.17229 29.92326 12.68328 77 A + Three Standard Errors (%) 6.62907 19.79062 5.28867 14.02492 38.25843 31.88490 14.02492 78 A + Four Standard Errors (%) 7.50542 21.38749 6.05156 15.36656 40.34458 33.84653 15.36656 79 80 81 E-formula [Underrepresentation] 82 83 District ID Composition (%) 10.00000 10.00000 10.00000 20.00000 30.00000 10.00000 10.00000 100.00000 84 Minimum Percent ID Needed: 85 A - One Standard Error (%) 3.12364 13.40313 2.23711 8.65836 29.91386 24.03837 8.65836 86 A - Two Standard Errors (%) 2.24729 11.80626 1.47422 7.31672 27.82771 22.07674 7.31672 87 A - Three Standard Errors (%) 1.37093 10.20938 0.71133 5.97508 25.74157 20.11510 5.97508 88 A - Four Standard Errors (%) 0.49458 8.61251-0.05156 4.63344 23.65542 18.15347 4.63344 89 99

Attachment E Page E-1 Hypothetical Small School District - Effect of One New Native American Student: General Education (GE) Enrollment = 1,001; Intellectual Disability (ID) Enrollment = 11 1 Native Asian Pacific Black Hispanic White Multiple Total/Overall 2 3 State Data 4 5 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 6 State GE Composition (Fraction) 0.00739 0.11240 0.00609 0.06983 0.51308 0.27529 0.01592 1.00000 7 State GE Composition (%) 0.73902 11.24017 0.60886 6.98307 51.30752 27.52918 1.59217 100.00000 8 9 State ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 10 State ID Composition (Fraction) 0.00692 0.07988 0.00544 0.11030 0.55202 0.23408 0.01136 1.00000 11 State ID Composition (%) 0.69207 7.98815 0.54395 11.02967 55.20200 23.40773 1.13642 100.00000 12 State ID Risk (Fraction) 0.00603 0.00458 0.00575 0.01017 0.00693 0.00548 0.00460 0.00644 13 State ID Risk (%) 0.60320 0.45777 0.57546 1.01739 0.69302 0.54769 0.45975 0.64413 14 15 16 District Data 17 18 District GE Enrollment (N) 41 150 30 100 320 260 100 1,001 19 District GE Composition (Fraction) 0.04096 0.14985 0.02997 0.09990 0.31968 0.25974 0.09990 1.00000 20 District GE Composition (%) 4.09590 14.98501 2.99700 9.99001 31.96803 25.97403 9.99001 100.00000 21 22 District ID Enrollment (N) 2 1 1 2 3 1 1 11 23 District ID Composition (Fraction) 0.18182 0.09091 0.09091 0.18182 0.27273 0.09091 0.09091 1.00000 24 District ID Composition (%) 18.18182 9.09091 9.09091 18.18182 27.27273 9.09091 9.09091 100.00000 25 District ID Risk (Fraction) 0.04878 0.00667 0.03333 0.02000 0.00938 0.00385 0.01000 0.01099 26 District ID Risk (%) 4.87805 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.09890 27 28 29 Relative Diff. in Composition [Overrepresentation or Underrepresentation] 30 31 Relative Diff. in ID Composition (%) 343.90244-39.33333 203.33333 82.00000-14.68750-65.00000-9.00000 0.00000 32 100

Attachment E Page E-2 Hypothetical Small School District - Effect of One New Native American Student: General Education (GE) Enrollment = 1,001; Intellectual Disability (ID) Enrollment = 11 33 Native Asian Pacific Black Hispanic White Multiple Total/Overall 34 35 Risk Ratio [Overrepresentation or Underrepresentation] 36 37 District ID Risk (%) 4.87805 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.09890 38 All Other ID in District 9 10 10 9 8 10 10 39 All Other GE in District 960 851 971 901 681 741 901 40 ID Risk for All Others (%) 0.93750 1.17509 1.02987 0.99889 1.17474 1.34953 1.10988 41 ID Risk Ratio 5.20325 0.56733 3.23667 2.00222 0.79805 0.28500 0.90100 42 43 44 Weighted Risk Ratio [Overrepresentation or Underrepresentation] 45 46 District ID Risk (Fraction) 0.04878 0.00667 0.03333 0.02000 0.00938 0.00385 0.01000 0.01099 47 (1-State GE Comp)*(Dist ID Risk) 0.04842 0.00592 0.03313 0.01860 0.00456 0.00279 0.00984 48 (State GE Comp)*(Dist ID Risk) 0.00036 0.00075 0.00020 0.00140 0.00481 0.00106 0.00016 0.00874 49 Sum of All Others in the Row Above 0.00838 0.00799 0.00853 0.00734 0.00393 0.00768 0.00858 50 Weighted ID Risk Ratio 5.78009 0.74076 3.88190 2.53421 1.16231 0.36300 1.14717 51 52 53 Alternate Risk Ratio [Overrepresentation or Underrepresentation] 54 55 District ID Risk (%) 4.87805 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.09890 56 State ID (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 57 All Other ID in State 38,887 36,030 38,945 34,839 17,542 29,992 38,713 58 All Other GE in State 6,034,320 5,395,929 6,042,233 5,654,729 2,960,136 4,405,680 5,982,455 59 Statewide All Others' ID Risk (%) 0.64443 0.66773 0.64455 0.61610 0.59261 0.68076 0.64711 60 Alternate ID Risk Ratio 7.56955 0.99841 5.17159 3.24621 1.58199 0.56498 1.54533 61 101

Attachment E Page E-3 Hypothetical Small School District - Effect of One New Native American Student: General Education (GE) Enrollment = 1,001; Intellectual Disability (ID) Enrollment = 11 62 Native Asian Pacific Black Hispanic White Multiple Total/Overall 63 64 E-formula Data 65 66 District ID Enrollment (N) 2 1 1 2 3 1 1 11 67 A = District GE Composition (%) 4.09590 14.98501 2.99700 9.99001 31.96803 25.97403 9.99001 100.00000 68 Std. Error = [A*(100-A)/N] (%) 5.97581 10.76168 5.14091 9.04132 14.06106 13.22103 9.04132 69 70 71 E-formula [Overrepresentation] 72 73 District ID Composition (%) 18.18182 9.09091 9.09091 18.18182 27.27273 9.09091 9.09091 100.00000 74 Maximum Percent ID Allowed: 75 A + One Standard Error (%) 10.07172 25.74669 8.13791 19.03133 46.02909 39.19506 19.03133 76 A + Two Standard Errors (%) 16.04753 36.50837 13.27882 28.07266 60.09016 52.41609 28.07266 77 A + Three Standard Errors (%) 22.02335 47.27005 18.41973 37.11398 74.15122 65.63712 37.11398 78 A + Four Standard Errors (%) 27.99916 58.03173 23.56063 46.15530 88.21228 78.85815 46.15530 79 80 81 E-formula [Underrepresentation] 82 83 District ID Composition (%) 18.18182 9.09091 9.09091 18.18182 27.27273 9.09091 9.09091 100.00000 84 Minimum Percent ID Needed: 85 A - One Standard Error (%) -1.87991 4.22334-2.14390 0.94869 17.90697 12.75299 0.94869 86 A - Two Standard Errors (%) -7.85573-6.53834-7.28481-8.09264 3.84591-0.46804-8.09264 87 A - Three Standard Errors (%) -13.83154-17.30002-12.42572-17.13396-10.21515-13.68907-17.13396 88 A - Four Standard Errors (%) -19.80735-28.06170-17.56663-26.17528-24.27622-26.91010-26.17528 89 102

Attachment F Page F-1 Hypothetical Medium Sized School District - Effect of One New Native American Student: General Education (GE) Enrollment = 10,001; Intellectual Disability (ID) Enrollment = 101 1 Native Asian Pacific Black Hispanic White Multiple Total/Overall 2 3 State Data 4 5 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 6 State GE Composition (Fraction) 0.00739 0.11240 0.00609 0.06983 0.51308 0.27529 0.01592 1.00000 7 State GE Composition (%) 0.73902 11.24017 0.60886 6.98307 51.30752 27.52918 1.59217 100.00000 8 9 State ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 10 State ID Composition (Fraction) 0.00692 0.07988 0.00544 0.11030 0.55202 0.23408 0.01136 1.00000 11 State ID Composition (%) 0.69207 7.98815 0.54395 11.02967 55.20200 23.40773 1.13642 100.00000 12 State ID Risk (Fraction) 0.00603 0.00458 0.00575 0.01017 0.00693 0.00548 0.00460 0.00644 13 State ID Risk (%) 0.60320 0.45777 0.57546 1.01739 0.69302 0.54769 0.45975 0.64413 14 15 16 District Data 17 18 District GE Enrollment (N) 401 1,500 300 1,000 3,200 2,600 1,000 10,001 19 District GE Composition (Fraction) 0.04010 0.14999 0.03000 0.09999 0.31997 0.25997 0.09999 1.00000 20 District GE Composition (%) 4.00960 14.99850 2.99970 9.99900 31.99680 25.99740 9.99900 100.00000 21 22 District ID Enrollment (N) 11 10 10 20 30 10 10 101 23 District ID Composition (Fraction) 0.10891 0.09901 0.09901 0.19802 0.29703 0.09901 0.09901 1.00000 24 District ID Composition (%) 10.89109 9.90099 9.90099 19.80198 29.70297 9.90099 9.90099 100.00000 25 District ID Risk (Fraction) 0.02743 0.00667 0.03333 0.02000 0.00938 0.00385 0.01000 0.01010 26 District ID Risk (%) 2.74314 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00990 27 28 29 Relative Diff. in Composition [Overrepresentation or Underrepresentation] 30 31 Relative Diff. in ID Composition (%) 171.62539-33.98680 230.06601 98.03960-7.16894-61.91546-0.98020 0.00000 32 103

Attachment F Page F-2 Hypothetical Medium Sized School District - Effect of One New Native American Student: General Education (GE) Enrollment = 10,001; Intellectual Disability (ID) Enrollment = 101 33 Native Asian Pacific Black Hispanic White Multiple Total/Overall 34 35 Risk Ratio [Overrepresentation or Underrepresentation] 36 37 District ID Risk (%) 2.74314 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00990 38 All Other ID in District 90 91 91 81 71 91 91 39 All Other GE in District 9,600 8,501 9,701 9,001 6,801 7,401 9,001 40 ID Risk for All Others (%) 0.93750 1.07046 0.93805 0.89990 1.04396 1.22956 1.01100 41 ID Risk Ratio 2.92602 0.62278 3.55348 2.22247 0.89802 0.31281 0.98912 42 43 44 Weighted Risk Ratio [Overrepresentation or Underrepresentation] 45 46 District ID Risk (Fraction) 0.02743 0.00667 0.03333 0.02000 0.00938 0.00385 0.01000 0.01010 47 (1-State GE Comp)*(Dist ID Risk) 0.02723 0.00592 0.03313 0.01860 0.00456 0.00279 0.00984 48 (State GE Comp)*(Dist ID Risk) 0.00020 0.00075 0.00020 0.00140 0.00481 0.00106 0.00016 0.00858 49 Sum of All Others in the Row Above 0.00838 0.00783 0.00838 0.00718 0.00377 0.00752 0.00842 50 Weighted ID Risk Ratio 3.25040 0.75569 3.95502 2.58987 1.21096 0.37061 1.16867 51 52 53 Alternate Risk Ratio [Overrepresentation or Underrepresentation] 54 55 District ID Risk (%) 2.74314 0.66667 3.33333 2.00000 0.93750 0.38462 1.00000 1.00990 56 State ID (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 57 All Other ID in State 38,887 36,030 38,945 34,839 17,542 29,992 38,713 58 All Other GE in State 6,034,320 5,395,929 6,042,233 5,654,729 2,960,136 4,405,680 5,982,455 59 Statewide All Others' ID Risk (%) 0.64443 0.66773 0.64455 0.61610 0.59261 0.68076 0.64711 60 Alternate ID Risk Ratio 4.25669 0.99841 5.17159 3.24621 1.58199 0.56498 1.54533 61 104

Attachment F Page F-3 Hypothetical Medium Sized School District - Effect of One New Native American Student: General Education (GE) Enrollment = 10,001; Intellectual Disability (ID) Enrollment = 101 62 Native Asian Pacific Black Hispanic White Multiple Total/Overall 63 64 E-formula Data 65 66 District ID Enrollment (N) 11 10 10 20 30 10 10 101 67 A = District GE Composition (%) 4.00960 14.99850 2.99970 9.99900 31.99680 25.99740 9.99900 100.00000 68 Std. Error = [A*(100-A)/N] (%) 1.95211 3.55285 1.69732 2.98498 4.64149 4.36443 2.98498 69 70 71 E-formula [Overrepresentation] 72 73 District ID Composition (%) 10.89109 9.90099 9.90099 19.80198 29.70297 9.90099 9.90099 100.00000 74 Maximum Percent ID Allowed: 75 A + One Standard Error (%) 5.96171 18.55135 4.69702 12.98398 36.63829 30.36183 12.98398 76 A + Two Standard Errors (%) 7.91381 22.10419 6.39435 15.96896 41.27978 34.72626 15.96896 77 A + Three Standard Errors (%) 9.86592 25.65704 8.09167 18.95394 45.92127 39.09070 18.95394 78 A + Four Standard Errors (%) 11.81803 29.20989 9.78900 21.93892 50.56275 43.45513 21.93892 79 80 81 E-formula [Underrepresentation] 82 83 District ID Composition (%) 10.89109 9.90099 9.90099 19.80198 29.70297 9.90099 9.90099 100.00000 84 Minimum Percent ID Needed: 85 A - One Standard Error (%) 2.05749 11.44565 1.30238 7.01402 27.35531 21.63297 7.01402 86 A - Two Standard Errors (%) 0.10538 7.89281-0.39495 4.02904 22.71382 17.26854 4.02904 87 A - Three Standard Errors (%) -1.84672 4.33996-2.09227 1.04406 18.07234 12.90410 1.04406 88 A - Four Standard Errors (%) -3.79883 0.78711-3.78960-1.94092 13.43085 8.53967-1.94092 89 105

Attachment G Page G-1 Hypothetical Medium Sized School District - Perfectly Homogeneous (100% Hispanic): General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 1 Native Asian Pacific Black Hispanic White Multiple Total/Overall 2 3 State Data 4 5 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 6 State GE Composition (Fraction) 0.00739 0.11240 0.00609 0.06983 0.51308 0.27529 0.01592 1.00000 7 State GE Composition (%) 0.73902 11.24017 0.60886 6.98307 51.30752 27.52918 1.59217 100.00000 8 9 State ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 10 State ID Composition (Fraction) 0.00692 0.07988 0.00544 0.11030 0.55202 0.23408 0.01136 1.00000 11 State ID Composition (%) 0.69207 7.98815 0.54395 11.02967 55.20200 23.40773 1.13642 100.00000 12 State ID Risk (Fraction) 0.00603 0.00458 0.00575 0.01017 0.00693 0.00548 0.00460 0.00644 13 State ID Risk (%) 0.60320 0.45777 0.57546 1.01739 0.69302 0.54769 0.45975 0.64413 14 15 16 District Data 17 18 District GE Enrollment (N) 0 0 0 0 10,000 0 0 10,000 19 District GE Composition (Fraction) 0.00000 0.00000 0.00000 0.00000 1.00000 0.00000 0.00000 1.00000 20 District GE Composition (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 100.00000 21 22 District ID Enrollment (N) 0 0 0 0 100 0 0 100 23 District ID Composition (Fraction) 0.00000 0.00000 0.00000 0.00000 1.00000 0.00000 0.00000 1.00000 24 District ID Composition (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 100.00000 25 District ID Risk (Fraction) #DIV/0! #DIV/0! #DIV/0! #DIV/0! 0.01000 #DIV/0! #DIV/0! 0.01000 26 District ID Risk (%) #DIV/0! #DIV/0! #DIV/0! #DIV/0! 1.00000 #DIV/0! #DIV/0! 1.00000 27 28 29 Relative Diff. in Composition [Overrepresentation or Underrepresentation] 30 31 Relative Diff. in ID Composition (%) #DIV/0! #DIV/0! #DIV/0! #DIV/0! 0.00000 #DIV/0! #DIV/0! 0.00000 32 106

Attachment G Page G-2 Hypothetical Medium Sized School District - Perfectly Homogeneous (100% Hispanic): General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 33 Native Asian Pacific Black Hispanic White Multiple Total/Overall 34 35 Risk Ratio [Overrepresentation or Underrepresentation] 36 37 District ID Risk (%) #DIV/0! #DIV/0! #DIV/0! #DIV/0! 1.00000 #DIV/0! #DIV/0! 1.00000 38 All Other ID in District 100 100 100 100 0 100 100 39 All Other GE in District 10,000 10,000 10,000 10,000 0 10,000 10,000 40 ID Risk for All Others (%) 1.00000 1.00000 1.00000 1.00000 #DIV/0! 1.00000 1.00000 41 ID Risk Ratio #DIV/0! #DIV/0! #DIV/0! #DIV/0! #DIV/0! #DIV/0! #DIV/0! 42 43 44 Weighted Risk Ratio [Overrepresentation or Underrepresentation] 45 46 District ID Risk (Fraction) #DIV/0! #DIV/0! #DIV/0! #DIV/0! 0.01000 #DIV/0! #DIV/0! 0.01000 47 (1-State GE Comp)*(Dist ID Risk) #DIV/0! #DIV/0! #DIV/0! #DIV/0! 0.00487 #DIV/0! #DIV/0! 48 (State GE Comp)*(Dist ID Risk) #DIV/0! #DIV/0! #DIV/0! #DIV/0! 0.00513 #DIV/0! #DIV/0! 0.00513 49 Sum of All Others in the Row Above 0.00513 0.00513 0.00513 0.00513 0.00000 0.00513 0.00513 50 Weighted ID Risk Ratio #DIV/0! #DIV/0! #DIV/0! #DIV/0! #DIV/0! #DIV/0! #DIV/0! 51 52 53 Alternate Risk Ratio [Overrepresentation or Underrepresentation] 54 55 District ID Risk (%) #DIV/0! #DIV/0! #DIV/0! #DIV/0! 1.00000 #DIV/0! #DIV/0! 1.00000 56 State ID (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 57 All Other ID in State 38,887 36,030 38,945 34,839 17,542 29,992 38,713 58 All Other GE in State 6,034,320 5,395,929 6,042,233 5,654,729 2,960,136 4,405,680 5,982,455 59 Statewide All Others' ID Risk (%) 0.64443 0.66773 0.64455 0.61610 0.59261 0.68076 0.64711 60 Alternate ID Risk Ratio #DIV/0! #DIV/0! #DIV/0! #DIV/0! 1.68746 #DIV/0! #DIV/0! 61 107

Attachment G Page G-3 Hypothetical Medium Sized School District - Perfectly Homogeneous (100% Hispanic): General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 62 Native Asian Pacific Black Hispanic White Multiple Total/Overall 63 64 E-formula Data 65 66 District ID Enrollment (N) 0 0 0 0 100 0 0 100 67 A = District GE Composition (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 100.00000 68 Std. Error = [A*(100-A)/N] (%) 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 69 70 71 E-formula [Overrepresentation] 72 73 District ID Composition (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 100.00000 74 Maximum Percent ID Allowed: 75 A + One Standard Error (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 76 A + Two Standard Errors (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 77 A + Three Standard Errors (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 78 A + Four Standard Errors (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 79 80 81 E-formula [Underrepresentation] 82 83 District ID Composition (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 100.00000 84 Minimum Percent ID Needed: 85 A - One Standard Error (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 86 A - Two Standard Errors (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 87 A - Three Standard Errors (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 88 A - Four Standard Errors (%) 0.00000 0.00000 0.00000 0.00000 100.00000 0.00000 0.00000 89 108

Attachment H Page H-1 Hypothetical Medium Sized School District - Almost Homogeneous (90% Hispanic): General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 1 Native Asian Pacific Black Hispanic White Multiple Total/Overall 2 3 State Data 4 5 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 6 State GE Composition (Fraction) 0.00739 0.11240 0.00609 0.06983 0.51308 0.27529 0.01592 1.00000 7 State GE Composition (%) 0.73902 11.24017 0.60886 6.98307 51.30752 27.52918 1.59217 100.00000 8 9 State ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 10 State ID Composition (Fraction) 0.00692 0.07988 0.00544 0.11030 0.55202 0.23408 0.01136 1.00000 11 State ID Composition (%) 0.69207 7.98815 0.54395 11.02967 55.20200 23.40773 1.13642 100.00000 12 State ID Risk (Fraction) 0.00603 0.00458 0.00575 0.01017 0.00693 0.00548 0.00460 0.00644 13 State ID Risk (%) 0.60320 0.45777 0.57546 1.01739 0.69302 0.54769 0.45975 0.64413 14 15 16 District Data 17 18 District GE Enrollment (N) 0 0 0 300 9,000 700 0 10,000 19 District GE Composition (Fraction) 0.00000 0.00000 0.00000 0.03000 0.90000 0.07000 0.00000 1.00000 20 District GE Composition (%) 0.00000 0.00000 0.00000 3.00000 90.00000 7.00000 0.00000 100.00000 21 22 District ID Enrollment (N) 0 0 0 9 85 6 0 100 23 District ID Composition (Fraction) 0.00000 0.00000 0.00000 0.09000 0.85000 0.06000 0.00000 1.00000 24 District ID Composition (%) 0.00000 0.00000 0.00000 9.00000 85.00000 6.00000 0.00000 100.00000 25 District ID Risk (Fraction) #DIV/0! #DIV/0! #DIV/0! 0.03000 0.00944 0.00857 #DIV/0! 0.01000 26 District ID Risk (%) #DIV/0! #DIV/0! #DIV/0! 3.00000 0.94444 0.85714 #DIV/0! 1.00000 27 28 29 Relative Diff. in Composition [Overrepresentation or Underrepresentation] 30 31 Relative Diff. in ID Composition (%) #DIV/0! #DIV/0! #DIV/0! 200.00000-5.55556-14.28571 #DIV/0! 0.00000 32 109

Attachment H Page H-2 Hypothetical Medium Sized School District - Almost Homogeneous (90% Hispanic): General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 33 Native Asian Pacific Black Hispanic White Multiple Total/Overall 34 35 Risk Ratio [Overrepresentation or Underrepresentation] 36 37 District ID Risk (%) #DIV/0! #DIV/0! #DIV/0! 3.00000 0.94444 0.85714 #DIV/0! 1.00000 38 All Other ID in District 100 100 100 91 15 94 100 39 All Other GE in District 10,000 10,000 10,000 9,700 1,000 9,300 10,000 40 ID Risk for All Others (%) 1.00000 1.00000 1.00000 0.93814 1.50000 1.01075 1.00000 41 ID Risk Ratio #DIV/0! #DIV/0! #DIV/0! 3.19780 0.62963 0.84802 #DIV/0! 42 43 44 Weighted Risk Ratio [Overrepresentation or Underrepresentation] 45 46 District ID Risk (Fraction) #DIV/0! #DIV/0! #DIV/0! 0.03000 0.00944 0.00857 #DIV/0! 0.01000 47 (1-State GE Comp)*(Dist ID Risk) #DIV/0! #DIV/0! #DIV/0! 0.02791 0.00460 0.00621 #DIV/0! 48 (State GE Comp)*(Dist ID Risk) #DIV/0! #DIV/0! #DIV/0! 0.00209 0.00485 0.00236 #DIV/0! 0.00930 49 Sum of All Others in the Row Above 0.00930 0.00930 0.00930 0.00721 0.00445 0.00694 0.00930 50 Weighted ID Risk Ratio #DIV/0! #DIV/0! #DIV/0! 3.87283 1.03236 0.89499 #DIV/0! 51 52 53 Alternate Risk Ratio [Overrepresentation or Underrepresentation] 54 55 District ID Risk (%) #DIV/0! #DIV/0! #DIV/0! 3.00000 0.94444 0.85714 #DIV/0! 1.00000 56 State ID (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 57 All Other ID in State 38,887 36,030 38,945 34,839 17,542 29,992 38,713 58 All Other GE in State 6,034,320 5,395,929 6,042,233 5,654,729 2,960,136 4,405,680 5,982,455 59 Statewide All Others' ID Risk (%) 0.64443 0.66773 0.64455 0.61610 0.59261 0.68076 0.64711 60 Alternate ID Risk Ratio #DIV/0! #DIV/0! #DIV/0! 4.86931 1.59371 1.25910 #DIV/0! 61 110

Attachment H Page H-3 Hypothetical Medium Sized School District - Almost Homogeneous (90% Hispanic): General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 100 62 Native Asian Pacific Black Hispanic White Multiple Total/Overall 63 64 E-formula Data 65 66 District ID Enrollment (N) 0 0 0 9 85 6 0 100 67 A = District GE Composition (%) 0.00000 0.00000 0.00000 3.00000 90.00000 7.00000 0.00000 100.00000 68 Std. Error = [A*(100-A)/N] (%) 0.00000 0.00000 0.00000 1.70587 3.00000 2.55147 0.00000 69 70 71 E-formula [Overrepresentation] 72 73 District ID Composition (%) 0.00000 0.00000 0.00000 9.00000 85.00000 6.00000 0.00000 100.00000 74 Maximum Percent ID Allowed: 75 A + One Standard Error (%) 0.00000 0.00000 0.00000 4.70587 93.00000 9.55147 0.00000 76 A + Two Standard Errors (%) 0.00000 0.00000 0.00000 6.41174 96.00000 12.10294 0.00000 77 A + Three Standard Errors (%) 0.00000 0.00000 0.00000 8.11762 99.00000 14.65441 0.00000 78 A + Four Standard Errors (%) 0.00000 0.00000 0.00000 9.82349 102.00000 17.20588 0.00000 79 80 81 E-formula [Underrepresentation] 82 83 District ID Composition (%) 0.00000 0.00000 0.00000 9.00000 85.00000 6.00000 0.00000 100.00000 84 Minimum Percent ID Needed: 85 A - One Standard Error (%) 0.00000 0.00000 0.00000 1.29413 87.00000 4.44853 0.00000 86 A - Two Standard Errors (%) 0.00000 0.00000 0.00000-0.41174 84.00000 1.89706 0.00000 87 A - Three Standard Errors (%) 0.00000 0.00000 0.00000-2.11762 81.00000-0.65441 0.00000 88 A - Four Standard Errors (%) 0.00000 0.00000 0.00000-3.82349 78.00000-3.20588 0.00000 89 111

Attachment I Page I-1 Hypothetical Medium Sized School District - Effect on High Incidence Rate in District: General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 200 1 Native Asian Pacific Black Hispanic White Multiple Total/Overall 2 3 State Data 4 5 State GE Enrollment (N) 44,927 683,318 37,014 424,518 3,119,111 1,673,567 96,792 6,079,247 6 State GE Composition (Fraction) 0.00739 0.11240 0.00609 0.06983 0.51308 0.27529 0.01592 1.00000 7 State GE Composition (%) 0.73902 11.24017 0.60886 6.98307 51.30752 27.52918 1.59217 100.00000 8 9 State ID Enrollment (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 10 State ID Composition (Fraction) 0.00692 0.07988 0.00544 0.11030 0.55202 0.23408 0.01136 1.00000 11 State ID Composition (%) 0.69207 7.98815 0.54395 11.02967 55.20200 23.40773 1.13642 100.00000 12 State ID Risk (Fraction) 0.00603 0.00458 0.00575 0.01017 0.00693 0.00548 0.00460 0.00644 13 State ID Risk (%) 0.60320 0.45777 0.57546 1.01739 0.69302 0.54769 0.45975 0.64413 14 15 16 District Data 17 18 District GE Enrollment (N) 400 1,500 300 1,000 3,200 2,600 1,000 10,000 19 District GE Composition (Fraction) 0.04000 0.15000 0.03000 0.10000 0.32000 0.26000 0.10000 1.00000 20 District GE Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 21 22 District ID Enrollment (N) 8 30 6 20 64 52 20 200 23 District ID Composition (Fraction) 0.04000 0.15000 0.03000 0.10000 0.32000 0.26000 0.10000 1.00000 24 District ID Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 25 District ID Risk (Fraction) 0.02000 0.02000 0.02000 0.02000 0.02000 0.02000 0.02000 0.02000 26 District ID Risk (%) 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 27 28 29 Relative Diff. in Composition [Overrepresentation or Underrepresentation] 30 31 Relative Diff. in ID Composition (%) 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 32 112

Attachment I Page I-2 Hypothetical Medium Sized School District - Effect on High Incidence Rate in District: General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 200 33 Native Asian Pacific Black Hispanic White Multiple Total/Overall 34 35 Risk Ratio [Overrepresentation or Underrepresentation] 36 37 District ID Risk (%) 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 38 All Other ID in District 192 170 194 180 136 148 180 39 All Other GE in District 9,600 8,500 9,700 9,000 6,800 7,400 9,000 40 ID Risk for All Others (%) 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 41 ID Risk Ratio 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 42 43 44 Weighted Risk Ratio [Overrepresentation or Underrepresentation] 45 46 District ID Risk (Fraction) 0.02000 0.02000 0.02000 0.02000 0.02000 0.02000 0.02000 0.02000 47 (1-State GE Comp)*(Dist ID Risk) 0.01985 0.01775 0.01988 0.01860 0.00974 0.01449 0.01968 48 (State GE Comp)*(Dist ID Risk) 0.00015 0.00225 0.00012 0.00140 0.01026 0.00551 0.00032 0.02000 49 Sum of All Others in the Row Above 0.01985 0.01775 0.01988 0.01860 0.00974 0.01449 0.01968 50 Weighted ID Risk Ratio 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 51 52 53 Alternate Risk Ratio [Overrepresentation or Underrepresentation] 54 55 District ID Risk (%) 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 2.00000 56 State ID (N) 271 3,128 213 4,319 21,616 9,166 445 39,158 57 All Other ID in State 38,887 36,030 38,945 34,839 17,542 29,992 38,713 58 All Other GE in State 6,034,320 5,395,929 6,042,233 5,654,729 2,960,136 4,405,680 5,982,455 59 Statewide All Others' ID Risk (%) 0.64443 0.66773 0.64455 0.61610 0.59261 0.68076 0.64711 60 Alternate ID Risk Ratio 3.10352 2.99524 3.10296 3.24621 3.37491 2.93790 3.09067 61 113

Attachment I Page I-3 Hypothetical Medium Sized School District - Effect on High Incidence Rate in District: General Education (GE) Enrollment = 10,000; Intellectual Disability (ID) Enrollment = 200 62 Native Asian Pacific Black Hispanic White Multiple Total/Overall 63 64 E-formula Data 65 66 District ID Enrollment (N) 8 30 6 20 64 52 20 200 67 A = District GE Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 68 Std. Error = [A*(100-A)/N] (%) 1.38564 2.52488 1.20623 2.12132 3.29848 3.10161 2.12132 69 70 71 E-formula [Overrepresentation] 72 73 District ID Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 74 Maximum Percent ID Allowed: 75 A + One Standard Error (%) 5.38564 17.52488 4.20623 12.12132 35.29848 29.10161 12.12132 76 A + Two Standard Errors (%) 6.77128 20.04975 5.41247 14.24264 38.59697 32.20322 14.24264 77 A + Three Standard Errors (%) 8.15692 22.57463 6.61870 16.36396 41.89545 35.30484 16.36396 78 A + Four Standard Errors (%) 9.54256 25.09950 7.82494 18.48528 45.19394 38.40645 18.48528 79 80 81 E-formula [Underrepresentation] 82 83 District ID Composition (%) 4.00000 15.00000 3.00000 10.00000 32.00000 26.00000 10.00000 100.00000 84 Minimum Percent ID Needed: 85 A - One Standard Error (%) 2.61436 12.47512 1.79377 7.87868 28.70152 22.89839 7.87868 86 A - Two Standard Errors (%) 1.22872 9.95025 0.58753 5.75736 25.40303 19.79678 5.75736 87 A - Three Standard Errors (%) -0.15692 7.42537-0.61870 3.63604 22.10455 16.69516 3.63604 88 A - Four Standard Errors (%) -1.54256 4.90050-1.82494 1.51472 18.80606 13.59355 1.51472 89 114

About the Author Lalit Roy is a former administrator at the California Department of Education, Special Education Division. He has worked in the area of racial/ethnic disproportionality in special education since the 1970 s. Prior to joining the California Department of Education, he taught mathematics and sciences in two high schools; worked as a consultant in education planning for developing countries under the auspices of the Ford Foundation; and was a visiting scholar at the Program in International Education Finance at the School of Education, University of California at Berkeley. He served as a member of several national advisory groups and initiatives in education, including the Education Information Management Advisory Committee (EIMAC) and Performance Based Data Management Initiative (PBDMI) under the U.S. Department of Education and National Accessible Reading Assessment Project (NARAP), sponsored by the Education Testing Service (ETS). His professional interests include education planning and policy analysis, education finance, quantitative methods, information systems development, and educational research and program evaluation. He holds a Ph.D. in Education Policy, Planning, and Administration from University of California, Berkeley. When not traveling around the world, he spends his time in Sacramento, California with his family and friends. He can be reached at lalitroy@comcast.net. 115