Achievement Plus at Cleveland Junior High School evaluation results

Transcription

1 Achievement Plus at Cleveland Junior High School evaluation results N O V E M B E R

2 Achievement Plus at Cleveland Junior High School evaluation results November 2008 Prepared by: Dan Mueller, Katie Broton, and Edith Gozali-Lee Wilder Research 451 Lexington Parkway North Saint Paul, Minnesota

3 Contents Summary... 1 Introduction... 7 Background on Cleveland... 7 Achievement Plus, a partnership for community schools... 8 Achievement Plus evaluation... 9 Contents of the report Student profile Overall enrollment and demographics Students included in academic results Achievement Plus implementation Standards-based curriculum and instruction Extended Learning Learning Supports Student results Academic achievement Attendance School climate Student behavior References Appendix Achievement Plus evaluation report Wilder Research, November 2008

4 Figures 1. Student profile : Cleveland Junior High School Cleveland students home languages, Percentage of English Language Learners, Changes in Cleveland s student enrollment, to Student profile : Cleveland Junior High School subgroup Cleveland Achievement Plus implementation efforts, , , and Extended Learning attendance indicators, Number of students attending Extended Learning per day range, MCA II results in spring 2007 and participation in Extended Learning in MCA-II reading results, 2006, 2007 and MCA-II math results, 2006, 2007, and Cleveland seventh graders MCA-II results by student characteristics, Cleveland eighth graders MCA-II results by student characteristics, Cleveland MCA-II reading results compared to demographically similar peers by student group, 2006, 2007, and Cleveland math results (MCA-II or MTELL) compared to demographically similar peers by student group, 2006, 2007, and Cleveland students MCA-II growth from 2006 to 2007, and 2007 to 2008, by student group Cleveland attendance for the last three school years Cleveland attendance by demographic characteristics, Cleveland attendance in by MCA-II proficiency level in Overall school rating, School atmosphere rating, Parental/community involvement rating by staff, Student behavior rating by staff, Achievement Plus evaluation report Wilder Research, November 2008

5 Acknowledgments We wish to thank RoseMary Enslin, Lynnell Thiel, and Cindy Torguson of Achievement Plus, and Jill Gebeke, principal of Cleveland Junior High School, for the information they provided in the development of this report. Mark Anton and Louann Graham of Wilder Research contributed to the data analysis and production of this report. Achievement Plus evaluation report Wilder Research, November 2008

6 Summary Achievement Plus is a public-private initiative working to improve the academic achievement of Saint Paul children by developing and strengthening community schools. In , Cleveland Junior High joined Achievement Plus in the initiative s first partnership with a junior high school. Enrollment at Cleveland has been decreasing the past several years. Slightly over 300 students were enrolled in Two-thirds of Cleveland students are English Language Learners, and almost all are racial/ethnic minorities and come from low-income families. Through Achievement Plus partnerships with community agencies, the school offers a variety of academic and social services to support these students and families. Ultimately, the school s partnership with Achievement Plus aims to improve students academic achievement. Achievement Plus Achievement Plus emphasizes the local community s role in supporting its students development. Participants initially included Saint Paul Public Schools, the Amherst H. Wilder Foundation, the City of Saint Paul, and Ramsey County. The State of Minnesota has provided financial support for the initiative, as have many foundations and corporations. Achievement Plus began in the school year in two Saint Paul elementary schools, Dayton s Bluff Achievement Plus Elementary School and Monroe Achievement Plus Community School. (Monroe is no longer an Achievement Plus school.) A third elementary school, John A. Johnson Achievement Plus Elementary School, was added in the school year, and Cleveland Junior High School joined the initiative in The initiative focuses on implementing programs and services in the following three areas: 1. Standards-based curriculum and instruction 2. Extended Learning opportunities for students 3. Learning Supports for students and families Significant instructional changes began in with the selection of the America s Choice model, a standards-based curriculum, for implementation in Achievement Plus classrooms. Achievement Plus has also been instrumental in the implementation of Saint Paul Public Schools own standards-based instructional reform program, the Project for Academic Excellence. Achievement Plus evaluation report 1 Wilder Research, November 2008

7 Evaluation Wilder Research serves as the independent evaluator of Achievement Plus with assistance from the Saint Paul Public Schools Department of Research, Evaluation and Assessment. The evaluation seeks to document Achievement Plus implementation efforts and assess students progress on key indicators of performance at Cleveland. The evaluation uses information from the school, the district, and Achievement Plus records; Achievement Plus staff and the Cleveland principal; school climate surveys conducted by school staff; and data on achievement test results from the district and state. Progress toward meeting the initiative s goal of improving academic achievement is measured using the Minnesota Comprehensive Assessments Series II (MCA-II), state tests administered annually to students in grades 3-8 (reading and math), 10 (reading), and 11 (math). In 2007 and 2008, Minnesota English Language Learner (ELL) students had the option of taking the Mathematics Test for English Language Learners (MTELL) instead of the MCA-II math test. Because of the high proportion of Cleveland students who were ELL, most Cleveland students took the MTELL rather than the MCA-II math test in 2007 and Academic results are only reported for students who were at Cleveland for most of the school year for which results are reported. Achievement Plus implementation Implementation efforts during the initiative s third year at Cleveland, , included the following: Standards-based curriculum and instruction. Staff members participated in professional development opportunities related to a wide variety of topics provided by the district. Achievement Plus had little involvement in this component during Extended Learning. The After School Club, which provides academic and enrichment activities, offered four programming days a week in and The number of days the program was offered increased from two to four days a week during due to the school s failure to make adequate yearly progress (AYP) toward Minnesota state standards as required by the federal No Child Left Behind Act. On average, 62 students attended the program per day in , or about one-fifth of enrolled students. A total of 184 students attended at least one day in , with 72 attending 30 or more days. Those who were not meeting MCA-II standards in reading and math were more likely to attend the program. Learning Supports. Students and families received support services as a result of Achievement Plus. Selected services in included the following: Achievement Plus evaluation report 2 Wilder Research, November 2008

8 Wilder Foundation staff provided week programs for male and female students dealing with emotional or behavioral issues, with an average weekly attendance of 6 female and 6 male students. Wilder also provided a 12-session program called the Hmong Family Program, attended by 15 families. This included a family dinner and discussion groups for parents and students at each session. The Eastside Family Center s family resource room provided assistance through more than 500 contacts with Cleveland families. Reasons for seeking assistance most commonly related to school issues, housing, health and dental needs, clothing, furniture, and the need for computer or phone use. Cleveland families continued to receive medical and dental services at John A. Johnson Achievement Plus Elementary School on-site clinics. Achievement plus hired a consultant to work with African American boys on social-behavioral skills. Achievement Plus staff also contributed to and helped coordinate other services in the school. Student results Academic achievement Generally, academic achievement based on the MCA-II/MTELL results was not as strong at Cleveland in 2008 as it was in Some groups of students, however, had encouraging results in 2008 as described below. Overall results MCA-II results in reading indicated that 19 percent of seventh graders and 28 percent of eighth graders met the standard for proficiency in spring These relatively low percentages are related to the high numbers of English Language Learners attending Cleveland. However, these percentages are somewhat lower than those for the previous two years. For math, MCA-II/MTELL results in spring 2008 showed that 16 percent of seventh graders met the standard for proficiency and 29 percent of eighth graders did so. Compared to the previous two years, the percentage proficient decreased for seventh graders and stayed about the same for eighth graders. Achievement Plus evaluation report 3 Wilder Research, November 2008

9 Comparisons to demographically similar students statewide One way students academic achievement can be viewed is in the context of apples-toapples comparisons to other students with similar characteristics. Generally, students with the characteristics of those at Cleveland low-income, racial/ethnic minorities, and English Language Learners tend to have lower achievement than higher-income, White, and native English speakers. Therefore, analyses comparing Cleveland students to similar students statewide may be more instructive for the Achievement Plus evaluation. Cleveland students were compared to students statewide who were the same as them in grade level, income level (i.e., free/reduced-price lunch status), English Language Learner status, and race/ethnicity. It was determined whether students MCA-II scores were higher, the same as, or lower than the average score for their demographically similar peer group. Results for reading in 2008 indicated that overall nearly half of Cleveland students surpassed the average score compared to their demographically similar peers. This suggests that Cleveland students are performing almost as well as their demographically similar peers statewide in reading. Latino students at Cleveland performed somewhat better than these overall results with slightly over half surpassing the average for their peers statewide in reading while White students had a weaker performance with only about one-third surpassing the average for their peers. Results of these reading comparisons the previous year were slightly more favorable toward Cleveland students. Somewhat stronger results were found for math in 2008, with more than half of Cleveland students surpassing their demographically similar peers statewide. More specifically, percent of Cleveland students across demographic groups surpassed the average math score for their peers statewide (with the exception of White students). Black students in particular have done comparatively well with percent surpassing their peers during the past two years. One-year growth Another way to look at academic achievement is with regard to the progress or growth students make in their reading and math skills while at Cleveland. The percentage of students making expected growth in these subject areas over the past year was examined in relation to the school district s performance targets. In reading, over the one-year period from spring 2007 to spring 2008, about half of the students in the largest demographic groups at Cleveland (low income, English Language Learners, and Asian students) were making expected growth, meeting the district target. Among the smaller demographic groups of students (Latino, Black, White, and Special Education students) percent made expected growth. These results were not as strong as those for the previous year. Achievement Plus evaluation report 4 Wilder Research, November 2008

10 Again, math results were somewhat stronger than those for reading. Over half of Cleveland students in the largest demographic groups were making expected growth (53%-62%) from spring 2007 to spring In the smaller demographic groups percent were making expected growth. Attendance The percentage of students frequently absent (11 or more days) has decreased at Cleveland over the past three years resulting in Cleveland doing better than district targets in this area. Although overall rates of frequent absences are low, some groups of students have relatively high rates. Highlights of attendance rates are as follows: A smaller percentage of Cleveland students were absent 11 or more days in (19%) than in the previous two years (23%, and 29% respectively). A much lower percentage of English Language Learners were frequently absent (11%) than non-english Language Learners (39%) in Compared to the other racial/ethnic groups, Asian students had significantly better attendance, with only 7 percent frequently absent. White students had the highest percentage frequently absent (38%), followed by Black (34%) and Latino (21%) students. School climate A school climate survey administered by the district is completed by students and staff at Cleveland every spring. Both student and staff ratings of school climate have improved over the past four years. This includes overall ratings of the school and ratings of school atmosphere (safety, treatment of students, and support services). In addition, staff ratings of parent and community involvement with the school increased. Behavior Staff ratings of student behavior as part of the school climate survey indicated improvement over the past four years. Staff saw improvements in such areas as student absenteeism, student apathy, and disorderly conduct such as fighting and bullying. They also saw improvements in the effectiveness of school policies and rules regarding student behavior. Student responses from the school climate survey suggested that decreases had occurred in suspensions and detentions, and in students being teased, picked on, or bullied. Similarly, data provided by the principal indicates reductions in behavior incidents (fewer referrals for defiance, fighting, and disruptive behavior) over the past four years. Achievement Plus evaluation report 5 Wilder Research, November 2008

11 Strengths and challenges Results from the school year can be used to inform future planning of Cleveland s Achievement Plus initiative. These results suggest some strengths to build on as well as some challenges to address. Strengths Student behavior. Staff and student perceptions, as well as statistics kept by the school, point to improvement in student behavior over the past four years. These results suggest that a student behavior plan implemented four years ago has had a positive impact. Additional student services provided through Achievement Plus may also have contributed to the improvement. Attendance. Overall the proportion of Cleveland students frequently absent is quite low, lower than the district target for excelling in this area. English Language Learner and Asian students had especially low rates. These results suggest that current attendance-related policies and practices are effective with many Cleveland students. School climate. Ratings by both students and staff also indicated improvement in school climate over the past four years. One of the areas included in these improved ratings is parent and community involvement in the school. Achievement Plus is likely to have contributed to this area through bringing new community partners to Cleveland and through its support of parent involvement activities. Challenges Academic performance. The overall level of academic achievement at Cleveland decreased somewhat in based on the MCA-II tests in reading and math. The percentages of students meeting standards in reading or math in spring 2008 were lower than 30 percent across grades. While the student population served (mostly low-income and English Language Learners) is a challenging one, there is a need to improve student performance. During the first two years of the grant administration, grant funds were used to provide additional academic materials in the classroom and to assist with math coaching. Professional development for staff is now implemented through the school district. However, this is clearly an area that needs more attention. Stronger teacher training in and implementation of the Project for Academic Excellence strategies (disciplinary literacy) may be one area to emphasize. Absenteeism among some students. While Cleveland had an overall low rate of frequent absences among students, several groups had quite high rates. The proportions of non-english Language Learner, White, Black, and Special Education students with frequent absences (11 or more days) ranged from 34 to 41 percent. The attendance of these groups may warrant additional attention. Achievement Plus evaluation report 6 Wilder Research, November 2008

12 Introduction Cleveland Junior High School is the first junior high to participate in Saint Paul s Achievement Plus initiative. Located on the east side of St. Paul, Cleveland is a relatively small school with a student population of 312. Enrollment has declined over the past few years, down from 571 in (Minnesota Department of Education, n.d.). Cleveland s School Comprehensive Improvement Plan (SCIP) describes the school as providing a small school atmosphere with big school opportunity. The average class size is 18 students, according to the SCIP (SPPS SCIP, n.d.). Many Cleveland students are English Language Learners (67% in ), and almost all are eligible for free or reduced-price lunch (94% in ) (Minnesota Department of Education, n.d.). The school offers a variety of academic and social support services for these students through grants and partnerships with community agencies. It is in this context of a small, urban school environment with a diverse student population and foundation of support services and community linkages that Cleveland and Achievement Plus have entered into partnership (SPPS SCIP, n.d.). Background on Cleveland Cleveland s partnership with Achievement Plus builds on an array of school services aimed at supporting students and families. Several were in place before Achievement Plus, and some have been newly added or enhanced. In some cases, Achievement Plus has also added a coordinating role. These services include academic supports, social supports, and a variety of enrichment activities. In direct support of academics, for example, Cleveland offers gifted and talented courses; the Multicultural Excellence Program for students of color; the Get Ready! program focusing on preparation for college; recruitment of volunteers and tutors by an AmeriCorps Promise Fellow; and enrichment activities funded by a federal 21 st Century Community Learning Centers grant. School policies have been developed to support an emphasis on reading and each day students engage in 40 minutes of independent reading (SPPS SCIP, n.d.). Partnerships with community agencies provide social supports for students and families, as well as opportunities for engaging in community service. For example, St. Paul Youth Services staff work with students on behavioral issues, and the St. Paul Police Department delivers Gang Resistance Education and Training (G.R.E.A.T.). The Fresh Force program connects students with community service projects and partners with other community organizations to provide hands-on environmental education Achievement Plus evaluation report 7 Wilder Research, November 2008

13 opportunities. With a family resource room housed in the school building, the Eastside Family Center connects Cleveland families to a variety of community resources (SPPS SCIP, n.d.). Through the Achievement Plus grant, Wilder Foundation staff offer weekly groups for male and female students that provide opportunities to learn social and interpersonal management skills and to build self-esteem. Sports teams, music programs, Math Team, and other activities provide additional enrichment opportunities for students. The school s partnership with Achievement Plus aims to improve students achievement by strengthening and building on these academic and social supports, and by promoting the district s model for standards-based curriculum and instruction. Programs and services supported or enhanced by Achievement Plus are described in the Achievement Plus implementation section of the report. Achievement Plus, a partnership for community schools Achievement Plus is a public-private initiative formed to develop and strengthen community schools in order to improve the academic achievement of Saint Paul children. Initial participants included Saint Paul Public Schools, the Amherst H. Wilder Foundation, the City of Saint Paul, and Ramsey County. The State of Minnesota has provided significant financial support for the initiative, as have many foundations and corporations. The current mission statement of the initiative follows: Achievement Plus community schools are the foundation of a comprehensive urban education reform model that integrates the school community, families, and the resources of public and private organizations to ensure academic achievement for all students. Achievement Plus began in the school year in two Saint Paul elementary schools, Dayton s Bluff Achievement Plus Elementary School and Monroe Achievement Plus Community School. A third elementary school, John A. Johnson Achievement Plus Elementary School, was added in the school year, and Cleveland Junior High School joined the initiative in The schools were chosen based on their location in geographic areas with the following characteristics: high concentrations of children under 16 years of age, diverse family income levels, low student achievement rates, high dropout rates, racially diverse neighborhoods, inadequate recreational facilities, and crowded classrooms. The primary goal of Achievement Plus is to increase student achievement. Other supporting goals include strengthening family and community participation in the Achievement Plus evaluation report 8 Wilder Research, November 2008

14 education of young children; offering parents and community members additional educational options; providing a means for the school district, city, and county to efficiently integrate their resources with those of private and community organizations; and developing a large-scale, comprehensive public education reform initiative with a rigorous evaluation of outcomes. To date, the initiative has implemented a number of programs and services. These programs and services are organized into three areas: 1. Standards-based curriculum and instruction 2. Extended Learning opportunities for students 3. Learning Supports for students and families The focus on standards-based curriculum and instruction began in earnest in the initiative s fourth year ( ) with the decision to implement America s Choice in Achievement Plus schools. America s Choice is a standards-based curriculum reform model developed through the National Center on Education and the Economy. In the past several years, standards-based instructional reform has continued through the Saint Paul Public Schools Project for Academic Excellence. In the area of Extended Learning, after-school programs are offered at Achievement Plus schools. Learning Supports for students and families include family resource rooms or centers and related family programming, as well as health and wellness services (e.g., medical services, dental services, and mental health services). Achievement Plus evaluation The evaluation of Achievement Plus is an important element in the overall initiative. Primary objectives of the evaluation include the following: Document Achievement Plus activities being implemented Assess progress of students on key indicators of performance (e.g., academic achievement, attendance, behavior) Wilder Research serves as the independent evaluator of Achievement Plus, working in collaboration with Saint Paul Public Schools Department of Research, Evaluation and Assessment. At Cleveland, the evaluation draws on information from school, district, and Achievement Plus records; Achievement Plus staff and the Cleveland principal; school climate surveys conducted by school staff; and data on achievement test results from the district and state. Achievement test results presented in this report include data available on district and state websites, as cited in the References section, as well as analyses performed by Wilder Research staff based on data files provided by the school district. Achievement Plus evaluation report 9 Wilder Research, November 2008

15 Contents of the report This report describes implementation and results for the Achievement Plus initiative s third year at Cleveland Junior High School, Activities and results are compared to those for the initiative s previous two year at Cleveland, and The report is divided into three sections: 1) a profile of Cleveland students, 2) a description of Achievement Plus implementation efforts at the school, and 3) an analysis of results. The results section presents findings on student academic achievement, attendance, and behavior, including changes over the past three years ( to ), and sometimes over the past four years. Cleveland s academic results are also compared to those for the Saint Paul district as a whole and the State of Minnesota. Trends in school climate are also presented. The Appendix provides additional information on Cleveland student demographic trends since and school climate survey results. Achievement Plus evaluation report 10 Wilder Research, November 2008

16 Student profile This section presents enrollment and student demographic characteristics for the school year. It also examines differences between Cleveland and the overall district, as well as changes seen at Cleveland in recent years. The section ends with a figure presenting comparable data for a subset of students who form the basis for academic analyses in this report. Because some students attend the school for only a short time, academic results are presented for a more stable subset of the population. Overall enrollment and demographics Cleveland is a relatively small school with high proportions of low-income and ELL students. Looking at data for the school year, Cleveland has an enrollment of 312. Almost all students are low-income, defined here as eligible for free or reducedprice lunch (94%). Two-thirds are English Language Learners (67%). Asian students are the school s largest racial/ethnic group (43%), followed by Black (29%), Latino (21%), White (5%), and American Indian (1%) students. Approximately one in six students receive Special Education services (16%) (Figure 1). 1. Student profile : Cleveland Junior High School Cleveland Saint Paul district Total enrollment K ,050 Race/ethnicity American Indian 1% 2% Asian 43% 30% Black 29% 30% Latino 21% 13% White 5% 25% Students receiving special services Eligible for free or reduced-price lunch 94% 71% English Language Learners 67% 37% Receiving Special Education services 16% 16% Note: Includes all students enrolled on October 1, Source: Minnesota Department of Education, n.d. Achievement Plus evaluation report 11 Wilder Research, November 2008

17 Figure 2 illustrates the diversity of languages spoken at Cleveland. Based on data from , the most common home language among Cleveland students is Hmong (33% of students), followed by English (29%), and Spanish (20%). An additional 19 percent of students have other home languages. 2. Cleveland students home languages, (N=312) Other 19% English 29% Spanish 20% Hmong 33% Note: Includes all students enrolled at Cleveland on October 1, Source: SPPS Data Center, n.d. Differences from the overall district Cleveland s student population differs from that of the district overall in some important ways. These differences may hold important implications for academic performance, and need to be considered in comparing Cleveland students performance with that of other students. Looking at data, one major difference between Cleveland and other schools is Cleveland s exceptionally high rate of ELL students, which far exceeds the overall Saint Paul district and statewide percentages (Figure 3). Another difference is the proportional representation of different racial/ethnic groups. Most notably, Cleveland has a higher proportion of Asian and Latino students and a lower proportion of White students than the Saint Paul district overall. Cleveland s percentage of low-income students is also notably higher than the overall district s (Figure 1). Achievement Plus evaluation report 12 Wilder Research, November 2008

18 3. Percentage of English Language Learners, % 37% 8% Cleveland Saint Paul Public Schools Minnesota Note: Includes all students enrolled on October 1, Source: Minnesota Department of Education, n.d. Changes in recent years Some of Cleveland s noteworthy characteristics have grown more pronounced in recent years. These include its relatively small enrollment, racial/ethnic diversity, and representation of ELL and low-income students. As shown in Figure 4, the school s enrollment has been decreasing, down from 498 in to 312 in Changes in Cleveland s student enrollment, to Note: Source: Includes all students enrolled at Cleveland on October 1 of each year. SPPS Data Center, n.d. Achievement Plus evaluation report 13 Wilder Research, November 2008

19 During the same time period, the percentage of students receiving English-language services increased from 53 to 67 percent. The proportion of students who are racial/ethnic minorities and the proportion who are low-income also showed increases during this time. In , 11 percent of the school s students were White, compared to only 5 percent in Similarly, the percentage of low-income students increased from 87 to 94 percent. Figure A1 and A2 in the Appendix show demographic changes over the past five years. Students included in academic results Academic results presented in the following section are based on a somewhat smaller subset of students, depicted in Figure 5. These students were at Cleveland on October 1, 2007 and also when they were tested with the Minnesota Comprehensive Assessments Series II (MCA-II) in the spring. Therefore, these students represent a more stable population than the full student body depicted in Figure 1. The two groups are similar in their demographic characteristics, however. While almost all of the students in the subset attended Cleveland from fall to spring, a few may have left and come back during the year. Such students are included in some academic analyses, but are excluded from the analyses of student change or growth in MCA-II scores. Students who enrolled at Cleveland later in the year for the first time, and those who left before the spring and did not return, are excluded from all the academic analyses. Achievement Plus evaluation report 14 Wilder Research, November 2008

20 5. Student profile : Cleveland Junior High School subgroup a Cleveland N % Total number of students in subgroup 270 Grade Seventh grade % Eighth grade % Race/ethnicity American Indian 2 1% Asian % Black 71 26% Latino 58 21% White 13 5% Gender Female % Male % Students receiving special services Eligible for free or reduced-price lunch % English Language Learners % Receiving Special Education services 42 16% a Includes only students who were enrolled at Cleveland on October 1, 2007, and who took at least one portion of the MCA-II (or MTELL) at Cleveland in spring Source: Saint Paul Public Schools, analysis conducted by Wilder Research. Achievement Plus evaluation report 15 Wilder Research, November 2008

21 Achievement Plus implementation This section describes Achievement Plus implementation efforts at Cleveland during Activities are summarized for the three main components of the initiative, including standards-based curriculum and instruction, Extended Learning opportunities for students, and Learning Supports for students and families. Figure 6 below summarizes implementation efforts in the first three years of the initiative, , , and Cleveland Achievement Plus implementation efforts, , , and Standards-based curriculum and instruction Extended Learning staff participated in a total of 22 professional development sessions After School Club offered 2 programming days/week About 100 students attended per day All teachers participated in at least 2 learning walks Teachers could also participate in 1 or 2 districtwide learning walks 13 staff participated in approximately 200 hours of professional development After School Club offered 4 programming days/week (2 additional days as a result of the school not making adequate yearly progress toward Minnesota state standards) About students attended per day Students attending provided community service (see Learning Supports below) Staff participated in professional development opportunities related to a variety of subjects* After School Club offered 4 programming days/week Achievement Plus funded two people to provide Hmong culture and literacy programming Hmong American Partnership provided six tutors to work individually with students 2 days/week On average, 62 students attended per day Students attending provided community service (see Learning Supports below) * Achievement Plus did not fund the professional development opportunities. Achievement Plus evaluation report 16 Wilder Research, November 2008

22 6. Cleveland Achievement Plus implementation efforts, , , and (continued) Learning Supports Wilder offered mental health groups beginning in spring; 25 students participated Eastside Family Center (ESFC) family resource room opened in mid-may; 51 people visited to learn about its services and 4 families received assistance Before opening of Cleveland site, families were referred to Johnson ESFC Families accessed medical services at Johnson clinic Wilder offered weekly programs for female (28 weeks) and male (26 weeks) students; average weekly attendance was 11 and 7 students, respectively Wilder provided 12-session Strengthening Multicultural Families parenting program; 9 families attended ESFC family resource room provided assistance through more than 200 contacts with Cleveland families Families accessed medical and dental services at Johnson clinics Achievement Plus staff contributed to and helped coordinate a variety of family involvement activities and evening parent events An AmeriCorps Promise Fellow funded through the Achievement Plus grant organized community and volunteer activities and student service learning opportunities; 41 adult volunteers and 18 high school student volunteers tutored 213 students; Cleveland students provided more than 1,000 hours of community service Wilder offered weekly programs for male and female students for 28 and 29 weeks, respectively; average weekly attendance was 6 male and 6 female students Wilder provided 12-session Hmong Family Program; 15 families attended ESFC family resource room provided assistance through more than 500 contacts with Cleveland families Families accessed medical and dental services at Johnson clinics Achievement Plus staff contributed to and helped coordinate a variety of family involvement activities and evening parent events Achievement Plus hired a consultant to work with 30 African American boys on social and leadership skills An AmeriCorps Promise Fellow funded through the Achievement Plus grant organized community and volunteer activities and student service learning opportunities; 333 tutoring sessions provided by 96 adult volunteers; Cleveland students provided more than 900 hours of community service Achievement Plus evaluation report 17 Wilder Research, November 2008

23 Standards-based curriculum and instruction Achievement Plus supports implementation of Saint Paul Public Schools Project for Academic Excellence, the district s standards-based curriculum reform model. As part of the model, teachers are trained in standards-based instruction. In , staff members participated in professional development opportunities provided by the district on a wide range of topics. Examples of topics include how to effectively integrate technology in a variety of subjects, professional learning communities, disciplinary literacy, and online school assessments. Extended Learning The Extended Learning program, referred to as the After School Club, is designed to primarily serve children who are below grade level in reading or math, although all students are eligible to enroll. During the school year, the After School Club offered four programming days a week. The two mandatory programming days were divided into either Area Learning Center programming for students seeking general improvement or Credit Recovery for students who were failing or behind in credits. The Area Learning Center programming consisted of one hour of math and reading and one hour of enrichment activities. Examples of enrichment activities include book or craft club; learning to cook, knit or dance; participating in exercise programs; service projects; and Lego robotics team. Credit Recovery students spent the full two hours working on the subject they were behind in. The two optional days consisted of Supplemental Education Services focusing on math and reading or Credit Recovery. The Promise Fellow, who was funded through the Achievement Plus grant, assisted with a variety of functions in the after school program including helping students provide more than 900 hours of service to the community. Achievement Plus also partnered with the Hmong American Partnership to provide six tutors to work with students after school two days per week for a total of 199 hours. Finally, Achievement Plus funded two people to provide Hmong culture and literacy programming. Transportation and a snack were provided daily by the school with support from the 21 st Century Community Learning Centers grant. During the school year, the Extended Learning program was offered on 90 days. A total of 184 students participated in one or more days. On average, about one in five students (62 of 312 enrolled students) attended Extended Learning each day (Figure 7). The majority of students attended 29 or fewer days of Extended Learning with only 72 students attending 30 or more days (Figure 8). Achievement Plus evaluation report 18 Wilder Research, November 2008

24 7. Extended Learning attendance indicators, Attendance indicator Extended Learning Day Total number of days offered 90 Number of students attending at least one day 184 Average (mean) number of students attending each day 62 Range in number of students attending each day Source: Saint Paul Public Schools, analysis conducted by Wilder Research. 8. Number of students attending Extended Learning per day range, Students Days Number Percent % % % % Total % Source: Saint Paul Public Schools, analysis conducted by Wilder Research. While all students are welcome to attend, the Extended Learning program is designed to primarily serve children who are below grade level in reading or math. Of those students who did not meet or partially met MCA II reading standards, 26 percent attended 30 or more days compared to 11 percent of those who met or exceeded standards. Similarly, of those students who did not meet or partially met MCA II math standards, 28 percent attended 30 or more days compared to 15 percent of those that met or exceeded standards (Figure 9). In sum, those who did not meet or partially met the standards were more likely to participate in Extended Learning and to attend more days. Achievement Plus evaluation report 19 Wilder Research, November 2008

25 9. MCA II results in spring 2007 and participation in Extended Learning in Subject MCA II results in spring 2007 Reading Math Does not meet or partially meets standards N Number of days attended Extended Learning None 1-29 days 30 or more days Total % 34% 26% 100% Meets or exceeds standards 44 61% 27% 11% 100% Does not meet or partially meets standards % 37% 28% 100% Meets or exceeds standards 67 64% 21% 15% 100% Source: Saint Paul Public Schools, analysis conducted by Wilder Research. Learning Supports To support students and families at Cleveland, Achievement Plus provides mental and physical health services and a family resource room. Overall, the number of contacts with Cleveland families in the resource room was higher than the previous years. Following are descriptions of specific services in the area of Learning Supports. Wilder Foundation services Wilder Foundation Violence Prevention and Intervention Services staff worked with groups of students on issues such as social skills, anger management, and depression. In , Wilder staff offered the Cleveland Girls program for female students (formerly known as Voices & Choices) and the Young Males program for male students. The Young Males met once a week for 28 weeks while the Cleveland Girls met for 29 weeks. Looking at attendance, from two to eight students attended Cleveland Girls each week, with an average weekly attendance of six students. Attendance at Young Males program ranged from two to nine students each week, with an average weekly attendance of six students. Through the Achievement Plus grant, the Wilder Foundation also provided a program called the Hmong Family Program in This 12-session program was attended by 15 families (15 adults and 16 students) and included a family dinner and discussion groups for parents and students at each session. Achievement Plus evaluation report 20 Wilder Research, November 2008

26 Eastside Family Center services As part of Achievement Plus, the Eastside Family Center operates a family resource room inside the Cleveland building. The resource room provides assistance to families with housing, clothing, furniture, health and dental referrals, school registration and other school-related issues, and the use of a computer and telephone, for example. Assistance was provided through more than 554 contacts with Cleveland families in Medical and dental services In , Cleveland families continued to receive medical and dental services at John A. Johnson Achievement Plus Elementary School on-site clinics. Services are provided by Achievement Plus partners, including St. Mary s Clinic and Children s Dental Services. African American groups At the request of teachers and administrators, Achievement Plus hired a consultant who works with African American boys on social and leadership skills. Most students had discipline issues and were frequently sent to the office for behavioral problems. Thirty boys participated in weekly group sessions and bi-weekly check-in sessions. Collaboration with and coordination of other school services Achievement Plus staff also contribute to and help coordinate other services in the school. For example, Achievement Plus staff participate in school-wide family involvement activities such as open houses, conferences, family events, site council, and the 21 st Century Community Learning Center after-school program. During the merging phase between Cleveland and Farnsworth Aerospace in , Achievement Plus staff participated in a school committee facilitating the transition process. The Achievement Plus grant has funded an AmeriCorps Promise Fellow who organizes community and volunteer activities. In , 333 student tutoring sessions were provided by adult volunteers. The total number of hours provided by these volunteers was 773 hours. Achievement Plus evaluation report 21 Wilder Research, November 2008

27 Student results This section presents results for Cleveland students in the areas of academic achievement, attendance, school climate and behavior. Results are presented for the initiative s third year, , with comparisons to the previous 1-2 years in many cases. The Achievement Plus evaluation considers to be Cleveland s baseline year because components of the initiative began being implemented during, and in some cases late, in that year and continued to be implemented in and This section compares results over 2-3 years to determine whether student outcomes are changing at Cleveland. It should be noted that some of the results reported here also reflect measures followed as part of the district s strategic plan. To assess progress toward district goals, the Strategic Plan for Continued Excellence established measurable outcomes as well as performance targets used to track progress toward those outcomes. Charts on the district s website provide results for targets addressing student achievement and attendance. Where applicable, this section notes that results presented here also reflect measures in the district s strategic plan (SPPS Data Center, n.d.). Academic achievement The Achievement Plus evaluation uses results from the state-mandated Minnesota Comprehensive Assessments Series II (MCA-II) to measure students academic achievement in reading and mathematics. The MCA-II measures student progress toward state academic standards and is administered to students in grades 3-8, 10, and 11. The MCA-II has been administered to students statewide for three years now, in the spring of 2006, 2007, and Therefore, it is possible to compare scores over this period. Sources of MCA-II results used in this report include the following: Minnesota Department of Education web site (see References) Saint Paul Public Schools Data Center web site (see References) Wilder Research analyses of MCA-II data from files received from Saint Paul Public Schools Achievement Plus evaluation report 22 Wilder Research, November 2008

28 Interpreting MCA-II results The following descriptions provide information important for interpreting the academic results presented in this section. Results categories MCA-II results are categorized into four levels with higher levels indicating higher achievement. Students scoring at Level 1 are not meeting the standards, students scoring at Level 2 are partially meeting the standards, students scoring at Level 3 are meeting the standards, and students scoring at Level 4 are exceeding the standards. Students are considered to be proficient if they score at Level 3 or 4 (i.e., meeting or exceeding standards). Students included in academic results Results reported here include only students who were in the school on October 1 and also when they were tested in the spring (Figure 5). These students are similar to Cleveland s overall student population in their enrollment and demographic characteristics, but represent a more stable group. Almost all of these students stayed in the school from fall to spring, although a few students may have left and come back to the school. As described in the previous section, students who left and came back are included in some analyses here but excluded from the analyses of student change or growth in MCA-II scores. Students who enrolled in the school later in the year for the first time, and those who left before the spring and did not return, are excluded from all the academic analyses. Changes in ELL testing In interpreting results, it is important to note that changes from 2006 to 2007 and 2008 could in part reflect changes in the testing of ELL students. Minnesota ELL students had the option of taking the Mathematics Test for English Language Learners (MTELL) instead of the MCA-II math test in 2007 and The MTELL was not an option for ELL students in The MTELL includes simplified language and other accommodations for ELL students, but is designed to be as rigorous and measure the same standards as the MCA-II math test (Minnesota Department of Education, 2007). The Saint Paul district made the decision to test almost all ELL students with the MTELL in 2007 and Factoring Cleveland s student characteristics This section begins by presenting information on Cleveland s overall test results in 2006, 2007, and 2008 compared to the district s and state s, and on the differences and similarities in Cleveland s results by student characteristics. However, it is important to note that generally, students with the characteristics of those at Cleveland low-income, Achievement Plus evaluation report 23 Wilder Research, November 2008