EVALUATING THE EFFECTIVENESS OF EDUCATIONAL BLOCK GRANTS TO ORPHANS AND VULNERABLE CHILDREN

Transcription

1 Research Report Boston University OVC CARE Project EVALUATING THE EFFECTIVENESS OF EDUCATIONAL BLOCK GRANTS TO ORPHANS AND VULNERABLE CHILDREN September 2011 Center for Global Health and Development Boston University School of Public Health & Boston University School of Education Boston, Massachusetts, USA The USAID Project SEARCH, Orphans and Vulnerable Children Comprehensive Action Research (OVC CARE) Task Order, is funded by the U.S. Agency for International Development under Contract No. GHH I , beginning August 1, OVC CARE Task Order is implemented by Boston University.

2 EVALUATING THE EFFECTIVENESS OF EDUCATIONAL BLOCK GRANTS TO ORPHANS AND VULNERABLE CHILDREN SEPTEMBER 2011 Funding Source: PEPFAR USAID IQC No. GHH I Task Order No. 01 Principal Investigator: Co investigators (BU): Malcolm Bryant, MBBS, MPH Center for Global Health and Development Mary Shann, PhD Boston University School of Education Bram Brooks, MPH Center for Global Health and Development Co investigators (Uganda): Paul Bukuluki, PhD Makerere University Denis Muhangi, PhD Makerere University Co investigators (Tanzania): Joe Lugalla, PhD Centre for Strategic Research and Development (CESTRE) Gideon Kwesigabo, MD Centre for Strategic Research and Development (CESTRE) Contact information: Malcolm Bryant Center for Global Health and Development, Boston University 801 Massachusetts Ave, Crosstown Center 3 rd Floor, Boston, MA 02118, USA [email protected]

3 Table of Contents LIST OF ACRONYMS... 1 EXECUTIVE SUMMARY... 2 INTRODUCTION AND OVERVIEW BACKGROUND AND RATIONALE METHODOLOGY AND STUDY SITES SAMPLE DESIGN AND SIZE STUDY PROCEDURES RESULTS DISCUSSION REFERENCES APPENDECES INSTRUMENT: QUANT 1(STUDENT COMPARISON) INSTRUMENT: QUANT 2 (SCHOOL COMPARISON) INSTRUMENT: FGD (COMMUNITY REPRESENTATIVES) INSTRUMENT: FGD (OVC STUDENTS) INSTRUMENT: SSI (NGOS) INSTRUMENT: SSI (HEADMASTER / DEPUTY HEADMASTER) CAREGIVER CONSENT FORM CHILD ASSENT FORM OTHER CONSENT FORM

4 List of Acronyms AIDS AVSI BG BU CBO CGHD COPE CRS FBO ICOBI IRB NGO OVC PEPFAR S SSN USAID USE Acquired Immune Deficiency Syndrome Association of Volunteers in International Service Block Grant Boston University Community Based Organization Center for Global Health and Development Community Based Orphan Care, Protection & Empowerment Catholic Relief Service Faith Based Organization Integrated Community Based Initiatives Institutional Review Board Non Governmental Organization Orphans and Vulnerable Children President s Emergency Plan for AIDS Relief Scholarship Program Social Safety Net United States Agency for International Development Universal Secondary Education 1

5 EXECUTIVE SUMMARY Introduction Boston University, in collaboration with Makerere University in Uganda and the Center for Strategic Research and Development (CESTRE) in Tanzania, implemented a mixed methods study to determine the effectiveness and cost of different approaches to providing educational support to OVC at secondary schools. The study had four objectives: to compare block grants (BG) versus individual scholarship (S) approaches; to explore the differences in effectiveness among various block grant approaches; to calculate the costs per child of each approach, and to measure the impact of each approach on educational outcomes for all children in the targeted schools. Four NGOs funded by PEPFAR through USAID Track 1 funds were studied. Two used block grants as a means to support OVC in secondary school (Africare and ICOBI), while two used scholarship approaches (AVSI and CRS). One NGO providing block grants operated in both countries (Africare). Each of the five NGO programs studied was distinct in its characteristics, administration, and adaptations to the local environment. The study was a retrospective cohort study and used a mixed methods design adopting both qualitative and quantitative approaches to research. The qualitative components drew on interviews, focus groups, observational techniques, and document analysis to learn about the intervention used by each NGO and its context from the point of view of the primary recipients and other stakeholders who were knowledgeable about the OVC block grants and scholarship programs. The quantitative components of the study took three forms. The first (QUANT1) addressed how well the OVC fared under different approaches, with a specific focus on comparing outcomes for those in block grant and scholarship programs over time as compared to their OVC peers and non OVC peers who did not receive any targeted benefit. The second quantitative component (QUANT2) addressed how individual schools that have received financial assistance through block grants have fared over time as compared with a matched set of schools that did not receive any form of block grants. The third quantitative component of the study (QUANT3) examined the relative cost effectiveness of different models of disbursing the funding to schools to aid OVC by examining direct costs, indirect costs, and imputed costs of each NGO program. Results A total of 5,738 students were recruited to the study in both countries. Table 1 outlines the overall characteristics of the sample size. Gender characteristics of the students were virtually identical in both countries with 52.5% male in Tanzania and 52.6% male in Uganda. Students at every grade level in Tanzania were older than their counterparts in Uganda, a statistic that reflects the difference in school 2

6 enrolment age in the two countries. In Uganda children begin primary school at 6 years of age, whereas in Tanzania they begin primary school at 7 years of age. Table 1: Study Sample Characteristics Tanzania (n = 1930) Uganda (n = 3808) Mean Student Age (years) F1/S1 Student (Std Dev) F2/S2 Student (Std Dev ) F3/S3 Student (Std Dev ) F4/S4 Student (Std Dev ) Gender Male (%) Female (%) Student Status OVC Ever Supported (%) OVC Never Supported (%) Non OVC (%) Student NGO Support Africare (%) ICOBI (%) AVSI (%) CRS (%) None (%) Availability of Student Records 2006 (%) 2007 (%) 2008 (%) 2009 (%)* 15.8 (1.4) 16.8 (1.4) 17.6 (1.4) 18.3 (1.2) 1013 (52.5%) 917 (47.5%) 498 (25.8%) 721 (37.4%) 711 (36.8%) 243 (12.6%) 255 (13.2%) 1432 (74.2%) 232 (12.0%) 747 (38.7%) 1464 (75.9%) 1723 (89.3%) *Student records for ICOBI (Uganda) were captured up until (2.0) 15.6 (1.9) 16.6 (1.9) 17.5 (1.9) 2003 (52.6%) 1805 (47.4%) 777 (20.4%) 1531 (40.2%) 1500 (39.4%) 256 (6.7%) 257 (6.7%) 264 (6.9%) 3031 (79.6%) 662 (17.4%) 1130 (29.7%) 2670 (70.12%) 2605 (68.4%) Table 2 identifies the specific sub populations in the sample by country and type of approach: Table 2: Sample Characteristics by Intervention Uganda Total number of students 3,808 Block Grant (Africare) supported OVC 256 Block Grant (ICOBI) supported OVC 257 Scholarship (AVSI) supported OVC 264 Control OVC (never supported) 1,531 Non OVC 1,500 Tanzania Total number of students 1,930 Block Grant (Africare) supported OVC 243 Scholarship (CRS) supported OVC 255 Control OVC (never supported) 721 Non OVC 711 3

7 QUANT1 Mean absenteeism rate was calculated by averaging absenteeism rates between the different groups. The results, presented in Table 3, show a significant difference overall between groups, with ever supported OVC (5.1%) having much better attendance rates than never supported OVC (6.4%). Also, never supported OVC (6.4%) had much worse attendance rates than non OVC (5.5%). However, we found marked differences between Tanzania and Uganda. In Tanzania, children who were ever supported (6.5%) demonstrated worse attendance rates than their never supported counterparts (5.7%) and no significant difference as compared to non OVC. On the other hand, in Uganda, ever supported OVC (3.6%) showed much better attendance rates than either their never supported OVC (6.8%) or non OVC (6.1%) counterparts. Table 3: Mean Student Absenteeism Rate, Stratified by Country OVC Ever OVC Never Supported Supported Mean Student Absenteeism Rate ( ) Tanzania (n=1783) Uganda (n=2792) Total (n=4575) Non OVC 6.5%* 3.6%* 5.1%* 5.7%* 6.8%* 6.4%* + 6.1% 5.1%* 5.5% + Paired t test compares the means of two groups of observations: OVC Ever Supported vs. OVC Never Supported; OVC Never Supported vs. Non OVC; OVC Ever Supported vs. Non OVC. Paired significance is noted accordingly. * Student status significantly different at p < Student status significantly different at p < 0.05 When we examined absenteeism results across NGOs to compare block grants and scholarships, we found some interesting differences (Table 4). Firstly, in Tanzania there is no significant difference in absenteeism in the aggregate or when broken down by gender between block grant or scholarships, although it is interesting to note that absenteeism among male OVC at scholarship schools (supported or not) is much higher than females. Secondly, in Uganda, all three NGOs show dramatic reductions in absenteeism over both comparison groups, however children supported through Africare s block grant approach did not fare statistically better than OVC who attended the same schools and were not supported. 4

8 Table 4: Mean Student Absenteeism Rate, Stratified by NGO Support and Gender OVC Ever Supported OVC Never Supported Non OVC Mean Absenteeism Rate ( ) Africare (BG) UG (n=721) Male (n=404) Female (n=317) ICOBI (BG) UG (n=1010) Male (n=515) Female (n=495) AVSI (S) UG (n=1059) Male (n=542) Female (n=517) Africare (BG) TZ (n=1152) Male (n=614) Female (538) CRS (S) TZ (n=631) Male (n=338) Female (n= %* 2.5%* + 4.0% 3.7%* 4.0%* 3.6%* 3.6%* 4.3%* 2.7%* + 6.2% 6.0% 6.3% 7.0% 8.1% 5.6% 5.6% 5.7%* 5.5% 8.4%* + 8.9%* + 7.9%* + 6.1%* 7.2%* + 5.2%* 5.8% 5.6% 6.0% 5.4% 6.2% 4.4% 6.1%* 5.8% + 6.4% 4.5% + 4.9% + 4.1% + 5.0%* 5.3% + 4.7% + 6.3% 6.5% 6.0% 5.7% 5.7% 5.7% Paired t test compares the means of two groups of observations: OVC Ever Supported vs. OVC Never Supported; OVC Never Supported vs. Non OVC; OVC Ever Supported vs. Non OVC. Paired significance is noted accordingly. * Student status significantly different at p < Student status significantly different at p < 0.05 Drop out rates were significantly different by country and showed considerable variation among NGO and approaches to supporting students. It is striking in Table 5 that the drop out rates for supported OVC are significantly worse for Africare supported schools in both countries. Further investigation is required to understand this, but the same pattern is not reflected in the other block grant program, so that it is most likely an effect of Africare s approach rather than the block grant approach as a whole. A multivariate model was created to predict student drop out (Table 6). Secondary school students in Uganda are eleven times more like to drop out compared to students in Tanzania. This is not surprising since dropout rates in Uganda are considerably higher than dropout rates in Tanzania. When analyzing the effect of gender, there seems to be no difference in dropout rates between male and female students. Finally, when comparing the impact of educational support, we see that never supported OVC students are almost twice as likely to drop out when compared to non OVC students OR 1.83 (95% CI ). The adjusted odds ratio is slightly lower for ever supported OVC students compared to non OVC students OR 1.46 (95% CI ) suggesting that the support they receive may facilitate a slight reduction in the likelihood of dropping out of school. 5

9 Table 5: Drop Out Rate by Student Status and Approach Proportion (%) Odds Ratio (95% CI) NGO Support Africare UG Block Grant Non OVC [Ref.] 48/491 (9.8%) 1.00 OVC Never Supported 88/493 (17.9%) 2.01 ( )* OVC Ever Supported 39/247 (15.8%) 1.73 ( )* ICOBI UG Block Grant Non OVC [Ref.] 43/475 (9.1%) 1.00 OVC Never Supported 98/517 (19.0%) 2.35 ( )* OVC Ever Supported 31/255 (12.2%) 1.39 ( ) AVSI UG Scholarship Non OVC [Ref.] 35/508 (6.9%) 1.00 OVC Never Supported 36/501 (7.2%) 1.05 ( ) OVC Ever Supported 14/264 (5.3%) 0.76 ( ) Africare TZ Block Grant Non OVC [Ref.] 3/466 (0.6%) 1.00 OVC Never Supported 4/469 (0.9%) 1.33 ( ) OVC Ever Supported 7/242 (2.9%) 4.60 ( )* CRS TZ Scholarship Non OVC [Ref.] 1/244 (0.4%) 1.00 OVC Never Supported 3/250 (1.2%) 2.95 ( ) OVC Ever Supported 5/524 (2.0%) 2.34 ( ) * Student status significantly different at p < 0.05 Table 6: Multivariate Analysis to Predict Drop Out (N=5679) Adjusted Odds Ratio Ɨ (95% CI) Country Tanzania [Ref.] 1.00 Uganda ( )* Gender Male [Ref.] 1.00 Female ( ) Student Status Non OVC [Ref.] 1.00 Never Supported OVC 1.83 ( )* Ever Supported OVC 1.46 ( )* Ɨ Adjusted odds ratio by logistic regression analysis * Significantly different at p <

10 A second multivariate model looked at predictors of passing the national Form IV exams (Table 7). The observation that students in Uganda are eleven times more likely to pass their From IV exams compared to their Tanzanian counterparts is largely an artifact of several things. Firstly, the two countries use different assessment criteria to determine pass/fail; secondly, the ministry of education advised NGOs to support the schools with the greatest problems and poorest performance; finally, because of the later start of schooling in Tanzania, large numbers of students aged out of the program, leaving a very small proportion (less than 10%) of students taking Form IV exams. However the most plausible reason behind the difference in passing rates on Form IV exams between the two countries is rooted in the difference in language policy. In both countries, Form IV exams are given in English. In Uganda where many tribal languages are spoken and no one language dominates, English is widely used and is the language of instruction throughout elementary and secondary schools. In Tanzania, Swahili has been adopted as the national language, and it is the language of instruction in elementary schools. However, in secondary school in Tanzania, there is an abrupt change to the use of English as the language of instruction. Students take their final Form IV examinations in English, a language they have only used for four years. In assessing gender as a predictor of passing From IV exams, stark differences can be seen between male and females students. Girls are half as likely to pass their Form IV exams compared to boys OR 0.55 (95% CI ). The impact of educational support can be seen as having a positive effect on passing the Form IV exams. Whereas never supported OVC are 20% less likely to pass their exam OR 0.78 (95% CI ), there is no statistical significance between ever supported OVC and non OVC students. This finding suggests that the educational support received by OVC students puts them almost on the same passing rate compared to non OVC students. Table 7: Multivariate Analysis to Predict Passing of National Form IV Examination (N=1752) Adjusted Odds Ratio Ɨ (95% CI) Country Tanzania [Ref.] 1.00 Uganda ( )* Gender Male [Ref.] 1.00 Female 0.55 ( )* Student Status Non OVC [Ref.] 1.00 Never Supported OVC 0.78 ( )* Ever Supported OVC 1.10 ( ) Ɨ Adjusted odds ratio by logistic regression analysis * Significantly different at p < 0.05 There were indications that children in block grant supported programs performed better than scholarship schools, but none of the differences were significant at the alpha 0.05 level. 7

11 Table 8 Table 10 explore the Form II examination results (Tanzania) and the Form II and Form IV examination results (Uganda and Tanzania) by subject stratified by gender. With the exception of Form IV results from Tanzania, these tables emphasize the gender discrepancies and confirm that they exist for every subject and exist between the two countries. Table 8: Form II National Examination Scores between Male and Female Students, Tanzania Female Male Mean Exam Scores Ɨ ( ) English (n=1340) Civics (n=1341) Geography (n=613) Math (n=1341) Physics (n=1275) Chemistry (n=1294) Biology (n=613) Kiswahili (n=1341) 26.7 (13.5) 30.2 (12.4)* 44.8 (14.6)* 14.7 (13.9)* 33.7 (14.1) 32.6 (13.7)* 35.0 (12.5)* 46.9 (19.4) 28.1 (16.1) 33.0 (14.2)* 47.8 (16.3)* 17.9 (17.1)* 34.8 (16.7) 34.3 (15.3)* 38.9 (14.8)* 47.0 (17.1) * Student status significantly different at p < 0.05 Ɨ Exam scores are out of 100 points (higher points indicate better exam performance) Note: Form II National Examinations are not conducted in Uganda Table 9: Form IV National Examination Scores between Male and Female Students, Tanzania Female Male Mean Exam Scores Ɨ ( ) English (n=152) Civics (n=152) Math (n=153) Physics (n=45) Chemistry (n=45) Kiswahili (n=151) 4.2 (0.8) 4.2 (0.8) 4.8 (0.5) 4.2 (0.8) 4.3 (0.7) 4.2 (0.8) 4.3 (0.8) 4.2 (0.8) 4.6 (0.9) 4.2 (0.8) 4.1 (0.9) 4.2 (0.7) * Student status significantly different at p < 0.05 Ɨ Exam scores are out of 5 points: 1=A 2=B 3=C 4=D 5=F (lower points indicate better exam performance) Table 10: Form IV National Examination Scores between Male and Female Students, Uganda Female Male Mean Exam Scores Ɨ ( ) English (n=1667) Geography (n=841) Math (n=1667) Physics (n=1664) Chemistry (n=1661) Biology (n=838) 6.0 (1.8) 6.0 (1.6)* 7.1 (1.7)* 8.1 (1.5)* 8.3 (1.3)* 7.4 (1.6)* 6.1 (1.7) 5.5 (1.7)* 6.6 (1.9)* 7.5 (1.7)* 7.9 (1.6)* 6.8 (1.8)* * Student status significantly different at p < 0.05 Ɨ Exam scores are out of 9 points: 1=D1 2=D2 3=C3 4=C4 5=C5 6=C6 7=P7 8=P8 9=F (lower points indicate better exam performance) 8

12 QUANT2 This part of the study was conducted in response to the repeated observation by school teachers, head teachers, and NGO staff, that the improvements that block grants brought to the school improved the quality of the school grounds, increased the number of class rooms, brought physical improvements such as water, electricity, latrines, and also purchased essential laboratory and other equipment. It was frequently observed that these changes had improved the standing of the school, attracted new students, and improved overall school performance. We hoped to be able to observe if the interventions had made any noticeable difference to the overall functioning of the school. In order to study whether these changes had occurred, observations were taken for four years prior to when the interventions began. Because of the small sample size and large standard deviations, no statistical significance tests were conducted, and only trends can be observed. Anecdotal stories of increases in enrolment because of block grants are not born out by the data presented in QUANT2 and do not show variance with the national trends. Increases noted by staff at assisted schools, are therefore more likely to be as a result of pre existing trends augmented by Universal Secondary Education (USE). Overall, with the exception of AVSI supported schools, there was a steady rise in student to teacher ratios, with the ratios noticeably higher for Africare and ICOBI. The similarities between Africare and ICOBI are not surprising as they both serve rural areas in the Southwest part of the country. The National data represent both urban and rural schools, but the majority of schools in Uganda are still rural, thus trends are more likely to resemble those seen with Africa and ICOBI. In addition to the rise in student to teacher ratios, national examination scores fell dramatically over the observation period in national schools, once again being paralleled in Africare and ICOBI schools. Observations from Western settings have demonstrated a direct relationship between student to teacher ratios and the quality of education. We speculate that the increasing ratios were at least in part, responsible for the decreasing exam scores. Most importantly, block grants did not seem to have influenced this downward trend line. QUANT3 Each of the NGOs kept financial data in a different format, which made analysis of the cost data complex, and impossible to provide identical categorization of financial expenditures and costs for each organization. There was wide variation in the total number of children served by each program, and even pinning this number down was difficult because of the PEPFAR reporting requirements which call for both direct and indirect support to be noted. Each NGO had a slightly different way of calculating the percentage of resources that go to direct vs. indirectly supported OVC. 9

13 Table 11 shows the comparative costs per child of block grants vs. scholarships. Costs of scholarship programs are significantly and consistently higher than the block grant programs in both countries. Table 11: Comparative Costs of Block Grants vs. Scholarships Uganda Tanzania Block Grant average cost per child per year $324 $293 Scholarship average cost per child per year $412 $356 Difference between approaches per child per year $88 $63 When examined by NGO (Table 12) there is considerable variation between NGOs with some obvious inefficiencies being present. One means of reducing the inefficiencies would be to increase the number of children served as this would appear to gain significant economies of scale. However, even with large numbers of children, 40% of the total expenditure per child is going to administration costs. Table 12: Breakdown of Program Costs by NGO Africare TZ USD (% of total) CRS TZ USD (% of total) Africare UG USD (% of total) AVSI UG USD (% of total) ICOBI UG USD (% of total) Total children 1,150 1, ,352 3,108 served Direct Benefit $109 (37%) $130 (37%) $127 (37%) $199 (61%) $184 (60%) to Child Program costs $127 (43%) $140 (39%) $150 (44%) $87 (27%) $76 (25%) Indirect costs $56 (19%) $86 (24%) $66 (19%) $42 (13%) $45 (15% and donations Total $293 (100%) $356 (100%) $343 (100%) $412 (100%) $305 (100%) Finally, Table 13 compares the cost effectiveness of the two approaches. With the exception of the variation in drop out rates there is little difference between the approaches in general in terms of their ability to produce educational outcomes. The qualitative data suggests the variation in drop out rates is directly related to the amount of money given as direct benefit to the child rather than the difference between block grant or scholarship (e.g. Africare with a higher drop out rate in Uganda only provides partial funding and spends between $110 $130 per child, while AVSI provides far in excess of the actual school fees and spends $199 per child). Table 13: Comparison of Effectiveness of Different Approaches to Providing Support to OVC vs. No Support by Country UGANDA Cost per child Attendance Drop out Academic Achievements per year Block Grants $324 Significantly Improved No difference No difference to non OVC Scholarship $412 Significantly Improved Significantly improved No difference to non OVC TANZANIA Block Grants $293 No difference Significantly higher rates Notably better performance (not significant) Scholarship $356 No difference No difference No difference to non OVC 10

14 Based on the above, we come to the conclusion that block grants are more cost effective than scholarships, although we observed in the qualitative data that as the number of children included in a school supported by a block grant becomes small (less than ten), the cost effectiveness approaches that of scholarships. QUALITATIVE Individual interviews or focus group discussions were conducted with district education officers, NGO chiefs and their staff, headmasters, community representatives, and both supported and non supported OVCs separated by gender. A descriptive summary of the qualitative summary is presented in Table 14. Some of the questions yielded brief responses that could be tallied. Others yielded more detailed explanations. In the latter cases themes were abstracted and direct quotes presented that anchored the range and common ground of the sentiments. Table 14: Descriptive Summary of Participants in the Qualitative Study District Headmasters Community NGO Reps. OVC Education (IDI) Reps. (FGD) (IDI) (FGD) Officers (IDI) Tanzania Gender Male Female Total Uganda Gender Male Female Total Grand Total Notes: IDI = In Depth Interview FGD = Focus Group Discussion Focus Groups with OVC Students Although there are varying perceptions of how they felt as an OVC, in general, most students felt negatively about being an OVC. Students in Tanzania and Uganda listed different problems faced by OVC, however, issues such as discrimination, stigma, lack of educational materials, and shortages in food were commonly raised. When asked the question of how OVC students fared compared to their non OVC counterpart, it s interesting to note that the majority of OVC felt as though they either performed equally or studied harder than non OVC. (More importantly, the quantitative achievement data bear up this claim.) Furthermore, in both countries, many of the supported OVC who claimed they had done well academically responded with lofty goals when asked about their aspirations for the future, including wanting to be a pilot, accountant, doctor, engineer, teacher, or banker

15 Key Informant Interviews with Headmasters Headmasters noted that the main issues faced by schools included insufficient classroom and facilities, lack of electricity and water, and teen pregnancy. In regards to the selection process associated for educational support, different NGOs had different criteria for the selection of schools. Some schools applied for block grant support while other schools were asked to participate and were not part of the decision process. When asked about the strengths of the block grant support, most headmasters responded favorably, especially when support was translated into material goods such as desks, tables, and laboratory supplies. A wide range of weaknesses of the block grant were also highlighted by the headmasters, including delays in payments, the small number of OVC students that can be supported, and the school administration s limited ability to prioritize their needs. Overall, headmasters noted that the benefits of the block grant were mostly positive and that block grant support benefitted the entire school. Key Informant Interviews with NGO Personnel Some of the positive benefits for the school as a result of the various educational support included increased school reputation and the ability for the school to invest on capital developments that they would otherwise be unable to afford. In regards to positive benefits for children, NGO personnel mentioned that support increased school attendance and that it was beneficial to students who were unable to pay for school fees. Various strengths were mentioned of the different educational support that NGOs provided, from improved school infrastructure, increased enrollment, and the ability of block grants to support OVC and non OVC students. Although optimistic about the positive impact of the support that they have provided, NGO personnel were realistic of the limitations of their intervention. Educational support does not address the other needs of a child that influence retention and academic performance. In addition, it was noted that some caregivers show reduced responsibilities towards OVC as they believe that the NGOs will provide all the necessary assistance to a supported child. Key Informant Interviews with District Education Officers DEOs highlighted some of the main challenges faced by the district, from the material deficiencies in schools to the issue of high poverty and illiteracy rates within their district. In terms of specific problems faced by OVC, DEOs reiterated the stigma and negative attitudes towards OVC and the educational issues faced by OVC. In regards to the process to select schools and children for educational support, DEOs noted the involvement of local communities in selecting schools and children to be supported. When asked about ways to improve educational support in their respective districts, it was interesting to note that one DEO requested more evaluation and follow up of different support programs. As OVC support is targeted for children under 18 and programs have a limited implementation timeline, it was common to hear complaints associated with the fact that supported students were unable to finish the four year O Level education cycle due to these issues. 12

16 Focus Group Discussion with Community Representatives Community representatives repeated some of the main OVC problems that were already highlighted by earlier interviews, including lack of food and education issues. When asked about the process for identifying OVC for selection of educational support, there was no consistent way in which this was done reflecting the different approaches that NGOs took in selecting OVC. OVC identification is usually done at the community level, either through community leaders or village governments. It was interesting to note that factors such as poverty were considered as a selection criteria as preference was usually given to children who are needy and badly off. Finally, when asked about ways to improve OVC support programs, community representatives stressed the importance of close collaboration with local governments. In addition, they thought that it was important to follow up supported OVC to monitor their progress and find effective ways to select OVC for educational support. Discussion While there were two main approaches for the delivery of services to OVC block grants and scholarships the five NGOs delivering the services had distinctive programs for targeting the aid to needy students. Therefore we examined them individually. Each of the five programs supporting OVC to attend secondary school that we studied was successful in allowing limited numbers of OVC who would not have been able to receive secondary education, go to school. From that perspective, every organization attained its programmatic goals. However, there are limitations presented by each approach, particularly in relation to the costs of the programs and the inability to address the underlying challenges faced by the educational systems in both countries we studied. The original question that this study was tasked with was to compare block grants to schools with the provision of individual scholarships. We look at the comparison of block grants versus scholarships as follows: 13

17 Block Grants Scholarship Did the program succeed in increasing access to secondary education? Yes. Partly. Every child sponsored by Africare and ICOBI In Tanzania, significant numbers of children were verified as attending school and their registered in the scholarship program could not records examined. be verified as attending school and their records could not be found. In Uganda, the same administrative issues were encountered at first leading to significant modification of the program. Did the program provide equity in access to secondary education? No Limited funds were available and priorities had to be made about who would be chosen to receive secondary education and who would not. However, because of the process used to select OVC the community was generally aware of the number of OVC in need and additional sources were sought out for OVC not supported by the block grants. Most importantly, the schools were aware of who was sponsored and who was not and could pay increased attention to them. No As with block grants, limited funds were available and priorities had to be made. The process of selection was not perceived to be transparent. Schools often were unaware of children selected, and thus could not provide attention to special needs or follow up on absenteeism or drop outs. Did the program result in improved attendance at school? Partially Partially Two of the three block grant programs One of two scholarship programs demonstrated showed significant improvements of significant improvements of supported OVC supported OVC over their non OVC and nonsupported OVC counterparts. counterparts. over their non OVC and supported OVC Did the program result in a decrease in dropout rates? Mixed Two of the three block grant programs saw significantly increased drop out rates when compared to non OVC, while the third saw lower drop out rates. No There were no observed differences in either scholarship approach between supported OVC and their non OVC counterparts. Did the program result in better academic performance? Possibly No Supported OVC in Tanzania performed better There were no differences on Form II or Form on their aggregate scores for Form II, but the IV aggregate scores. Numbers were too small to result was not statistically significant. There look at individual subjects. were no measurable differences on aggregate scores for form IV and individual subject scores showed that OVC performed at the same level as non OVC. 14

18 Block Grants Scholarship Did the program result in measurable changes in the school s infrastructure? Yes No Schools documented construction of new There were no known benefits to the schools. classrooms, latrines, water tanks, electric Interviews with teachers showed that in the generators, and purchase of laboratory majority of cases teachers were unaware that equipment and other school materials for all assistance was being provided to the school. block grant approaches. Did the program result in measurable improvements in the school s performance No Overall school performance decreased over time, but this is explained by the rapidly rising enrolments and increasing student to teacher ratios. There were no differences observed between intervention schools and control schools. No This was not part of the design or intent of the program. Did the program result in improvements in psychosocial wellbeing of OVC? Yes Partially Students reported being able to use school Students reported receiving support at home, resources made available as a result of the but few students were aware of there being a conditionality of block grants. teacher trained in psychosocial care at the school. Was the program cost effective? More effective Costs of block grant programs were between $60 and $90 dollars less per child than scholarships. However, it was noted that cost effectiveness is directly related to the number of students enrolled in the block grant program. Accountability Less effective Scholarship programs were more expensive than block grants, with no consistent difference in educational performance, absenteeism, or continuation for children supported by block grants and scholarships. In addition to the comments above, we made several observations about accountability and the potential for diversion of funds. 1. Africare s BG approach was the most regimented approach, leaving little or no freedom for schools to make their own decisions about the use of funds beyond the relatively narrow criteria that Africare set for use of the block grant money. This approach was criticized by several local key informants in other NGOs and the Ministry of Education because it was perceived to be disempowering of the schools and exhibited a mistrust of schools capacity to procure goods and manage money. However, the approach was never criticized by staff or informants the schools that were studied. 2. AVSI s S approach was less regimented than Africare, but still limited what funds could be spent on and conducted detailed financial audits on expenditures. No complaints were ever raised about their approach. 15

19 3. ICOBI s BG approach was designed to contrast with Africare s regimented approach and provide much more freedom for schools to choose and be free of administrative burden. This approach was extremely popular with schools, but it was observed that ICOBI is now considering moving to a more rigid model aligned with Africare s approach because of concerns about accountability. 4. CRS s S approach was the most open of all. However, it was also observed to be the most open to abuse, and several cases were observed which raised concerns. Firstly, when schools were visited to collect data from supported student s records there were several instances where the student had never attended the school and could not be traced. This led the local investigation team to coin the term Ghost students (Watoto yatima hewa in Swahili), coming to the conclusion that families had received funding, but money had not been used for school. Secondly, there were several reports by teachers that students only had part of their funds paid by the local FBO, with the student forced to leave school partway through the term because of non payment. Finally, in several instances local FBO collaborators simply disappeared after questions were raised about funding, leading to concern that they feared that the research team were auditors. Absenteeism and gender There is a great deal of discussion in both the popular and the scientific literature about the effect of menstruation on girls ability to maintain school attendance and perform adequately in school. The argument is made that the lack of sanitary napkins prevents girls from attending school during menstruation, and that even when sanitary napkins are available, lack of privacy and adequate sanitation makes it difficult for girls to change napkins. Some authors have argued that this may result in between days of school lost in a 180 day school year, Mooijman et al (2005). Many NGOs (including all the NGOs in this study) provide sanitary napkins to OVC girls they are supporting as part of their program. Our results are therefore somewhat surprising in that for every group supported OVC, non supported OVC, and non OVC, girls have a lower rate of absenteeism than boys. Menstruation appears to be less of an issue for girls than previously thought (some estimates put girls absenteeism as high as 18%, with the cause being an inability to deal with menses). This is the first study to report quantitative data on girl s absenteeism in Africa, and indicates that further study is required to clarify the issue. Further conclusions cannot be drawn as this was not a specific focus of the research and questions to differentiate the reasons for girls and boys absenteeism were not asked. The findings of reduced absenteeism of girls when compared to boys were consistent in control schools where there was no intervention to provide sanitary napkins. Our findings are in line with the recent publication by Oster and Thornton (2011), which concluded that actual lost time at school due to menses was very small (approximately half a day a year in Nepal). This is not to suggest that provision of sanitary napkins, good hygiene and privacy are not an important human right for all young women, simply that our observations would reinforce the fact that menstruation does not result in higher absenteeism for girls. 16

20 Drop out and gender There was no difference in the drop out rates between boys and girls in any of the schools in our study (6% for boys and 6% for girls). However, OVC who were supported did not have a significantly higher rate of drop out than non OVC, while non supported OVC of both genders had significantly higher dropout rates. We can conclude therefore that the support being provided to OVC was protective against dropping out. Academic performance and gender Results indicate that on average, female students are half as likely to pass their National Form II and Form IV exams compared to male students. Across the different support groups, the same pattern holds true, female students are less likely to pass their national exams. Male OVC ever supported students are more likely to pass their Form IV exams compared to non OVC students, while female OVC ever supported students are less likely to pass the equivalent exam compared to their non OVC counterpart. Data suggests that educational support to OVC seems to have better impact on the academic performance of male students as opposed to female students. Costs and efficiencies The wide variation in costs per program largely reflects the variation in administrative oversight and follow up across the organizational approaches, as well as the fact that each organization contributes different percentages of the total education costs (ranging from 50% for Africare s block grant approach in Uganda to 90% for AVSI s scholarship program in Uganda). It is not clear how decision making should be made about what proportion of the overall costs should be covered by NGOs and this requires further research. It was observed that the larger the number of children in the program, the lower the proportion of administrative costs suggesting noticeable economies of scale and that large programs are more cost effective. It was also noted during qualitative interviews that block grant programs with small numbers of students did not provide sufficient funds to the school to provide the leverage required to enable the entire school to benefit from the grant. It was concluded that the larger the number of students enrolled in a block grant program the more cost effective the approach, while block grant programs with smaller numbers rapidly become as inefficient as scholarship programs. Limitations of the Study Lack of knowledge of the households from which the supported OVC came. Many contextual factors which would help to explain issues raised in this study cannot be answered because funding was not sufficient to conduct a household survey of supported OVC to understand their social, economic, parental, living, and other conditions. This lack of 17

21 information does not affect the relevance or quality of the data collected at schools or in focus groups, but it does limit the ability to interpret some of the findings around absenteeism and drop out. It also limits the ability to provide direct comparisons between schools as the details of the populations served are not well known. Children aging out of the program. Because of early statements about who constituted an OVC by PEPFAR, decisions were made early on by the various programs that educational support would not be continued beyond the age of 18 years (in the last year of project implementation this was reversed, but did not affect the cohort of students being studied). The consequence of this is that children who initiated the program found themselves losing their educational support in Form IV (and sometimes earlier). In Tanzania this resulted in the majority of Form IV students being lost to follow up and this inturn affected the results for the final year of the cohort particularly for Form IV examinations. Lack of concentrated populations of supported OVC. The finding that both of the scholarship programs (CRS and AVSI) did not have populations of 50 or more supported OVC in any individual secondary school only became clear as individual recipient lists were examined. This required that a much larger sample size of schools be made for both NGOs. In turn, directly matched control schools could not be chosen for all schools in the CRS and AVSI samples. At CRS schools there were often only five or six study students, and therefore the control students were picked from the same schools. Differences in the educational systems between Uganda and Tanzania. Children start school one year younger in Uganda than in Tanzania, with the result that the OVC in our sample in Tanzania were all one year older than their equivalent grade in Uganda. Fortunately, while a year s age difference is significant in younger age groups, it has little bearing on the ability to perform academically after mid adolescence and we do not anticipate that age would result in any differences in pass rates between the two countries. However, the later start date in Tanzania meant that many students in Form IV were over 18 and therefore no longer eligible for OVC support. This is reflected in the very low numbers of exam results for students in Form IV in Tanzania which in turn limited our ability to draw significant conclusions on Form IV academic performance in this environment. In 2007, Uganda introduced Universal Secondary Education, eliminating tuition fees for secondary school. In contrast, fees for secondary education are still charged in Tanzania. The fees in Tanzania are 20,000 TZS per term, which is then matched by the government. However, secondary education in Uganda is not free, and fees in Tanzania are not limited to the official government fees. The reality is that in both countries, the cost of actually attending secondary school is approximately the same (in excess of 100,000 Shillings). The actual fees charged are not standardized and may differ from school to school, and even from year to year in the same school. We therefore do not believe that USE in Uganda presents a problem in making countryto country comparisons. 18

22 Lack of detailed information on country wide school performance. The information needed to conduct performance comparisons of all intervention schools (regardless of NGO) in the country versus matching schools has not been made available from the Ministry of Education. This has severely limited the conclusions we can draw from the QUANT2 study; we can only observe trends rather than make clear statements about the broader impact of assistance to secondary schools. This constraint does not affect the other conclusions and recommendations being made in this report. Conclusions We can conclude from our study that providing support to children to attend secondary school can be successful whether through block grants or scholarships, and that in general, supported performed as well as their non OVC peers and sometimes better. We also concluded that of the two approaches (block grant and scholarship), block grants are simpler to administer, require less administrative costs and oversight to ensure that students receive the benefits, and are more cost effective than scholarships per student. However, we did observe that block grants become less effective with smaller numbers of students enrolled as not only are there losses in economies of scale, the leverage on the school is much reduced when only small amounts are provided. We were unable to determine the exact point at which block grants become most efficient. It was clear throughout our observations that provision of free access to schools alone is insufficient to ensure that OVC attend regularly or perform well. Additional support was required that addressed physical health, nutrition, social setting, economic well being, and other aspects of the child's overall well being. Tanzania specific: It is obvious that the Government of Tanzania's focus on increasing access to secondary education in rural areas through the building of ward level schools has been successful. However, the extremely poor performance at certificate level is of enormous concern reflecting serious problems with the quality of education, which needs to be addressed urgently. We noted that Tanzania's use of a national examination at Form II provides an extremely important benchmark for early decision making about appropriate interventions to improve student performance. It was observed to the research team on several occasions that this examination is no longer may even be phased out which would remove this excellent tool. Uganda specific: Because Uganda only collects national standardized achievement data at Form IV, the Ministry has no way of knowing how the system is performing over time, as a whole, and earlier decision making about appropriate interventions to improve student performance is problematic. 19

23 Recommendations General Recommendations: Investment in secondary school education for OVC is of value and is a program that PEPFAR and other stakeholders should consider supporting while encouraging local Ministries of Education to take on this role. In designing secondary school education programs for OVC, Governments, PEPFAR and other development partners should give a high priority to block grant programs. However, these should be done within a comprehensive approach that addresses the physical, mental and social wellbeing of children. To design their OVC block grant programs more efficiently, PEPFAR and other development partners should focus on areas with concentrations of OVC (especially in rural settings), and within those populations seek to serve larger numbers of students in smaller numbers of schools. Selection criteria for selecting OVC when resources are limited, should be clarified and standardized to ensure equity and transparency; should OVC be chosen on the basis of the most needy, or the most likely to succeed? Tanzania Specific Recommendations: In Tanzania access to education for OVC needs to be expanded; the Government of Tanzania, civil society, and the private sector, should work together to eliminate specific financial and other barriers to education. The Government of Tanzania should continue to use Form II examinations as a means of determining educational effectiveness. If this is impossible, then sampling of students and content areas can be used making the evaluation of effectiveness less costly. The Ministry of Education needs to standardize the collection, storage, and maintenance of school records (including attendance registers and grade reports), and these standards should be applied to private and church schools as well. Uganda Specific Recommendations: In Uganda, in addition to providing Universal Secondary Education, the Government of Uganda should eliminate financial and other barriers to education for OVC. The Government of Uganda should give serious consideration to establishing additional standardized national examinations for earlier grades at secondary school to enable improved decision making about the quality of education. If this is impossible, then sampling of students and content areas can be used making the evaluation of effectiveness less costly. The Ministry of Education needs to standardize the collection, storage, and maintenance of school records (including attendance registers and grade reports), and these standards should be applied to private and church schools as well. 20

24 Recommendations for Further Research: Research is required to determine what proportion of the total cost of education is the optimal amount to assure the best outcomes from support programs. Future research should employ comprehensive outcome measures and longitudinal approaches to study improvements in student access, teacher effectiveness, and student performance over time. 21

25 INTRODUCTION AND OVERVIEW At the request of the PEFPAR Technical Working Group for Orphans and Vulnerable Children, Boston University (BU) undertook a study to provide evidence to facilitate PEPFAR decisionmakers to make informed choices as new investments are made in educational support to Orphans and Vulnerable Children (OVC). This work was conducted through the Orphans and Vulnerable Children Comprehensive Action Research Project (OVC CARE), funded by USAID through Project SEARCH (Contract No. GHH I , Task Order 1) In early 2011, in collaboration with research partners Makerere University in Uganda and the Center for Center for Strategic Research and Development (CESTRE) in Tanzania, BU implemented a mixed methods study to evaluate the effectiveness of educational support provided to secondary school children through the President s Emergency Fund for AIDS Relief (PEPFAR) by a variety of Non Governmental Organizations (NGOs), using different approaches, in Uganda and Tanzania. The design was a retrospective, longitudinal study, and used a mixed methods approach, adopting both qualitative and quantitative methodologies. The qualitative components drew on interviews, focus groups, observational techniques, and document analysis to learn about the intervention used by each NGO and its context from the point of view of the primary recipients and other stakeholders who are knowledgeable about the OVC block grants programs. The quantitative components of the study took three forms. The first addressed how well the OVC fared on attendance, drop out, and achievement measures under different approaches, with a specific focus on comparing block grant and scholarship programs over time as compared to their OVC peers and non OVC peers who did not receive any targeted benefit. The second quantitative component addressed how individual schools that have received financial assistance through block grants have fared over time as compared with a matched set of schools that did not receive any form of block grants. The matches were based on school size, class grade level, locale, and SES characteristics. The third quantitative component of the study examined the relative cost effectiveness of different models of disbursing the funding to schools to aid OVC by examining direct costs, indirect costs, and imputed costs of each NGO program. BACKGROUND AND RATIONALE In addition to the intrinsic importance of education and its important role in economic growth, a causal link has long been established between education and a range of health outcomes (Lynch 2003, Chou 2010). More recently it has been shown that for every year of increase in the education of women of reproductive age, child mortality decreased by 9.5% (Gakidou 2010). Therefore increasing access to education and retention at school is important not only to the health and wellbeing of the individual, but also to their future children and families. 22

26 Many factors contribute to children s access to education in developing countries, including: gender, health status, family structure, poverty, distance to school, and perceived value of education (Goodsmith, 2004). Being orphaned or made vulnerable can also play a role in whether a child goes to school. Studies have indicated that where national school attendance is high, there is little difference between OVC and non OVC in terms of school attendance. However, in 38 of 59 surveys reviewed (most in Sub Saharan Africa), low school attendance was the norm, and in these settings orphans were significantly less likely to attend school (Akwara, 2010). Equally troubling is that Africa has one of the lowest education completion rates in the world. In one out of four African countries, half the children enrolled at the end of primary school do not continue to the secondary level (UNESCO, 2006). The low transition rate into secondary schools is partially a result of caregivers not being able to afford to pay for a child s education. Secondary schools are rarely free and students must pay for tuition as well as other school related expenditure such as books, uniforms, and examination fees. Various mechanisms of funding are applied by USAID and other development partners to support the education of orphans and vulnerable children. These include scholarships and block grants. Scholarships are individual payments made to, or on behalf of, a child to pay for part or all of their education. Block grants are fixed sum grants to local or regional educational systems or institutions that give the recipient organization broad flexibility to design and implement designated programs. Scholarships are usually given to the family or a trustee on behalf of the child, to pay for school fees. Block grants are paid in advance to the school and use the attraction of a lump sum of money to leverage admission to school for less than the regular admission fee. In most cases, with the payment of a block grant, a certain number of children are guaranteed admission to the school under a contractual basis. While both mechanisms have been used in various developing countries, their relative impact on both the quantity and quality of education for OVC has not been well evaluated. It is also not clear when it is appropriate to apply one or the other. The goal of this research was to compare educational access that results from implementation of the two approaches in secondary schools among OVC. The research also provides information on the ability of each approach to influence educational outcomes. Beginning in 2005, many large Non Governmental Organizations (NGOs) were awarded grants by the United States Agency for International Development (USAID) under so called Track One funding to provide core services to OVC in PEPFAR focus countries. However, PEPFAR was an emergency response to the AIDS epidemic with the intention of reaching the maximum number of children in need as quickly as possible. The direction provided to the NGOs was broad in terms of options for educational access with the result that numerous models of support emerged. At the same time, monitoring and evaluation of educational support programs focused largely on numbers of OVC served with insufficient data to ascertain the effectiveness of different approaches in achieving specific educational outcomes. The funding for these programs ended June 30, Therefore it was considered timely and important to study the programs and their impact to inform the development of policies and programs for the future, for both the US and the international community of donors. 23

27 Review of the Literature Impact of orphanhood on education Being orphaned or made vulnerable can play a role in whether a child goes to school. OVC may lose access to school for several reasons including poverty, need for domestic labor, need for income generation activities, stigmatization, and parental sickness or death. School enrollment inequities among all types of orphans have been documented throughout sub Sahara Africa (Monasch and Boerma, 2004; Case et al., 2004; Nyamukapa and Gregson, 2005; Miller et al., 2006; Case and Ardington, 2006; Bicego et al., 2003). One estimate suggests that orphans are approximately 13% less likely to attend school than non orphans (Monasch and Boerma, 2004). Studies have also shown that disparities in grade progression were found in several countries including Botswana, Niger, Ghana, Kenya, Tanzania, and Zimbabwe (Miller et al., 2006; Bicego et al., 2003). Appropriate grade progression is important because losing ground in school places children at higher risk for failing and dropping out. Household factors that negatively affect orphan schooling include poverty, low education among household heads, male household heads, and high dependency ratios (Miller et al., 2006). In some countries, differences between household poverty and orphan based disparities were small, indicating that the negative effects of orphanhood are aggravated by poverty, as orphans are more likely to live in poorer households (Sengendo and Nambi, 1997). One study looking at communities affected by AIDS in Tanzania and Burkina Faso noted that educational status among orphans and non orphans were similar (Kurzinger et al., 2008). A recent meta analysis indicates that while orphan status alone can explain differences in education in some settings, the only consistent correlate with school attendance and length of education is household economic status (Akwara, 2010) Several child level factors negatively impact schooling, including weak family ties with caregiver and orphan type. The lower enrolment of orphans can be explained by the tendency of orphans to live with distant relatives or unrelated caregivers (Case et al., 2004; Kurzinger et al., 2008). Maternal orphans are less likely to be enrolled in school and have completed fewer years of schooling than children whose mothers are alive (Case and Ardington, 2006). Although paternal orphans experience inequalities, double and maternal orphans seem to be at the greatest disadvantage. The evidence base for gender disparities among orphans is limited. A multicountry study that looked at 10 sub Saharan countries showed that after controlling for gender, age, and household resources, both boys and girls were equally disadvantaged (Case et al., 2004). While these studies show that orphan based disparities do exist and that maternal and double orphans face the greatest inequity in educational attainment, there are several limitations. Available data are often too old to accurately determine the extent of disparities and some studies do not control for covariates. In addition, educational indicators in these studies are limited to enrollment and grade progression which make it difficult to understand the different mechanisms that undermine access to education. Given that education is an important 24

28 pathway towards poverty reduction and improved health outcomes, a better understanding of the mechanisms that lead to OVC educational failure and dropout is essential. Educational interventions and block grants programs There is growing literature that evaluates some of the determinants of schooling decision making. Interventions that improve quality of education, especially in lower income countries where average quality is often poor (Pritchett, 2004), do not seem to increase student participation levels in the short term (Banerjee et al., 2007; He and MacLeod, 2007; Muralidharan and Sundararaman, 2006). On the other hand, studies have shown that interventions aimed at reducing the cost of attending school seem to result in improved participation levels either through subsidies to attend private schools (Angrist et al., 2002, 2006), reduced user fees (Barrera Osorio et al., 2007), scholarships (Kremer et al., 2007), or direct cash incentives (Vermeersch and Kremer, 2005; Schultz 2004; Glewwe and Olinto, 2004; Schady and Araujo, 2006; Attanasio et al., 2005). Several block grant programs have been implemented in developing countries with the goal of improving educational outcomes. From 1995 to 2000, the World Bank conducted research that took advantage of an opportunity in Kenya to perform randomized evaluations to improve primary education (World Bank, 2001). Out of the 100 primary schools, 25 were selected to receive block grants that could be spent on several options such as textbooks, other school supplies, or construction of new classrooms. The purpose of this intervention was to see whether funds were used more effectively when schools were given a choice on how to spend them. Analysis of the results indicated a small but statistically significant impact on test scores after one year, however the impact seems to diminish over time. As with supplying textbooks, the better students seem to have benefited the most. The results show statistically significant effects in reducing dropout rates and increasing passing rates (World Bank, 2001). In 1999, the World Bank conducted a rapid assessment of Indonesia s social safety net (SSN) Scholarship and Block Grant programs in 4 provinces (World Bank, 2004). The National Compulsory Education Program of 1994 made 9 years of education compulsory for all students in Indonesia. The SSN Scholarship and Block Grant programs targeted students from poor families and offered operational support for their schools with the goal of improving educational attainment. This report focused more on programmatic issues instead of looking into specific educational indicators. The assessment indicated that block grants had not been very effective except in some disadvantaged private schools, public primary schools and madrasahs. It was noted that Block Grants were more effective in schools located in remote areas. Providing SSN Scholarship versus Block Grant programs has had no impact on the reduction of student dropout rates. Some of the important issues surrounding successful implementation of SSN Scholarships and Block Grant programs include mechanisms for targeting recipients, sufficient levels of funding, the use and disbursement of funding, and the effectiveness of the committees in program monitoring and supervision (World Bank, 2004). 25

29 A review of block grants in Kenya, Uganda, Tanzania, and Mozambique indicated that enrollment rates increased rapidly as children had greater access to education (Mushemeza, 2005). Disparities in enrollment rates between rural and urban, rich and poor, male and female decreased. However, the quality of teaching and learning seemed to have worsened. This was attributed to several factors: sharp increase in the student teacher ratio, increase in unqualified teachers in schools, low teacher salaries, and threat of AIDS pandemic increasing absenteeism in schools. Several issues that affected the success of the program included political interference, poor management, lack of community involvement, poor education infrastructure, and a shortage of trained teachers. Rationale for conducting the study in Uganda and Tanzania The first key criterion for this study was that it should be conducted in multiple settings to enable comparison of approaches in different educational and national environments. The second criterion was that there should be both block grant and individual scholarship approaches offered in each country. Uganda and Tanzania were selected because both have similar (but not identical) educational systems inherited from their colonial past. Children receive seven years of primary education and then pass on to secondary education (class grades referred to as Form I VI in Tanzania; Secondary I VI in Uganda), the first four years of which are spent studying for Ordinary Level (O Level) exams, and the final two years spent studying for Advanced Level (A Level) exams. Both countries provide free primary education but only Uganda provides free secondary education. However, the term free education does not reflect the reality that while there may be no tuition exacted by the government, individual schools are free to impose fees for items such as text books, equipment, desks and chairs, laboratory use, building maintenance, etc. These fees can exceed tuition and provide a real barrier to access. A site visit to both countries revealed that there were no absolutely pure models of support for OVC at secondary school because of the sharing of promising practices and adaptation during five years of implementation. Heads of NGOs communicated with each other and compared practices, incorporating what they thought were best practices into their own methods of dispersing funds, each one arriving at a unique, blended approach. Thus, there was sufficient variation in approaches to enable country to country comparison and also to enable approach to approach comparison. Specific comparisons: Country to country comparison: Africare s Community Based Orphan Care, Protection & Empowerment (COPE) project (Blackett Dibinga et al, 2006) implemented similar block grant programs in both Tanzania and Uganda. 26

30 Approach to approach comparison: o ICOBI in Uganda and Tanzania in both countries implemented block grant programs, but with significant differences in the way the grants were administered; o Catholic Relief Services in Tanzania implemented a strictly scholarship based program, with money being given directly to the family to pay school fees and support other needs; o AVSI in Uganda implemented a modified scholarship based program, in which individual fees were paid directly to the school, which then took on responsibility for providing additional support to the OVC and their family. Characteristics of the four NGO s approach to secondary education support A. Block grants with control of the funds retained by the implementing organization as implemented by Africare in both Uganda and Tanzania Africare adapted its COPE approach from Zimbabwe (Blackett Dibinga et al, 2006) and began to apply it in Uganda and Tanzania in Because of the differences in the educational systems in the two countries, there are slight variations between the countries, but the overall approach is the same. The main characteristics are: 1. Individual OVC have a percentage (30% in Uganda, 100% in Tanzania) of their overall school fees paid for by Africare, leaving the caregivers responsible for funding the balance. 2. The school makes a commitment to admit and retain the selected OVC from S1/F1 through S4/F4. In Uganda this was a four year commitment, while in Tanzania it was negotiated as two two year commitments. 3. Africare makes the total amount of money available to the schools to improve school functioning as block grants but limits the scope of what the school can choose to do with the funds to physical renovations; the purchase of equipment, desks, chairs, curriculum materials; and other capital investments. Operating costs or salary top ups are not allowed. 4. The schools determine their prioritized needs from a menu of choices offered by Africare. 5. After selection of the priorities, Africare procures services or materials to respond to the needs. This may vary from procurement of generators, solar panels, books, laboratory materials, or services to renovation or construction of classrooms, lavatories, etc. 6. Individual assistance is provided to specific OVC for uniforms, pens, and other materials (such as sanitary pads for girls). 7. In school support is provided to OVC through anti AIDS youth groups called COPE Clubs. 8. Additional educational support is provided to OVC through the purchase of school uniforms, pens, notebooks, school materials, and sanitary pads for girls. The educational support is part of a larger, more comprehensive approach to supporting OVC which includes economic strengthening for households with OVC through micro enterprise activities, engaging OVC in workforce development, and building capacity of caregiver associations to increase income; building the capacity of OVC committees and other local structures to monitor and support OV households; conducing life skills training and peer 27

31 education to OVC and their families; providing psychosocial monitoring and support to OVC and their families. Africare believes that its approach to education provides the following benefits: School enrolment is increased because OVC who would not have been able to attend school are now able to attend. OVC who attend as part of the COPE program have a continuous education compared to their non supported OVC peers who are able to attend school, because their attendance is guaranteed, while non supported OVC have to find their own funds and may be asked to miss periods of time if their parents cannot pay fees for a particular month. Paying block grants is a more sustainable approach than paying direct school fees as it guarantees that children attend school. Block grants reduce stigmatization of OVC in their schools. B. Block grants with full control of the funds given to the schools, as implemented by ICOBI in Uganda ICOBI began its work with education later than the other organizations, receiving its first grant in It examined other forms of assistance that were being used in Uganda before coming up with the design of its program. 1. At the start of the academic year, a lump sum of 200,000 UGX was paid to the school on behalf of every OVC being sponsored by ICOBI to attend secondary school. 2. Every child for whom the lump sum was paid was guaranteed secondary school education for the entire year and no additional charges were to be made to the child. 3. The school dispensed the money through its normal financial and procurement systems, providing reports to ICOBI. 4. Of the grant money, 60% was restricted to being used for the purchase of direct support to students (such as textbooks) or infrastructure or institutional support. It was for the school to determine what these funds are spent on examples have included solar panels, computers, rehabilitation of classrooms lavatories, school desks and furniture, generators for lighting, and rain water collectors. 5. Schools could use up to 40% of the funds on any activity they deemed appropriate, such as creating orphan clubs, feeding students, or topping up teacher salaries. 6. Additional, individual support was provided to children by ICOBI through the purchase of school uniforms, pens, math sets, notepads, and sanitary pads for girls. ICOBI saw its educational support as being part of a larger family centered approach which also identified other needs of households; they then provided targeted services such as socioeconomic strengthening, psychosocial assessments and support, informing children and caregivers about legal protections, and support through working with community care committees. Overall, ICOBI believed that its approach would empower households to reduce vulnerability. 28

32 ICOBI believed that its approach to education provided the following benefits: School enrolment was increased because OVC who would not have been able to attend school were then guaranteed a place at school. By providing cash to the schools at the start of the academic year, financial bottlenecks that arose from slow arrival of government funds were overcome, enabling the school to respond quickly to their needs By allowing schools to make their own purchases and procurements, the schools were empowered. The provision of cash sums enabled the schools to make changes that benefited all children, not simply those receiving support. C. Provision of individual scholarships to attend school with the funds provided directly to the family, as implemented by CRS in Tanzania CRS provided individual scholarships to students in three regions, Arusha, Tanga, and Manyama. However, as CRS s program expanded in the south to Iringa and Ruvuma regions, they switched to a block grant approach. In Tanga (the focus for this research), the program was administered through local community based and faith based groups (7 Catholic and 1 Anglican). 1. Individual OVC in need of educational support were identified by the CBO/FBO. 2. Funding was provided from CRS to the CBO/FBO based on the assessment. 3. Funding was provided directly to the family to pay for school fees and examination fees. 4. CRS (through the CBO/FBO) provided school uniforms, books, pens, notepads and other basic requirements. 5. One teacher in each school was trained in psychosocial support of OVC. CRS s overall approach was family and child centered, and educational support was not provided in isolation from the full range of core OVC areas of intervention. Services included paying for health insurance for families, providing supplemental food if required, strengthening household economics through income generating activities, and shelter. An extremely heavy reliance was placed on the local CBO/FBO to administer grants and scholarship money, conduct monitoring and follow up, provide a large amount of volunteer time and contributions in kind from the community to support OVC. CRS believed that its approach to education provided the following benefits: By addressing the range of needs of OVC and not limiting themselves to only three interventions, comprehensive needs of the child were addressed, which would result in better overall outcomes. Because school performance was affected by hunger, health, whether the child slept well, and other factors, this approach would result in better performance at school. Children who would not normally be able to attend secondary school were able to do so because their fees were paid. 29

33 Engagement by the broader community with the family reduced stigma and provided a more secure household for the OVC. D. Provision of individual scholarships to attend school with the funds provided directly to the school, as implemented by AVSI in Uganda AVSI provided a somewhat contrasting model of support that was similar to a scholarship program, but had important differences to the model developed by CRS. This came about because the managers of AVSI drew on the perceived successes of block grant programs, and attempted to emulate them. AVSI is also a faith based group, and its program focused for the most part on religious schools, however, they also covered government run schools. The faithbased schools became the administering partner for the child whether the child attended school at that school or another school. 1. Individual parishes identified OVC in need of secondary school educational support. 2. The parish recommended the children to the local church school. 3. The church school conducted assessments of the children and their families to determine their educational needs. 4. The school put together a budget for each child which includes tuition fees, exam fees, and other fees; psychosocial support; school uniforms, books, notebooks, and other personal needs; capital needs of the school; transportation needs; capacity building needs of the school to address OVC; and, community sensitization. 5. Where students were not attending the church school, the school paid the fees to the government school the student was attending. 6. The church school acted as the implementing agent, procuring and providing all services to the OVC. AVSI s approach to education was also embedded in a holistic approach; they did not believe that services to OVC should be limited to only a few of the core services. They therefore provided support for health care, psychosocial evaluation and support, income generating activities for households, community sensitization about legal issues, and they also worked with families to ensure good housing. AVSI believed that its approach to education provided the following benefits: The focus within the faith based community ensured the best quality educational services. Providing a holistic approach to all the needs of an OVC and their family would benefit not only their attendance at school but their performance as well. Making the grants directly to the school and making the school responsible empowered the schools and built the capacity of the schools to evaluate the needs of children and more effectively engage with the community. 30

34 METHODOLOGY AND STUDY SITES The study employed a four year, retrospective, longitudinal design and followed a mixed method approach adopting both qualitative and quantitative methodologies. The target population was children who attend publically funded schools in both countries, and was divided into five groups: Group 1 Group 2 Group 3 Group 4 Group 5 Children attending intervention schools, determined to be orphaned or vulnerable by local standards, who needed support, and who had been the recipients of direct educational support to ensure their attendance at secondary school, either block grants or scholarships. Children attending intervention schools, determined to be orphaned or vulnerable by local standards who had not received any direct educational support. Children attending intervention schools that were not classified as orphans or vulnerable by local standards and had not received any form of educational support. Children determined to be orphans or vulnerable by local standards (regardless of need) who were attending control schools. Children who were not classified as orphans or vulnerable by local standards who were attending control schools. The selected districts were: Uganda Africare: Ntungamo and Isingiro District ICOBI: Mbarara and Ibanda Districts AVSI: Mpigi, Mukuno, Kampala, Luwero, and Masaka Districts Tanzania Africare: Dodoma District CRS: Tanga and Muheza District 31

35 DEFINITIONS For the purpose of this study the following definitions apply: (a) Orphans A child, 0 17 years old, who has lost one or both parents (b) Vulnerable child Is vulnerable because of any or all of the following factors Is HIV positive; Lives without adequate adult support (e.g., in a household with chronically ill parents, a household that has experienced a recent death from chronic illness, a household headed by a grandparent, and/or a household headed by a child); Lives outside of family care (e.g., in residential care or on the streets); or Is marginalized, stigmatized, or discriminated against; Suffers with a physical or mental disability; Is identified as vulnerable by the local Child Care Committee, OVC committee, or other community group. QUANTITATIVE COMPONENTS OF THE STUDY The quantitative components of the study took three forms QUANT 1 looked at individual student level data from OVC. QUANT 2 looked at aggregate data from individual schools, while QUANT 3 looked at cost data to calculate the cost per child for each intervention. QUANT 1 addressed how well individual OVC fared under the block grant and scholarship programs over time as compared to their OVC peers and non OVC peers who did not receive any targeted benefit. Outcome measures included: Attendance Retention Pass rates National examination scores A limited sample of intervention schools was selected because of the nature of the evaluation, the limitations posed by the available funding, and limited availability and access to reliable data. Lists of schools from intervention districts were obtained from the local education authorities. The schools were then stratified into those served by individual intervention NGOs and those who did not receive any intervention. For each NGO, five intervention schools were initially selected by random sampling, and this number was expanded until the target number of intervention OVC was identified. Five matching schools were selected in each district using the criteria noted above, to serve as control schools. 32

36 Numeric data for the last four years ( ) were then collected from each school from the following sources: class attendance registers; individual student files; school registers; and published examination scores. These data were compared for OVC with support, versus OVC without support, versus non OVC. The data included measures of attendance, retention, pass rates, and national achievement examinations. QUANT 2 addressed how well entire schools receiving block grant support performed in comparison to schools receiving no support. Each school that received NGO educational support was matched with a school in the same district that did not received block grants. Schools were matched for size, grades taught, and socio economic status. Data were collected over the previous eight years to determine if enrollments had increased and performance had improved. QUANT 3 examined the relative cost effectiveness of different models of disbursing the funding to schools to aid OVC. Cost data were gathered from the intervention NGOs about the costs required for schooling, additional benefits provided, volunteer time, and the overhead costs to run the operations that NGO s use to disseminate, monitor, and account for the effectiveness of block grant spending. From this, direct, indirect, and imputed costs of secondary school education support were calculated. The methodology used for QUANT 3 was developed earlier in the OVC CARE project and is described in Larson, This procedure followed a six step process: 1. Access and organize the financial reports of each organization. 2. Reorganize the financial report sub categories into input cost categories. 3. Estimate the annual equivalent payment for program equipment. 4. Document donations to the NGO for program implementation. 5. Include the portion of NGO costs not attributed to specific programs. 6. Include the results of steps 3 5 into an expanded cost profile. Qualitative Methodologies drew on interviews, focus groups, observational techniques, and document analysis to learn about the intervention and its context from the point of view of the primary participants and other stakeholders who are knowledgeable about the OVC block grants programs. Contextual accounts were captured and triangulated by comparing perspectives and by contrasting process and product measures. The sources for qualitative data included: A structured interview with the lead staff of each NGO A structured interview with each school headmaster and director of studies A structured interview with the district education officer Five focus groups: o The first three at intervention schools. With a group of eight community leaders who work with OVC or who are involved in the management of the school through the school management committees or parent teachers associations. 33

37 SAMPLE DESIGN AND SIZE With a group of eight male OVC students receiving educational support With a group of eight female OVC students receiving educational support o The second two at matched non intervention schools With a group of eight male OVC students With a group of eight female OVC students A sample design framework is described in Figure 1. Figure 1: Sample Design Framework Sample size was calculated based on the availability of intervention schools, the presence of a sufficient number of OVC at each school, the criteria for inclusion in the evaluation, and the resources available for the study. At each intervention school a four year retrospective record was collected to include attendance, retention, pass rates, and national exam scores for 150 selected children (50 supported OVC, 50 unsupported OVC, and 50 non supported, non OVC). Similarly, at each control school a four year retrospective record review of attendance, retention, pass rates, and national exam scores for 100 selected children (50 OVC, and 50 non OVC) was collected. Educational indicators among OVC in secondary schools were scarce. National education statistics from Tanzania indicated that the failure rate of students taking O level examinations ranged from 16.3% 24.4% in 2008 (Basic Educational Statistics in Tanzania, 2009). Using this 34

38 information, we assumed an examination failure rate of 30% (based on the observation that routine statistical information systems can underestimate actual data by as much as 50%) for OVC who are more likely to be disadvantaged compared to the average secondary school student. As one of the key objectives of the study was to determine whether or not OVC that attended block grant supported schools performed better at school, we calculated the power we would need to detect meaningful differences between intervention OVC (treatment subjects) versus non intervention OVC (control subjects) based on national exam failure rates. Country Level Analysis In Uganda, the study had 750 treatment subjects (50 intervention OVC x 3 NGOs x 5 schools per NGO) and 750 control subjects (50 non intervention OVC x 3 NGOs x 5 schools per NGO). In Tanzania, we had 500 treatment subjects (50 intervention OVC x 2 NGOs x 5 schools per NGO) and 500 control subjects (50 non intervention OVC x 2 NGOs x 5 schools per NGO). Using the assumption that the examination failure percentage among non intervention OVC (control subjects) is 30% and using an alpha of 0.05, we had 75% power to detect a 25% reduction in the failure rate (i.e. relative risk of 0.75) for intervention subjects relative to controls in Tanzania and 90% power in Uganda. School Level Analysis At the school level, we had 250 treatment subjects (50 intervention OVC x 5 school per NGO) and 250 control subjects (50 non intervention OVC x 5 school per NGO). Using the assumption that the examination failure percentage among non intervention OVC (control subjects) is 30% and using an alpha of 0.05, we have 86% power to detect a 40% reduction in the failure rate (i.e. relative risk of 0.60) for intervention subjects relative to controls within schools supported by different NGOs. STUDY PROCEDURES Prior to survey implementation, ethical approvals were obtained from the Boston University Institutional Review Board (IRB), the Ugandan National Research Council, and the Tanzanian Commission for Science and Technology. In addition, formal approvals were obtained from the Ugandan Ministry of Education and Sports, and the Tanzanian Ministry of Education and Vocational Training. Letters of approval were also received from each NGO. Consent forms and study instruments were prepared in English. In Tanzania the study was administered in Swahili and all instruments and consents were translated and back translated to ensure consistency. In addition, official certifications of consent form translations were provided to the BU IRB. In Uganda, study instruments and consent forms were administered in English. Data collectors underwent three days of training in data collection for quantitative and qualitative methods prior to the study. Training was conducted jointly by the local research 35

39 partner and BU. During this training field testing of the instruments was also conducted and any necessary revisions were made. Data collectors visited schools in teams of 2, taking approximately 3 days to collect all required data from each school. Focus group discussions were run by one of the senior researchers from each local research team. Recruitment & enrollment of participants Intervention schools were selected randomly from lists of intervention schools maintained by the selected NGOs. All children who had been recipients of assistance from the NGO (based on NGO records) were assigned to Group 1. The head teacher was interviewed and asked to identify those students known to fit within the local definition of orphaned or vulnerable. Any OVC who had received direct support to attend school from other sources were also assigned to Group 1, noting the source and type of assistance. OVC who had not received any form of assistance were assigned to Group 2 All other children were assigned to Group 3 Control schools were matched for geographic location, size, grade level taught, and government expenditure per child. In one location (Ntungamo, there were insufficient control schools that had not received some form of assistance. In this case schools were selected from Bushenyi district, which has the same ethnic, geographic, and socio economic characteristics. The head teacher was interviewed and asked to identify those students who fit within the classification of OVC. All OVC were assigned to Group 4; All non OVC were assigned to Group 5 Data collected on school attendance and individual performance are all a matter of public record and are not confidential or privileged information. However, because of the stigma associated with HIV/AIDS and OVC status, codes were assigned to individual students and after the data were gathered, the key to the codes was destroyed. For focus group discussions, informed consent was obtained from guardians and individual OVC aged provided assent. A written signature or thumbprint was used for consent or assent forms. All Individuals were assigned a code for the recording of focus groups and not identified by name. A standard plan was developed for all researchers so that if any report of health problems, physical, mental, or sexual abuse were to arise, information on where to access appropriate assistance was available. 36

40 All quantitative data forms were transported to either Dar es Salaam (Tanzania) or Kampala (Uganda). Data were entered into a CSPro data base through double data entry, and files were transmitted electronically to Boston for cleaning and analysis. Qualitative data were translated and transcribed into Microsoft word, and analyzed using Nvivo to identify key themes. In addition to the individual school data collected for QUANT2, data were provided by the Ministry of Education and the National Examination Board in each country to enable comparison of schools at the district level. Data analysis plan All quantitative analyses were performed using SAS version 9.1. Basic descriptive statistics frequencies, means, and standard deviations where appropriate were calculated on main variables of interest to describe key characteristic of the study sample. For QUANT1, primary outcome of interest included student absenteeism, dropout rate, national examination pass rate and individual examination subjects scores. Proportions were compared using chi square analysis and comparison of means through one way analysis of variance. A bivariate analysis was conducted using simple logistic regression to assess the associations between the primary outcome and predictor variables. Odds ratios (OR) and the corresponding 95% Confidence Intervals (CI) are presented. Statistically significant associations of P < 0.05 are highlighted accordingly in the different analyses. A multivariate analysis examining the influence of country, gender, and student support status was performed through multiple logistic regression. For QUANT2, temporal trends of key indicators were assessed using Microsoft Excel. Due to limited sample size, no tests for statistical significance were conducted in any of the school level analyses. For the process of analyzing cost data, QUANT3, standard costing methods was employed to estimate the costs at each school of providing educational support for selected OVC beneficiaries over a determined time period. To estimate the average costs to educate a child in the intervention school, full program and implementation costs within an intervention school are divided by the number of directly supported OVC beneficiaries. For the qualitative component of the study, key informant interviews and focus group discussions were analyzed using grounded theory analysis to identify key recurring themes associated with different educational support programs. Qualitative analysis was conducted using Nvivo 9. RESULTS A total of 5,738 children were recruited to the study, including 1,930 in Tanzania and 3,808 in Uganda. As indicated in Table 1, 1,275 (22.2%) students were ever supported OVC (498 in Tanzania and 777 in Uganda), 2,252 (39.3%) were never supported OVC (721 in Tanzania and 1,531 in Uganda), while 2,209 (38.5%) represented Non OVC (710 in Tanzania and 1,499 in Uganda). 37

41 Table 1: Sample Characteristics by Intervention Uganda Total number of students 3,808 Block Grant (Africare) supported OVC 256 Block Grant (ICOBI) supported OVC 257 Scholarship (AVSI) supported OVC 264 Control OVC (never supported) 1,531 Non OVC 1,500 Tanzania Total number of students 1,930 Block Grant (Africare) supported OVC 243 Scholarship (CRS) supported OVC 255 Control OVC (never supported) 721 Non OVC 711 Two unforeseen challenges were posed with the sample size for the CRS students in the Tanga region of Tanzania. Firstly, the concentration of students receiving scholarship support was very low in any given area, with the result that no schools in the originally designated area of Tanga Urban had the required 50 students. This necessitated a much larger school sample of 34 schools, and made it impossible to obtain matched schools. Thus the total sample size in Tanzania was reduced accordingly. Secondly, in order to reach the desired sample size of 250 supported OVC, sampling was required in the much more rural Muheza district, which introduced a wider variation in the characteristics of students. During the data collection, it was also observed that for children receiving direct scholarships there were discrepancies between the records of children supported, and the records of schools concerning children actually attending school. This is discussed in detail later in this report. An unforeseen challenge also arose with AVSI in Uganda, where the number of students in rural districts was inadequate to be able to achieve the sample size required, and the schools selected were changed to the largely urban and peri urban areas around Kampala and Entebbe, with approximately half the schools being church schools and the other half being government schools. General Characteristics of the Population As noted above, a total of 1,930 students were studied in Tanzania and 3,808 in Uganda. Gender characteristics of the students were similar in both countries with 52.5% male in Tanzania and 52.6% male in Uganda. Students at every grade level in Tanzania were older than their counterparts in Uganda which reflects the difference in school enrolment age in the two countries. In Uganda, children begin primary school at 6 years of age, whereas in Tanzania they begin primary school at 7 years of age. The total number of intervention children for each NGO met the sampling framework requirements. The breakdown of OVC ever supported vs. OVC never supported vs. Non OVC in Uganda matched the expected sampling frame, while the proportion in Tanzania was skewed 38

42 towards the OVC ever supported, reflecting the problems in recruiting control schools to match CRS schools. The availability of records turned out to be a major challenge, especially in relation to attendance, performance, and drop out information. The more recent the data sought, the higher the percentage of records available. On questioning of school teachers, headmasters, and administrators, this was explained by the fact that many teachers kept records at their homes instead of at the school. Although we were able to retrieve many of these record books, the turn over of teachers was high, with many moving within two or three years of appointment. When that happened, the school records frequently moved with the teacher and were lost. Additionally, because ICOBI did not start its programming at the same time as the other NGOs, data were collected retrospectively for 2009 and QUANT1 Table 2: Study Sample Characteristics Mean Student Age (years) F1/S1 Student (Std Dev) F2/S2 Student (Std Dev ) F3/S3 Student (Std Dev ) F4/S4 Student (Std Dev ) Gender Male (%) Female (%) Student Status OVC Ever Supported (%) OVC Never Supported (%) Non OVC (%) Student NGO Support Africare (%) ICOBI (%) AVSI (%) CRS (%) None (%) Availability of Student Records 2006 (%) 2007 (%) 2008 (%) 2009 (%)* Tanzania (n = 1930) Uganda (n = 3808) 15.8 (1.4) 16.8 (1.4) 17.6 (1.4) 18.3 (1.2) 1013 (52.5%) 917 (47.5%) 498 (25.8%) 721 (37.4%) 711 (36.8%) 243 (12.6%) 255 (13.2%) 1432 (74.2%) 232 (12.0%) 747 (38.7%) 1464 (75.9%) 1723 (89.3%) *Student records for ICOBI (Uganda) were captured up until 2010 Student absenteeism rates 14.5 (2.0) 15.6 (1.9) 16.6 (1.9) 17.5 (1.9) 2003 (52.6%) 1805 (47.4%) 777 (20.4%) 1531 (40.2%) 1500 (39.4%) 256 (6.7%) 257 (6.7%) 264 (6.9%) 3031 (79.6%) 662 (17.4%) 1130 (29.7%) 2670 (70.12%) 2605 (68.4%) Absenteeism rates are presented as the total number of days a student was absent compared to the total number of days in the academic year. These data were collected from individual 39

43 class registers in which attendance was noted on a daily basis for both morning and afternoon attendance. Although the intention had been to compare absences for students in the morning and the afternoon, it became clear that despite the policy of schools to mark attendance twice daily, teachers took attendance only once a day. Therefore, students who were present when attendance was taken were marked as present for the entire day, even if they absented themselves for a later part of the day. An example of a class attendance register can be seen in Figure 2. Figure 2: Example of Class Attendance Register Absenteeism rates were not collected by the Ministry of Education in either country. Instead the number of days present per class per month were totaled and reported, so it is not possible to compare our results to national statistics. Mean absenteeism rate was calculated by averaging absenteeism rate between the different groups. The results, presented in Table 3, show a significant difference overall between groups, with ever supported OVC (5.1%) having much better attendance rates than never supported OVC (6.4%). Also, never supported OVC (6.4%) had much worse attendance rates than non OVC (5.5%). However, we found marked differences between Tanzania and Uganda. In Tanzania, children who were ever supported (6.5%) demonstrated worse attendance rates than their never supported counterparts (5.7%) and no significant difference as compared to non OVC. On the other hand, in Uganda, ever supported OVC (3.6%) showed much better attendance rates than either their never supported OVC (6.8%) or non OVC (6.1%) counterparts. 40

44 Table 3: Mean Student Absenteeism Rate, Stratified by Country OVC Ever Supported OVC Never Supported Non OVC Mean Student Absenteeism Rate ( ) Tanzania (n=1783) Uganda (n=2792) Total (n=4575) 6.5%* 3.6%* 5.1%* * Student status significantly different at p < %* 6.1% 6.8%* 5.1%* 6.4%* + 5.5% + + Student status significantly different at p < 0.05 Figure 3 shows a graphical representation of overall mean student absenteeism rate between the different groups from 2006 to The data indicates that non supported OVC have higher absenteeism rates compared to OVC supported and non OVC students, which is statistically significant from 2006 to Figure Figure 3: Mean 2: Student Absenteeism Rate by by Type of of Student Status in in Uganda and Tanzania 7.0% * + 6.5% 6.0% * + * + 5.5% 5.0% 4.5% * + * + * + 4.0% 2006 (n=689) (n=1389) (n=3098) 2009 (n=3369) 2009 OVC Supported OVC Non Supported Non OVC * Student status significantly different at p < Student status significantly different at p < 0.05 If these numbers are disaggregated by country, we notice quite different trends between Uganda and Tanzania. The Uganda data (Figure 4), clearly illustrate that supported OVC have lower absenteeism rates than either non supported OVC, or their non OVC counterparts. On the other hand we can see 41

45 that OVC who receive no support have significantly higher absenteeism (p <.05) rates than either non OVC, and supported OVC. Figure 4: Mean Student Absenteesm Rate by Type of Student for Uganda 8.0% 6.0% 4.0% 2.0% Ever Supported Never Supported Figure 5: Mean Student Absenteeism Rate by Type of Student for Tanzania 8.0% 6.0% 4.0% 2.0% Ever Supported Never Supported Non OVC This is in contrast to the findings in Tanzania (Figure 5), which present a much more confusing picture, with the curves beginning where we would expect in 2006, but then overlapping during the next three years. There is no statistical significance between the curves in Tanzania and we can only conclude that there are factors affecting attendance that were beyond our study to measure. Table 4 shows the breakdown of absenteeism by gender, and overall, we observe that female students have slightly lower absenteeism rates compared to male students throughout the different groups, although there was no statistical significance between genders. Table 4: Mean Student Absenteeism Rate by Student Status, Stratified by Gender Male Mean Student Absenteeism Rate (Uganda and Tanzania) Non OVC (n=1809) 5.7% OVC Never Supported (n=1871) 5.3% OVC Ever Supported (n=895) 6.3% Mean Student Absenteeism Rate (Uganda) Non OVC (n=1142) OVC Never Supported (n=1198) OVC Ever Supported (n=452) Mean Student Absenteeism Rate (Tanzania) Non OVC (n=667) OVC Never Supported (n=673) OVC Ever Supported (n=443) * Gender significantly different at p < %* 7.4% 5.8% 5.4% 3.8% 7.1% Female 5.3% 4.9% 5.9% 6.0%* 6.2% 5.5% 4.7% 3.5% 6.0% If the data are disaggregated by country, we see that in Uganda, female non OVC students (6.0%) had significantly lower mean absenteeism rate compared to the absenteeism rate found among male non OVC students (6.8%). 42

46 If the absenteeism rates are examined over a four year period broken down by gender we see that the lower rates of female absenteeism in Uganda were consistent over the entire period. Interestingly however, in Tanzania we observe that females had a higher absenteeism rate than males (although not statistically significant) in 2006, but by 2009 that rate had reversed such that girls have a statistically significant lower rate of absenteeism than boys. Figure 6: Uganda Absenteeism by Gender over Time Figure 7: Tanzania Absenteeism by Gender over Time 43

47 Absenteeism results across NGOs provide interesting differences (Table 5). Firstly, neither of the NGOs in Tanzania demonstrates any significant difference in absenteeism in the aggregate or when broken down by gender, although absenteeism among OVC at CRS supported schools (supported or not) is much higher for males than females. Secondly, in Uganda, all three NGOs show dramatic reductions in absenteeism over both comparison groups, however children supported by Africare did not fare statistically better than OVC who attended the same schools and were not supported. Table 5: Mean Student Absenteeism Rate, Stratified by NGO Support and Gender OVC Ever Supported OVC Never Supported Non OVC Mean Absenteeism Rate ( ) Africare UG (n=721) Male (n=404) Female (n=317) ICOBI UG (n=1010) Male (n=515) Female (n=495) AVSI UG (n=1059) Male (n=542) Female (n=517) Africare TZ (n=1152) Male (n=614) Female (538) CRS TZ (n=631) Male (n=338) Female (n= %* 2.5%* + 4.0% 3.7%* 4.0%* 3.6%* 3.6%* 4.3%* 2.7%* + 6.2% 6.0% 6.3% 7.0% 8.1% 5.6% * Student status significantly different at p < % 5.7%* 5.5% 8.4%* + 8.9%* + 7.9%* + 6.1%* 7.2%* + 5.2%* 5.8% 5.6% 6.0% 5.4% 6.2% 4.4% 6.1%* 5.8% + 6.4% 4.5% + 4.9% + 4.1% + 5.0%* 5.3% + 4.7% + 6.3% 6.5% 6.0% 5.7% 5.7% 5.7% + Student status significantly different at p < 0.05 The fact that girls had similar, or in many cases, significantly better attendance rates than boys is surprising given the repeated assertion that girls miss more school because of their monthly menstrual cycle and either the lack of sanitary pads, or inadequate facilities to change sanitary pads at schools. It is also important to note that the differences are present whether there was an NGO providing assistance to girls with their menstrual cycle or not. Student dropout rates Dropout rates in this study are significantly below the national average rates (Tanzania 2.2%, Uganda estimated at 14 15%). This is an anticipated result of the sampling approach, which biases the result toward representing those who go to school and have fewer drop outs than would be expected. However, the same bias applies across each group of children sampled and therefore direct comparisons are possible. In addition, the sampling size is sufficiently large that statistically significant results can be demonstrated. 44

48 Table 6: Student Drop Out Rate, Stratified by Country OVC Ever Supported OVC Never Supported Non OVC Drop Out Rate ( ) Tanzania (n=1926) Uganda (n=3753) Total (n=5679) 12/496 (2.4%)* 85/768 (11.1%)* 97/1264 (7.7%)* * Significantly different at p < /720 (1.0%)* 222/1511 (14.7%)* 229/2231 (10.3%)* 4/710 (0.6%)* 126/1474 (8.6%)* 130/2184 (6.0%)* In addition, the striking difference between the two countries is not easily explained. Anecdotal evidence was that with the introduction of USE in Uganda, there has been an increase in overall school enrolments (which is borne out in QUANT 2 data), but that because there are still significant fees present, many parents find that they cannot continue to pay for schooling. Conversely in Tanzania, where students know that they have to pay for their entire education, there is a better understanding of the costs before entering into school. In addition, there has been a very high level political campaign in the study districts in the last four years firstly in building new schools, and then by senior politicians visiting districts and wards to emphasize the importance of attendance and reinforce the penalties for truancy and drop out. It is likely that one or both of these factors have affected the dropout rates, but the extent is not clear. Dropout rates for OVC in general are higher than for non OVC, and never supported OVC have a higher dropout rate than supported OVC. All these differences are significant at the alpha 0.05 level. Figure 8 illustrates that despite the differences between the groups, there is a disturbing upward trend in dropout rates for all groups over the four years with non supported OVC faring the worst. 45

49 Figure Figure 8: Student 5: Drop Drop Out Out Rate Rate by by Type of of Student Status in Uganda and Tanzania 8.0% 7.0% 6.0% 5.0% 4.0% 3.0% 2.0% 1.0% 2006 (n=900) (n=1890) (n=4159) 2009 (n=4358) 2009 OVC Supported OVC Non Supported Non OVC When the annual trends are disaggregated by country however, the results are surprising. In Tanzania, ever supported OVC have a noticeably higher rate of drop out than either of the comparison groups, although the numbers are not enough to produce statistical significance on an annual basis. However, the overall trend would appear to show a declining rate of drop out. In Uganda however, the numbers reflect the aggregate trend. 2.0% 1.0% Figure 9: Mean Student Dropout Rate by Type of Student for Tanzania 20.0% 10.0% Figure 10: Mean Student Dropout Rate by Type of Student for Uganda 0.0% Ever Supported OVC Never Supported OVC Non OVC 0.0% Ever Supported OVC Never Supported OVC Non OVC 46

50 We also calculated odds ratios of the likelihood of dropping out for ever supported and never supported OVC compared to their non OVC peers. As shown in Table 7, never supported OVC in Tanzania were not statistically more likely to drop out. Table 7: Drop Out Rate by Student Status, Proportion (%) Odds Ratio (95% CI) Country Tanzania Non OVC [Ref.] 4/710 (0.6%) 1.00 OVC Never Supported 7/720 (1.0%) 1.73 ( ) OVC Ever Supported 12/496 (2.4%) 4.38 ( )* Uganda Non OVC [Ref.] 126/1474 (8.6%) 1.00 OVC Never Supported 222/1511 (14.7%) 3.20 ( )* OVC Ever Supported 85/768 (11.1%) 1.33 ( ) Total Non OVC [Ref.] 130/2184 (6.0%) 1.00 OVC Never Supported 229/2231 (10.3%) 1.81 ( )* OVC Ever Supported 97/1264 (7.7%) 1.31 ( ) Gender Male Non OVC [Ref.] 71/1194 (6.0%) 1.00 OVC Never Supported 116/1174 (9.9%) 1.73 ( )* OVC Ever Supported 44/615 (7.2%) 1.22 ( ) Female Non OVC [Ref.] 59/990 (6.0%) 1.00 OVC Never Supported 113/1057 (10.7%) 1.89 ( )* OVC Ever Supported 53/649 (8.2%) 1.40 ( ) NGO Support Africare UG Non OVC [Ref.] 48/491 (9.8%) 1.00 OVC Never Supported 88/493 (17.9%) 2.01 ( )* OVC Ever Supported 39/247 (15.8%) 1.73 ( )* ICOBI UG Non OVC [Ref.] 43/475 (9.1%) 1.00 OVC Never Supported 98/517 (19.0%) 2.35 ( )* OVC Ever Supported 31/255 (12.2%) 1.39 ( ) AVSI UG Non OVC [Ref.] 35/508 (6.9%) 1.00 OVC Never Supported 36/501 (7.2%) 1.05 ( ) OVC Ever Supported 14/264 (5.3%) 0.76 ( ) Africare TZ Non OVC [Ref.] 3/466 (0.6%) 1.00 OVC Never Supported 4/469 (0.9%) 1.33 ( ) OVC Ever Supported 7/242 (2.9%) 4.60 ( )* CRS TZ Non OVC [Ref.] 1/244 (0.4%) 1.00 OVC Never Supported 3/250 (1.2%) 2.95 ( ) OVC Ever Supported 5/524 (2.0%) 2.34 ( ) * Student status significantly different at p <

51 However, ever supported OVC were 4.4 times (95% CI ) more likely to drop out. Conversely, never supported OVC in Uganda were 3.2 times (95% CI ) more likely to drop out. A multivariate model was created to predict student drop out (Table 8). Secondary school students in Uganda are eleven times more like to drop out compared to students in Tanzania. This is not surprising since dropout rates in Uganda are considerably higher than dropout rates in Tanzania. When analyzing the effect of gender, there seems to be no difference in dropout rates between male and female students. Finally, when comparing the impact of educational support, we see that never supported OVC students are almost twice as likely to drop out when compared on non OVC students OR 1.38 (95% CI ). The adjusted odds ratio is slightly lower for ever supported OVC students compared to non OVC students OR 1.46 (95% CI ) suggesting that the support they receive may facilitate a slight reduction in the likelihood of dropping out of school. Table 8: Multivariate Analysis to Predict Drop Out (N=5679) Adjusted Odds Ratio Ɨ (95% CI) Country Tanzania [Ref.] 1.00 Uganda ( )* Gender Male [Ref.] 1.00 Female ( ) Student Status Non OVC [Ref.] 1.00 Never Supported OVC 1.83 ( )* Ever Supported OVC 1.46 ( )* Ɨ Adjusted odds ratio by logistic regression analysis * Significantly different at p < 0.05 National Examination Scores Tanzania offers two nationally scored examinations for all students. The first is at the end of Form II, the second at the end of Form IV. Uganda only offers one nationally scored exam at the end of Standard IV. Because these exams are externally marked, it is possible to compare student performance between schools because there should be no variation in the quality of the grading. Each country has a core curriculum which includes English, Math, Physics, and Chemistry, with electives possible in Civics and Kiswahili (Tanzania), and Geography and Biology (Uganda). In addition to the individual grades for each subject, the scores are aggregated to a single letter grade. Each country applies a different scoring system, so cross country comparison is inappropriate. Both countries also award an aggregate score to students at Form IV/Standard IV which is used to determine an overall success at this level. 48

52 Form II Standard Exams Table 9 and Table 10 analyze Form II exams in Tanzania. Table 9 presents the national aggregate scores and demonstrates a surprisingly low overall pass rate of Form II exams for all groups of students. However never supported OVC score statistically worse than their peers OR 0.69 (95% CI ). This is reflected in the numbers for both male and female non supported OVC, although only male students score significantly worse than others OR.65 (95% CI ). Table 9: Tanzanian Form II National Exam Pass Rate by Student Status, Proportion (%) Odds Ratio (95% CI) Country Tanzania Non OVC [Ref.] 118/511 (23.1%) 1.00 OVC Never Supported 85/497 (17.1%) 0.69 ( )* OVC Ever Supported 51/227 (22.5%) 0.97 ( ) Gender Male Non OVC [Ref.] 76/276 (27.5%) 1.00 OVC Never Supported 58/290 (20.0%) 0.65 ( )* OVC Ever Supported 31/113 (27.3%) 0.99 ( ) Female Non OVC [Ref.] 42/235 (17.9%) 1.00 OVC Never Supported 27/207 (13.0%) 0.69 ( ) OVC Ever Supported 20/114 (17.5%) 0.98 ( ) * Student status significantly different at p < 0.05 Form II exams are marked out of 100 points with the following ranking: = F = D = C = B = A In looking at the individual examination scores (Table 10) results are uniformly low, however, with the exception of Math (with an average of F), most students obtain passing grades (D or C). However, supported OVC consistently outscored their non supported peers on every subject with the results being statistically significant for English, Civics, Physics, Chemistry, and Biology. 49

53 Table 10: Tanzanian Form II National Examination Scores, OVC Supported OVC Non Supported Non OVC Mean Exam Scores (SD) Ɨ English (n=1339) Civics (n=1340) Geography (n=613) Math (n=1340) Physics (n=1274) Chemistry (n=1293) Biology (n=613) Kiswahili (n=1340) 30.5 (17.91)* (14.5)* 47.1 (15.5) 19.2 (18.2)* (15.5)* 36.1 (13.9)* 37.6 (14.3) 45.8 (18.4) 25.9 (12.9)* 31.5 (12.9) 46.2 (16.7) 15.9 (15.5)* 33.0 (15.7)* 32.1 (15.1)* 36.8 (15.4) 47.9 (17.7) * Student status significantly different at p < 0.05 between designated groups + Student status significantly different at p < 0.05 between designated groups Ɨ Exam scores are out of 100 points (higher points indicate better exam performance) Note: Form II National Examinations are not conducted in Uganda 27.5 (15.1) (13.5)* 46.1 (14.7) 15.5 (14.7) (15.5) 33.6 (14.5) 36.9 (12.3) 46.6 (18.5) Finally, looking at scores by the type of NGO support OVC received, we observed that there was no difference between supported children from scholarship and block grant programs, however, never supported OVC at Africare scored statistically worse than their supported OVC counterparts OR 0.65 (95% CI ). Table 11: Tanzanian Form II National Exam Pass Rate by Student Status, Proportion (%) Odds Ratio (95% CI) NGO Support Africare TZ Block Grant Non OVC [Ref.] 83/400 (20.8%) 1.00 OVC Never Supported 57/390 (14.6%) 0.65 ( )* OVC Ever Supported 27/154 (17.5%) 0.81 ( ) CRS TZ Scholarship Non OVC [Ref.] 35/111 (31.5%) 1.00 OVC Never Supported 28/107 (26.2%) 0.77 ( ) OVC Ever Supported 24/73 (32.9%) 1.06 ( ) * Student status significantly different at p < 0.05 As Form II pass rates are disaggregated by gender (Table 12), we see that throughout the three different types of students, female students are half as likely to pass their Form II exams as compared to their male counterparts. This gender analysis is statistically significant for both non OVC students OR 0.57 (95% CI ) and never supported OVC students OR 0.60 (95% CI ). Although the gender analysis between ever supported OVC students is not statistically significant, the results indicate a strong association that fewer female students pass their Form II exams compared to male students. 50

54 Table 12: Tanzanian Form II National Exam Pass Rate by Gender, Proportion (%) Odds Ratio (95% CI) Student Status Non OVC Male [Ref.] 76/275 (27.6%) 1.00 Female 42/234 (18.0%) 0.57 ( )* OVC Never Supported Male [Ref.] 58/287 (20.2%) 1.00 Female 27/206 (13.1%) 0.60 ( )* OVC Ever Supported Male [Ref.] 31/113 (27.4%) 1.00 Female 20/114 (17.5%) 0.56 ( ) * Student status significantly different at p < 0.05 This is also reflected in Table 13 which breaks down individual subject scores by gender and shows girls scoring worse than boys in every subject with the difference being statistically significant in 5 of 8 subjects. Table 13: Form II National Examination Scores between Male and Female Students, Tanzania Female Male Mean Exam Scores Ɨ ( ) English (n=1340) Civics (n=1341) Geography (n=613) Math (n=1341) Physics (n=1275) Chemistry (n=1294) Biology (n=613) Kiswahili (n=1341) 26.7 (13.5) 30.2 (12.4)* 44.8 (14.6)* 14.7 (13.9)* 33.7 (14.1) 32.6 (13.7)* 35.0 (12.5)* 46.9 (19.4) 28.1 (16.1) 33.0 (14.2)* 47.8 (16.3)* 17.9 (17.1)* 34.8 (16.7) 34.3 (15.3)* 38.9 (14.8)* 47.0 (17.1) * Student status significantly different at p < 0.05 Ɨ Exam scores are out of 100 points (higher points indicate better exam performance) Note: Form II National Examinations are not conducted in Uganda Form IV Standard Exams in Tanzania Form IV exams do not receive a percentage mark and are simply graded as A, B, C, D and F. The national examination board calculates an average score and then assigns an aggregate score which is ranked as Division I, Division II, Division III, Division IV, or Fail. As with Form II examinations, the overall aggregate pass rate for Form IV was extremely low (Table 14), less than 20%. Unfortunately, so few students in Tanzania sat for the Form IV examinations that tests of statistical significance were not possible. However, when gender is taken into account, female students had a significantly lower pass rate than male students (12.5% versus 24.3%). The low numbers also prevent us from identifying significant differences between the NGO approaches (Table 15). 51

55 Table 14: Tanzanian Form IV National Exam Pass Rate by Student Status, Proportion (%) Odds Ratio (95% CI) Country Tanzania Non OVC [Ref.] 12/61 (19.7%) 1.00 OVC Never Supported 2/20 (10.0%) 0.45 ( ) OVC Ever Supported 2/36 (5.6%) 0.24 ( ) Gender Male Non OVC [Ref.] 9/37 (24.3%) 1.00 OVC Never Supported 1/9 (11.1%) 0.39 ( ) OVC Ever Supported 1/14 (7.1%) 0.24 ( ) Female Non OVC [Ref.] 3/24 (12.5%) 1.00 OVC Never Supported 1/11 (9.1%) 0.70 ( ) OVC Ever Supported 1/22 (4.6%) 0.33 ( ) * Student status significantly different at p < 0.05 Table 15: Tanzanian Form IV National Exam Pass Rate by Student Status, Proportion (%) Odds Ratio (95% CI) NGO Support Africare TZ Block Grant Non OVC [Ref.] 12/60 (20.0%) 1.00 OVC Never Supported 2/19 (10.5%) 0.47 ( ) OVC Ever Supported 2/32 (6.3%) 0.27 ( ) CRS TZ Scholarship Non OVC [Ref.] 0/1 (0%) 1.00 OVC Never Supported 0/1 (0%) OVC Ever Supported 0/4 (0%) * Student status significantly different at p < 0.05 As indicated in Table 16, we allotted ordinal scores to the letter grades (1=A, 2=B, 3=C, 4=D, 5=F) and calculated the mean. Table 16: Form IV National Examination Scores, Tanzania OVC Supported OVC Non Supported Non OVC Mean Exam Scores (SD) Ɨ English (n=152) Civics (n=152) Math (n=152) Physics (n=45) Chemistry (n=45) Kiswahili (n=151) 4.2 (0.7) 4.3 (0.8) (0.9) 4.2 (0.7) 4.3 (0.7) 4.4 (0.8) + * Significantly different at p < Student status significantly different at p < 0.05 Ɨ Exam scores are out of 5 points: 1=A 2=B 3=C 4=D 5=F (lower points indicate better exam performance) 4.5 (0.7)* 4.5 (0.7)* 4.9 (0.4) 3.8 (0.8) 3.8 (0.8) 4.5 (0.7)* 4.1 (0.7)* 3.9 (0.8)* (0.6) 4.2 (0.8) 4.1 (0.9) 3.9 (0.7)* + 52

56 Generally, OVC in Tanzania scored worse than their non OVC counterparts, and with the exception of Physics and Chemistry, supported OVC scored worse than non supported OVC. It is perhaps important to note that the two subjects deemed by teachers and key informants as the most important subjects for students (Kiswahili and Civics) scored significantly worse for supported OVC than other groups. The overall pass rates above are not representative of the national statistics, which showed a 49% pass rate for Form IV in It should be noted however, that the schools assigned to Africare were all deemed to be the most failing schools in the district and therefore the most in need. The numbers for 2009 were the first Form IV exams that students in these schools took, so that there are no historical data for comparison. It is also intriguing that students can perform so well at Form II but so badly at Form IV. To explore this further we engaged in a key informant interview with Eugine Kafanabo, Professor of Education at the School of Education in the University of Dar es Salaam. She indicated that in the last two years (during this study period), the Form IV exams have changed to reflect a competency based curriculum, whereas the Form II exams have remained a more general measure of knowledge and rote learning. In addition, it was pointed out that with the recruitment of large numbers of inexperienced teachers who are awaiting formal training, their own competency covers subject matter on Form II, but not Forms III and IV, with the result that their students can be ill prepared for the exams. The gender analysis for Form IV exam pass rate is presented in Table 17 below. Due to the small sample size, no statistical significance can be found between male and female students through the different types of students. It is interesting to note that the trend indicates the females are less likely to pass their Form IV exams when compared to male students. Table 17: Tanzanian Form IV National Exam Pass Rate by Gender, Proportion (%) Odds Ratio (95% CI) Student Status Non OVC Male [Ref.] 9/37 (24.3%) 1.00 Female 3/24 (12.5%) 0.44 ( ) OVC Never Supported Male [Ref.] 1/9 (11.1%) 1.00 Female 1/11 (9.1%) 0.80 ( ) OVC Ever Supported Male [Ref.] 1/14 (7.1%) 1.00 Female 1/22 (4.6%) 0.62 ( ) * Student status significantly different at p <

57 Overall, individual scores for girls were equal to those for boys with no statistical significance. Table 18: Form IV National Examination Scores between Male and Female Students, Tanzania Female Male Mean Exam Scores Ɨ ( ) English (n=152) Civics (n=152) Math (n=153) Physics (n=45) Chemistry (n=45) Kiswahili (n=151) 4.2 (0.8) 4.2 (0.8) 4.8 (0.5) 4.2 (0.8) 4.3 (0.7) 4.2 (0.8) 4.3 (0.8) 4.2 (0.8) 4.6 (0.9) 4.2 (0.8) 4.1 (0.9) 4.2 (0.7) * Student status significantly different at p < 0.05 Ɨ Exam scores are out of 5 points: 1=A 2=B 3=C 4=D 5=F (lower points indicate better exam performance) Standard IV National Exams in Uganda As noted above, Uganda does not offer standardized examinations at the national level in Standard II. All children sit for 9 Standard IV exams of which six are compulsory subjects and three are electives. Exams are scored out of a possible 9: Distinction 1 = 1 Distinction 2 = 2 Credit 3 = 3 Credit 4 = 4 Credit 5 = 5 Credit 6 = 6 Pass 7 = 7 Pass 8 = 8 Fail = 9 Once again, an aggregate score is calculated by the national examinations board. To calculate the aggregate score the worst score is discarded and the remaining 8 subjects are totaled. A score of 8 is therefore the maximum possible aggregate score, while a score of 64 represents a failure in every single subject. Failing aggregate scores are extremely rare (less than.5% of all students). In Table 19 below it was decided to use an aggregate score of 56 or less to represent successful academic performance. However because of the difference between the two country systems, this passing rate cannot be compared to the rates from Tanzania. The number of students taking Standard IV exams in Uganda was much larger than in Tanzania, which helps to draw better conclusions, and we see that supported OVC perform as well as their non OVC peers, while non supported OVC performed statistically worse than their non OVC counterparts OR 0.78 (95% CI ). What is notable is that male OVC who received support scored significantly better than male students in the other groups OR 1.6 (95% CI ) and also scored better than female students. 54

58 Table 19: Ugandan Standard IV National Exam Pass Rate by Student Status, Proportion (%) Odds Ratio (95% CI) Country Uganda Non OVC [Ref.] 461/710 (64.9%) 1.00 OVC Never Supported 320/541 (59.1%) 0.78 ( )* OVC Ever Supported 259/386 (67.1%) 1.10 ( ) Gender Male Non OVC [Ref.] 289/421 (68.7%) 1.00 OVC Never Supported 194/296 (65.5%) 0.87 ( ) OVC Ever Supported 154/198 (77.8%) 1.60 ( )* Female Non OVC [Ref.] 172/289 (59.5%) 1.00 OVC Never Supported 126/245 (51.4%) 0.72 ( ) OVC Ever Supported 105/188 (55.9%) 0.86 ( ) * Student status significantly different at p < 0.05 When looked at by type of NGO support (Table 20), never supported students at Africare and ICOBI generally did much worse than non OVC or supported OVC, but the difference was only significant for students at ICOBI OR 0.38 (95% CI ) Table 20: Ugandan Standard IV National Exam Pass Rate by Student Status, Proportion (%) Odds Ratio (95% CI) NGO Support Africare UG Block Grant Non OVC [Ref.] 182/285 (63.9%) 1.00 OVC Never Supported 87/145 (60.0%) 0.56 ( ) OVC Ever Supported 133/213 (62.4%) 0.94 ( ) ICOBI UG Block Grant Non OVC [Ref.] 94/156 (60.3%) 1.00 OVC Never Supported 49/135 (36.3%) 0.38 ( )* OVC Ever Supported 38/59 (64.4%) 1.19 ( ) AVSI UG Scholarship Non OVC [Ref.] 185/269 (68.8%) 1.00 OVC Never Supported 184/261 (70.5%) 1.09 ( ) OVC Ever Supported 88/113 (77.9%) 1.60 ( ) * Student status significantly different at p < 0.05 Table 21 presents the scores for the individual core subjects in Uganda, and it is very striking in this setting that there is no noticeable difference between any of the groups. The scores between OVC supported, OVC non supported and non OVC are almost identical with very consistent standard deviations. We believe that this consistency in Uganda represents the fact that all the approaches in Uganda provided their funding directly to the school with clear criteria for how the funding should be used. This in turn supports the frequently stated observation during interviews that the block grant approach provided noticeable benefits to all children, not just the children supported by the program. 55

59 Table 21: Standard IV National Examination Scores, Uganda OVC Supported OVC Non Supported Non OVC Mean Exam Scores (SD) ǂ English (n=738) Geography (n=356) Math (n=738) Physics (n=738) Chemistry (n=736) Biology (n=355) 6.2 (1.7) 6.1 (1.5) 7.2 (1.5) 8.3 (1.2) 8.4 (1.1) 7.6 (1.4) 6.0 (1.9) 6.1 (1.8) 7.2 (1.8) 8.0 (1.7) 8.2 (1.6) 7.4 (1.8) * Significantly different at p < 0.05 ǂ Exam scores are out of 9 points: 1=D1 2=D2 3=C3 4=C4 5=C5 6=C6 7=P7 8=P8 9=F (lower points indicate better exam performance) 5.9 (1.8) 5.9 (1.6) 7.1 (1.7) 8.0 (1.5) 8.2 (1.3) 7.4 (1.5) It should, however, be noted that the mean scores are all very low and do not reflect outstanding academic performance in any setting. The gender analysis for Standard IV exam pass rate in Uganda is presented in Table 22 below. While we were unable to get statistical significance in Tanzania due to the small number of students taking the Form IV exam, there is sufficient data from Uganda to compare the effect of gender between the different types of students. In all the different types of students non OVC, OVC never supported, and ever supported OVC students females students are significantly less likely to pass their Form IV exams compared to male students. Table 22: Ugandan Standard IV National Exam Pass Rate by Gender, Proportion (%) Odds Ratio (95% CI) Student Status Non OVC Male [Ref.] 289/421 (68.6%) 1.00 Female 172/289 (59.5%) 0.67 ( )* OVC Never Supported Male [Ref.] 193/295 (65.4%) 1.00 Female 126/245 (51.4%) 0.56 ( )* OVC Ever Supported Male [Ref.] 154/198 (77.8%) 1.00 Female 105/188 (55.9%) 0.36 ( )* * Student status significantly different at p < 0.05 It is very interesting to note that in Uganda girls perform consistently worse in each subject with the exception of English (Table 23). This is in contrast to Tanzania where the scores are equal. It would appear therefore that the difference seen when Tanzania and Uganda are combined represents the weighting of Uganda. As noted in the results for Tanzania, the scores in the selected schools are extremely low, and it is possible that with such low scores it is simply not possible to observe gender differences. 56

60 Table 23: Form IV National Examination Scores between Male and Female Students, Uganda Female Male Mean Exam Scores Ɨ ( ) English (n=1667) Geography (n=841) Math (n=1667) Physics (n=1664) Chemistry (n=1661) Biology (n=838) 6.0 (1.8) 6.0 (1.6)* 7.1 (1.7)* 8.1 (1.5)* 8.3 (1.3)* 7.4 (1.6)* 6.1 (1.7) 5.5 (1.7)* 6.6 (1.9)* 7.5 (1.7)* 7.9 (1.6)* 6.8 (1.8)* * Student status significantly different at p < 0.05 Ɨ Exam scores are out of 9 points: 1=D1 2=D2 3=C3 4=C4 5=C5 6=C6 7=P7 8=P8 9=F (lower points indicate better exam performance) A multivariate analysis model was constructed to look at predictors of passing the national Form IV exams (Table 24). As the national exams between the two countries have different scoring mechanisms, it s not surprising to see that students in Uganda are eleven times more likely to pass their From IV exams compared to their Tanzanian counterparts. In assessing gender as a predictor of passing From IV exams, we see stark differences between male and females students. Girls are half as likely to pass their Form IV exams compared to boys OR 0.55 (95% CI ). The impact of educational support can be seen as having a positive effect on passing the Form IV exams. Whereas never supported OVC are 20% less likely to pass their exam OR 0.78 (95% CI ), there is no statistical significance between ever supported OVC and non OVC students. This finding suggests that the educational support received by OVC students puts them almost on the same passing ratio as non OVC students. Table 24: Multivariate Analysis to Predict Passing of National Form IV Examination (N=1752) Adjusted Odds Ratio Ɨ (95% CI) Country Tanzania [Ref.] 1.00 Uganda ( )* Gender Male [Ref.] 1.00 Female 0.55 ( )* Student Status Non OVC [Ref.] 1.00 Never Supported OVC 0.78 ( )* Ever Supported OVC 1.10 ( ) Ɨ Adjusted odds ratio by logistic regression analysis * Significantly different at p <

61 QUANT2 School level data collected at intervention and control schools in Uganda The data presented below in Tables were collected at the individual schools through record review. This part of the study was in response to the repeated observations by school teachers, head teachers, and NGO staff, that the improvements that block grants brought to the school improved the quality of the school grounds, increased class rooms, brought physical improvements such as water, electricity, latrines, and also purchased essential laboratory and other equipment. It was frequently observed that these changes had improved the standing of the school, attracted new students and improved school performance. We hoped to be able to observe if the interventions had made any noticeable difference to the overall functioning of the school. In order to observe if changes had occurred, measures were taken for four years prior to when the interventions began. Because of the small sample size and large standard deviations, we have paired each table below with a graph to illustrate the trends over time. The sample size is too small to conclude statistical significance. Table 25 and Figure 11 represent the control schools in Uganda and show that after a reduction in total student enrolment between the years 2004 and 2005, student enrolment rose steadily after 2006, with the curve becoming steeper after Universal Secondary Education (USE) was introduced in With implementation of USE, student enrollment increased dramatically while the number of teachers and classrooms remained stagnant. With slight variability, the number of OVC registered in schools also rose steadily in schools since Notably, while the number of students increased, the resources available to the schools did not match the increased enrolment. Available budget initially rose after USE in 2007 but flattened the next year; average number of school teachers also rose slightly, but also reached a plateau the next year, while the student teacher ratio has been flat. Finally, during this time the average Standard IV pass rate (aggregate score of 56 or lower) has dropped from 60% in 2002 to 45% in Table 25: Non Supported Schools in Uganda (n=14) YEAR Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million UGX) Student/Teacher Ratio

62 Figure Figure 11: 8: Key KeySchool School Level Level Indicators Indicators for Non Supported for Non Supported Schools Schools in Uganda in Uganda (N=14) (N=14) Implementationof Universal Secondary Education (USE) Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million UGX) Table 26 and Figure 12 represent Africare schools in Uganda. The trends of total enrolment, number of OVC, funding, level of infrastructure and academic performance closely resemble those for the rest of the country. However, the student to teacher ratio has greatly increased since the introduction USE. Table 26: Africare Supported Schools in Uganda (n=5) YEAR Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million UGX) Student: Teacher Ratio Interven on Began 59

63 Figure 9: 12: Key School Level Indicators for for Africare Africare Supported Supported Schools Schools in Uganda in Uganda (N=5) (N=5) Approximate start date of support Implementationof Universal Secondary Education (USE) Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million UGX) Table 27 and Figure 13 present the results from the schools supported by ICOBI which is very similar to the trends for Africare. Table 27: ICOBI Supported Schools in Uganda (n=16) YEAR Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million UGX) Student: Teacher Ratio Interven on Began 60

64 Figure 10: 13: Key School Level Indicators for ICOBI for ICOBI Supported Supported Schools Schools in Uganda in Uganda (N=16) (N=16) Implementationof Universal Secondary Education (USE) Approximate start date of support Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million UGX) Schools supported by AVSI show some minor differences in characteristics. Firstly, the total enrolment is generally higher than the national average for Africare or ICOBI, and secondly, the average Form IV pass rate has remained constant for all eight years. Finally, student teacher ratios have also remained constant. Table 28: AVSI Supported Schools in Uganda (n=13) YEAR Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million UGX) Student: Teacher Ratio Interven on Began 61

65 Figure 11: 14: Key School Level Indicators for AVSI for AVSI Supported Supported Schools Schools in Uganda in Uganda (N=13) (N=13) Approximate start date of support Implementation of Universal Secondary Education (USE) Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million UGX) Table 29 and Figure 15 present the student to teacher ratios for the country as a whole, and the three NGOs. Both Africare and ICOBI are very similar and roughly parallel the national trend, although both have markedly higher student to teacher ratios than the national average. AVSI on the other hand stands out as having a consistent student to teacher ratio during the course of the last 8 years. Table 29: Student to Teacher Ratio in Uganda Schools (Pupils per Teacher) Non Supported Schools (N=14) Africare Supported School (N=5) ICOBI Supported Schools (N=16) AVSI Supported Schools (N=13)

66 Figure 12: Student to Teacher Ratio Among Different Schools in Uganda Figure 15: Student to Teacher Ratio Among Different Schools in Uganda Implementationof Universal Secondary Education (USE) Non Supported Schools (N=14) ICOBI Supported Schools (N=16) Africare Supported School (N=5) AVSI Supported Schools (N=13) Anecdotal stories of increases in enrolment because of block grants are not borne out by the data presented here, as the two most pure block grant schemes (Africare and ICOBI) do not show variance with the national trends. Increases noted by staff at assisted schools, are therefore more likely to be as a result of pre existing trends augmented by USE. Secondly, overall, with the exception of AVSI supported schools, there was a steady rise in student to teacher ratios, with the ratios noticeably higher for Africare and ICOBI. The similarities between Africare and ICOBI are not surprising as they both serve rural areas in the Southwest region of the country. The National data represent both urban and rural schools, but the majority of schools in Uganda are still rural, thus trends are more likely to resemble those seen with Africa and ICOBI. In addition to the rise in student to teacher ratios, national examination scores fell dramatically over the observation period in national schools, once again being paralleled in Africare and ICOBI schools. Observations from Western settings have demonstrated a direct relationship 63

67 between student to teacher ratios and the quality of education. We speculate that the increasing ratios were at least in part, responsible for the decreasing exam scores. Most importantly, block grants did not seem to have influenced this downward trend line. In formal discussions with the Ministry of Education in both countries, the topic of student teacher ratios was raised. Both Ministries recognize that this is an acute problem. Uganda has instituted an intense teacher training program, but it was reported that teachers leave the system at the same rate at which they are trained. Tanzania has attempted to address the challenge by training teachers for 3 weeks only (instead of the normal 4 years), using peacecorps volunteers, and by currently recruiting Cuban teachers. The fact that AVSI responded so differently is explained by two factors. Firstly, the urban and periurban nature of the sample means that it is easier to recruit and retain teachers, thus increasing and maintaining the student to teacher ratio. Secondly, the large percentage of private church schools in the sample means that performance was not comparable to other schools because of (1) Bias in student selection due to the religious nature of the school; and (2) The private fee structure of the school meaning that even when budgets from the state were limited, the school was able to raise additional funds that public schools could not. School level data collected at intervention and control schools in Tanzania Tables 30 to 32 present the data collected in Tanzania for the same factors. Unfortunately, the data are incomplete. On the national scale, with the advent of a new government in 2005, education policy was introduced that dictated the construction of a school in every ward. In Dodoma, where Africare has been active, all the schools were constructed during 2006 and Thus there are no historical records before 2007 making it impossible to formulate the same comparison as was done in Uganda. Tables and Figures present the comparison for control schools and Africare intervention schools in Dodoma district. These data suggest that there was little difference between the control schools and intervention schools for the three years of observations. Table 30: Control Schools in Tanzania (n=5) YEAR Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % 73 Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million TZS) Student: Teacher Ratio

68 Table 31: Africare Schools in Tanzania (n=5) YEAR Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % 64 Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million TZS) Student: Teacher Ratio Figure Figure 16: 13: School Key School Level Indicators Level Indicators for Non Supported for Schools Schools in Tanzania in Tanzania (N=5) (N=5) 250 Figure 17: 14: School Key School Level Level Indicators Indicators for Africare for Africare Supported Schools in Tanzania in (N=5) 250 Approximate start date of support Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million TZS) Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million TZS) The sampling framework for CRS schools was changed because of the highly dispersed nature of the OVC who received scholarships, making it impossible to find matching control schools. However, there was an additional year of data available as many schools were older. The schools had constant enrolment numbers over the period, with a gradual increase in the number of OVC in each school. Budgets were relatively flat or slightly increasing in line with student enrolment, while infrastructure and number of teachers appeared to decrease with time. Finally the pass rates worsened over time as student to teacher ratios increased. While it is not possible to draw conclusions from this small sample, it is interesting to note that the majority of CRS schools were in Muheza district, which is largely rural. This was not originally planned, but the number of children in the Tanga urban area were insufficient to achieve the desired sample size. The enrolment numbers for CRS schools and much lower budgets (approximately half of the Dodoma schools) may explain many of the differences seen between Africare and CRS schools in Tanzania. 65

69 Table 32: CRS Supported Schools in Tanzania (n=34) YEAR Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million TZS) Student: Teacher Ratio Figure 18: 15: Key Key School School Level Level Indicators Indicators for CRS for Supported CRS Supported Schools Schools in Tanzania in Tanzania (N=34) (N=34) 250 Approximate start date of support Avg Total # of Students Avg # of OVC Avg Form IV Pass Rate % Avg # of Teachers Avg # of Classrooms Avg Budget ( x 1 million TZS) 66

70 QUANT3 Each of the NGOs kept financial data in a different format, which made analysis of the cost data complex, and impossible to provide identical categorization of financial expenditures and costs for each organization. In addition, detailed financial reports were not provided by AVSI and ICOBI. AVSI did provide summary data to enable an approximation of direct costs, while ICOBI did not provide any data beyond their own calculation of costs per child. Table 33: Program Costs by NGO for the Secondary School Education Programs Africare TZ CRS TZ Africare UG AVSI UG ICOBI UG Children served 1,150 1, ,352 3,108 Personnel Costs $73,417 $50,746 $68,495 $282,367 $144,431 Fringe Benefits $18,576 $20,162 $18,576 $9,912 Travel Costs $23,158 $5,745 $7,081 $12,971 Equipment Costs $2,770 $1,660 $9,568 $16,587 Capacity Building Costs $15,343 $22,231 $4,482 $49,481 Block Grants $68,842 $82,199 $570,885 Uniforms/Materials/Tuition $57,429 $145,273 $6,720 $666,000 Other forms of support $12,465 $ 57,610 $4,944 $3,125 TOTAL DIRECT COSTS $271,999 $301,767 $194,157 $957,935 $807,393 Indirect and Organizational donations $64,790 $96,089 $46,248 $142,000 $139,405 TOTAL COSTS $336,789 $397,856 $240,405 $1,099,935 $946,798 Direct Cost per child $ $ $ $ $ Total Cost per child $ $ $ $ $ There is wide variation in the total number of children served by each program, and even pinning this number down is difficult because of the PEPFAR reporting requirements that call for both direct and indirect support to be noted. Each NGO has a slightly different way of calculating the percentage of resources that go to direct vs. indirectly supported OVC. With the exception of AVSI, each organization spends similar amounts of money on their programs with a cost per child in direct costs that ranges from $236 to $277. These costs include expatriate and local staff, fringe benefits such as housing, insurance, and pension, travel for volunteers, staff, and supervisors, equipment, and training of staff, volunteers, and community members (categorized as capacity building). The most expensive items are the direct supports provided to support children through school. All children, regardless of whether 67

71 their tuition was paid as a scholarship or block grant, received school uniforms, education materials, and books as well. Table 34: Breakdown of Program Costs by NGO Africare TZ USD (% of total) CRS TZ USD (% of total) Africare UG USD (% of total) AVSI UG USD (% of total) ICOBI UG USD (% of total) Total children 1,150 1, ,352 3,108 served Direct Benefit $109 (37%) $130 (37%) $127 (37%) $199 (61%) $184 (60%) to Child Program costs $127 (43%) $140 (39%) $150 (44%) $87 (27%) $76 (25%) Indirect costs $56 (19%) $86 (24%) $66 (19%) $42 (13%) $45 (15% and donations Total $293 (100%) $356 (100%) $343 (100%) $412 (100%) $305 (100%) When examined by NGO (Table 34) we see considerable variation between NGOs and some obvious inefficiencies. One means of reducing the inefficiencies would appear to be increase the number of children as this would appear to gain significant economies of scale. However, even with large numbers of children, 40% of the total expenditure per child is going to administration, which is an unusually high percentage. Table 35 shows the comparative costs per child of block grants vs. scholarships. Costs of scholarship programs are significantly and consistently higher than the block grant programs in both countries. Table 35: Comparative Costs of Block Grants vs. Scholarships Uganda Tanzania Block Grant average cost per child per year $324 $293 Scholarship average cost per child per year $412 $356 Difference between approaches per child per year $88 $63 Finally, Table 36 compares the cost effectiveness of the two approaches. With the exception of the variation in drop out rates there is little difference between the approaches in general in terms of their ability to produce educational outcomes. The qualitative data suggest the variation in drop out rates is directly related to the amount of money given as direct benefit to the child rather than the difference between block grant or scholarship (e.g. Africare with a higher drop out rate in Uganda only provides partial funding and spends between $110 $130 per child, while AVSI provides far in excess of the actual school fees and spends $199 per child). Based on the above, we come to the conclusion that block grants are more cost effective than scholarships. However, we observed in the qualitative data that as the number of children included in a school supported by a block grant becomes small (less than ten), the cost effectiveness approaches that of scholarships. 68

72 Table 36: Comparison of Effectiveness of Different Approaches by Country UGANDA Cost per child Attendance Drop out Academic Achievements per year Block Grants $324 Significantly Improved No difference No Difference to non OVC Scholarship $412 Significantly Improved Significantly improved No Difference to non OVC TANZANIA Block Grants $293 No difference Significantly higher rates Better performance (not statistically significant) Scholarship $356 No difference No difference No Difference to non OVC 69

73 QUALITATIVE STUDY Individual interviews or focus group discussions were conducted with district education officers, NGO chiefs and their staff, headmasters, community representatives, and both supported and non supported OVCs separated by gender. A descriptive summary of participants in the qualitative summary is presented in Table 37. Some of the questions yielded brief responses that could be tallied. Others yielded more detailed explanations. In the latter cases themes were abstracted and direct quotes presented that anchored the range and common ground of the sentiments. The focus groups with students will be reported here. These are followed by table formats that are used to present the results from other groups of respondents. Table 37: Descriptive Summary of Participants in the Qualitative Study Headmasters Community Reps. District Education NGO Reps. (IDI) (FGD) Officers (IDI) (IDI) Tanzania Gender Male Female OVC (FGD) Total Uganda Gender Male Female Total Grand Total Notes: IDI = In Depth Interview FGD = Focus Group Discussion Focus Groups with OVC Students Some of the major points and key quotes from the OVC focus group discussions are found in Table 38. Although there are varying perceptions of how they felt as OVC, in general, most students felt negatively. Students in Tanzania and Uganda listed different problems faced by OVC, however, issues such as discrimination, stigma, lack of educational materials, and shortages in food were commonly raised. When asked the question of how OVC students fared compared to their non OVC counterpart, it s interesting to note that the majority of OVC felt as though they either performed equally or studied harder than non OVC. In both countries, many of the supported OVC who claimed they had done well academically responded with lofty goals when asked about their aspirations for the future, including wanting to be a pilot, accountant, doctor, engineer, teacher, or banker. 70

74 Table 38: Key Points and Highlighted Quotes from OVC Focus Group Discussion Perceptions of being an OVC Tanzania I personally feel good when I am recognized as an orphan because there are people who heartedly support orphans Chinangali Female OVC When someone calls me an orphan, I feel weakened and disheartened Tanga Male OVC Uganda This is something that is just natural, we just accept what happened and what we are right now Nyakayojo Male OVC Whenever you get a challenge you feel like if I had my parents I would not have seen this Birere Female OVC Top 3 problems faced by OVC Tanzania 1. Stigma, discrimination & differential treatment by caregivers (n=30) 2. Lack of educational expenses & materials (n=15) 3. Lack of food (n=8) Uganda 1. Lack of basic needs such as food, shelter or clothing (n=13) 2. Lack of educational expenses & materials (n=10) 3. Child labor (n=7) Perception of academic performance compared to non OVC Tanzania I can say the performance is balanced because we are getting the same marks with those regular students Female OVC There is a difference so many non OVC don t bother studying hard because they know they have parents to depend on Male OVC The performance of OVCs is weaker than non OVCs Female OVC Uganda We are disciplined thus we perform better Female OVC Non OVC do not care about their studies because their parents do have money but for OVC they do concentrate a lot on their studies Male OVC We do have a lot of psychological problems so the attendance is not so regular, therefore the non OVC perform better Female OVC Majority of respondents felt as though they either performed equally or studied harder than non OVC Future aspirations Tanzania My dream is to become a member of parliament, and if possible I will be a speaker Male OVC I want to be a teacher and assist in improving education in Tanzania Male OVC Uganda I would like to become a journalist Male OVC I want to work in an organization that helps children Female OVC Focus Groups with Supported OVC Stigma and Basic Needs Facilitators opened their focus group discussions with OVC asking them if they knew what the term OVC meant. All of the respondents in all groups answered appropriately, referencing the loss of one or both parents. Some gave extended responses about the extent and depth of problems OVC experienced and why. One female responded that she was not an orphan but 71

75 nonetheless vulnerable. She described a regrettable encounter with a community leader as follows: I am not an orphan but I am vulnerable. One person (I won t mention his name) who is a leader came to our house and wanted me to have sex with him for TShs 200,000 for only half an hour. That person is like my father, so I refused and did not tell him anything. Such people take advantage on our problems; I could have agreed but that could have affected me after having sex with him. A female fourth former from Tanzania also spoke of this special vulnerability the girls face: Sometimes these children are cheated, particularly girls are cheated by men and are impregnated and they fail to continue with the school. Asked if people in their community knew they were OVC, their replies were mixed, some saying: they cannot know simply by looking at your appearance, and others claiming: To be frank the community understands our situation. We are known to the community. One assigned figures: 20% can, 80% cannot identify the orphans. Whether or not they faced a stigma was a question that produced mixed results. Some simply described it as a fact: We don t encounter stigma because to be an orphan is not a thing we asked for. It can happen to anybody, as one female in Tanga, Tanzania put it. A male OVC from Uganda stated: This is something that is just natural, we just accept what happened and what we are right now. One Chinangali Female OVC claimed I personally feel good when I am recognized as an orphan because there are people who heartedly support orphans. However many others reported feeling a stigma associated with the label. One third former, a male in Tanzania explained it this way: Yes there is a stigma to the vulnerable children; the one who is most vulnerable is likely to be stigmatized because his or her parents perhaps have died due to HIV/AIDS. They stigmatize him or her because they think that he or she might be infected with HIV/AIDS. Although this may not be true because having parents who have died because of HIV/AIDS is not necessarily regarded as children being infected. Sometimes relatives of the deceased parent tend to take the properties and chase away the children. These relatives leave those children without any property, and due to that, those children will be forced to go in the streets looking for something to eat while those relatives are enjoying (the assets) and refusing to give support to those children. In this situation I think those children are stigmatized. A male OVC from Tanga personalized it this way: Stigma is happening to us all. A step mother cannot give you everything you need as an orphan the same way s/he does to his/her own children. A step mother cannot be like your biological parent. If you need something and s/he reacts negatively, you think because s/he is not your biological mother. That is also a stigma to your caregiver. There is always a stigma in our families. 72

76 Older male OVC aged 19 to 24 in the Mbarara region of Uganda reported feeling stung by derogatory name calling and the associated stigma. In translation, they were labeled variously: those who survive on handouts, the poor, helpless, the vulnerable, those infected by AIDS/HIV, or those who have no importance in society. The following exchange between a moderator and a female OVC illustrates how teachers can set the tone. It depends on some students. Some discriminate; others comfort them. So yes, there is a difference among students. Asked about teachers: They are the same like students; if someone hates you, then nothing can be done. You will be stigmatized if it happens so. If someone loves you, then you will experience love. Does it happen at this school that some teachers stigmatized some of you? I have not seen that at this school. Do you think at this school teachers treat you well? Yeah. Another student chimed in: They don t differentiate OVCs and non OVCs, they treat all of them equally. This sentiment was echoed by a male OVC in Dodoma: Teachers do regard us as equal as other ordinary students; they teach us equally and no stigma is experienced from them. And they teach us very well; sometimes they sit with us and try to advice the way we are supposed to live and behave for the sake of our academic performance. A male OVC in Tanzania pointed to the positive regard the OVC experienced in his school. Other children in this community tend to distance themselves from us because we don t have parents like them; hence we are not equal economically. On the other hand, there are children who consider us just their friends. These regular children sometimes take the trouble of seeking some support from their parents that can be given to other orphans. S/he can go and explain to his/her parents that his friend has got some problems so if there is any possibility of supporting him in terms of education, health, shelter and other important things, do so. Other parents support us when they are persuaded by their children. His classmate concurred: Within this school, our fellow students consider us as their colleagues, who are living in the same family. Generally in this school we are living peacefully and other regular students do care about us in a good manner. Most supported OVC reported that fellow students did not discriminate against them. There is no stigma at our school. We are stigmatized by the people we are living with, our families or relatives. Death is always there, it can happen to anybody, at the school or in the streets. Our fellow children do not bother about the status of other people. They just mind their own business. But a stigma lingers that OVC find very hurtful: Although stigma is not seen much nowadays, but sometimes it irritates you when your colleague refuses to give you his book by saying go and tell your father to buy one for you. It irritates because you don t have a father to do that. Stigma is seen when OVCs lack some school materials. A person asking about your father while knowing that you don t have one is something that hurts so much. Asked to identify the top three problems faced by OVC, fully 30 of the 54 OVC interviewed in Tanzania mentioned stigma, discrimination and differential treatment by caregivers, despite the widely reported claim by others that the stigma was not a big problem for them. 73

77 Additionally 15 mentioned the lack of educational expenses and materials, while 8 cited the lack of food. Once female in Tanga describe it this way: For me I am already suffering discrimination by living with caregivers who do not care.you go to school hungry, you stay in the class hungry, and you go back home hungry too. Can you really gain anything in the class?...you cannot listen to anything in the class. From the focus groups in Uganda, 13 cited the lack of basic needs such as food, shelter, or clothing; 10 mentioned the lack of educational expenses and materials; and 7 specified child labor. Asked about problems they faced as orphans, a focus group from Dodoma responded with a long and diverse list of needs: proper education, shelter, clothes, food, a proper place to live, spiritual services, health services, lack of love from the community, lack of happiness, and stigmatization, perhaps from your step parent. Academic Performance and Aspirations Especially in light of the relatively superior academic performance of OVCs in Tanzania on Form II examinations and their equivalent performance to non OVCs on Form IV exams in Uganda, it was of special interest to note how OVCs compared their own academic performance to that of their non OVC counterparts. Fully 22 out of 26 respondents in Tanzania thought they either performed equally or studied harder than non OVCs. A female OVC replied: I can say the performance is balanced because we are getting the same marks with those regular students. A male observed: There is a difference.so many non OVCs don t bother studying hard because they know they have parents to depend on. Another Tanzanian female explained: Academic performance for OVCs can be good because they always think about the school, whereas regular students can be fooled by the richness of their parents. This perception appeared to be widely held. Another female OVC in Tanga explained: It depends on each student responsible for studying. On the other hand, it also depends on the ability to understand lessons; sometimes OVCs perform better than non OVCs. Non OVCs don t care because even if they fail, their parents are there to care for them. For OVC s/he thinks of having no father and no mother so s/he studies hard. S/he remembers of having no parents so struggles to be like others. Her female classmate offered the minority perspective: You cannot perform equally with someone who has parents who meet all school requirements. They can pay tuition fees, which is difficult for us, so we cannot be equal academically. They are better when compared to us. OVC in Uganda responded in similar fashion to the Tanzanian majority. A female noted: We are disciplined, thus we perform better. A male felt: Non OVC do not care about their studies because their parents do have money but for OVC, they do concentrate a lot on their studies. A focus group of females in Uganda provided several powerful reasons as to why they went to school: because I love to study; to become literate and important in future; to have a better future (3); to get a job and start helping others; because of peoples jealousy and 74

78 rumors of the community; to add to what my parents do not have; and to become an important person since I got in an accident and am now disabled. In both countries, many of the supported OVC who claimed they had done well academically responded with lofty goals when asked about their aspirations for the future. Female OVCs reported wanting to be a pilot, accountant, doctor, engineer, teacher, or banker. One male from Tanzania replied: My dream is to become a member of parliament, and if possible I will be a speaker. Another male said: I want to be a teacher and assist in improving education in Tanzania, and another a military general. OVC in Uganda aspired to be a journalist, banker, lawyer, nurse, engineer, teacher, president, or doctor. And those were just the females. Their male counterparts named becoming a teacher, journalist, entrepreneur, radio personality, veterinarian, bishop, accountant, or researcher. Perceptions of Supported OVC on Non supported OVC OVC receiving support were very sensitive and empathetic toward the needs of OVC who did not receive any support. As a female in Chinangali noted: Truly they feel very bad because we were given the exercise books and bags while they were given nothing. Her classmate said: There are others who have passed their standard seven exams but failed to join secondary school due to being poor. Therefore, seeing us joining the school and they were not joining. They felt very bad and they were thinking why not them? Male students responded in a similar fashion: To be honest, I feel very bad when I see my fellow children are not with me at the school. We studied together at the primary school and fortunately they passed the exams; just imagine I m coming to school while they at home doing nothing! His classmate noted: Those who are left at home, they feel very bad. I would like to advice Africare not only to visit students at the school but rather they have also to extend their support to the street children and there are many who are in need of this kind of support. A student with support from CRS reported: They feel isolated when they see us going to school. They have no uniforms and no school fees, so they stay at home miserably. Positive Gains at School and at Home A female OVC in Chinangali, Dodoma reported getting a desk, stationery, school requirements and other contributions from Africare. Her classmate cited school fees and other school requirements. Another cited sports facilities and the teacher s table. Additional support came in the way of books, posters and training materials for educating each other about HIV/AID. Students were clearly very appreciative of these gains. They expressed this gratitude in a variety of ways. I will not forget Africare because they have supported my studies. The thing which I have liked most in this school and even if I will quit, I will remember the cooperation that I have with my fellow students. The thing that I won t forget in this school is my teachers. I will remember them very much, and the officials from Africare will always be remembered 75

79 because they are the ones who discovered that I have very critical problems for which I should be supported. Another group of respondents echoed the same sentiments. They will remember the support they got from Africare, their teachers, and their sponsors. OVC s Recommendations for Future Investments in the School Despite their gratitude, OVC noted that their school was missing some important facilities including a library and a science laboratory. One expanded that wish list: I would like the school to be supplied with piped water, more books, and increase the number of teachers. Another group of students complained about the lack of books, laboratory, and teachers, as well as piped water and the availability of a laboratory. One young man emphasized the need for sports gear. Finally, several requested transport to school since they lived at a distance. Focus Groups with Non Supported OVC In addition to their focus groups with samples of OVC who received support from one of the five NGOs, our field research teams also conducted separate focus groups with OVC who were attending school but who were not supported by any agency. Again the groups were addressed separately by gender. A focus group of females in Uganda that served as controls for supported OVC in Ntungamo reported a bleak and disturbing array of responses to the issue of whether or not they felt a stigma as an OVC. Their responses, save the first, were much more negative than those reported by the supported OVC. Here is a sampling: I feel comfortable. I feel bad with a lot of anger as if I am the one who killed them. You feel bad especially when someone is calling their parents mum or dad. You do not feel confident and life becomes difficult. Whenever you get a challenge you feel like if I had my parents, I would not have seen this. When like you are in the village and you see your peers going to school, you feel bad that if your father was alive you would be going to school. If you do not have your parents you can fall sick and lose weight. If you do not have your parents and you are on the way playing, children can discriminate you, thinking that your parents died of HIV/AIDS, yet it is not what killed them. When you are an orphan you feel bad knowing that you will not be able to get what ever you need like clothes and school fees. When they call you an orphan you feel bad especially in class and you start regretting as if you are the one who wanted them to die. When you are an orphan, you feel bad especially when someone abuses you that kumanyoko, you start recalling your parents. You lose hope, feel uncomfortable and you remember your parents. 76

80 When you are an orphan you are rated unequal to the rest of the children you are staying with and especially on Christmas, they buy clothes for others and you are missed out or even others are given school fees and yours is missing. If you are an orphan and staying at other people s homes, you feel bad, and instead you think of killing yourself. The kids ask OVC where they come from especially on visiting days when the OVC have not been visited. They use words like; okaretwa omutunga (meaning, you were brought by the flood ). The male counterparts of the group reported above were more accepting and not so devastated as their female classmates with their OVC status. They reported: I feel good about it because it is my status; I cannot change anything about it. It depends on who is identifying me as an orphan; some see it as a way of insulting me. I feel sad because when someone is identified as an orphan one thinks that he or she lives in very poor conditions. It s OK with me and, as my colleague said, we are identified as people living in poor conditions; I want to live to change that perception. Another group of females from Uganda who were non supported OVCs expressed downcast views, more negative than those of their OVC counterparts who were supported, but not as negative as the responses of the female group reported above: on the positive side, one reported I do not feel bad because it is normal to be one. Another said: At times I feel happy because I could benefit from my status. But most responses were negative: I feel sad because I live in the world without parents, my needs cannot be met. And, I feel bad because I do not get parental love, I also feel insecure as I lack parental guidance. Some students identify us as fees defaulters. Their male counterparts in both control groups offered much less bleak and more matter offact assessments of their status and any stigma that might be attached. Typical responses were: I feel OK since that what I am. I do not feel OK because I cannot get all the necessities I need. With regard to stigma it depends where you are; at school we are not discriminated, but within the community we are discriminated. These male, non supported OVC reported that the other students treated them well. They are really concerned about our status, they befriend us, and they are so attached to us. Another summed it up: They really do associate with us. In assessing the regard teachers hold for them, they praised the teachers but noted teachers cannot overcome the negative views of others or their inability to pay school fees. Teachers really care for us and they can offer help at any time I need it. Teachers do care, but other staff see us as a burden; if it so happens that I do not get food, no one will care, but if another child misses out on food then it s a big issue. At times when we fail to get school dues we are offered hard labor at school which makes us lose concentration on our studies. We are not given priority and we are never listened to. 77

81 Despite these seemingly insurmountable problems, these unsupported male OVC still hold high aspirations for their futures. Upon finishing school they would like to become: an entrepreneur, journalist, lawyer, and engineer. Their goals were no less ambitious as those of their OVC peers who were supported. Focus Group Discussions with Community Representatives Focus groups were conducted in six districts, three each in Tanzania and Uganda, by trained moderators who facilitated discussion among community representatives. The representatives had special knowledge and responsibility for the selection of OVC. Their ages ranged from 22 to 84, but most were in their thirties and forties. Citing their occupation or role in the community, they listed peasant former, catechist, teacher, parent, youth leader, secretary, and care giver. Between 6 and 12 participants of mix gender were assembled in each group except in Chinagali district where the respondents were all male. Sessions ran, on average, about 75 to 90 minutes, generating 40 pages of transcripts. Defining and Describing OVC The purpose of the focus group discussions was to hear from the representatives their views on who constitute OVC, how they were identified as worthy of support to go to secondary school, what progress they were making, and their assessment of the procedures. As did all of the groups representing each of the six districts, the representatives in the Chinagali group identified the loss of a father, mother, or both parents as the core aspect of OVC s identity, but the oldest member of the Chinagali group captured fuller dimensions of their plight: Perhaps I can add something, an orphans is a needy one in the community because orphans do not have people to support them. They have no people to care for them; they don t have food, educational support and other basic needs. On the other hand, an orphan needs happiness because to him or her anybody who will console her/him will leave him/her with happiness. I can say orphans always depend on God s wishes if nobody in the community will regard them. Lastly, orphans are children, so they don t have plans, they need somebody to plan and focus for them. So they need the community to care for them for their future to be fruitful. Others too reflected on the larger dimensions of vulnerability that these children face: the lack of adequate food, clothing, shoes, uniforms, and more generally psychosocial support to ward off depression and malaise. Girls were particularly vulnerable according to some respondents: Orphans, girls in particular, are in very vulnerable environments more than their counterpart boys. This is so due to the temptation they encounter. If they are not provided the needs they want, like uniforms, fees and other important needs, it will be easy for them to be affected with diseases such as HIV and sometimes impregnated, hence will not complete their studies. 78

82 Community representatives from Uganda offered these additional perspectives about OVC: They tend to work hard and engage in difficult and risky jobs. If the children are not at school, they suffer and become hooligans, they also lack guidance at school because even at school, the rest of the children who need special attention are many. They get blamed for what they have not done because they are always seen at home. Had they been in school they would not get into such problems. Asked if the orphans were well known in their communities, the respondents answered variously, sometimes by describing the process by which the community groups obtain the information, and not whether they knew them personally. In this community we visit homes and see them with their guardians. It is also known when someone dies and lives behind orphaned children. We find them with their families or relatives. As you walk around in the village, you can easily identify the vulnerable child, according to the type of shelter / house they have or he/she has, the type of food they eat and how they dress. In another community in Tanzania, a representative noted: these children are well known by the community, particularly to the place where they are coming from. For example, when we were identifying orphans and vulnerable children, we cooperated with the community. Therefore we met with the hamlet chairpersons who told us about orphans and vulnerable children that they thought could be better supported in their studies. Then we got the list of those children, this was shared also with the administration of Africare, so these children are known to the community. The problems of these OVCs were reported by chairpersons of the hamlets, including those in our records were first listed by those chairpersons and later on were proved by the Africare. This is to say the community is aware and the orphans are well known. But still another representative of the same community noted that not all orphans could be readily identified: These people who have lost their parents are of many types at home. There are children that have lost their parents but are not in this group of the needy ones. The reason is that they are not in the vulnerable environments. I m living with them and other people do think that those are my biological children. 79

83 Several community representatives from Muheza would agree with this perspective: Some of us have orphans but we don t tell anyone. We tell orphans that even if they lost their parents, we are there to help them. Therefore an orphan should be only a child without any support, and the community should understand in that way. There are some people who look for orphans and take care of them. Sometimes they are not known to anyone as they are just wondering in the streets as no one takes trouble to identify them. Most orphans are not known to the community. Some OVCs are just living with relatives without being made known to the community. Some OVCs do not even know whether caregivers are their real parents or not, especially those who lost their parents when they were still babies. Most community members don t know a number of OVCs. Asked how to clarify how they identified the OVC, an earlier respondent continued: Yes first of all, we ask and announce about the students with problems because there are others that have been identified at primary level and have been supported with different NGO s. For example, there is COMPASSION then there is HPC and HPPCT. So when these children join in our school, we ask them about the support they are given. So after recognizing and identifying those who are supported, we ask them their names and the kind of parent that is still alive. Therefore we can understand the status of the orphan, if the orphan has lost both parents, we also ask about the one taking care identification is done through the Village Executive Officer and the Ward Executive Officer has to prove that the selected ones are really the needy ones. Not only that, but also the Ward Executive Officer has to prove that the listed children can be supported by the village governments or the Dodoma municipal council. After that those names are brought here at our school. It was clear that the community representatives wanted to be informed on a regular basis about the progress that the OVC selected for support were making in school. Those who were not performing well should be interviewed with a view toward remedying basic needs or directing the support to other deserving OVC. I suggest supporting NGOs should concentrate on making follow ups to know performance of OVCs they support. If they support someone who is not seriously studying, s/he should be better not supported. OVCs must know that their sponsors are making close follow ups. A community representative from Tanga thought that the responsibility to follow up rested with the NGO. Moreover, he faulted the OVC for being lax in their studies. Yet this assessment 80

84 is contrary to what OVC had to say about themselves and especially their outcome measures that rivaled or surpassed even the non OVC. Stigma They should be making follow ups on their academic progress. Because of lack of academic follow ups from the supporting NGOs, OVCs tend to be lazy and make no efforts in studying. We told sponsors that it is imperative for them to know how supported OVCs are progressing academically and warn them that if they don t struggle they want get the support. OVCs are just relaxing, to be frank. There were differences among the community representatives in their assessment of whether or not a stigma was attached to being an OVC. Some generalized the attribution of shame and humiliation to all OVC, while others were circumscribed in their remarks, for example: We have only one example. There is one child that both her parents passed away, in Kondoa district. Her aunt decided to take care and brought her here. Her aunt also had her (own) children; therefore they were living together as a family. Those children thought that the girl was a foreigner; they started stigmatizing her, they wanted her to vacate their house. After all naughty acts, they decided to collect her belongings and dumped (them) outside the house as means of telling her that she is not needed at their house... So she feels inferior within the family. This girl is psychologically tortured. There are other children that tell the orphans frankly saying: what do you want here? Why can t you go at you home? Don t you know that this is not your house? Therefore there is a stigma. Despite this heart wrenching story, it is important to remember the accounts given by the OVC themselves as reported in an earlier section. The negative, hurtful stigma was evident to them only in certain cases, and almost never from the teachers. In fact the problem seemed to diminish in the school environment. The following comment from a community representative would corroborate that observation: Principally I can see this issue in two sides: there are orphans that are stigmatized at the family level, and these children feel happy whenever they are at the school and their follow students do not stigmatize them. Stigma at the school level starts when that orphan has not been provided all important needs required by the school, like school contributions or fees. If somebody will help that person in fulfilling his/her needs that stigma will be vanish. Selection Criteria Finally, the researchers wanted to know more about the selection criteria the community representatives used to choose among the larger group of orphans which ones would get support through the NGOs. It became evident from the scoping trip in May 2010 that two 81

85 points of view could be found: Should support go to the neediest, or should support go to those OVC most likely to succeed in school? While there were some dissenters, It appeared from the focus group interviews conducted early in 2011 that most representatives strove to identify the most needy, but they were not satisfied with the process. The following responses illustrate their concerns: This is a serious problem because OVCs are living with caregivers who are unable to support them. You see girls are giving birth to children without knowing exactly where their fathers are. This is also an orphan! Some OVCs introduce themselves. Some are brought by their caregivers and introduced to us. Some are identified by Village Executive Officers but we don t know what criteria are used to select them for support. There is a problem because we may end up supporting who do not deserve, as the criteria are only known to leaders. There are criteria used to identify an orphan. I am a Ward Executive Officer; I get a letter from the district council instructing me to list OVCs in my ward. I forward a letter to Village Executive Officers who work with other village leaders to identify OVCs in their respective areas. There is a problem of listing children who are not OVCs that result into having a wrong population of OVCs. It has happened village leaders listing their relatives in expectation of receiving support to benefit their families. Representatives were acutely aware of the problem of selection in the case of limited funds: I was asked to list OVCs at this school and I got 70 of them. This is a huge burden for Chief Mang enya Secondary School as all of them cannot pay school fees and other school contributions. Even the few that are supported where listed secretly because if we were to decide to advertise them, then it could be chaos. The remaining groups of respondent were engaged in semi structured interviews that did not provide the rich context and detail that the Community Representatives and especially the OVC did about their own situations. Perhaps this was owing to the format of their interviews. They were asked direct questions that yielded responses that could be tallied for their content and frequency. Summaries of responses from each group are given in the paragraphs that follow, while tallies and representative quotes are offered in tabular formats. Key Informant Interviews with Headmasters To provide perspective from local school administrators, major points and key quotes from the headmaster interviews are found in Table 39. Headmasters noted that the main issues faced by schools included insufficient classroom and facilities, lack of electricity and water, and teen pregnancy. In regards to the selection process associated for educational support, different NGOs had different criteria for the selection of schools. Some schools applied for block grant support while other schools were asked to participate and were not part of the decision 82

86 process. When asked about the strengths of the block grant support, most headmasters responded favorably, especially when support was translated into material goods such as desks, tables, and laboratory supplies. A wide range of weaknesses of the block grant were also highlighted by the headmasters, including delays in payments, the small number of OVC students that can be supported, and the school administration s limited ability to prioritize their needs. Overall, headmasters noted that the benefits of the block grant were mostly positive and that block grant support benefitted the entire school. Table 39: Key Points and Highlighted Quotes from Headmaster Interviews Tanzania Uganda Top 5 Issues or problems faced by schools Top 5 Issues or problems faced by schools 1. Lack of electricity and water (n= 22) 1. Insufficient classrooms and facilities (n=37) 2. Insufficient classrooms (n=19 ) 2. Lack of electricity and water (n=32) 3. Teen Pregnancy (n= 17) 3. Old infrastructure (n=21) 4. Old infrastructure (insufficient latrines) (n= 10) 4. Teen pregnancy (n=14) 5. Lack of laboratories (n=2) 5. Not enough teachers (n=13) Schools Receiving Block Grant Support Selection process for educational support Selection process for educational support The school applied to the grant through ward leaders, school board and municipal council leaders We listed the items we needed by showing priorities Chinangali Headmaster The school was asked to participate and so it did The process was to approve names of OVC and give them the grant Lukundo Headmaster We received [the block grant] through application [We identified OVC] through writing their names and a letter forwarded by the Ward Executive Officer, School Board and District Education Officer Kisasa Headmaster We did not apply for the grant, we were just informed by ICOBI that they were to support some students, we just had to agree on the conditions Kashaka Headmaster We were consulted by Africare and we were requested to provide a list of students who were OVC Rwamanyoni Headmaster The parents/guardians applied for the grant and we were notified of who was to receive support. We were not involved in the selection St. Peter s Katukuru Headmaster Strengths of block grant support It is the best kind of support, especially when it was turned into materials... like desks, tables, books, shelves and staff chairs Miyuji Headmaster It helped more OVCs who hadn t got support from other organizations. Lukundo Headmaster Weakness of block grant support It is specific on bringing abstract materials instead of money hence the school has no other choice like paying part time teachers Lukundo Headmaster The children demand a lot of psychological treatment more than material support Mnandani Headmistress Few students are supported than the demand for support Miyuji Headmaster Strengths of block grant support It comes in a lump sum and eases planning Muyenga Headmaster It has supported a good number of students who would be on the streets begging Kashari Headmaster It eases school fees collection fatigue Nyakayojo Headmaster Weakness of block grant support It left out the priorities of the school Nyakyeera Headmaster ICOBI has had little interaction with these students thus many parents have abused the generosity many children are getting this money yet they come from able families Nyamitanga Headmaster There are delays in payment Rutooma Headmaster 83

87 Key Informant Interviews with NGO Personnel Findings from key informant interviews, showcased in Table 40, provide some interesting perspectives. Some of the positive benefits for the school as a result of the various educational support included increased school reputation and the ability for the school to invest on capital developments that would otherwise be unable to afford. In regards to positive benefits for children, NGO personnel mentioned that support increased school attendance and that it was beneficial to students who were unable to pay for school fees. Various strengths were mentioned of the different educational support that NGOs provided, from improved school infrastructure, increased enrollment, and the ability of block grants to support OVC and non OVC students. Although optimistic about the positive impact of the support that they have provided, NGO personnel were realistic of the limitations of their intervention. Educational support does not address the other needs of a child that influence retention and academic performance. In addition, it was noted that some caregivers show reduced responsibilities towards OVC as they believe that the NGOs will provide all the necessary assistance to a supported child. Table 40: Key Points and Highlighted Quotes from NGO Interviews Schools Supported by Africare TZ Schools Supported by CRS TZ Schools Supported by Africare UG Schools Supported by ICOBI UG Positive benefit for schools Positive benefit for schools Positive benefit for schools Positive benefit for schools School reputation Access to laboratory chemicals, desks, textbooks Deputy Chief of Party, Africare School s reputation, attracted more students, better academic performance, there are more than 200 HIV+ who are studying Project Manager, CRS Also there were increments in student enrollment and also schools were able to access materials that they would not have bought using meager funds. Grants Manager, Africare The schools receive money at once (for a whole year) and therefore they are able to plan for it properly. They can afford to spend on expensive capital developments that they would otherwise never afford. Program Manager, ICOBI Positive benefit for children Positive benefit for children Positive benefit for children Positive benefit for children It is helping since some could not afford to pay school fees Deputy Chief of Party, Africare Improved attendance and some are continuing to higher education and colleges Project Manager, CRS Yes it helped the OVCs. There was regular attendance even if payments delayed By the time the final evaluation was done, 540 out of the 750 OVC supported in secondary education had completed S4. Grants Manager, Africare The child is able to concentrate on studies with no worries about school fees if they are paid for a whole year. Program Manager, ICOBI 84

88 Schools Supported by Africare TZ Schools Supported by CRS TZ Schools Supported by Africare UG Schools Supported by ICOBI UG Strength of educational support Strength of educational Strength of educational support Strength of educational support program program support program program It supports both MVC and non MVC and improves school learning environment Deputy Chief of Party, Africare Weakness of educational support program It doesn t address other needs which could influence retention and performance access to food, distance to schools Deputy Chief of Party, Africare Improve attendance, child protection, health and nutrition, income generating activities to caregivers, and some are supported with shelter. Finally psychosocial support has been successful Project Manager, CRS Weakness of educational support program There is a problem with identifying OVC, untrained volunteers and duplication of support services Project Manager, CRS It reduced on the discrimination of the OVCs within schools and it offered schools enough time to concentrate on other developments within the schools. Grants Manager, Africare Weakness of educational support program Even though support was provided, some OVCs did not finish school, dropped out without any skill. Also, we were not able to support children beyond S4. Grants Manager, Africare There are schools which had very poor infrastructure but now they have improved. One example is Kyangyenyi S.S, which had stopped the boarding section due to lack of facilities, but now they have constructed dormitories and re introduced the boarding section. School enrolment in this school was only about 120, now it is over 400. Program Manager, ICOBI Weakness of educational support program Some caregivers have sat back and relaxed, leaving all the responsibility to ICOBI. For some, even when a child falls sick or gets another problem, they call ICOBI staff or volunteers and say your child is sick you better do something. In a sense, the approach de links caregivers from schools, and they expect ICOBI to handle all school matters for them. Program Manager, ICOBI 85

89 Key Informant Interviews with District Education Officers Some of the major points and key quotes from the District Education Officer (DEOs) interviews are found in Table 41. DEOs highlighted some of the main challenges faced by the district, from the material deficiencies in schools to the issue of high poverty and illiteracy rates within their district. In terms of specific problems faced by OVC, DEOs reiterated the stigma and negative attitudes towards OVC and the educational issues faced by OVC. In regards to the process to select schools and children for educational support, DEOs noted the involvement of local communities in selecting schools and children to be supported. When asked about ways to improve educational support in their respective districts, it was interesting to note that one DEO requested more evaluation and follow up of different support programs. As OVC support is targeted for children under 18 and programs have limited implementation timeline, it was common to hear complaints associated with the fact that supported students were unable to finish the four year O Level education cycle due to these issues. Table 41: Key Points and Highlighted Quotes from District Education Officer Interviews Challenges faced by the district Infrastructure of schools is insufficient, and teaching and learning materials are not enough. Muheza District Education Officer (TZ) Deficit of classrooms, libraries, laboratories, desks, teachers, staff houses and administration blocks. Dodoma District Education Officer(TZ) High poverty levels, high school drop outs, teen pregnancies, inaccessibility to some areas, high illiteracy rates. Mbarara District Education Officer (UG) Understaffing within the local government, local government does not have enough funds, high poverty levels, and inaccessible water sources. Ntungama District Education Officer (UG) Specific problems that OVC face in the district Failure to get school fees, uniforms, nutrition and learning materials are major problems. Muheza District Education Officer (TZ) Inefficiency of financial support for them to get education. Dodoma District Education Officer (TZ) Negative attitude of the community towards OVCs. Mbarara District Education Officer (UG) Dropping out of school, heading of households, child labor. Ntungama District Education Officer (UG) Selection process for schools and children to be supported Village governments assist in choosing schools and identifying OVC together with Ward Development Committees. Muheza District Education Officer (TZ) The community was involved. Individual students are identified at the ward level and the names are forwarded to the Municipal Director. Dodoma District Education Officer (TZ) Selection was done at community level and students were sent to selected schools mainly in rural areas. As the district office we were just informed on the progress of the project but we were not directly involved in the selection process, so emphasis was put on schools in the rural communities. Ntungama District Education Officer (UG) Ways to improve educational support More evaluation and follow up is needed. Schools should be supported directly. Muheza District Education Officer (TZ) Supported students have not been able to finish the educational cycle (O Level) as support has ceased. Mbarara District Education Officer (UG) 86

90 Focus Group Discussion with Community Representatives The final set of qualitative interviews focused on key perspectives from community representatives, shown in Table 42. Community representatives repeated some of the main OVC problems that were already highlighted by earlier interviews, including lack of food and education issues. When asked about the process for identifying OVC for selection of educational support, there was no consistent way in which this was done reflecting the different approaches that NGOs took in selecting OVC. OVC identification is usually done at the community level, either through community leaders or village governments. It was interesting to note that factors such as poverty were considered as a selection criteria as preference was usually given to children who are needy and badly off. Finally, when asked about ways to improve OVC support programs, community representatives stressed the importance of close collaboration with local governments. In addition, they through that it was important to follow up supported OVC to monitor their progress and find effective ways to select OVC for educational support. Table 42: Key Points and Highlighted Quotes from Community Representative Focus Group Discussions Problems faced by OVC While at school they face psychosocial problems they see themselves worthless in the society Muheza Rep. (TZ) These OVC are extremely psychologically stressed Those who are in classes are totally stressed because they come here hungry Muheza Rep. (TZ) If he or she has no food definitely will not attend the classes Chinangali Rep. (TZ) There are different issues to look at, some have parents but have no support/guidance due to the poverty of the parents. Ntungamo Rep. (UG) We also have many drop outs in the community because parents cannot afford school fees and even those that are being supported in some cases they also drop out of schools because the support they get from ICOBI is not full school fees. Mbarara Rep. (UG) Process of identifying OVC for school support Identification is done through the Village Executive Officer and the Ward Executive Officer has to prove that the selected ones are really the needy ones Chinangali Rep. (TZ) Some OVC introduce themselves some are brought by their caregivers some are identified by village executive officers There is a problem because we may end up supporting who do not deserve Muheza Rep. (TZ) The mechanism to identify OVC is coordinated by village governments we also conduct census across villages to identify and record OVC Tanga Rep. (TZ) Poverty was weighed heavier than other factor when making decision to support children. Ntungamo Rep. (UG) Through community leaders. We look for the children who are badly off. Mbarara Rep. (UG) Improvements to OVC support Close cooperation between city council and wards should be strengthened Tanga Rep. (TZ) Supporting NGOs should concentrate on making follow ups to know performance of OVC they support Muheza Rep. (TZ) You have to propose other effective criteria to identify and decide which OVC to be supported Muheza Rep. (TZ) The support would have continued for more years but overall, we and our children appreciate the support though we still need more. Ntungamo Rep. (UG) 87

91 DISCUSSION Each of the five approaches to supporting OVC to attend secondary school that we studied was successful in allowing limited numbers of OVC who would not have been able to receive secondary education, to go to school. From that perspective, every organization attained its programmatic goals. However, there are limitations presented by each approach, particularly in relation to the costs of the programs and the inability to address the underlying challenges faced by the educational systems in both countries we studied. The original question that this study was tasked with was to compare block grants to schools with the provision of individual scholarships. This is a question that has rapidly become largely irrelevant in the two country settings we studied because of contamination among organizations. In 2006 when the Track 1 programming began, there were two very distinct approaches to educational support. Africare had pioneered block grants in Zimbabwe and introduced them in both Tanzania and Uganda. CRS had a long history of providing scholarship funds to families and continued this in Tanzania. AVSI in Uganda began a scholarship program similar to the CRS program in Tanzania, but immediately were in communication with Africare and modified their program so that while they continued to provide individual scholarships, the money was not given to the child or the family, but to the school to administer. ICOBI, when it began programming two years after the start of Track 1 programming, took stock of the different approaches in Uganda and selected a model of block grants similar to Africare s, but with some significant variations about how the funds were controlled by the school. Finally, as CRS in Tanzania has come to the end of its Track 1 funding, it has phased out the scholarship approach and is now instituting block grants through its work with the Tunajali project and its own work in the South. 88

92 We look at the comparison of block grants versus scholarships as follows: Block Grants Scholarship Did the program succeed in increasing access to secondary education? Yes. Partly Every child sponsored by Africare and ICOBI were In Tanzania, significant numbers of children verified as attending school and their records registered in the scholarship program could not be examined. verified as attending school and their records could not be found. In Uganda, the same administrative issues were encountered at first leading to significant modification of the program. Did the program provide equity in access to secondary education? No Limited funds were available and priorities had to be made about who would be chosen to receive secondary education and who would not. However, because of the process used to select OVC the community was generally aware of the number of OVC in need and additional sources were sought out for OVC not supported by the block grants. Most importantly, the schools were aware of who was sponsored and who was not and could pay increased attention to them. No As with block grants, limited funds were available and priorities had to be made. The process of selection was not perceived to be transparent. Schools often were unaware of children selected, and thus could not provide attention to special needs or follow up on absenteeism or drop outs. Did the program result in improved attendance at school? Partially Partially Two of the three block grant programs showed One of two scholarship programs demonstrated significant improvements of supported OVC over significant improvements of supported OVC over their non OVC and non supported OVC their non OVC and supported OVC counterparts. counterparts. Did the program result in a decrease in dropout rates? Mixed No Two of the three block grant programs saw There were no observed differences in either significantly increased drop out rates when scholarship approach between supported OVC and compared to non OVC, while the third saw lower their non OVC counterparts. drop out rates. Did the program result in better academic performance? Possibly Supported OVC in Tanzania performed better on their aggregate scores for Form II, but the result was not statistically significant. There were no measurable differences on aggregate scores for form IV and individual subject scores showed that OVC performed at the same level as non OVC. No There were no differences on Form II or Form IV aggregate scores. Numbers were too small to look at individual subjects. Did the program result in measurable changes in the school s infrastructure? Yes No Schools documented construction of new There were no known benefits to the schools. classrooms, latrines, water tanks, electric Interviews with teachers showed that in the majority generators, and purchase of laboratory equipment of cases teachers were unaware that assistance was and other school materials for all block grant being provided to the school. approaches. 89

93 Block Grants Scholarship Did the program result in measurable improvements in the school s performance No No Overall school performance decreased over time, This was not part of the design or intent of the but this is explained by the rapidly rising program. enrolments and increasing student to teacher ratios. There were no differences observed between intervention schools and control schools. Did the program result in improvements in psychosocial wellbeing of OVC? Yes Students reported being able to use school resources made available as a result of the conditionality of block grants. Was the program cost effective? More effective Costs of block grant programs were between $60 and $90 dollars less per child than scholarships. However, it was noted that cost effectiveness is directly related to the number of students enrolled in the block grant program. Partially Students reported receiving support at home, but few students were aware of there being a teacher trained in psychosocial care at the school. Less effective Scholarship programs were more expensive than block grants, with no consistent difference in educational performance, absenteeism, or continuation for children supported by block grants and scholarships. Accountability In addition to the above we made several observations about accountability and the potential for diversion of funds. 1. Africare s approach was the most regimented approach, leaving little or no freedom for schools to make their own decisions about the use of funds beyond the relatively narrow criteria that Africare set for use of the block grant money. This approach was criticized by several local key informants in other NGOs and the Ministry of Education because it was perceived to be disempowering of the schools and exhibited a mistrust of schools capacity to procure goods and manage money. However, the approach was never criticized by staff or informants the schools that were studied. 2. AVSI s approach was less regimented than Africare, but still limited what funds could be spent on and conducted detailed financial audits on expenditures. No complaints were ever raised about their approach. 3. ICOBI s approach was designed to contrast with Africare s regimented approach and provide much more freedom for schools to choose and be free of administrative burden. This approach was extremely popular with schools, but it was observed that ICOBI is now considering moving to a more rigid model aligned with Africare s approach because of concerns about accountability. 4. CRS s approach was the most open of all. However, it was also observed to be the most open to abuse, and several cases were observed which raised concerns. Firstly, when schools were visited to collect data from supported student s records there were several instances where the student had never attended the school and could not be traced. This led the local investigation team to coin the term 90

94 Ghost students (Watoto yatima hewa in Swahili), coming to the conclusion that families had received funding, but money had not been used for school. Secondly, there were several reports by teachers that students had only had part of their funds paid by the local FBO, and the student was forced to leave school partway through the term because of non payment. Finally, in several instances local FBO collaborators simply disappeared after questions were raised about funding, leading to concern that they feared that the research team were auditors. Gender Issues Absenteeism and gender There is a great deal of discussion in both the popular and the scientific literature about the effect of menstruation on girls ability to maintain school attendance and perform adequately in school. The argument is made that the lack of sanitary napkins prevents girls from attending school during menstruation, and that even when sanitary napkins are available, lack of privacy and adequate sanitation makes it difficult for girls to change napkins. Some authors have argued that this may result in between days of school lost in a 180 day school year, Mooijman et al (2005). Many NGOs (including all the NGOs in this study) provide sanitary napkins to OVC girls they are supporting as part of their program. Our results are therefore somewhat surprising in that for every group supported OVC, non supported OVC, and non OVC, girls have a lower rate of absenteeism than boys. If menstruation was a significant cause of absenteeism we would have expected to see the opposite. In fact, menstruation appears to be less of an issue for girls than previously thought (some estimates put girls absenteeism as high as 18%, with the cause being an inability to deal with menses). This is the first study to report quantitative data on girl s absenteeism in Africa, and indicates that further study is required to clarify the issue. Further conclusions cannot be drawn as this was not a specific focus of the research and questions to differentiate the reasons for girls and boys absenteeism were not asked. The findings of reduced absenteeism of girls when compared to boys were consistent in control schools where there was no intervention to provide sanitary napkins. Our findings are in line with the recent publication by Oster and Thornton (2011), which concluded that actual lost time at school due to menses very small (approximately half a day a year in Nepal). This is not to suggest that provision of sanitary napkins, good hygiene and privacy are not an important human right for all young women, simply that our observations would reinforce the fact that menstruation is not a significant contributor to absenteeism. Drop out and gender There was no difference in the drop out rates between boys and girls in any of the schools in our study (6% for boys and 6% for girls). However, OVC who were supported did not have a significantly higher rate of drop out than non OVC, while non supported OVC of 91

95 both genders had significantly higher dropout rates. We can conclude therefore that the support being provided to OVC was protective against dropping out. Academic performance and gender Results indicate that on average, female students are half as likely to pass their National Form II and Form IV exams compared to male students. Across the different support groups, the same pattern holds true, female students are less likely to pass their national exams. Male OVC ever supported students are more likely to pass their Form IV exams compared to non OVC students, while female OVC ever supported students are less likely to pass the equivalent exam compared to their non OVC counterpart. Data suggests that educational support to OVC seems to have better impact on the academic performance of male students as opposed to female students. Costs and efficiencies and administrative burden The wide variation in costs per program largely reflects the variation in administrative oversight and follow up across the organizational approaches, as well as the fact that each organization contributes different percentages of the total education costs (ranging from 50% for Africare s block grant approach in Uganda to 90% for AVSI s scholarship program in Uganda). It is not clear how decision making should be made about what proportion of the overall costs should be covered by NGOs and this requires further research. It was observed that the larger the number of children in the program, the lower the proportion of administrative costs suggesting noticeable economies of scale and suggesting that large programs are more cost effective. It was also noted during qualitative interviews that block grant programs with small numbers of students did not provide sufficient funds to the school to provide the leverage required to enable the entire school to benefit from the grant. It was concluded that the larger the number of students enrolled in a block grant program the more effective the approach, while administering smaller numbers in block grant programs rapidly become as inefficient as running scholarship programs. As noted in the discussion about accountability, when funds are decentralized to communities or schools, there is an increasing administrative burden to oversee the program. The scholarship program in Tanzania did not invest the additional funds and administrative oversight necessary to ensure that all children enrolled in the program received an education. The scholarship program in Uganda took those steps and all children were accounted for. However, this was accomplished at significant additional cost, and raises questions of cost effectiveness of the approach. Limitations of the Study Lack of knowledge of the households from which the supported OVC came Many contextual factors which would help to explain issues raised in this study cannot be answered because funding was not sufficient to conduct a household survey of supported 92

96 OVC to understand their social, economic, parental, living, and other conditions. This lack of information does not affect the relevance or quality of the data collected at schools or in focus groups, but it does limit the ability to interpret some of the findings around absenteeism and drop out. It also limits the ability to provide direct comparisons between schools as the details of the populations served are not well known. Children aging out of the program Because of early statements about who constituted an OVC by PEPFAR, decisions were made early on by the various programs that educational support would not be continued beyond the age of 18 years (in the last year of project implementation this was reversed, but did not affect the cohort of students being studied. The consequence of this is that children who were begun in the program found themselves losing their educational support in Form IV (and sometimes below). In Tanzania this resulted in the majority of Form IV students being lost to follow up and this in turn affected the results for the final year of the cohort particularly for Form IV examinations. Lack of concentrated populations of supported OVC The finding that both of the scholarship programs (CRS and AVSI) did not have populations of 50 or more supported OVC in any individual secondary school only became clear as individual recipient lists were examined. This required that a much larger sample size of schools be made for both NGOs. In turn, directly matching control schools could not be chosen for all schools in the CRS and AVSI samples. At CRS schools there were often only five or six study students, and therefore the control populations were picked from the same schools. Differences in the educational systems between Uganda and Tanzania Children start school one year younger in Uganda than in Tanzania, with the result that the OVC in our sample in Tanzania were all one year older than their equivalent grade in Uganda. Fortunately, while a year s age difference is significant for development in younger age groups, it has little bearing on the ability to perform academically after the age of 14 and we do not anticipate that age would result in any differences in pass rates between the two countries. However, the later start date in Tanzania meant that many students in Form IV were over 18 and therefore no longer eligible for OVC support. This is reflected in the very low numbers of exam results for students in Form IV in Tanzania which in turn limited our ability to draw significant conclusions on Form IV academic performance in this environment. In 2007, Uganda introduced Universal Secondary Education, in which tuition fees were no longer charged for secondary school. In contrast, fees for secondary education are still charged in Tanzania. The fees in Tanzania are 20,000 TZS per term, which is then matched by the government. However, secondary education in Uganda is not free, and fees in Tanzania are not limited to the official government fees. The reality is that in both countries, the cost of actually attending secondary school comes to approximately the same (in excess of 100,000 Shillings). The actual fees charged are not standardized and 93

97 may differ from school to school, and even from year to year in the same school. We therefore do not believe that USE in Uganda presents any problem in making country tocountry comparisons. Lack of detailed information on country wide school performance The information needed to conduct performance comparisons of all intervention schools (regardless of NGO) in the country versus matching schools has not been made available from the Ministry of Education. This has severely limited the conclusions we can draw from the QUANT2 study and we can only observe trends rather than make clear statements about the broader impact of assistance to secondary schools. This does not affect the conclusions and recommendations being made in this report. Conclusions We can conclude from our study that providing support to children to attend secondary school can be successful whether through block grants or scholarships, and that in general, supported performed as well as their non OVC peers and sometimes better. We also concluded that of the two approaches (block grant and scholarship), block grants are simpler to administer, require less administrative costs and oversight to ensure that students receive the benefits, and are more cost effective than scholarships per student. However, we did observe that block grants become less effective with smaller numbers of students enrolled as not only are there losses in economies of scale, the leverage on the school is much reduced when only small amounts are provided. We were unable to determine the exact point at which block grants become most efficient. It was clear throughout our observations that provision of free access to schools alone is insufficient to ensure that OVC attend regularly or perform well. Additional support was required that addressed physical health, nutrition, social setting, economic well being, and other aspects of the child's overall well being. Tanzania specific: It is obvious that the Government of Tanzania's focus on increasing access to secondary education in rural areas through the building of ward level schools has been successful. However, the extremely poor performance at certificate level is of enormous concern reflecting serious problems with the quality of education, which needs to be addressed urgently. We noted that Tanzania's use of a national examination at Form II provides an extremely important benchmark for early decision making about appropriate interventions to improve student performance. It was observed to the research team on several occasions that this examination is no longer may even be phased out which would remove this excellent tool. 94

98 Uganda specific: Because Uganda only collects national standardized achievement data at Form IV, the Ministry has no way of knowing how the system is performing over time, as a whole, and earlier decision making about appropriate interventions to improve student performance is problematic. Recommendations General Recommendations: Investment in secondary school education for OVC is of value and is a program that PEPFAR and other stakeholders should consider supporting while encouraging local Ministries of Education to take on this role. n designing secondary school education programs for OVC, Governments, PEPFAR and other development partners should give a high priority to block grant programs. However, these should be done within a comprehensive approach that addresses the physical, mental and social wellbeing of children. To design their OVC block grant programs more efficiently, PEPFAR and other development partners should focus on areas with concentrations of OVC (especially in rural settings), and within those populations seek to serve larger numbers of students in smaller numbers of schools. Selection criteria for selecting OVC when resources are limited, should be clarified and standardized to ensure equity and transparency; should OVC be chosen on the basis of the most needy, or the most likely to succeed? Tanzania Specific Recommendations: In Tanzania access to education for OVC needs to be expanded; the Government of Tanzania, civil society, and the private sector, should work together to eliminate specific financial and other barriers to education. The Government of Tanzania should continue to use Form II examinations as a means of determining educational effectiveness. If this is impossible, then sampling of students and content areas can be used making the evaluation of effectiveness less costly. The Ministry of Education needs to standardize the collection, storage, and maintenance of school records (including attendance registers and grade reports), and these standards should be applied to private and church schools as well. Uganda Specific Recommendations: In Uganda, in addition to providing Universal Secondary Education, the Government of Uganda should eliminate financial and other barriers to education for OVC. 95

99 The Government of Uganda should give serious consideration to establishing additional standardized national examinations for earlier grades at secondary school to enable improved decision making about the quality of education. If this is impossible, then sampling of students and content areas can be used making the evaluation of effectiveness less costly. The Ministry of Education needs to standardize the collection, storage, and maintenance of school records (including attendance registers and grade reports), and these standards should be applied to private and church schools as well. Recommendations for Further Research: Research is required to determine what proportion of the total cost of education is the optimal amount to assure the best outcomes from support programs. Future research should employ comprehensive outcome measures and longitudinal approaches to study improvements in student access, teacher effectiveness, and student performance over time. 96

100 REFERENCES Akwara, P., Nourbary, B, AhKen, P, Johnson, K, Yates, R, Winfrey, W, et al. (2010) Who is the Vulnerable Child? Using survey data to identify children at risk in the era of HIV and AIDS. AIDS Care Vol.22, No Angrist, J. D., E. Bettinger, E. Bloom, E. M. King and M. Kremer. (2002) Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment, American Economic Review 92: Angrist, Joshua D., Eric Bettinger, and Michael Kremer. (2006) Long Term Educational Consequences of Secondary School Vouchers: Evidence from Administrative Records in Colombia. American Economic Review 96: Attanasio, Orazio, Erich Battistin, Emla Fitzsimons, Alice Mesnard and Marcos Vera Hernández. (2005). How Effective are Conditional Cash Transfers?: Evidence from Colombia. The Institute of Fiscal Studies Briefing Note No. 54. Banerjee, A., S. Cole, E. Duflo and L. Linden. (2007) "Remedying Education: Evidence from Two Randomized Experiments in India," Forthcoming Quarterly Journal of Economics. Barrera Osorio, Felipe, Leigh L. Linden, Miguel Urquiola. (2007) The Effects of User Fee Reductions on Enrollment: Evidence form a Quasi Experiment, Columbia University Department of Economics Mimeo. Bicego G, Rutstein S, Johnson K. (2003). Dimensions of the emerging orphan crisis in sub Saharan Africa. Social Science & Medicine. 56: Blackett Dibinga K, Anah K, Matinhure N. (2006) Innovations in education: the role of the education sector in combating HIV/AIDS. Africare s Office of Health and HIV/AIDS. Blasie Ayako A. (2006) Lessons of the Experience with Direct Support to Schools Mechanism: A Synthesis. Association for the Development of Education in Africa. Case A, Ardington C. (2006). The impact of parental death on school enrollment and achievement: Longitudinal evidence from South Africa. Demography. 43: Case A, Paxson C, Ableidinger J. (2004). Orphans in Africa: Parental Death, Poverty, and School Enrollment. Demography. 41: Chou S.Y, Liu J.T, Grossman M, Joyce T, Parental education and child health: Evidence from a natural experiment in Taiwan. Am Econ J Appl Econ : Gakidou, E, Cowling K, Lozano, R, Murray C, Increased educational attainment and its effect on child mortality in 175 countries between 1870 and 2009: a systematic analysis. Lancet 2010: 376: Glewwe, Paul and Pedro Olinto (2004). Evaluating the Impact of Conditional Cash Transfers on Schooling: An Experimental Analysis of Honduras PRAF Program. Final Report for USAID. International Food Policy Research Institute. Goodsmith, L (2004). The role of education in the protection, care and support of orphans and vulnerable children living in a world with HIV and AIDS. Geneva, UNICEF. 97

101 He, F., L. Linden and M. MacLeod. (2007) "Teaching What Teachers Don't Know: An Assessment of the Pratham English Language Program" Columbia University Department of Economics Mimeo. Kremer, Michael, Edward Miguel and Rebecca Thornton. (2007). Incentives to Learn, Manuscript. University of California at Berkeley, Department of Economics. Kurzinger ML, Pagnier J, Kahn JG, Hampshire R, Wakabi T, Dye TDV. (2008). Education status among orphans and non orphans in communities affected by AIDS in Tanzania and Burkina Faso. AIDS Care. 20(6): Larson B (2010). Costing of OVC Service Delivery in South Africa and Zambia. Center for Global Health and Development, Boston University Lynch S.M, Cohort and life course patterns in the relationship between education and health: a hierarchical approach. Demography : Miller CM, Gruskin S, Subramanian SV, Rajaraman D, Heymann SJ. (2006). Orphan care in Botswana's working households: growing responsibilities in the absence of adequate support. Am J Public Health. 96: Ministry of Education and Vocational Training (2009). Basic Educational Statistics in Tanzania (BEST). Dar es Salaam, MOEVT, United Republic of Tanzania Mooijman A, Snel M, Ganguly S, Shordt K. Strengthening Water, Sanitation and Hygiene in Schools A WASH guidance manual with a focus on South Asia. The Hague: IRC International Water and Sanitation Centre. Monasch R, Boerma JT. (2004). Orphanhood and childcare patterns in sub Saharan Africa: an analysis of national surveys from 40 countries. AIDS. 18:S55 S65 S55 S65 Muralidharan, Kartik and Venkatesh Sundararaman. (2006) Teacher Incentives in Developing Countries: Experimental Evidence from India, Working Paper. Harvard University Department of Economics Mushemeza ED. (2005) Direct Support to Schools Through the Block Grant Mechanism: Lessons From the Ugandan Experience. CODESRIA/ADEA Working Group on Education and Finance (WGEF). Nyamukapa C, Gregson S. (2005). Extended family's and women's roles in safeguarding orphans' education in AIDS afflicted rural Zimbabwe. Social Science & Medicine. 60: Oster E, Thornton R (2011). Menstruation, Sanitary Products, and School Attendance: Evidence from a Randomized Evaluation. AEJ: Applied Economics. 3(1): President's Emergency Plan for AIDS Relief (PEPFAR), Office of the Global AIDS Coordinator (2006). Orphans and Vulnerable Children: Programming Guidance for United States Government In Country Staff and Implementing Partners. Washington DC, PEPFAR Pritchett, L. (2004). Towards A New Consensus for Addressing the Global Challenge of the Lack of Education Copenhagen Consensus Challenge Paper in Education 98

102 Schady, Norbert and Maria Caridad Araujo (2006). Cash Transfers, Conditions, School Enrollment, and Child Work: Evidence from a Randomized Experiment in Ecuador. World Bank Policy Research Working Paper Schultz, T. Paul. (2004) School Subsidies for the Poor: Evaluating the Mexican Progresa Poverty Program, Journal of Development Economics, 74(1): Sengendo J, Nambi J. (1997) The psychological effect of orphanhood: a study of orphans in Rakai district. Health Trans Rev. 7 (Suppl): Social Monitoring and Early Response Unit (SMERU). SMERU s Rapid Assessment of Education Problems, and the JPS Scholarships and Block Grants Program in Four Provinces. World Bank Sept UNESCO (2006). Fact sheet: How many children in Africa reach secondary education? Paris, UNESCO. Vermeersch, Christel and Michael Kremer. (2004) School Meals, Educational Achievement, and School Competition: Evidence from a Randomized Evaluation World Bank Policy Re search Working Paper: No. 3523, World Bank, The Partnership for Child Development, UNICEF, UNAIDS (2002). Ensuring education access for orphans & vulnerable children. Washington DC, World Bank. World Bank. Improving Primary Education in Kenya. [Internet] 2001 Dec. Available from: 99

103 APPENDECES 1. Instrument: QUANT 1(Student Comparison) Quantitative Component # 1 (Student Comparison) Date of record review / / Data Collector: 1. School Information Obtain information through record review 1.1 Country (Select one) 1.2 Secondary School (Select one) Secondary Schools in Uganda (1) Africare 1 Intervention (2) Africare 2 Intervention (3) Africare 3 Intervention (4) Africare 4 Intervention (5) Africare 5 Intervention (6) Africare 1 Control (7) Africare 2 Control (8) Africare 3 Control (9) Africare 4 Control (10) Africare 5 Control (11) ICOBI 1 Intervention (12) ICOBI 2 Intervention (13) ICOBI 3 Intervention (14) ICOBI 4 Intervention (15) ICOBI 5 Intervention (16) ICOBI 1 Control (17) ICOBI 2 Control (18) ICOBI 3 Control (19) ICOBI 4 Control (20) ICOBI 5 Control (21) AVSI 1 Intervention (22) AVSI 2 Intervention (23) AVSI 3 Intervention (24) AVSI 4 Intervention (25) AVSI 5 Intervention (26) AVSI 1 Control (27) AVSI 2 Control (28) AVSI 3 Control (1) Uganda (2) Tanzania Secondary Schools in Uganda (31) Africare 1 Intervention (32) Africare 2 Intervention (33) Africare 3 Intervention (34) Africare 4 Intervention (35) Africare 5 Intervention (36) Africare 1 Control (37) Africare 2 Control (38) Africare 3 Control (39) Africare 4 Control (40) Africare 5 Control (41) CRS 1 Intervention (42) CRS 2 Intervention (43) CRS 3 Intervention (44) CRS 4 Intervention (45) CRS 5 Intervention (46) CRS 1 Control (47) CRS 2 Control (48) CRS 3 Control (49) CRS 4 Control (50) CRS 5 Control 100

104 1.3 Primary NGO Support for Secondary School Intervention (Select one) 1.4 Academic Start Date (MM/YYYY) 1.5 Academic End Date (MM/YYYY) (29) AVSI 4 Control (30) AVSI 5 Control (1) Africare (2) ICOBI (3) AVSI (4) CRS (5) None (6) Other: / / 2. Student Information Obtain information through record review 2.1 Year of Birth (YYYY) 2.2 Gender (Select one) (1) Male (2) Female 2.3 Student Status (Select one) 2.4 Student s class level (Select one) (1) OVC supported (2) OVC non supported (3) non OVC YEAR 1: 2006 YEAR 2: 2007 YEAR 3: 2008 YEAR 4: 2009 (1) OVC supported (2) OVC nonsupported (3) non OVC (1) F1 / S1 (2) F2 / S2 (3) F3 / S3 (4) F4 / S4 (5) Not in school (1) OVC supported (2) OVC nonsupported (3) non OVC (1) F1 / S1 (2) F2 / S2 (3) F3 / S3 (4) F4 / S4 (5) Not in school (1) OVC supported (2) OVC nonsupported (3) non OVC (1) F1 / S1 (2) F2 / S2 (3) F3 / S3 (4) F4 / S4 (5) Not in school (1) OVC supported (2) OVC nonsupported (3) non OVC (1) F1 / S1 (2) F2 / S2 (3) F3 / S3 (4) F4 / S4 (5) Not in school 2.5 Number of school days in a year 2.6 Number of absent mornings only 2.7 Number of absent afternoons only 2.8 Number of absent mornings and afternoons on same day 2.9 Did student drop out from school? (Select one) 2.10 If yes, what were the reasons? A. Inability to pay school fees (Select one) (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes 101

105 B. Poor academic performance (Select one) C. Moved out of area (Select one) D. Death (Select one) E. Illness (Select one) F. Marriage (Select one) G. Pregnancy (Select one) H. Other (Select one) 2.11 Did student pass the academic year? (Select one) (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing Other: (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing Other: (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing Other: (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing Other: (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing 2.12 If no, what were the reasons? A. Drop out (Select one) B. Poor academic performance (Select one) C. Other (Select one) (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing Other: (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing Other: (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing Other: (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing (0) No (1) Yes (99) Unknown / Missing Other: (0) No (1) Yes (99) Unknown / Missing 102

106 3. Exam Scores Obtain information through record review National exam score percentages (indicate O level exam type; if exam was not taken during the year, select N/A. If a particular exam was not taken, write 999) YEAR 1: (YYYY) (1) F2/S2 Exam (2) F4/S4 Exam (3) N/A YEAR 2: (YYYY) (1) F2/S2 Exam (2) F4/S4 Exam (3) N/A YEAR 3: (YYYY) (1) F2/S2 Exam (2) F4/S4 Exam (3) N/A YEAR 4: (YYYY) (1) F2/S2 Exam (2) F4/S4 Exam (3) N/A A. English % % % % B. CRE % % % % C. History % % % % D. Geography % % % % E. Mathematics % % % % F. Agriculture % % % % G. Physics % % % % H. Chemistry % % % % I. Biology % % % % J. Commerce % % % % K. Other L. Division Rank (Select one) Other: % (1) Division I (2) Division II (3) Division III (4) Division IV (5) Failed (99) Unknown / Missing Other: % (1) Division I (2) Division II (3) Division III (4) Division IV (5) Failed (99) Unknown / Missing Other: % (1) Division I (2) Division II (3) Division III (4) Division IV (5) Failed (99) Unknown / Missing Other: % (1) Division I (2) Division II (3) Division III (4) Division IV (5) Failed (99) Unknown / Missing Check that all sections are filled appropriately Date Name Signature Data Collector Supervisor Data Entry 1 Data Entry 2 103

107 2. Instrument: QUANT 2 (School Comparison) Quantitative Component # 2 (School Comparison) Date of record review (dd/mm/yy) / / Data Collector: 1. School Information Obtain information through record review 1.6 Country (Select one) 1.7 Region (Name) 1.8 District (Name) 1.9 Secondary School (Name) 1.10 Starting Year of Operation (YYYY) 1.11 Primary NGO Support for Secondary School Intervention (Select one) 1.12 Starting Year of NGO support (YYYY) (1) Uganda (2) Tanzania (1) Africare (2) ICOBI (3) AVSI (4) CRS (5) None (6) Other: 2A. School Information ( ) Obtain information through record review 2.1 Academic Year Gender M F M F M F M F 2.3 Total student enrollment 2.4 Number of OVC 2.5 Form II O level exam pass rate (%) 2.6 Form IV O level exam pass rate (%) 2.7 Form IV graduation rate (%) 2.8 Students enrolled in Form V 2.9 Number of teaching staff 2.10 Number of classrooms 2.11 Budget,,,, 104

108 (in thousands) 2B. School Information ( ) Obtain information through record review 2.1 Academic Year Gender M F M F M F M F 2.3 Total student enrollment 2.4 Number of OVC 2.5 Form II O level exam pass rate (%) 2.6 Form IV O level exam pass rate (%) 2.7 Form IV graduation rate (%) 2.8 Students enrolled in Form V 2.9 Number of teaching staff 2.10 Number of classrooms 2.11 Budget (in thousands),,,, Check that all sections are filled appropriately Date Name Signature Data Collector Supervisor Data Entry 1 Data Entry 2 105

109 3. Instrument: FGD (Community Representatives) Focus Group Discussions Community Representatives Date of interview (dd/mm/yy) / / Country School District Region NGO Support Participant Name Age Gender Occupation (Community Role) Interviewer s Initials 1. Selection of OVC There are more OVC than funds can support we will ask a few questions about selecting OVC for aid and support. 1.1 How do you define OVC? Within your community, do people know who is an OVC and who is not? How do community members know and find out? If they don t know, who does and how do they find out? What is the OVC burden in your community what are the problems faced by OVC? How many OVC What type of OVC (maternal, paternal, double) Food shortages 106

110 Shelter and crowding Limited psychosocial support Issues with education (primary and secondary) OVC doing additional work (livestock and field) Looking after siblings Stigma Other How do they decide which children should get support? Are there factors that are considered? Both parents deceased One parent deceased Households headed by grandparents Living parent is bedridden (very ill) Child headed household Evidence of abuse (sexual, physical, emotional) Child labor exploited Substandard household structure Do certain factors hold more weight when making the decision on which child gets support? How do you follow the progress of selected children? What type of educational support program is implemented in the school (block grant, scholarship, etc)? Do you think the educational support program is benefiting the individual children? Do you think this support has benefitted the school as a whole? What are the strengths of this support program? What are the weaknesses of the support program? 107

111 1.13 What improvements would to recommend to make the support program better? Check that all sections are filled appropriately. Supervisor Data Entry 1 Date Signature 108

112 4. Instrument: FGD (OVC Students) Focus Group Discussions OVC Students Date of interview (dd/mm/yy) / / Country School District Region NGO Support Name Age Gender Grade level Interviewer s Initials 1. Orphans and vulnerable children (OVC) I would like to ask a few questions about being an orphan and/or vulnerable children. 1.1 How do you define OVC? How do you feel about being identified as an OVC? Is there any stigma? How do other kids feel about OVC within the community? School? How do they treat you? How do adults feel about OVC within the community? School (teachers, headmaster, etc)? What do they say/do to show this? 109

113 1.5 What are the broad problems you face as an OVC? Food security Shelter (sleep at night) Physical, sexual abuse Education Legal protections Other 2. School Information I would like to ask a few questions about your education Why did you decide to go to school? Did you decide on your own? Did somebody else decide for you (who)? How do you feel about school? Do you like it why or why not? Describe the support you received from school. Uniforms and shoes Books HIV awareness Support clubs Other Do you know whether your tuition and other fees were supported by this block grant? Do you think children who are not supported are resentful if so, how? How well do OVCs as a group perform academically equal, better or worse compared to other students? With the educational support you have received, are there any changes in circumstances at school? 110

114 With the educational support you have received, are there any changes in circumstances at home? What do you like best about this support program? What do you like least about this support program? What changes do you think should be made to improve this program? What do you expect to do after you finish school? Did you receive any other form of external support outside of school? Check that all sections are filled appropriately. Supervisor Data Entry 1 Date Signature 111

115 5. Instrument: SSI (NGOs) Semi structured Interview Non Governmental Organizations (NGOs) Date of interview (dd/mm/yy) / / Country NGO Interviewer s Initials 1. Participant Information I would like to ask a few questions about you. 1.1 Name 1.2 Age 1.3 Gender 1.4 Title of position 1.5 Length of block grant program 1.6 Areas of block grant support 2. Educational Support Program for OVC I would to ask a few questions about the educational support program that your organization has implemented How many schools do you support under this program? How many secondary schools do you support under this program? Number of children served throughout entire program? Total number of children OVC with direct support OVC without direct support Non OVC 112

116 Number of children served within secondary schools? Total number of children OVC with direct support OVC without direct support Non OVC Please describe the model of fund disbursement: Give to child or family Give cash to school Give material support to school Other Please explain how this works. How do schools use the money? School level: school construction school improvement desk and chairs laboratory facilities and equipment textbooks lavatories water tanks generators lighting computers Child level: Tuition Uniforms Shoes Desk fees Exam fees Other fees Food Sanitary pads 113

117 What was the process of disbursing funds to the school? How did you decide the amount each receives? When do schools receive this support (beginning of term, monthly, etc) How many OVC do schools need to support? Who makes the decisions to allocate funds? Please describe the level of involvement your organization contributes in this process? How much does tuition cost for one student? For the students that are supported, what percentage of their tuition is covered in this program? Are there any other fees for students? Please list the cost of the various fees (if known). Salary top off PTA fees Sport fees Desk fees Exam fees Building fees Other Does the program provide any additional fee payments for students that are supported? Do you think that the block grant is helping OVC? Are they attending school more? Are they doing better in school? How has the block grant affected the school in general and the performance of other children not specifically supported? School s reputation Attracted more students Better academic 114

118 performance Other Will schools continue to receive educational support in the future? If not what does the organization intend to do with the children it has supported that are still in school? In your experience, what are the strengths of the block grant program? In your experience, what are the weaknesses of the block grant program? If you were redesigning the program, what would you do differently? Check that all sections are filled appropriately. Supervisor Data Entry Date Signature 115

119 6. Instrument: SSI (Headmaster / Deputy Headmaster) Semi structured Interview Headmaster / Deputy Headmaster Date of interview (dd/mm/yy) / / Country School District Region NGO Support Interviewer s Initials 1. Participant Information I would like to ask a few questions about you. 1.1 Name 1.2 Age 1.3 Gender 1.4 Title of Position 1.5 Total years as a teacher Total years as headmaster of this school Total years as headmaster of any school 1.8 What are your qualifications 2. School Information I would like to ask a few questions about the school. 2.1 When was this school founded? 116

120 What are some of the important issues or problems faced by the school? Lack of Electricity Lack of Water Old infrastructure Insufficient class rooms Teen pregnancy Other What is the cost of tuition for one year of school? Are there any other fees for students? Please list the cost of the various fees. Salary top off PTA fees Sport fees Desk fees Exam fees Building fees Other When are fees collected? Start of term, monthly, weekly, etc. What happens when all or part of fees are unpaid? Please describe how fees are managed. Do you have separate account for each fee category? Do you pool fees into one account? How are expenditures accounted for? 3. Block Grant Support for OVC [NOTE: ONLY FOR INTERVENTION SCHOOLS] I would like to ask a few questions about the block grant that the school receives Whom did you receive block grant from? Please describe how you received the block grant. Was school asked to participate, did school apply, etc? 117

121 What was the process of the block grant? How much money does the school receive? How many children were supported? How does the school use the money? Unrestricted cash Conditional grant (if so what are the conditions) Scholastic material (textbooks, laboratory equipment, etc) Food, uniforms, shoes, sanitary pads, etc Other How does the school make the decisions to allocate funds? What percentage of the school budget is comprised of educational support received on an annual basis? Do you think that the block grant is helping OVC? Are they attending school more? Are they doing better at school? How has the block grant affected the school in general and the performance of other children not specifically supported? School s reputation Attracted more students Better academic performance Other How long has the school received this block grant program? 118

122 Will it continue to receive block grant support in the future? If not what does the school intend to do with the children it supported? What are the strengths of the block grant program? What are the weaknesses of the block grant program? What would you do differently? 4. Other Educational Support I would like to ask a few questions about other educational support that the school receives Does the school receive any other educational support? What is the form of this support? Paying tuition Paying other fees Scholastic material Food, uniforms, shoes, sanitary pads, etc Other 4.3 Who were the donors? How does the school make the decision to use / allocate this educational support? How many children are supported and how does the school decide which children get support? What percentage of the school budget is comprised of this type of educational support received on an annual basis? How long have you had this support for? Will you continue getting this support in the future? How has this educational support benefitted children? How has this educational support benefitted the school? 119

123 Check that all sections are filled appropriately. Supervisor Data Entry Date Signature 120

124 7. Caregiver Consent Form CAREGIVER CONSENT FORM Focus Group Discussions EVALUATING THE EFFECTIVENESS OF EDUCATIONAL BLOCK GRANTS TO ORPHANS AND VULNERABLE CHILDREN Background I am from and Boston University in the United States. You are being asked to participate because you are a caregiver to an orphan or a vulnerable child that is currently going to secondary schools. Many factors play a role in deciding whether or not children go to school, including gender, health status, and poverty. Being orphaned or made vulnerable can also play a role in whether a child goes to school. A team of researchers from Boston University and is conducting a study to help better understand whether programs aimed at providing educational support to OVC is making a difference to the schools and children that receive them. Purpose The purpose of this study is to find out views and experiences that your child has as a student going to secondary school. What Happens In This Research Study You will be one of approximately subjects to be asked to participate in this study. All or part of the research in this study will take place at the following location(s): Boston University Medical Center. If you agree to be in this study, I will ask your child some questions about their views and experiences as an orphan or a vulnerable child going to school. They will be asked questions in the presence of other people in a group setting. Risks and Discomforts The interview/discussion may take time to complete and this may cause your child some inconvenience. They may also feel uncomfortable answering the questions in the presence of other participants. They do not have to answer questions that make them feel uncomfortable. There may be unknown risks/discomforts involved. Study staff will update you and your child in a timely way on any new information that may affect your health, welfare, or decision to stay in this study Potential Benefits You will receive no direct benefit from your participation in this study. However, your participation may help the investigators better understand the benefits of educational programs to the children they serve. Alternatives Your alternative is to not participate in the study. Subject Costs and Payments There are no costs to you for participating in this research study. You will not be paid to participate in this research study. 121

125 Confidentiality Information from this study may be reviewed and photocopied by state and federal regulatory agencies such as the Office of Human Research Protection as applicable, and the Institutional Review Board of Boston University Medical Center. Information from this study may be used for research purposes and may be published; however, your child s name will not be used in any publications. Subject's Rights By consenting to participate in this study you do not waive any of your legal rights. Giving consent means that you have heard or read the information about this study and that you agree to participate. You will be given a copy of this form to keep. If at any time you withdraw from this study you will not suffer any penalty or lose any benefits to which you are entitled. You may obtain further information about your rights as a research subject by calling the Office of the Institutional Review Board of Boston University Medical Center at If this study is being done outside the United States you can ask the investigator for contact information for the local Ethics Board. The investigator or a member of the research team will try to answer all of your questions. If you have questions or concerns at any time, or if you need to report an injury while participating in this research, contact at. Right to Refuse or Withdraw Taking part in this study is voluntary. You have the right to refuse to take part in this study. If you decide to be in the study and then change your mind, you can withdraw from the research. Your participation is completely up to you. Your decision will not affect your being able to get health care at this institution or payment for your health care. It will not affect your enrollment in any health plan or benefits you can get. If you choose to take part, you have the right to stop at any time. If there are any new findings during the study that may affect whether you want to continue to take part, you will be told about them as soon as possible. The investigator may decide to discontinue your participation without your permission because he/she may decide that staying in the study will be bad for you, or the sponsor may stop the study. Signature / Thumb Print Signing this consent form indicates that you have read this consent form (or have it read to you), that your questions have been answered to your satisfaction, and that you voluntarily agree to participate in this research study. You will receive a copy of this signed assent form. Subject (Signature / Thumb Print and Printed Name) Person Obtaining Consent (Signature and Printed Name) Date Date 122

126 8. Child Assent Form CHILD ASSENT FORM Focus Group Discussions EVALUATING THE EFFECTIVENESS OF EDUCATIONAL BLOCK GRANTS TO ORPHANS AND VULNERABLE CHILDREN Background I am from and Boston University in the United States. You are being asked to participate because you are an orphan or a vulnerable child that is currently going to secondary schools. Many factors play a role in deciding whether or not children go to school, including gender, health status, and poverty. Being orphaned or made vulnerable can also play a role in whether a child goes to school. A team of researchers from Boston University and is conducting a study to help better understand whether programs aimed at providing educational support to OVC is making a difference to the schools and children that receive them. Purpose The purpose of this study is to find out views and experiences that you have as a student going to secondary school. What Happens In This Research Study You will be one of approximately subjects to be asked to participate in this study. All or part of the research in this study will take place at the following location(s): Boston University Medical Center. If you agree to be in this study, I will ask you some questions about your views and experiences as an orphan or a vulnerable child going to school. You will be asked questions in the presence of other people in a group setting. Risks and Discomforts The interview/discussion may take time to complete and this may cause you some inconvenience. You may also feel uncomfortable answering the questions in the presence of other participants. You do not have to answer questions that make you feel uncomfortable. There may be unknown risks/discomforts involved. Study staff will update you in a timely way on any new information that may affect your health, welfare, or decision to stay in this study Potential Benefits You will receive no direct benefit from your participation in this study. However, your participation may help the investigators better understand the benefits of educational programs to the children they serve. Alternatives Your alternative is to not participate in the study. Subject Costs and Payments There are no costs to you for participating in this research study. You will not be paid to participate in this research study. 123

127 Confidentiality Information from this study may be reviewed and photocopied by state and federal regulatory agencies such as the Office of Human Research Protection as applicable, and the Institutional Review Board of Boston University Medical Center. Information from this study may be used for research purposes and may be published; however, your name will not be used in any publications. Subject's Rights By consenting to participate in this study you do not waive any of your legal rights. Giving consent means that you have heard or read the information about this study and that you agree to participate. You will be given a copy of this form to keep. If at any time you withdraw from this study you will not suffer any penalty or lose any benefits to which you are entitled. You may obtain further information about your rights as a research subject by calling the Office of the Institutional Review Board of Boston University Medical Center at If this study is being done outside the United States you can ask the investigator for contact information for the local Ethics Board. The investigator or a member of the research team will try to answer all of your questions. If you have questions or concerns at any time, or if you need to report an injury while participating in this research, contact at. Right to Refuse or Withdraw Taking part in this study is voluntary. You have the right to refuse to take part in this study. If you decide to be in the study and then change your mind, you can withdraw from the research. Your participation is completely up to you. Your decision will not affect your being able to get health care at this institution or payment for your health care. It will not affect your enrollment in any health plan or benefits you can get. If you choose to take part, you have the right to stop at any time. If there are any new findings during the study that may affect whether you want to continue to take part, you will be told about them as soon as possible. The investigator may decide to discontinue your participation without your permission because he/she may decide that staying in the study will be bad for you, or the sponsor may stop the study. Signature / Thumb Print Signing this consent form indicates that you have read this assent form (or have it read to you), that your questions have been answered to your satisfaction, and that you voluntarily agree to participate in this research study. You will receive a copy of this signed assent form. Child Assent (Signature / Thumb Print and Printed Name) Person Obtaining Consent (Signature and Printed Name) Date Date 124

128 9. Other Consent Form RESEARCH CONSENT FORM Semi-Structured Interviews and Focus Group Discussions EVALUATING THE EFFECTIVENESS OF EDUCATIONAL BLOCK GRANTS TO ORPHANS AND VULNERABLE CHILDREN Background I am from and Boston University in the United States. You are being asked to participate because of your involvement with educational assistance programs. Many factors play a role in deciding whether or not children go to school, including gender, health status, and poverty. Being orphaned or made vulnerable can also influence whether a child goes to school. A team of researchers from Boston University and is conducting a study to help better understand whether programs aimed at providing educational support to OVC is making a difference to the schools and children that receive them. Purpose The purpose of this study is to find out the views and experiences of headmasters, community representatives and nongovernmental organization (NGO) staffs that are involved in running these educational support programs. What Happens In This Research Study You will be one of approximately subjects to be asked to participate in this study. All or part of the research in this study will take place at the following location(s): Boston University Medical Center. If you agree to be in this study, I will ask you some questions about your views and experiences regarding educational support programs for OVC. You may be asked questions in the presence of other people in a group setting. Risks and Discomforts The interview/discussion may take time to complete and this may cause you some inconvenience. You may also feel uncomfortable answering the questions in the presence of other participants. You do not have to answer questions that make you feel uncomfortable. There may be unknown risks/discomforts involved. Study staff will update you in a timely way on any new information that may affect your health, welfare, or decision to stay in this study Potential Benefits You will receive no direct benefit from your participation in this study. However, your participation may help the investigators better understand the benefits of educational programs to the children they serve. Alternatives Your alternative is to not participate in the study. Subject Costs and Payments There are no costs to you for participating in this research study. You will not be paid to participate in this research study. 125

129 Confidentiality Information from this study may be reviewed and photocopied by state and federal regulatory agencies such as the Office of Human Research Protection as applicable, and the Institutional Review Board of Boston University Medical Center. Information from this study may be used for research purposes and may be published; however, your name will not be used in any publications. Subject's Rights By consenting to participate in this study you do not waive any of your legal rights. Giving consent means that you have heard or read the information about this study and that you agree to participate. You will be given a copy of this form to keep. If at any time you withdraw from this study you will not suffer any penalty or lose any benefits to which you are entitled. You may obtain further information about your rights as a research subject by calling the Office of the Institutional Review Board of Boston University Medical Center at If this study is being done outside the United States you can ask the investigator for contact information for the local Ethics Board. The investigator or a member of the research team will try to answer all of your questions. If you have questions or concerns at any time, or if you need to report an injury while participating in this research, contact at. Right to Refuse or Withdraw Taking part in this study is voluntary. You have the right to refuse to take part in this study. If you decide to be in the study and then change your mind, you can withdraw from the research. Your participation is completely up to you. Your decision will not affect your being able to get health care at this institution or payment for your health care. It will not affect your enrollment in any health plan or benefits you can get. If you choose to take part, you have the right to stop at any time. If there are any new findings during the study that may affect whether you want to continue to take part, you will be told about them as soon as possible. The investigator may decide to discontinue your participation without your permission because he/she may decide that staying in the study will be bad for you, or the sponsor may stop the study. Signature / Thumb Print Signing this consent form indicates that you have read this consent form (or have it read to you), that your questions have been answered to your satisfaction, and that you voluntarily agree to participate in this research study. You will receive a copy of this signed consent form. Subject (Signature / Thumb Print and Printed Name) Person Obtaining Consent (Signature and Printed Name) Date Date 126