Financial Education for Children and Youth: A Systematic Review and Meta-analysis

Transcription

1 Financial Education for Children and Youth: A Systematic Review and Meta-analysis Aflatoun Working Paper C Support: This review was made possible through the generous support of Aflatoun Child Social and Financial Education. Suggested Citation: O Prey L and Shephard D (2014) Financial Education for Children and Youth: A Systematic Review and Meta-analysis. Aflatoun Working Paper C. Accessed at Corresponding Author: Llorenc O Prey, School for Policy Studies, University of Bristol, 8 Priory Road, Bristol, BS8 1TZ, United Kingdom: [email protected] Working Paper C 1

2 Executive Summary This paper reports on the findings of the systematic review and meta-analysis exploring the effectiveness of financial education aimed at children and youth. Overall, 21 independent experimental studies were uncovered, suggesting that the evidence base is broader than previously thought. The studies reported on a broad range of programmes, including the content that was taught, the way it was delivered, and the context in which children and youth experienced their education. There exists a varying degree of methodological quality across included studies, with some of the most rigorous studies being the most recent suggesting that the evidence base is improving over time. Where there was sufficient and comparable data to do so, the outcomes reported in each study were combined to produce an overall effect-size. The results suggest that financial education is effective in improving knowledge, attitudes and behaviours. The strongest and most significant finding was regarding knowledge gains. The meta-analysis found overall modest improvements in attitudinal and behaviour change. This confirms that generally, financial education does produce tangible gains in financial capability amongst children and youth. Summary of Overall Effects of Financial Education: Knowledge gains: 0.18 (0.09, 0.27) Attitudinal change: 0.08 (0.01, 0.15) Behaviour change: 0.07 (0.03, 0.11) Standard Mean Differences (95% Confidence Intervals) Random Effects Model Compared to research exploring other educational programmes, this metaanalysis presents robust findings in support of financial education. Across the studies included in this review, there was variation in the effectiveness of different programmes, with some providing greater effects than others. The paper goes onto explore and expand on these issues, and their potential implications for practice. Working Paper C 2

3 Background This review collects and synthesizes all the available experimental evidence that analyses the change brought about by financial education programmes aimed at children and youth. Through meta-analysis the review will explore the links between education programmes and intended outcomes. The Systematic Review seeks to: 1. Explore the efficacy and effectiveness of financial education programmes aimed at children and youth. 2. Explore causal links between financial education and outcomes. 3. Within different approaches and programmes, identify which types of programmes are showing promise and which are less effective. 4. Provide a definitive statement of the quality and scope of the evidence regarding financial education aimed at children and youth. Search of the Literature The authors of the review completed a systematic search of the literature, which produced over 1400 potential studies. These studies were then screened against a number of criteria, including: Intervention: Financial Education Population: Children and Youth Study Design: Randomized Control Trail Design Studies that met all of the inclusion criteria were included in the review. The screening process found 21 unique studies reported in 15 papers. Two papers were found to report on 8 independent studies. A summary of the literature search can be found overleaf. Working Paper C 3

4 Table, summary of the literature search: Ongoing studies that could not be included: The search also uncovered a number of studies that are ongoing, and were unable to be included in the review because they were still under completion. Tajikistan CRCT of the efficacy of a programme called Aflateen+ that combines a youth social and financial programme with a sexual and reproductive health intervention designed for young women at risk of early marriage. Data collection is complete with the final analysis expected in Peru CRCT of two variations of the youth social and financial education programme called Aflateen being led by Matthew Bird. The study seeks to determine if a different goal setting mechanism in the curriculum affects savings behavior. Data collection is complete with the final analysis expected in Colombia RCT looking into the effectiveness of youth access to accounts combined with text messaging once or twice a month, with one group having financial education oriented text messaging instead of just reminders to save. China CRCT looking into the effectiveness of the Aflatoun and Aflateen Working Paper C 4

5 social and financial education curriculum by comparing a control group that receives no programme, with the standard programme group plus individual savings, and a third group using the same curriculum but combining it with a group savings mechanism. Rwanda CRCT of a social and financial education for children called Aflatoun. The study is using both children outcomes and teacher observation methods to determine causal pathways between teacher methodology and children s social and financial outcomes. A CRCT of a school-based financial education programme in Turkey, conducted by Alan Ertaç. A cluster randomized control trial evaluating the YouthSave Ghana Experiment, with results expected later in 2014 (Chowa et al, 2012). Suubi Bridges a follow-up experiment looking into the Suubi project in Uganda. Led by Fred Ssewamala. Characteristics of Included Studies The 21 unique studies display a broad range of characteristics in the way they were designed and executed. Methodology Nine studies reported clustering at the classroom or school level, and 11 at the individual level, and one used an encouragement design analysed at the individual level. Most studies employed a treatment as usual control, in which participants continue as normal (f = 14). One study reported using a wait-list control, in which participants assigned to the control have financial education delayed until after the study (Supanantaroek, 2013). Three studies used multiple trial arms to explore the effectiveness of different treatments. Most studies sought to create a balance sample size between intervention and control. Most studies used pre- and post- testing to explore change over time. Some studies reported collecting data at multiple time-points, including after the programme had finished. Participants Studies were located across the globe, although most were conducted in the US (f = 11). Uganda (f = 5), Ghana (f = 1), Tunisia (f = 1), Tanzania (f = 1), Brazil (f = 1), and Italy (f = 1) were also represented. Studies varied in size, the smallest involving just 67 young people, and the largest initially involving over 240,000. All studies sought to understand co-educational programmes that were delivered to both boys and girls without exception. Intervention All studies provided at least partial descriptions of the financial education components under investigation. 11 unique programmes were investigated, Working Paper C 5

6 including two trials that explored the effectiveness of Aflatoun programme in Ghana and Uganda, three trials looking into the effectiveness of the SEED/Suubi Project in Uganda, and six trials of variations of Visa s online credit education program. Across the different programmes, there was considerable diversity in the content and mode of delivery. The majority were classroom based, in which children and youth would access content through exercises and materials delivered by a teacher (f = 12). Other studies explored media delivered content, including the web based materials (f = 6), computer based materials linked to classroom content (f = 2), and one explored the effectiveness of edutainment, a TV show which included a financial education component. All studies included some form of financial education that looked to develop either knowledge and skills, or the importance of saving and asset accumulation. For an outline of study characteristics, please see overleaf. Working Paper C 6

7 Table: Summary of Included Studies: Study 1 Becchetti et al (2012) 2 Berry et al (2013) 3 Bruhn et al (2013) 4 Faircloth et al (1986) 5 Gartner et al (2005a) 6 Gartner et al (2005b) Type of Mode of Country Focus Duration Trial Delivery N CRCT Italy Financial 16 hours Classroom 944 education CRCT Ghana Mixed, 20 hours Classroom 5,000 Financial and Social Education CRCT Brazil Financial 72 hours Classroom 26,000 education CRCT USA Financial education RCT USA Financial education RCT USA Financial education 3 hours Classroom, on computer At own Media, pace web At own pace Media, web ,112 80,982 7 Gartner et al (2005c) 8 Gartner et al (2005d) 9 Gartner et al (2005e) 10 Gartner et al (2005f) 11 Hinojosa et al (2009) 12 Jamison et al (2012) RCT USA Financial education RCT USA Financial education RCT USA Financial education RCT USA Financial education RCT USA Financial education CRCT Uganda Financial education At own pace Media, web At own Media, pace web At own Media, pace web At own Media, pace mail and web 15 hours Classroom, on computer 82,600 8,190 11,686 39, Classes 15 hours Classroom 2, Jorgensen and Tonsburg (2011) Encouragement CRCT Tanzania Financial education 14 Karimli (2013) CRCT Uganda Mixed, Financial and Health Education 15 McLean (2010a) RCT USA Financial education 14 hours Media, TV 2, to 20 hours Classroom hours Classroom McLean (2010b) 17 McNeil et al (2008) 18 Premand et al (2012) 19 Ssewamala et al (2008) 20 Ssewamala et al (2009) 21 Supanantaroek et al (2012) RCT USA Financial education RCT USA Financial education RCT Tunisia Mixed, Financial and Entrepreneurial Education CRCT Uganda Mixed, Financial and Health Education CRCT Uganda Mixed, Financial and Health Education CRCT Uganda Mixed, Financial and Social 12 hours Classroom 75 2 hours Classroom weeks intensive Classroom 1, hours Classroom 96 Not stated Classroom hours Classroom 1,746 Working Paper C 7

8 Risk of Bias in Included Studies Before analysing and synthesizing the results of each study, it is important to understand how each study was completed, and if any potential weaknesses in its design or execution could have influenced (biased) the results. A bias is a systematic error in results or inferences. Biases can operate in either direction: different biases can lead to an underestimation or overestimation of the true intervention effect. Cochrane Handbook (2014) Against each study, assessments were made across a number of potential risks by the review authors using the Cochrane Collaboration s risk of bias tool (Higgens et al, 2011). With each potential risk, reviewers made an independent assessment either a low risk of bias, one that was unlikely to significantly alter the results, or a high risk of bias, one that seriously weakens confidence in the results. In cases where there was insufficient information from a paper to make an accurate assessment, attempts were made to seek clarification from authors. Several authors did not respond, in which case the judgment was left as an unclear risk of bias: a plausible bias that raises some doubt about the results. In completing the risk of bias assessment, a number of methodological weaknesses were identified. Random sequence generation: All studies reported randomly assigning participants to either intervention or control groups. Eight studies provided sufficient information to ascertain that the methods used to randomly allocate schools or participants were at a low risk of bias. A number of studies (f = 3) reported using stratification as a means of ensuring the comparability of intervention and control groups. This process involved dividing schools or participants into groups according to certain characteristics, such as geographical location (eg, urban and rural) prior to random allocation. The precise method of randomisation was not reported in 10 studies, and we were unable to clarify with authors. This makes it difficult to ascertain if selection bias was likely to have occurred and, if so, in which direction it may have biased the results. Therefore an assessment of unclear risk of bias was made. Allocation Concealment: Allocation concealment was not practically possible in the majority of studies, especially where allocation occurred at the school or district level. Where adequate random sequence generation methods were used, a low risk of bias has been assumed, as it would ensure that allocation was down to chance. Working Paper C 8

9 Bruhn et al (2013) reported that after allocation, some school districts requested that at least one school be allocated into the intervention school, requiring three schools, chosen at random, to be transferred over from control to intervention group. In this instance a low risk of bias has been assumed as steps were taken to ensure a measure of chance in the final allocation. Of the remaining studies, it was not possible to determine whether allocation concealment had been appropriately carried out, so therefore an unclear risk of bias was assumed. Blinding of participants and personnel: In many instances, it was not feasible to completely blind participants or personnel as to which arm the school or individual had been allocated, especially for those in clusters or in the intervention arm. In this instance, a low risk of bias has been assumed. Hinojosa et al (2009) reported that allocation influenced attrition, in which teachers that found themselves in the control group either requested to be in the intervention group or decided not to continue with the study. In this instance a high risk of bias has been assumed as it was clear that an absence of blinding significantly influenced data collection. Blinding of outcome assessment: Three studies reported that it was not possible to blind data collectors as to the assignment of particular schools or participants, however took steps to control for sampling or measurement error. In this instance a low risk of bias has been assumed. It was not clear from 18 studies that steps were taken to either blind data collectors as to the allocation of intervention and control arms, or to take measures to control for reporting bias. In this instance, an unclear risk of bias is assumed as it is not possible to rule out if and how this bias may have influenced results. Incomplete data: Most studies reported high rates of attrition (drop-out). The way in which missing data from attrition is handled can have a significant impact on the results. Two studies reported handling attrition and missing data by analysing outcomes based on intention-to-treat analysis. This form of analysis attempts to control for missing data, and therefore assumes a low risk of bias. Similarly, in studies that had low or no reported attrition (f= 4) a low risk of bias is assumed. In cases (f = 3) where it was unclear how attrition was handled, and therefore the potential to bias the results, an unclear risk of bias was assumed. In one case (Jamison et al 2014), the amount of attrition did not differ between groups; Working Paper C 9

10 however, a mixture of analytical methods to deal with missing data made the existence and direction of any bias difficult to interpret as such an unclear risk of bias was assumed. Two studies appeared to report a treatment-on-treated analysis of the results. This ignores drop-out, and only analyses data for those participants that completed the study. This type of analysis is likely to overestimate the effectiveness of a programme to the extent to which those that dropped out display different characteristics and outcomes to those that finished the programme, for example, lower levels of motivation or knowledge gains. In this instance a high risk of bias has been assumed. All of the Gartner et al (2005) studies reported extremely high rates of attrition. This was due to the nature of the intervention, however it does present significant potential to bias the final results. Therefore a high risk of bias has been assumed. For Becchetti et al (2013), it was not clear exactly how sample size and attrition were reflected in the final analysis, with different analyses and papers reporting different sample sizes. Attempts to contact the authors to clarify were unsuccessful, so in this instance a high risk of bias is assumed. The effect of Becchetti on the Financial Knowledge meta-analysis was subjected to sensitivity analysis but did not substantively affect the overall findings. Selective reporting: We were unable to find published protocols for any included study. This made it difficult to determine selective reporting bias when a study only publishes data that favours a particular hypothesis. All studies reported outcomes that are consistent with the study objectives, and where there was no reason to suggest selective reporting such as not reporting negative findings, a low risk of bias is assumed. Where there was insufficient information, an unclear risk of bias was assumed. With Berry et al (2013), it was unclear if the composite scores used to communicate the results accurately reflected all the data that was collected. Without clarification, a high risk of bias has been assumed. Other sources of bias: Included studies were also screened for a variety of other potential risks. Most studies reported using survey questionnaires (instruments) to collect data that were designed specifically for the programme under investigation. It was not always clear if assessments were made to ensure that instruments were both valid and reliable. Validity, the extent to which an instrument produces stable and consistent results, and reliability, the extent to which it accurately measures the concepts, skills and competencies under investigation, can influence data collection in a number of ways. Without a process of validation assessing whether an instrument is both accurate (valid) and consistent (reliable) it is difficult to determine how the results may have been affected. Another aspect that was assessed for potential bias was implementation fidelity Working Paper C 10

11 the degree to which a programme was delivered as intended. Through exploring and measuring whether an intervention has been implemented with fidelity, it is possible to understand how and why a programme works, and the extent to which outcomes could be improved. It raises confidence in the results, especially in those studies that report no effect. When fidelity is not assessed, it is difficult to determine whether the observed results are due to the intervention itself, or an undocumented counterfactual. Very few studies reported taking steps to ensure implementation fidelity. Most reported the programmed number of hours children and youth were expected to attend, but did not go onto explore the actual number of hours they received. This is perhaps the most basic measure of implementation fidelity in education research, and included studies were marked as being potentially open to bias if, at a minimum, participant exposure was not reported to have been observed. A summary of risks can be viewed in this graph, and a table overleaf. Table: Risk of Bias Graph Working Paper C 11

12 Working Paper C 12

13 Publication Bias: Another potential bias is that of publication bias, the possibility that publication of research depends on the nature and direction of the results. For example, some areas of effectiveness research tend to only make studies that report positive outcomes towards an intervention publically available, over those that do not. Assuming the assumption holds that meta-analysed data displays a normative distribution, then we are able to assess symmetry in the data. Though subjective assessment of funnel plots, it appears that data is broadly symmetrical for knowledge and attitudes, suggesting that there is a low risk of publication bias in the sample of included studies. However, it is notable that all studies of behaviour were on the positive end. Table: Funnel Plot of Knowledge Outcomes: Table: Funnel Plot of Attitude Outcomes: Working Paper C 13

14 Table: Funnel Plot of Behaviour Outcomes: Working Paper C 14

15 Results Of the included studies, we extracted all outcome data that could be located using published and unpublished sources, including contacting authors. It was not possible to include data from some studies as there was insufficient information to fully synthesis the data within a meta-analysis. The Gartner et al studies (2005) and Ssewamala (2008) did not provide sufficient data to include their analyses, and were contacted to provide clarification without success. In the case of Ssewamala, discussions with members of the research group indicated that the SEED study did not collect financial outcomes for both treatment and control groups because the study was focused on psychosocial outcomes only. Knowledge Eleven studies provided comparable knowledge outcomes. In comparing the results between the intervention and control groups after the programme had finished, six studies found clear knowledge gains for the intervention group. Five studies produced inconclusive results, in which the 95% confidence interval crosses the line of no effect. Table: Knowledge Outcomes, Random Effects Model: Combining the results of all studies together using a random effects model, we find that financial education has a moderate effect on knowledge outcomes (0.18). Statistical heterogeneity is high and significant (I 2 = 88%, p<0.001), suggesting that the results should be interpreted with caution, and that effect sizes reported may be accounted for by the differences within the included studies. Note: The highest effect size of 0.95 (Faircloth et al, 1986) tested knowledge immediately after the intervention which is likely to inflate the outcome in favour of intervention. The study was comparing computer assisted financial education with reading based financial education. Working Paper C 15

16 Table: Knowledge Outcomes, Fixed Effects Model: Using a fixed effects model to combine the results, we find similar positive results (0.18, CI 0.16, 0.20). Again, statistical heterogeneity is high and significant (I 2 = 88%, p<0.001). Working Paper C 16

17 Attitudes Nine studies provided comparable attitudinal outcomes. We found that three studies produced clear results in favour of the financial education, and six studies produced inconclusive results. Table: Attitude outcomes, Random Effects Model: In combining the results together, the intervention improves attitudinal outcomes by 0.08 standard deviations in favour of financial education. The effect size is small with the 95% confidence interval approaching the line of no effect (0.01 to 0.15). Heterogeneity was moderate and statistically significant (I 2 = 79%, p < 0.001). Table: Attitude outcomes, Fixed Effects Model: Using a fixed effects model to combine the results, we find a small effect size (0.06, CI 0.04, 0.09). Note: The data reported here for Ssewamala et al (2009) has been treated slightly differently. Due to significant variation in baseline differences between groups, gain scores were used to establish an overall effect size. This gives a more accurate indication of the overall change brought about by the programme. Working Paper C 17

18 Behaviours Six studies provided comparable behaviour outcomes, of which five produced results in favour of the financial education. One study, Premand et al, reported whether participants accessed loans as a result of attending the intervention. The other four studies assessed self-reported savings behaviour. Table: Behaviour outcomes, Random Effects Model: When all six studies results were combined applying a Random Effects model the effect-size was 0.07 (95% CI = 0.03, 0.11, p < 0.001). Statistical heterogeneity was relatively low and statistically insignificant (I 2 = 53%, p = 0.06), suggesting that the results are more consistent than knowledge and attitude findings. Table: Behaviour outcomes, Fixed Effects Model: The use of a Fixed Effects model reduced the overall effect-size to 0.06 with small variance (95% CI: 0.04, 0.07). This is due the fact that the one study, Bruhn et al (2013), dominated the results in this model with 81.7% of the weight, indicating that the Random Effects model is preferable. Note: Karimli (2013) data was transformed from an Odds Ratio to SMD to include into the meta-analysis using the method suggested in Chinn However, sensitivity analysis indicated that the inclusion of this data did not substantively change the overall findings due to the wide confidence interval and low weighting of the study. Working Paper C 18

19 Sub-group Analyses A number of sub-group analyses were conducted using a random effects model to explore trends within the data. Study Design: The use of an active control had the expected effect on the knowledge outcomes. Studies with an active control (f = 3) showed no statistically significant treatment effect (Z = 0.78, p = 0.43) while studies with a control group not receiving any form of financial education showed a modest but robust treatment effect of 0.18 standard deviations. The same is true for attitudinal outcomes, where studies with active controls (f = 2) showed no overall effect (Z = 1.42, p = 0.16) while studies with an inactive or TAU control showed an overall treatment effect of 0.12 standard deviations. Risk of bias found that attrition bias was the most significant risk within included studies. Studies with high risk of attrition bias were compared to those with inconclusive and low risk studies, with mixed results. High-risk studies reported slightly higher treatment effect and a broader confidence interval (SMD 0.19, CI 0.03, 0.36). Studies with a high risk of bias that exploring attitudinal change appear to have overestimated effects (0.30 compared to the overall effect of 0.06 in low risk studies). There was insufficient data to explore the effect of study design on behaviours, as all included studies shared the same characteristics around type of control and low risk of attrition bias. Table: Summary of study design sub group analyses (random effects): Knowledge Attitudes Type of Control Incomplete Outcome Data Inactive Ctrl Active Ctrl High RoB Other RoB No. of measures Effect Point Estimate CI (Lower, 0.09, , , , 0.26 Upper) Z (p) 3.99 (0.0001) 0.78 (0.43) 2.32 (0.02) 1.73 (0.08) I 2 90% 85% 87% 84% No. of measures Effect Point Estimate CI (Lower, 0.02, , , , 0.11 Upper) Z (p) 2.50 (0.01) 1.42 (0.16) 0.90 (0.37) 2.34 (0.02) I 2 82% 0% 96% 51% Working Paper C 19

20 Intervention: There was considerable variation in the composition and the mode of delivery of content across studies. Those programmes delivering just financial education were compared with those that delivered financial education plus another component, including social, health or entrepreneurship education. The results suggest that for knowledge outcomes, those programmes that delivered just financial education were more effective (financial only: 0.19, financial plus: 0.16), although the results should be interpreted with caution as the observations of financial plus cross the line of no effect. However, for both attitudinal and behavioural outcomes, the results suggested that financial plus produced larger effects compared to financial only. For attitudinal outcomes, studies of financial plus programmes found a treatment effect of 0.17 standard deviations while those of financial only found no effect. For behavioural outcomes, financial plus programmes had an effect of 0.13 standard deviations while the single study of financial only had a smaller effect of Comparing the number of hours of tuition children and youth receive within programmes suggested that those delivering less than 40 hours of tuition produced greater knowledge, attitudinal and behaviour change than those delivering more than 40 hours. Observations for attitudes were not significant however. In a further sub-group analysis, financial education delivered in the classroom was compared to content delivered through media, such as TV, computers or the internet. The results found higher knowledge gains for media driven interventions (0.47, compared to 0.14 for classroom based programmes), although two of the media studies took place in a classroom (Faircloth et al 1986) and in one case included complementary in class content (Hinojosa et al 2009). There is also unpublished data to suggest that the Jorgensen and Tonsburg (2011) study produced unintended outcomes, as those in the intervention group reported lower attendance at school and lower test scores at follow-up. There was insufficient data to explore attitudes and behaviour outcomes. Table on following page: Working Paper C 20

21 Table: Summary of intervention sub group analyses (random effects): Knowledg e Attitudes Behaviour Content Length of Programme Mode of Delivery Financial Plus Financial Only <40 Hours >40 Hours Class Media No. of measures Effect Point Estimate CI (Lower, -0.07, , , , , , 0.89 Upper) Z (p) 1.36 (0.17) ( 0.17) 3, (0.0004) (0.06) (0.002) (0.03) I 2 93% 85% 85% 93% 89% 68% No. of NA NA measures Effect Point Estimate CI (Lower, 0.03, , , -0.01, 0.12 Upper) 0.27 Z (p) 2.34 (0.02) 0.11 (0.91) (0.11) (0.19) I 2 85% 91% 80% 87% No. of NA NA measures Effect Point Estimate CI (Lower, 0.02, , , , 0.07 Upper) Z (p) 2.40 (0.02) 7.15 (<0.0001) 3.07 (0.002) 7.3 (0.0001) I 2 71% 0 38% 0% Working Paper C 21

22 Population Similarly, there was significant variation in the context and age range of participants found within studies. In exploring the role of context in shaping outcomes, the influence of human development was analysed. Drawing on the Human Development Index (HDI, UNDP, 2013), studies that took place in countries not rated as having very high HDI, were compared with those that were. The results suggest that programmes operating in countries with very high HDI produced similar knowledge gains, however, were significantly poorer at changing attitudes. There was insufficient data to explore behaviour change and country HDI. By separating those studies aimed at children (0-14) and youth (15+), sub-group analysis found greater impact across knowledge attitudes and behaviours for programmes aimed at younger children. With attitudinal outcomes, however, the effect size crosses the line of no effect. Table: Summary of population sub group analyses (random effects): Knowledge Attitudes Behaviour Context Age range Very High HDI Other HDI Children (0-14) Youth (15+) No. of measures Effect Point Estimate CI (Lower, -0.17, , , , 0.26 Upper) Z (p) 1.03 (0.30) 4.42 (0.001) 2.21 (0.03) 2.73 (0.006) I No. of measures Effect Point Estimate CI (Lower, -0.57, , , , 0.09 Upper) Z (p) 1.42 (0.16) 2.5 (0.01) 1.84 (0.07) 0.78 (0.44) I % 72% No. of NA NA 3 3 measures Effect Point Estimate CI (Lower, 0.09, , 0.07 Upper) Z (p) 3.89 (<0.0001) 7.39 (<0.0001) I 2 7% 0% Working Paper C 22

23 Conclusions Understanding the Results: Summary of Overall Effects of Financial Education: Knowledge gains: Modest robust positive effect 0.18 (0.09, 0.27) Attitudinal change: Small positive effect 0.08 (0.01, 0.15) Behaviour change: Small robust positive effect 0.07 (0.03, 0.11) Standard Mean Differences (95% Confidence Intervals) Random Effects Model The meta-analysis produced conclusive results suggesting financial education is effective in developing financial capabilities. Knowledge change represents a modest positive result, with small gains in both attitude and behaviour outcomes. Overall this indicates that financial education can be effective across a range of outcome domains and that the evidence base is much broader than is often supposed. The heterogeneity among the studies suggests that different interventions, combinations, contexts and targeting strategies will produce different results. Therefore, in order to assist in informing practice, the authors identified the strongest interventions in each outcome category. The strongest programmes for financial knowledge change were The Stock Market Game (Hinojosa et al 2009) and the Brazil high school programme (Bruhn et al 2013). Both of these interventions focused only on financial topics, included an in class component, and arguably included participants with stronger numeracy skills than some of the other study contexts. The best programme for changing financial attitudes was the Suubi Project that combined financial topics, health topics, and financial access with young single and double orphans in Uganda (Karimli 2013; Ssewamala et al 2009). The combined effect of these two studies on financial attitudes was 0.45 (0.03, 0.87). This combined intervention included younger participants (age 13), addressed health and psychosocial issues, and provided a 200% matched savings account to incentivise asset accumulation. Across studies, those that targeted younger populations had a stronger effect on all domains, although for attitudes this did not meet the standard cut-off for statistical significance. The strongest programme for changing financial behaviour was the Aflatoun curriculum which combines social and financial topics and targets younger populations (Berry et al 2013; Supanantoroek 2013). The combined effect of these two studies was 0.16 (0.08, 0.24), twice that of the overall effect. Across interventions, those that combined financial education with another component, targeted young beneficiaries and had a shorter overall intervention were more effective at changing behaviour. Working Paper C 23

24 Comparing the Results with other Educational Research: There is no universal guideline or rule of thumb for judging the practical importance or substantive significance of a standardized effect-size estimate for an intervention, especially when comparing them to other educational programmes. Within education, the US Department of Education defines an effect size of.25 of a standard deviation as a substantially important effect (2013, p27). This can be misleading however, as a particular effect-size may present a relatively small or large impact depending on the benchmark that is most relevant. For example, 0.1 may be considered to be a relatively small gain in reading test scores over an academic year, but may be considered a large effect if demonstrating improved school attendance over the same period. A meta-analysis of 181 studies exploring curriculum or broad instructional and free-standing programmes in the US found an average standard deviation of 0.32 (Lipsey et al, 2012). These included a range of programmes, such as reading, maths or science curricula, and multisession programmes in a general subject area. The figure should also be viewed with caution, however, as it draws on unstandardised data exploring a broad range of outcomes. It focuses on studies in the US, and does not control for other potential confounding variables including the amount of time children spend receiving tuition. In other educational research, a meta-analysis of over 800 meta-analyses found that on average, teachers in front of the class produced an effect of.24 of a standard deviation, and that programmed instruction on average produced an effect of.18 (Hattie, 2009). In comparison with other curriculum programmes and the US Dept. for Education s benchmark the overall effect of financial education observed in this review and meta-analysis are substantial for financial knowledge outcomes and modest for attitude and behaviour outcomes. However, it should be noted that most educational interventions measure outcomes with a focus on knowledge and skills rather than attitudes and behaviour, so we postulate that the smaller effect sizes found in this review may be of comparable size to attitude and behaviour shifts in other education programmes. Furthermore, the educational focus of the contexts should be taken into account. For children and youth themselves, the financial education programmes that they have been given access to represent a small and arguably peripheral part of their overall education. Gains in knowledge, attitudes and behaviours across financial educational programmes featured in this review therefore may be proportional to larger, and more intensive educational programmes. Implications for Research: The existing research base exploring the effectiveness of financial education is of varying methodological quality. Future research should seek to improve the overall quality of the literature. The application of robust experimental methods, the minimisation of potential bias, especially around allocation bias, should be a priority. The reporting of trials should also be improved, especially with regards Working Paper C 24

25 to methodology, implementation fidelity, and data analysis. This will help improve the comparability, transparency and utility of the research base. Although there exists a significant research base, the review has exposed significant gaps in our understanding. The paucity of experimental research exploring financial education in Europe and Asia should cause concern. There is a debate currently around the importance of numeracy in financial educational knowledge outcomes. Non-experimental studies have suggested that when you control for numeracy, financial education has no effect. Meanwhile, both Hinojosa et al (2009) and Bruhn et al (2013) found that financial education had improved overall math scores within the intervention group. This suggests that there is a bidirectional relationship, and that financial education may contribute to numeracy development while numeracy can be important in shaping successful financial education outcomes. The role of numeracy should be further investigated within experimental studies, in order to better understand that relationship. Working Paper C 25

26 Annex A: Summary of Included Studies Becchetti et al (2012) Published: Intervention: Target Group: N and Attrition: Location: Type of Study: Data collection: Primary outcomes: Results: Meta-analysis: Summary: Becchetti L, Caiazza S and Coviello D (2012) Financial Education and Investment Attitudes in High Schools in Italy Classroom-based financial education. Female and male students in their last year of high school (unclear, large variations among analyses) Schools across Rome, Milan, and Genova, Italy. Clustered RCT with treatment and control. Self-administered survey Financial literacy, including knowledge of financial concepts, savings preferences and investment attitudes. Impact on improving knowledge and attitudes, especially amongst those with the lowest financial literacy prior to programme. Knowledge The study explored how financial education affects financial literacy and investment attitudes amongst a sample of high school students in Italy. The treatment group received 16 hours of tuition relating to financial literacy. The study found that the treatment group displayed increased levels of financial literacy and the propensity to read and understand economic articles and principles, although the results are not statistically significant. The treatment group also recorded reduced risky behaviors, however they did not increase prudent behaviors. A sub-group analysis found differences based on location, starting financial literacy, and other characteristics. The analysis also uncovered baseline differences indicating imperfect randomization. Working Paper C 26

27 Berry et al (2013) Published: Berry J, Karlan D and Predhan M (2013) Evaluating the Efficacy of School Based Financial Education Programs in Ghana Intervention: Classroom-based financial education. Target Group: Female and male pupils aged between 9 and 14. N and Attrition: 5363 (75) Location: Schools in semi-urban and rural Ghana, including Greater Accra, Volta and in the Western District. Type of Study: Clustered RCT with two treatments, Aflatoun and Honest Money Box interventions, and a control group. Data collection: Primary outcomes: Results: Meta-analysis: Self-administered survey Financial well-being of students and their families, cognitive function, and perspectives on savings and time and risk preference. Students in HMB-assigned schools showed positive and significant increases in their overall attitudes toward savings, compared to students in control schools. However, students in both Aflatoun and HMB schools showed positive impacts on savings behavior compared to students in control schools: those students were more likely to save any money, and those students were also much more likely to save money at school. The pooled treatment effects are used in this metaanalysis. Knowledge, Attitudes and Behaviours Working Paper C 27

28 Bruhn et al (2013) Published: Intervention: Target Group: N and Attrition: Location: Type of Study: Data collection: Primary outcomes: Results: Meta-analysis: Summary: Bruhn M, Bilal Zia, Arianna Legovini, Rogelio Marchelli (2013) Financial Literacy for High School Students and Their Parents: Evidence from Brazil Classroom-based financial education. High school students and their parents. 17,680 (Attrition varied across time-points) Brazil Cluster randomised control trial. Surveys to individual pupils and at the household level Budgeting, savings, and general financial management. Significant impact on improving knowledge, attitudes and behaviours. Knowledge, Attitudes and Behaviors The study assessed the impact of high school financial education in Brazil. It included nearly 900 schools students. Administration of the program through schools allowed for a broad coverage of content in the curriculum. Separate training was provided to a group of parents of the students to examine whether inside-the-household interactions influenced behavior. Results found that the program increased student financial knowledge by 0.20 of a standard deviation, which led to a 1.4 percentage point increase in savings behaviours - a relatively large and economically relevant effect. Both current attitudes and forward-looking intentions to save improved. A complementary workshop for parents induced children to save even more. Working Paper C 28

29 Faircloth A (1986) Published: Faircloth A (1986) Effectiveness of Computer-Assisted Instruction for Teaching Consumer Credit in the USA. Intervention: Computer assisted instruction, vrs supervised reading for teaching consumer credit. Target Group: High School students in grades 10 through 12 N and Attrition: 68 Location: USA, large rural high school. Type of Study: Randomised Control Trial, with each arm testing either computer assisted instruction or supervised reading control group. Data collection: Pre and post tests, and administrative data. Primary Students IQ, grade point average, attitude towards computers outcomes: and familiarity with computers. Results: Computer assisted instuction is an effective intervention for teaching consumer credit with diverse types of high school students. Meta-analysis: N/A Summary: Using a random sample of 68 high school students learning about consumer credit in home economics classes, the study measured learning of an experimental group using computerassisted instruction and a control group using supervised reading methods. Pretests and posttests indicated greater learning gains in the experimental group. Working Paper C 29

30 Gartner et al (2005a) Published: Intervention: Target Group: Gartner K and Todd R (2005) Effectiveness of Online Early Intervention Financial Education for Credit Cardholders in the USA Online, credit-card education programme. New credit card holders or card holders reaching the point of first delinquency. N and Attrition: 241,112 Location: Across the USA Type of Study: RCT Data collection: Administrative data on customer behaviour, repayment performance, delinquency etc. Primary Behaviour change amongst credit card holders. outcomes: Results: Completion of online credit education correlates with more responsible credit card usage, however the experiments don t prove that the education causes this behavior. Meta-analysis: Summary: N/A The first of six independent RCTs, this was conducted by the Bank Wells Fargo. It tested whether offering online credit card education to credit cardholders is effective in changing behavior. The targeted populations were either new cardholders or cardholders reaching the point of first delinquency. The experimental groups received brief counseling plus a referral to the VISA online credit education web site MoneyChoices.com. Experiments with college student cardholders found much more responsible behavior by those who choose to complete the online education. The other studies, however, found smaller overall differences between the control and experimental groups. These results were not statistically significant. Working Paper C 30

31 Gartner et al (2005b) Published: Intervention: Target Group: N and Attrition: Location: Type of Study: Data collection: Primary outcomes: Results: Meta-analysis: Summary: Gartner K and Todd R (2005) Effectiveness of Online Early Intervention Financial Education for Credit Cardholders in the USA Online, credit-card education programme. New credit card holders or card holders reaching the point of first delinquency. 80,982 with significant attrition. Across the USA RCT Administrative data on customer behaviour, repayment performance, delinquency etc. Behaviour change amongst credit card holders. Completion of online credit education correlates with more responsible credit card usage, however the experiments don t prove that the education causes this behavior. N/A Conducted by Target. Working Paper C 31

32 Gartner et al (2005c) Published: Intervention: Target Group: Gartner K and Todd R (2005) Effectiveness of Online Early Intervention Financial Education for Credit Cardholders in the USA Online, credit-card education programme. New credit card holders or card holders reaching the point of first delinquency. N and Attrition: 82,600 Location: Across the USA Type of Study: RCT Data collection: Administrative data on customer behaviour, repayment performance, delinquency etc. Primary Behaviour change amongst credit card holders. outcomes: Results: Completion of online credit education correlates with more responsible credit card usage, however the experiments don t prove that the education causes this behavior. Meta-analysis: Summary: N/A Conducted by US Bank, aimed at cardholders past the point of delinquency on credit card payments. Working Paper C 32

33 Gartner et al (2005d) Published: Intervention: Target Group: Gartner K and Todd R (2005) Effectiveness of Online Early Intervention Financial Education for Credit Cardholders in the USA Online, credit-card education programme. New credit card holders or card holders reaching the point of first delinquency. N and Attrition: 8190 Location: Across the USA Type of Study: RCT Data collection: Administrative data on customer behaviour, repayment performance, delinquency etc. Primary Behaviour change amongst credit card holders. outcomes: Results: Completion of online credit education correlates with more responsible credit card usage, however the experiments don t prove that the education causes this behavior. Meta-analysis: Summary: N/A Completed by US Bank, the study aimed at understanding intervention aimed at non-delinquent but high risk college students. Working Paper C 33

34 Gartner et al (2005e) Published: Intervention: Target Group: N and Attrition: Location: Type of Study: Data collection: Primary outcomes: Results: Meta-analysis: Summary: Gartner K and Todd R (2005) Effectiveness of Online Early Intervention Financial Education for Credit Cardholders in the USA Online, credit-card education programme. New credit card holders or card holders reaching the point of first delinquency. 11,686 with significant attrition. Across the USA RCT Administrative data on customer behaviour, repayment performance, delinquency etc. Behaviour change amongst credit card holders. Completion of online credit education correlates with more responsible credit card usage, however the experiments don t prove that the education causes this behavior. N/A Completed by US Bank, the study aimed at understanding intervention aimed at non-delinquent and low risk college students. Working Paper C 34

35 Gartner et al (2005f) Published: Intervention: Target Group: Gartner K and Todd R (2005) Effectiveness of Online Early Intervention Financial Education for Credit Cardholders in the USA Online, credit-card education programme. New credit card holders or card holders reaching the point of first delinquency. N and Attrition: 241,112, with an average age of 18 Location: Across the USA Type of Study: Three independent RCT s each exploring support offered to credit card holders. Data collection: Administrative data on customer behaviour, repayment performance, delinquency etc. Primary Behaviour change amongst credit card holders. outcomes: Results: Completion of online credit education correlates with more responsible credit card usage, however the experiments don t prove that the education causes this behavior. Meta-analysis: Summary: N/A Completed by US Bank, the study aimed at understanding intervention aimed at non-delinquent but high risk college students. Working Paper C 35

36 Hinojosa et al (2009) Published: Hinojosa T, Miller S, Swanlund A, Hallberg K, Brown M, and O'Brien (2009) The Stork Market Game Study Intervention: Online stock market game delivered in the classroom Target Group: Students in grades 4 to 10 N and Attrition: 522 classrooms joined the study Location: Schools across the USA. Type of Study: Clustered RCT Data collection: pre- post test scores, administrative data. Primary Mathematics Ability and Investor Knowledge outcomes: Results: Overall, in both mathematics and investor knowledge, there is a significant difference between students who played The Stock Market Game and those who did not, with those who played substantially outperforming those who did not. Meta-analysis: Knowledge Summary: The program was designed to teach students the importance of saving and investing by building their financial literacy skills. Students manage imaginary investments online, competing against other individuals and teams both in their classroom and around the world. Teachers were invited to sign up to participate in the RCT, with approximately 1,200 teachers applying. Of those who met the eligibility requirements 568 teachers confirmed participation; 296 teachers and their students participated as the treatment classrooms and 272 teachers and their students participated as the control classrooms. Overall, results from the RCT showed that students who played The Stock Market Game significantly outperformed students who did not play the game on both mathematics and investor knowledge tests. Working Paper C 36

37 Jamison et al (2012) Published: Jamison J, Karlan D and Zinman J (2012) Starting a Lifetime of Saving: Teaching the Practice of Saving to Ugandan Youth Intervention: Classroom based financial curriculum, vrs specially designed youth group savings account. Target Group: Youth, both female and male, attending Church of Uganda Youth Fellowship Groups N and Attrition: 2,800 Location: Villages across Uganda Type of Study: Clustered RCT, with four trial arms, A) Financial literacy training with youth group savings account, B) Financial literacy training without youth group savings account, C) Youth group savings account without Financial literacy training D) Comparison group, with no intervention. Data collection: Pre- post- survey Primary Financial literacy and savings behaviour. outcomes: Meta-analysis: N/A Summary: Awaiting more data to be able to include in the review. Working Paper C 37

38 Jorgensen and Tonsburg (forthcoming) Published: Jorgensen and Tonsberg (2011) Edutainment and entrepreneurship: A field experiment on youth in Tanzania Intervention: Media based reality TV show exploring financial literacy and entrepreneurship. Target Group: In school and out of school youth between 15 and 30, averages awaiting confirmation. N and Attrition: 2,126 (199) Location: Urban and Rural Tanzania Type of Study: Encouragement Randomised Control Trial Data collection: Self-administered questionnaires Primary Knowledge gains, school attendance, school grades. outcomes: Results: Meta-analysis: Mixed, and awaiting precise confirmation, however early indications show that knowledge gains were inconclusive, with women having more positive results. The intervention group also reported lower school attendance and school grades. Knowledge Working Paper C 38

39 McLean (2010a) Published: McLean, Bill (2010) Increasing the Learning Effectiveness of Economics Education Intervention: Classroom based curriculum for teaching money and banking principles. Target Group: High school students N and Attrition: 221 (31) Location: USA Type of Study: RCT Data collection: Test-score data. Primary Knowledge and attitudes. outcomes: Results: Some improvements in learning achievements in the treatment group. Meta-analysis: Knowledge and Attitudes Summary: The first of two independent RCTs, the study evaluated new pedagogical innovations in financial curriculum delivery. Working Paper C 39

40 McLean (2010b) Published: McLean, Bill (2010) Increasing the Learning Effectiveness of Economics Education Intervention: Classroom based curriculum for teaching money and banking principles. Target Group: High school students N and Attrition: 102 (5) Location: USA Type of Study: RCT Data collection: Test-score data. Primary Knowledge and attitude gains. outcomes: Results: Some improvements in learning achievements in the treatment group. Meta-analysis: Knowledge and Attitudes Working Paper C 40

41 McNeil et al (2008) Published: McNeil N, Uttal D, Jarvin L and Sternberg R (2008) Should you show me the money? Concrete objects both hurt and help performance on mathematics problems. Intervention: Classroom based financial tools to help students develop maths problems. Target Group: 4 th and 6 th Grade Students N and Attrition: 308 Location: Connecticut, USA Type of Study: Two Randomised Experiments which randomly assigned students to two intervention groups and a control. Primary Mathematics test scores. outcomes: Results: Meta-analysis: Summary: Results suggest that the use of perceptually rich concrete objects conveys both advantages and disadvantages in children s performance in school mathematics. N/A Explored how concrete objects that cue real-world knowledge affect students performance on mathematics word problems. In the experimental group, fourth- and sixthgrade students (N 229) solved word problems involving money. Students in the experimental condition were given bills and coins to help them solve the problems, and students in the control condition were not. Students in the experimental condition solved fewer problems correctly. Experiment 2 tested whether this effect was due to the perceptually rich nature of the materials. Fifth-grade students (N 79) were given: perceptually rich bills and coins, bland bills and coins, or no bills and coins. Students in the perceptually rich condition made the most errors; however, their errors were least likely to be conceptual errors. This suggests that perceptually rich, realistic objects may also convey some advantages for students conceptualization of problems. Working Paper C 41

42 Premand et al (2012) Published: Premand, Patrick, Brodmann, Stefanie, Almeida, Rita, Grun, Rebekka and Barouni, Mahdi (2012) Entrepreneurship training and self-employment among university graduates: evidence from a randomized trial in Tunisia Intervention: Class-room based entrepreneurship training including financial literacy components such as budgeting and business planning. Target Group: University undergraduates N and Attrition: 1,702 (202) Location: Across universities in Tunisia Type of Study: RCT Data collection: Pre- post- survey Primary A range of indicators including labour market outcomes and outcomes: Results: Meta-analysis: access, and willingness to access credit. Observations suggested that the intervention was effective in promoting knowledge and skills gains across a range of measures including communication. Those in the intervention group, however, did not set up more businesses than those in the control. Knowledge Working Paper C 42

43 Ssewamala et al (2008) Published: Ssewamala F, Alicia S, Bannon W, Ismayiolova L (2008) A novel Economic Intervention to reduce HIV risks amongst School-going AIDS orphans in rural Uganda. Intervention: Classroom based, with access to savings accounts. Target Group: AIDS-orphaned adolescents, with an average age of N and Attrition: 96 youth drawn from 7public schools Location: Rakai District of southern Uganda Type of Study: Cluster RCT Data collection: Pre- and Post-test interviews, and actual savings behaviour into assigned bank accounts. Primary Savings behaviour, attitudes towards saving, educational outcomes: plans and aspirations, academic performance, sexual risktaking behaviour and attitudes. Results: Significant improvements in HIV prevention attitudes (17.2 to 18.5) and educational planning (88% to 96%) were observed in the intervention group. However, the effect sizes vis-a-vis the control group were not unambiguously reported and the author could not be contacted for clarification the partial η 2 for HIV attitudes reported as positive 0.1 but the 95% confidence interval was negative, meanwhile the confidence interval for educational plans includes the null (partial η 2 (CI)= 0.7 (-0.02 : 2.5)). Meta-analysis: N/A because financial outcomes were not reported for both groups. Summary: The study examined the introduction of matched-savings accounts, where savings deposits were matched by the school, amongst AIDS-orphaned young people. There were no comparable financial outcomes as the control group did not have access to savings accounts and the administrative data on the intervention accounts was the only measured financial outcome. Working Paper C 43

44 Ssewamala et al (2009) Published: Ssewamala F and Ismayilova L (2009) Integrating Children's Savings Accounts in the Care and Support of Orphaned Adolescents in Rural Uganda Intervention: Workshops focused on asset-building and future planning, monthly peer mentorship, with access to matched savings accounts. Target Group: AIDS-orphaned adolescents, with an average age of N and Attrition: 286 (19) youth drawn from 15 public schools Location: Rakai District of southern Uganda Type of Study: Cluster RCT Data collection: Pre- and Post-test interviews, and administrative data on Primary outcomes: Results: Meta-analysis: Summary: savings behaviour for the intervention group only. Savings behaviour, attitudes towards saving, educational plans and aspirations, academic performance, sexual risktaking behaviour and attitudes at 10 months. Savings attitudes improved in the intervention group, where as those in the control decreased. Actual savings behaviour was only available for the intervention group. Improvements on self-esteem Attitudes The study examined the introduction of matched-savings accounts, where savings deposits were matched by the school, amongst AIDS-orphaned young people. There were statistically significant differences between groups on attitudes towards savings, academic performance, educational aspirations and health-related behaviours. Working Paper C 44

45 Karimli (2013) Published: Karimli, L. (2013). Financial Asset Accumulation by Poor Adolescents Participating in Child Savings Accounts in Low Resource Communities in Uganda. Social Work. New York, Columbia University. Doctor of Philosophy: 146. Intervention: Career planning and financial management through ten 1-2 hour training sessions, monthly mentoring, and access to matched savings accounts in the child s name. Target Group: AIDS-orphaned adolescents, with an average age of N and Attrition: 346 dyads, children and guardians, 10 schools Location: Rakai District of southern Uganda Type of Study: Cluster RCT Data collection: Pre- and Post-test interviews, and actual savings behaviour into assigned bank accounts. Meta-analysis: Attitudes and Behavior Summary The intervention was assessed after months and months. At the later time point changes in savings behavior were noted. The intervention also explores personal and social determinants of savings behavior, finding links between both individual and family characteristics and the children s savings attitudes and behavior. Working Paper C 45

46 Supanantaroek S (2012) Published: Supanantaroek S (2012) Aflatoun/PEDN Uganda RCT Intervention: Classroom based Target Group: Primary school children, both female and male, with an average age of 12.5 years, with a Std Dev. of 1.46 N and Attrition: 1,746 Location: Uganda, government and private schools in Kampala and Wakiso Type of Study: Clustered Randomised Control Trial Data collection: Cross-sectional self-administered survey, classroom observation and pre and post tests. Primary outcomes: Financial awareness, budgeting and savings behaviour. Student s personal interest and participation in schooling. Results: Compared to the control, the results suggest that the programme treatment leads to positive results as measured by students awareness of money, budgeting and savings behaviour. Meta-analysis: Knowledge, Attitudes and Behaviours. Summary: An evaluation of Aflatoun s formal primary curriculum, the study explores that changes in knowledge, attitudes and behaviour around financial concepts including budgeting and saving. Employing a Clustered RCT, the study recruited 64 schools and assigned them to either a treatment or control group. The study found statistically significant results confirming the effectiveness of the curriculum against no intervention. Working Paper C 46

47 Bibliography of Included Studies Becchetti L, Caiazza S and Coviello D (2012) Financial education and investment attitudes in high schools: evidence from a randomized experiment, in Applied Financial Economics, 23:10, Berry J, Karlan D and Peadhan M (2013) Evaluating the Efficacy of School Based Financial Education Programs, Innovations for Poverty Action Bruhn M, Bilal Zia, Arianna Legovini, Rogelio Marchelli (2013) Financial Literacy for High School Students and Their Parents: Evidence from Brazil Faircloth A (1986) Effectiveness of Computer-Assisted Instruction for Teaching Consumer Credit Gartner K and Todd R (2005) Effectiveness of Online Early Intervention Financial Education for Credit Cardholders Hinojosa T, Miller S, Swanlund A, Hallberg K, Brown M, and O'Brien (2009) The Stork Market Game Study: Brief Report Jamison J, Karlan D and Zinman J (2012) Starting a Lifetime of Saving: Teaching the Practice of Saving to Ugandan Youth Jorgensen and Tonsburg (2011) Edutainment for entrepreneurship: Evidence from a field experiment on youth in Tanzania: Masters Thesis Karimli, L. (2013). Financial Asset Accumulation by Poor Adolescents Participating in Child Savings Accounts in Low Resource Communities in Uganda. Social Work. New York, Columbia University. Doctor of Philosophy: 146. McLean, Bill (2010) Two Essays on Increasing the Learning Effectiveness of Economics Education McNeil N, Uttal D, Jarvin L and Sternberg R (2008) Should you show me the money? Concrete objects both hurt and help performance on mathematics problems Premand P, Brodmann S, Almeida R, Grun R and Barouni M (2012) Entrepreneurship training and self-employment among university graduates: evidence from a randomized trial in Tunisia Ssewamala F and Ismayilova L (2009) Integrating Children's Savings Accounts in the Care and Support of Orphaned Adolescents in Rural Uganda Supanantaroek S (2012) Aflatoun/PEDN Uganda RCT Working Paper C 47

48 References: Chinn, S. (2000). A simple method for converting an odds ratio to effect size for use in meta-analysis. Statistics in Medicine 19: Chowa G, Ansong D, Masa R, Despard M, Osei-Akoto I, Richmond A, Agyei- Holmes A and Sherraden M (2012) Youth and Saving in Ghana: A baseline report from the YouthSave Ghana Experiment, YouthSave Ghana Cohen J (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Han, C-K, Ssewamala, F., Wang JSH (2013) Family economic empowerment and mental health among AIDS-affected children living in AIDS-impacted communities: evidence from a randomised evaluation in southwestern Uganda. J Epidemiol Community Health 67: Hattie, J. (2009). Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. NY: Routledge. Kagotho, N., Ssewamala, F. (2012) Correlates of depression among caregivers of children affected by HIV/AIDS in Uganda: Findings from the Suubi-Maka Family Study. AIDS Care: Psychological and Socio-medical Aspects of AIDS/HIV, 24(10): Lipsey M, Kelly Puzio, Cathy Yun, Michael A. Hebert, Kasia Steinka-Fry, Mikel W. Cole, Megan Roberts, Karen S. Anthony, and Matthew D. Busick (2012) Translating the Statistical Representation of the Effects of Education Interventions Into More Readily Interpretable Forms, US Department for Education. UNDP (2013) Human Development Roport, 2013, I index, UNDP, US Department for Education (2013) What Works Clearinghouse: Procedures and Standards Handbook, Working Paper C 48