Systematic review of a health system intervention:

Transcription

1 Systematic review of a health system intervention: lessons learnt from a review of results-based financing approaches in Africa and Asia. Emma Jolley, Juliet Milgate and Elena Schmidt

2 Format 1. Introduction and background 2. Methods 3. Findings 4. Reflections

3 Introduction and background

4 Sightsavers NGO established in 1950, currently works in over 30 countries in Africa and Asia Mission to eliminate avoidable blindness and promote equality of opportunity for disabled people Focus in the areas of eye health and health systems, education and social inclusion Research undertaken in partnership with academic institutions

5 Results-based financing Terminology Varies throughout the literature RBF: (for the purposes of this review) A pre-specified payment (financial or non-financial) from one party to individuals or individual health facilities, in return for the latter carrying out a pre-specified action or set of actions in a satisfactory manner. Not aid to governments Not incentives for relocation Not unconditional payments

6 Results-based financing Changing financing approach Input based financing Output based financing Salaries Medical equipment Drugs Building costs Utilities etc Health system Number of institutional deliveries Number of surgeries Number of children fully immunised

7 Results-based financing RBF incorporates a range of health initiatives that aim to achieve improvements in: Population health indicators Greater output of specific health services Increased use of health care services by people Changes in health-related behaviours Why so important now? MDGs Need to accelerate progress against the health-related MDGs (RBF a key policy response adopted by the Norwegian government) Economic austerity Linking payment to results

8 Sightsavers and RBF Sightsavers programmes still face challenges: Many districts have low service coverage Reaching the very poor with quality services remains an issue Current economic climate RBF could help address some of these issues, but RBF has not been evaluated in an eye health setting Sightsavers has no experience with RBF programmes

9 Objectives What Sightsavers wants to know What is RBF, and what can it do for us?

10 Existing evidence Other reviews exist Focus on the effect of RBF programmes (Witter, 2012) Focus on the effects of RBF on only particular health themes or particular types of RBF (USAID evidence synthesis, unpublished; Oxman, 2008) Focus on RBF programmes funded by certain donors (Toonen, 2009; Brenzel, 2009) Main findings Evidence generally low quality Generally short term outputs evaluated heterogeneity renders meta-analysis difficult Call for further and better quality research Out of date

11 Objectives Specific questions to address: 1. In what countries and in what health areas have RBF programmes been documented? 2. What are the characteristics of the different models of RBF that have been used in health care programmes? 3. What evaluations have been undertaken to assess the different models of RBF programmes, and what are the results? 4. What are the identified barriers and facilitators to RBF programmes operating successfully, and what challenges, externalities and other issues have been identified?

12 Methods

13 Methods Search strategy - databases Keywords: Results; performance; conditional; output; Funding; financing; pay; Geographical terms Databases searched: Medline, Embase, Global Health, ELDIS, WHO Global Health Library

14 Methodology Search strategy grey literature Websites searched: World Bank; WHO; African Development Bank; Global Fund; DFID; USAID; NORAD; GTZ; rbfhealth.org; Dutch Royal Tropical Institute; Swiss Tropical Institute; London School of Hygiene and Tropical Medicine; References Papers identified as eligible following abstract review

15 Methods Study selection inclusion and exclusion criteria Primary data describing an RBF intervention or evaluation Primary focus on RBF Health outcomes only Quantitative and qualitative data Description of data collection processes Geographic: Africa and Asia Date: published from 2006 onwards Language: English, French, Spanish

16 Methods Critical appraisal Appraising without scoring Checklists from the Critical Appraisal Skills Programme ( Focus on Study validity Results Applicability

17 Methods Data extraction 4 areas: 1.Document details 2.RBF design 3.RBF implementation details 4.Evaluation details

18 Methods Data synthesis Qualitative synthesis Thematic analysis If robust and comparable quantitative data existed, we would use metaanalysis techniques.

19 Findings

20 1,049 documents identified in databases 463 duplicates removes 586 unique documents identified 40 grey documents found and 43 references retrieved 525 did not meet inclusion criteria based on abstract review 144 documents identified for full-text review 93 excluded for not meeting inclusion criteria including reporting only secondary data, and 3 full-text documents could not be located 48 documents reporting primary data

21 Results 1. In what countries and in what health areas have RBF programmes been documented? 23 unique programmes in 17 countries Africa 13 programmes in 9 countries Asia 10 programmes in 8 countries Mostly focusing on improving maternal and child health outcomes, but also programmes focusing on: Tuberculosis HIV Malaria Family planning General clinical quality And mixtures

22 Results RBF programmes 2. What are the characteristics of the different models of RBF that have been used in health care programmes? Programmes can be categorised according to who is performing the incentivised actions

23 Results 3. What evaluations have been undertaken to assess RBF programmes? 9 cluster control trials (7 randomised) 8 qualitative studies 6 interrupted time series (2 with control elements) 3 population based surveys 2 randomised control trials

24 Results 4. What are the identified barriers and facilitators to RBF programmes operating successfully, and what challenges, externalities and other issues have been identified? Infrastructure investment HMIS, administration, clinical Equity universal vs means-tested incentives Unintended results staff relations

25 Interpretation What does this mean for Sightsavers Possibly applicable for certain interventions such as cataract and trichiasis surgery Theory of change needs to be well articulated Context is key Community involvement Linkage to existing RBF programme Whatever we undertake must be rigorously evaluated

26 Limitations Time pressures Search strategy may have overlooked some documents Grey literature search not exhaustive The same two reviewers did the majority of searching, reviewing and data extracting. Limitations of retrieved documents Qualitative synthesis difficult and open to bias Conclusions hard to draw Current (better designed) evaluations not included

27 Reflections

28 Reflections Limitations of review process for Sightsavers Relatively rapid (~12 months) still considered resource intensive Access to databases Excluded documents considered key Translating the findings in to policy and practice

29 Reflections Was this useful for Sightsavers? Answers the initial questions to some extent Raised awareness of the potential of RBF programmes for eye health Raised awareness of the key players in RBF programmes (in the countries in which we work) Raised awareness of design and implementation issues Raised awareness of the need to build comprehensive evaluation in to whatever programme goes forward. Raised awareness of the potential of systematic reviews for identifying and evaluating the state of evidence

30 Reflections Systematic reviews for international development Great potential for improving the evidence on which policies and programmes are developed Engagement with research means NGOs can contribute to formulating questions that matter to them Better implications for programmes than traditional reviews that may exclude studies on the basis of study design Resource intensive

31