Institute for Health Metrics and Evaluation (IHME) aidinfo use case. DevelopmentInitiatives

Transcription

1 Institute for Health Metrics and Evaluation (IHME) aidinfo use case DevelopmentInitiatives

2 Institute for Health Metrics and Evaluation (IHME) aidinfo use case Contents Executive summary... 3 Background... 4 Introduction... 4 What is IHME trying to elicit from the data, what is the purpose?... 5 What weaknesses has IHME identified in health aid data?... 6 Official aid flows: OECD-DAC database... 6 Private aid flows: Foundations... 7 Private aid flows: NGOs... 8 What would a better system of data on health flows look like?... 8 IHME s needs for better data... 9 DevelopmentInitiatives

3 Institute for Health Metrics and Evaluation (IHME) aidinfo use case Executive summary In July 2009 the Institute for Health Metrics and Evaluation (IHME) published its first annual report Financing Global Health 2009: Tracking Development Assistance for Health 1 which provides the most comprehensive analysis of global development assistance for health (DAH) to date. The key findings from the report are that: DAH funding has increased from $5.6 billion in 1990 to $21.8 billion in 2007 The fraction of DAH funding through multilateral organisations, such as the World Bank and the United Nations Agencies, has declined during the same period DAH from private initiatives such as the Global Alliance for Vaccinations and Immunisations (GAVI) and the Global Fund to Fight AIDS, Tuberculosis, and Malaria (GFATM) have significantly increased since 2002 Private philanthropy from sources such as the Bill & Melinda Gates Foundation, other foundations, individuals and corporate donations of drugs and medical supplies, contributed over 25% of health aid in For IHME, the process of collecting and analysing data on global health flows led to a number of challenges. These data challenges resulted in only a partial representation and understanding of the DAH financial architecture. The key challenges were as follows: Private resources from non-us non-governmental organisations (NGOs) and foundations were not tracked due to lack of comprehensive and comparable data Non-DAC donor contributions were not included because of poor access to and lack of sufficient data. Key donors such as China were not included because of the absence of reliable data Lack of timely data prevented IHME from analysing and presenting current trends in DAH for IHME was limited to DAH bilateral data up to 2007 because of the time lag in the publication of DAC data Definitions of key terminology varied within and across donors reporting. As a result of the challenges and discrepancies that IHME experienced when analysing DAH, this use case suggests a number of recommendations which could improve the quality, reliability and accuracy of current global DAH data. Key recommendations include timely reporting of data, data coverage for a wider range of donors, consistency in terminology used and more detailed and disaggregated data DevelopmentInitiatives

4 Background The Institute for Health Metrics and Evaluation (IHME) was established in 2007 at the University of Washington with funding from the Bill & Melinda Gates Foundation (BMGF) and the state of Washington. IHME is a research institute which specialises in producing comprehensive research on global health-related issues. The main objective of IHME s research is to provide scientifically reliable, timely information on health to help inform policy making and improve current global health systems. IHME have six distinct areas of research: health outcomes, health services, resource inputs, decision analytics, evaluations, and tools and instruments. This case study will focus on the resource inputs area, which attempts to measure and analyse three areas of health: development assistance for health, government health expenditure, and private health expenditure. This use case will specifically look at IHME s workstream development assistance for health (DAH). Introduction In order to help build an evidence base for transparent aid, aidinfo is developing case studies to document specific aid information needs for government departments, NGOs as well as sectoral, citizen, research and advocacy organisations. The use case aims to analyse key questions such as: What do users want? Why? How do they use aid information? How do they get aid data? What problems do they have accessing the aid information they need? What could they do with better data? Aidinfo s objective for use cases is to understand data issues faced by a wide variety of users. IHME s research on development assistance for health provides a sound example of the difficulties in accessing data and the implications and challenges this creates in producing a coherent and comprehensive analysis of global health aid flows. Particular reference is made to recent research conducted by IHME on DAH and the follow up policy report Financing Global Health 2009: Tracking Development Assistance for Health 3. Tracking health resources in the form of aid flows is IHME s primary goal for this workstream. The process will comprise of systematic research and analysis of global health spending building on the validity and reliability of existing data to produce consistent and comparable trends in global health spending through annual reports. In addition, IHME will complement this workstream with other relevant health related research as DAH only provides part of the picture and other variables need to be considered. Complimentary workstreams will a) analyse government (domestic) spending on health by looking at the role of national governments and other financers in the health system and b) research private health spending because of the dramatic impact that out-of-pocket spending has on the poor in developing countries. Analysis and research into these three distinct areas will contribute to a more comprehensive picture of overall global financing for health. This use case will focus on IHME s DAH study. 3

5 What is IHME trying to elicit from the data, what is the purpose? IHME defines DAH as all assistance for health channelled through public and private institutions whose primary purpose is to advance development in developing countries 4. IHME s research shows a considerable increase in DAH from with a significant injection of money for HIV and AIDS programmes. For IHME, the purpose of collecting and analysing data on development assistance for health is to capture timely and comprehensive health flows from all donors; governments, NGOs and foundations and produce annual reports of trends in DAH. IHME s rationale is that objective, comparable and comprehensive information on public and private resources for global health is needed for improving the quality of policy making and planning at all levels. It is also an essential ingredient for the effective monitoring and evaluation of global health initiatives and national health programs 5. In recent years, DAH has seen a rise in volume and changes in the donor architecture with an influx of new players and funders. The current downturn in the global financial climate has created an environment in which Overseas Development Assistance (ODA) could shrink and raises questions about continued sustainable funding in the future. Within this context, there has been an increase in demand for greater transparency and more efficiency and effectiveness in development assistance funding. However, in light of these shifting structures and the financial environment, the quality of data and knowledge of global health flows has remained unchanged. Because of the increase in volume, and the number and type of donors, fundamental questions still cannot be accurately answered such as who is giving what, how, to whom and to what end 6. At present, users are unable to truly estimate how much development assistance is flowing to the health sector in developing countries. More timely and accurate data on funding flows for health would help contribute to more effective national planning and policy making as well as hold donors accountable to commitments they have made. In recent years, international goals and targets have been set, which have exposed greater coverage of certain health outcomes. However, unless you can track the inputs such as how much is being invested in certain sectors such as health it is difficult to measure the success of the goal or commitments, i.e., the outcome and the impact. A comprehensive analysis and real understanding of DAH could highlight whether there were sufficient funds for particular health programmes or whether funds were being spent incorrectly. Currently, IHME cannot assess the amount of DAH that donors provide with 100% certainty due to weaknesses of existing data and lack of other important data on health aid. Without reliable statistics on all DAH inputs, the ability to accurately monitor and measure outputs and outcomes and evaluate the cost effectiveness of health funding is compromised. In turn, this undermines the ability to formulate and implement relevant policy procedures. The purpose of IHME s report, Financing Global Health 2009: Tracking Development Assistance for Health, is to provide high-quality and timely information on health aid to policy makers, researchers, donors, practitioners, local-decision makers and others. The research could be used to inform donor government policy making or enable recipient governments to understand how much DAH public and private donors have reported giving to them. IHME s research could help policy makers understand the distribution of health funding based on need and disease burden. IHME s report 4 pg. 8 5 ibid pg. 3 6 Ibid pg. 9

6 shows that some countries are ranked high on the disease burden list but receive relatively low assistance for health. For example, the Democratic Republic of Congo is ranked high for disease burden but low for the amount of assistance it receives, and in contrast, Uganda receives more assistance than one would expect when using the disease burden index 7. Disease burden and needsbased analysis is a complex process; one is not always correlated with the other. However, to a certain extent, tracking disease burden is only one measure of need. Disaggregated data on health spending could improve needs assessments by providing more concrete information on where money goes, for what purpose and what outcome. In this context, IHME s research and data on disease burden could be useful because it shows that country allocation of DAH appears to be driven by many considerations beyond the burden of disease, including historical, political and economic relationships between certain donors and recipient countries 8. A fundamental goal for IHME s research is to be able to disaggregate global health by health focus area. IHME s recent research on DAH was limited to four areas: HIV and AIDS, tuberculosis, malaria, and health sector support. Further analysis on the distribution of aid across different diseases and different interventions will be included in future reports. Evaluating health funding by disease type will enable a comparative analysis between funding for HIV and AIDS programmes and malaria, for example. This level of research will raise the question about the cost effectiveness of investing x amount of money in one disease with the intention to enable policy makers and donors to make informed decisions about whether funds should actually be spent on HIV prevention or HIV treatment, for example. Further work on cost effectiveness will be carried out through the Disease Control Priority Network. What weaknesses has IHME identified in health aid data? IHME s research highlights a number of weaknesses in DAH data. These weaknesses are present in data reported to the Organisation for Economic Cooperation of Development (OECD) Development Assistance Committee (DAC) database, and data from foundations and NGOs. Official aid flows: OECD-DAC database IHME s research relies heavily on the OECD-DAC database to gather data on DAH. IHME regards this database as the most comprehensive centralised repository for aid information as it enables users to identify how much DAC donor governments give to DAH at both an aggregated and disaggregated level. For example, the Creditor Reporting System (CRS) enables users to disaggregate aid information by project ID, sector, agencies, etc. However, as with many central repository data systems, there were issues with the data which posed challenges for IHME. Quality of reporting amongst donors: The standard of reporting amongst donors to the DAC database varies greatly. In 2007, IHME found that countries like the UK, Sweden, and Germany provided complete data for the channel of delivery variable. In contrast, the USA failed to report this information for over 30% of its health aid. The incompleteness of this field meant that it was unclear who the recipient of the grant was. Without this level of detail for health aid information, it was difficult for IHME to know whether health funding was going to recipient governments or civil society organisations (CSOs). The completeness 7 Financing Global Health 2009: Tracking Development Assistance for Health, pg 52 8 Ibid pg 53

7 of fields varied across different donor governments which affected the quality of the data. Poorquality data limited IHME s ability to analyse all the data in depth and provide a completely accurate picture of actual DAH flows. Timeliness IHME s report will be produced on an annual basis to show changing trends in global DAH flows. IHME needs timely information on DAH so that the data reflect actual disbursements in the past year. Unfortunately, data in the DAC database has a time lag which makes the data useful for retrospective trend analysis but does little to help produce a picture of current global flows. Without timely data the information has limited influence on policy making procedures and does not reflect the most current health aid architecture. If the data was released in a timely manner then it would be more relevant and beneficial for recipient countries, policy makers and advocates. Private aid flows: Foundations A limitation of the DAC database is it does not cover official flows or ODA from all donors; however this was never its intention. Currently, DAC donors, major multilateral organisations and some non- DAC donors report their ODA to the DAC database 9. In this sense, although relatively comprehensive within its own niche of donors, the DAC database cannot provide global DAH figures because the majority of private foundations, NGOs and non DAC donors do not report to the DAC. Due to the absence of comprehensive data on non-dac donor DAH flows, IHME was unable to capture all possible sources of DAH. Although IHME identified issues with data from the DAC database, it still proved an invaluable tool for their research enabling them to at least access data that captured a large portion of global health flows, even if the data were not complete. The increase in global players providing development assistance means that the global picture of health spending is not limited to government donors - in recent years NGOs and private foundations have made significant contributions. IHME was unable to provide a global figure of all private DAH due to the absence of a central repository or reliable data to track non-us private foundations and NGOs. To analyse DAH from US foundations, IHME used data from the Foundation Centre and supplemented it with DAH data from BMGF because it is the largest US foundation and funds a significant proportion of health related programmes. IHME s research shows that in 2007 global health disbursements from US based foundations (excluding BMGF) were nearly $300 million compared to over $1.2 billion from BMGF 10. Aggregate data Data for foundations were not disaggregated enough for IHME s research. In particular there was no distinction between commitments and disbursements. At present the Foundation Center is the only US organisation that collects grant information for foundations. However, the dataset provided by the Foundation Center did not disaggregate DAH from foundations by health focus area and by country. This background research into foundation funding was then supplemented with data from BMGF because it contributes the largest amount to global health of any US foundation. IHME collected data on BMGF s disbursements from its online grants database and IRS 990s, which included detailed information on grant recipients, country, and health focus area non DAC donors now report to the DAC (the 16 th includes Arab countries which comprises of Saudi Arabia, Kuwait and United Arab Emirates) 10 Financing Global Health 2009: Tracking Development Assistance for Health, pg 33

8 Definitions In order to collect more detailed data on DAH from foundations, IHME would have had to collect and analyse individual foundation s annual financial reports which would have been highly resourceintensive to collect and then compare. Definitions and fiscal years amongst foundations and other donors vary greatly and would therefore not be comparable. Private aid flows: non-us NGOs IHME found non-us NGO data difficult to collect and analyse which meant they excluded those data from the global DAH figure. The absence of raw non-us NGO data or a central repository for non-us NGO data meant that IHME would have to collect and analyse each NGO s accounts which would be prohibitively time-consuming. For IHME the existence of a central repository of data for NGOs would have been useful. This type of database would have enabled IHME to map out NGO contributions and identify the big NGO players on a global scale. Limited data for a limited number of NGOs The lack of a centralised database or easily accessible data for NGOs DAH meant IHME relied on the US Agency for International Development (USAID) Volag 11 report for NGO data. The Volag report only provides data for NGOs which receive grants from USAID and covers domestic and overseas expenditure, revenue from the US government and other public sources, private contributions and in-kind contributions. Again, the ability to analyse global flows was limited because the data only covered US NGOs that received funding from the US government every year. Another challenge with data from the Volag report was that non-us NGO coverage only started in To address this issue, IHME attempted to compile pre-1998 data on health expenditures of the top 10 non-us NGOs from financial reports on their websites and by contacting them directly. However, it proved extremely difficult to obtain reliable data before 2000 and as a result, only data available from US registered NGOs in the USAID reports were included in IHME s study from Due to the absence of US NGO data for 2007, IHME had to estimate expenditure volumes for DAH from these NGOs in 2007 based on annual growth rates from Lack of disaggregated data Another challenge with data from the USAID report was that the overseas expenditure information was not disaggregated by sector and the report failed to quantify how much NGOs spend on health. In addition, US NGO data were not disaggregated enough to show how much NGOs spend on different health focus areas in different countries. What would a better system of data on health flows look like? IHME found the DAC database to be the most useful source of information for research on DAH because it was a central repository, which although incomplete for certain donors and for certain fields, still enabled them to produce a comprehensive estimate figure of global health flows from DAC donors. However, there will always be challenges with the DAC database as DAC coverage is limited to donor governments and multilateral organizations and excludes financing from NGOs and foundations. An improved system for health data would ideally include data from foundations and NGOs which is formatted in the same way to ensure comparability. As the influence of foundations and NGOs grows and these actors contribute increasing amounts of DAH, their contributions must 11

9 be included in studies of development assistance for health. IHME suggests that the establishment of a central repository of data on development assistance flows from NGOs and foundations would be beneficial for their research enabling them to piece together a more comprehensive picture of donors global health funding. The standard could be modelled on the DAC database by building on existing definitions. IHME recognises the need for timely, coherent data on financial health flows. For IHME an ideal database or system would be one where data are publically available and comprehensive, with a standardised format, common definitions and specific reporting guidelines. This would enable researchers to analyse and produce information that is a more accurate reflection of real DAH flows. The outcome of more accurate data on DAH could inform policy making and planning in developed and developing countries. IHME s needs for better data In order for IHME to produce completely accurate and current research on actual DAH flows that reflect funding from a variety of donors, they need current aid data to be improved in the following ways: Better coverage IHME needs more comprehensive, high-quality, and comparable DAH data. Currently, comparable disaggregated data from foundations and NGOs are limited. Improved coverage of DAH from a wide range of donors, including non DAC donors would result in a more accurate picture of real global health spending. Consistency amongst terminology IHME wants to know how much money is committed and then actually disbursed. Terminology used for reporting development assistance varies amongst different donors. For example, there are differences between foundations that make grants and NGOs that implement programmes. Inconsistency in definitions by different donors means that IHME must invest time interpreting and recoding different datasets and terminology so it can be compared. IHME s research would benefit from: Standardized reporting: For example, commitments, disbursements, sector, purpose, recipient, and recipient country need to be reported consistently by all actors More comparable data: Many donors and organizations use different fiscal years and accounting methods which makes data comparison complex and can affect the validity of the analysis. For IHME, the disbursement date is important for donors and organizations to include Detailed data IHME s research was hindered by the lack of detail on DAH data. To improve future studies of DAH, IHME needs: Longer, more detailed project descriptions are important. A project described as an HIV and AIDS programme is too broad and limits the user s knowledge and understanding of the specific purpose of the programme

10 More detail on implementing agents is necessary. A principle recipient field identifying the agency receiving the funds, and a recipient country field showing the intended recipient country are important for IHME s research Greater detail on financial terms such as financial grants, loans status and purpose of the grant or loan are also important variables that donors and organizations should include Timely data Up-to-date and timely information on DAH is needed. The current time lag of two years for DAC data prevents the analysis of real time health funding. More timely data on DAH would facilitate the publication of up-to-date figures and would be more relevant for policy makers. Disaggregated data IHME needs data to be disaggregated by sector, such as health, and then further disaggregation at health-focus level. NGOs report an aggregate amount of total annual overseas expenditure which is not broken down by sector. This lack of disaggregated DAH data amongst some donors and organizations prevents IHME from understanding the allocation of this aid by country and health focus area.