MONITORING & EVALUATION PLAN FOR HEALTH SECTOR STRATEGIC & INVESTMENT PLAN 2010/ /15

Transcription

1 GOVERNMENT OF UGANDA Ministry of Health Promoting People s Health to Enhance Socio-economic Development MONITORING & EVALUATION PLAN FOR HEALTH SECTOR STRATEGIC & INVESTMENT PLAN 2010/ /15 Ministry of Health Plot 6 Lourdel Road, Wandegeya P. O. Box 7272, Kampala, Uganda MAY 2011 i

2 Any part of this document may be freely reviewed, quoted, reproduced or translated in full or in part, provided the source is acknowledged. It may not be sold or used in conjunction with commercial purposes or for profit. Government of Uganda, Ministry of Health: Monitoring & Evaluation Plan for the Health Sector Strategic and Investment Plan, 2010/ /15. Published by: Ministry of Health PO Box 7272 Kampala, Uganda Website: i

3 Executive Summary The Ministry of Health launched the Health Sector Strategic and Investment Plan 2010/ /15 which defines the medium term health agenda and operationalize Uganda s aspirations as outlined in the NDP and the Public Investment Plan (PIP) 2010/ /13. The development of M&E Plan for HSSIP 2010/ /15 has been largely informed by lessons from the Mid Term Review of the second Health Sector Strategic Plan (2005/ /10) and was in conjunction with the development of the HSSIP 2010/ /15, which took into consideration a wide range of policies, the new emerging diseases, the changing climatic conditions, issues of international health and international treaties and conventions to which Uganda is a signatory. The process of development of the HSSIP M&E plan was highly consultative, participatory and transparent. The Supervision, Monitoring, Evaluation and Research Technical Working Group (SME&R TWG) was responsible for overseeing the development of the HSSIP M&E plan. Stakeholders from Local Governments, Development Partners (DPs), Civil Society, private sector and academia were consulted during the development process. The M&E Plan for HSSIP 2010/ /15 aims at establishing a system that is robust, comprehensive, fully integrated, harmonized and well coordinated to guide monitoring of the implementation of the HSSIP and evaluate impact. It is envisaged that this comprehensive M&E plan to which all health partners subscribe shall be the basis for improving the quality of routine information systems and be used to institutionalize mechanisms and tools for measuring quality of both facility and community based services. It should also strengthen dissemination and use of information at both national and sub national levels. I wish to express my appreciation to all of you who worked tirelessly to develop the M&E Plan for HSSIP 2010/ /15 on behalf of the people of Uganda. I look forward to the acceleration of the implementation of the M&E Plan for the HSSIP towards attainment of our national and international health goals. For God and My Country DR. CHRISTINE ONDOA MINISTER OF HEALTH ii

4 TABLE OF CONTENTS Executive Summary... ii Table of Tables... vi Table of Figures... viii Acronyms... ix Operational Definitions... xi 1 Introduction Goal and Objectives of the HSSIP M&E Plan Goal Specific Objectives Key Outputs Outcomes Process of Development of the HSSIP M&E Plan The HSSIP 2010/ /15 Conceptual Framework Implications for Sector Monitoring and Evaluation Monitoring & Evaluation Mechanism for HSSIP 2010/ / The General M&E Framework The HSSIP Monitoring and Evaluation Tools Country Compact for Implementation of the HSSIP 2010/ / Core Indicators for HSSIP 2010/ / Programme / Project Specific Indicators... 9 i

5 2.2.4 HSSIP M&E Plan Programme and Project Specific M&E Plans Data Management Sources of Data for Health Sector Monitoring Critical gaps and challenges in data management Data Collection Methods and Tools Data Analysis and Synthesis Data Accuracy and Reliability Data Dissemination Data Communication Responsibilities for Data Management by Level The HSSIP 2010/ /15 Monitoring and Review Process Country mechanism for review and action Performance Monitoring and Review at Central Level Performance Monitoring and Review at Regional Level Performance Monitoring and Review in Local Governments/District Level Performance Monitoring and Review at Health Sub-District Level Performance Monitoring and Review at Health Facility Level Performance Monitoring and Review at Community Level Performance Monitoring and Review at VHT Level The National Health Accounts Joint Review Mission ii

6 3.2 Performance Monitoring by the MoFPED and OPM Performance Monitoring by the MoFPED and OPM at Central Level Performance Monitoring by the MoFPED at LG Level Monitoring Performance of the Supply Chain Management System Monitoring by the Health Development Partners Linkage between the health sector reviews, disease / programme / project specific reviews and global reporting Performance Monitoring and Review for Global Health Grants Performance Monitoring and Review for Civil Society Organisation and Private Sector Performance Monitoring and Review for Implementing Development Partners Sector Routine Feedback to Sub-national and Key Stakeholders Reporting Progress against the NDP 2010/ / Reporting Requirements on International Commitments and Resolutions HSSIP Evaluation Programme / Project Evaluation Mid Term Review HSSIP End Term Evaluation Surveys Surveys Commissioned by the MoH Service Availability and Readiness Assessment HIV/AIDS Epidemiological Surveillance iii

7 4.1.3 The Uganda HIV/AIDS Indicator Survey Availability of the six tracer medicines study Uganda Malaria Indicator Survey National Tuberculosis prevalence survey Client Satisfaction Surveys Non-Communicable Diseases (NCD) Survey Health related surveys by other institutions Uganda Demographic and Health Survey The Uganda National Panel Survey The Uganda National Household Survey National Service Delivery Survey (NSDS) Health Research and Evidence Generation Knowledge Management Use of Knowledge / Translation and Application Decision Support Tools and Approaches HSSIP 2010/ /15 M&E Plan Implementation Arrangements Strategies and Key Interventions HSSIP M&E Plan Performance Indicators The Key HSSIP M&E Plan Implementation Tasks Key Stakeholders Roles and Responsibilities for the HSSIP M&E Plan Implementation Calendar for Monitoring HSSIP 2010/ /15 M&E Plan Implementation iv

8 5.7 Monitoring the Implementation of the HSSIP M&E Plan Budget Annexes... I 7.1 HSSIP 2010/ /15 Indicators with targets... I 7.2 Key NDP Indicators... XVI 7.3 WHO Statistical Data... XVII 7.4 UNGASS National Indicators... XXII 7.5 ECSA-HC Regional Core Set of Indicator... XXIII 7.6 Responsibilities, Frequency and Tools for Data Management... XXVI 7.7 HC IV Performance Assessment... XLI 7.8 Hospital Performance Assessment... XLII 7.9 District Ranking... XLIII 7.10 Proposed Time Table for the HSSIP 2010/ /15 Midterm Review XLV 7.11 Proposed Time Table for the HSSIP 2010/ /15 End term Evaluation... XLVII 8 REFERENCES... XLIX v

9 Table of Tables Table 1: HSSIP 2010/ /15 Core Indicators by Domain Table 2: HSSIP 2010/ /15 Core Performance Indicators and corresponding reporting commitments Table 3: Poverty Dimensions and Analysis Table 4: HSSIP Monitoring and Review process Table 5: Timeframe for the DHO HMIS reporting Table 6: Timeframe for HSD HMIS Reporting Table 7: Timeframe for Health Facility Reporting Table 8: Timeframes for Central Government Quarterly Performance Report Submission Table 9: Local Government OBT indicators Table 10: Timeframes for Local Government Quarterly Performance Report Submission Table 11: Medicines Management Performance Summary Matrix Table 12: M&E Calendar for HSSIP 2010/ / Table 13: Estimated Cost of M&E and Knowledge Management Table 14: HSSIP 2010/ /15 M&E Plan Implementation Budget Table 15: Key Results Area Matrix... XVI Table 16: Responsibility, Frequency and Tools for HSSIP Performance Data Collection and Processing... XXVI Table 17: Responsibility, Frequency and Tools for HSSIP Performance Data Analysis and Synthesis... XXX Table 18: Responsibility, Frequency and Tools for HSSIP Performance Data Quality Assessment... XXXIV Table 19: Responsibility, Frequency and Tools for Data Dissemination and Communication... XXXV Table 20: Responsibility, Frequency and Tools for HSSIP Performance Data Translation into Policy and Decision-Making... XL Table 21: Indicators showing functionality of HC IVs... XLI Table 22: Performance Assessement Outputs from General Hospitals... XLII Table 23: Performance Assessement Outputs of Regional Referral and Large PNFP Hospitals... XLII Table 24: District Rankings for the Various Poverty Dimensions... XLIII vi

10 vii

11 Table of Figures Figure 1: HSSIP Conceptual Framework... 5 Figure 2: Monitoring & Evaluation Framework... 7 Figure 3: Country Health Systems Strengthening Platform... 8 Figure 4: Data Flow Diagram Figure 5: Framework for reviewing health progress and performance Figure 6: Country-led platform for monitoring & review of the national health strategy Figure 7: The knowledge management process Figure 8: Translating knowledge into policy and action viii

12 Acronyms AHSPR BFP CAO CCM CHeSS CPD CSO DHIS DHMT DHO DHT ECSA-HC GAVI GIS GoU HC HDP HMB HMIS HRIS HSD HSSP HSSIP HPA HPAC HUMC ICT IFMS IHP IDSR JAF JBSF JRM KM LAN LC LG LTIA MDAs MDGs MPS M&E MoH MoFPED MoLG MTR Annual Health Sector Performance Report Budget Framework Paper Chief Administrative Officer Country Coordinating Mechanism Country Health Systems Surveillance Continuous Professional Development Civil Society Organisation District Health Information System District Health Management Team District Health Office District Health Team Eastern, Central and Southern Africa Health Community Global Alliance for Vaccines and Immunisation Geographical Information System Government of Uganda Health Center Health Development Partners Hospital Management Board Health Management Information System Human Resource Information System Health Sub-District Health Sector Strategic Plan Health Sector Strategic and Investment Plan Hospital / HC IV Performance Assessment Health Policy Advisory Committee Health Unit Management Committee Information and Communication Technology Integrated Financial Management System International Health Partnerships and related Initiatives Integrated Disease Surveillance and Response Joint Assessment Framework Joint Budget Support Framework Joint Review Mission Knowledge Management Local Area Network Local Council Local Government Long Term Institutional Arrangements Ministries, Departments and Agencies Millennium Development Goals Ministerial Policy Statement Monitoring and Evaluation Ministry of Health Ministry of Finance, Planning and Economic Development Ministry of Local Government Mid Term Review ix

13 NCD NDP NHA NHP NSDS NMS NPA NRH OBT OPM PEAP PHP PNFP QAD RC RRH SCMS SMC SWAp TPC TRM TWG UDHS UNGASS UNHRO UNPS VFM VHT WHO Non-Communicable Diseases National Development Plan National Health Accounts National Health Policy National Service Delivery Survey National Medical Stores National Planning Authority National Referral Hospital Output Budgeting Tool Office of the Prime Minister Poverty Eradication Action Plan Private Health Practitioners Private-Not-For-Profit Organizations Quality Assurance Department Resource Center Regional Referral Hospital Supply Chain Management System Senior Management Committee Sector Wide Approach Technical Planning Committee Technical Review Mission Technical Working Group Uganda Demographic Health Survey United Nations General Assembly Special Session on HIV/AIDS Uganda National Health Research Organisation Uganda National Panel Survey Value-For-Money Village Health Team World Health Organisation x

14 Operational Definitions Civil Society Organization: any organisation except the government and the UN system. Data Management: comprises all processes related to data collection, analysis, synthesis and dissemination. Data Quality Assurance: The process of profiling data to discover inconsistencies, and other anomalies in the data cleansing activities (e.g. removing outliers, missing data interpolation) to improve the data quality Evaluation: The rigorous, science-based collection of information about program activities, characteristics, outcomes and impact that determines the merit or worth of a specific program or intervention. Impact: Fundamental intended or unintended changes in the conditions of the target group, population, system or organization. Knowledge Management: Is a set of principles, tools and practices that enable people to create knowledge, and to share, translate and apply what they know to create value and improve effectiveness. Monitoring: The routine tracking and reporting of priority information about a program and its intended outputs and outcomes. Monitoring & Evaluation Plan: Is an integral part of the component of the national health strategy that addresses all the monitoring and evaluation activities of the strategy. Monitoring & Evaluation Framework: Refers to the performance based framework for monitoring and evaluation of health systems strengthening. Outcome: Actual or intended changes in use, satisfaction levels or behaviour that a planned intervention seeks to support. Performance: The extent to which relevance, effectiveness, efficiency, economy, sustainability and impact (expected and unexpected) are achieved by an initiative, programme or policy. Performance measurement: The ongoing monitoring and evaluation of the results of an initiative, programme or policy, and in particular, progress towards pre-established goals. Performance management: Reflects the extent to which the implementing institution has control, or manageable interest, over a particular initiative, programme or policy. Review: Is an assessment of performance or progress of a policy, sector, institution, programme or project, periodically or on an ad hoc basis. Reviews tend to emphasize operational aspects, and are therefore closely linked to the monitoring function. xi

15 1 Introduction The Health Sector Strategic Investment Plan Monitoring and Evaluation (HSSIP M&E) plan 2010/ /15 has been developed to operationalise the strategic orientation provided for comprehensive Monitoring and Evaluation (M&E) in HSSIP 2010/ /15. M&E aims at informing policy makers about progress towards achieving targets as set in the HSSIP. In combination with other initiatives, the M&E Plan will focus attention of stakeholders and direct efforts towards the ultimate goal of the sector: to attain a good standard of health for all people in Uganda in order to promote a healthy and productive life. In order to do so, the M&E plan needs to provide strategic information to decision-makers, who will combine this information with other strategic information to make evidence-based decisions. This is relevant to both national and subnational (district, sub-district and institutional/facility) levels. At national level, strategic information will be used by the management and partnership/governance structures described in the HSSIP 2010/ /15 for improved management and service delivery. In addition, selected information will be provided to the Office of the Prime Minister (OPM), Ministry of Finance, Planning and Economic Development (MoFPED), National Planning Authority (NPA) and others, as well as to Health Development Partners (HDPs) in line with government procedures and partnership commitments. Providing this information is often key for obtaining funds and other resources for the sector. The MoH also has reporting obligations towards international institutions such as the World Health Organisation (WHO) and the Eastern, Central and Southern Africa Health Community (ECSA-HC, among others). This plan is based on principles intended to institutionalize the use of M&E as a tool for better public sector management, transparency and accountability, so as to support the overall direction of the HSSIP and achievement of the results. The underpinning principles include; i) simplicity; ii) flexibility; iii) progressiveness; iv) harmonization; v) alignment; and vi) enhance ownership The HSSIP M&E plan describes the processes, methods and tools that the sector will use to collect, compile, report and use data, and provide feed-back as part of the national Health Sector M&E System. It translates these processes into annualized and costed activities, and assigns responsibilities for implementation. However, as the MoH assumes its stewardship role vis-a-vis the health-related ministries with regard to other health determinants which are part of the responsibility of these ministries, the HSSIP M&E Plan also describes how key-information will be obtained from these ministries. The HSSIP 2010/ /15 M&E plan has been developed in a participatory manner and shall guide all HSSIP 2010/ /15 M&E activities. The M&E plan specifies the type of monitoring, monitoring reports, timing of evaluations, roles and responsibilities for the overall process and how they interact with the reporting each implementer is required to perform (clear roles and responsibilities with respect to data gathering and reporting;). It focuses on the main M&E activities and aligns them to the existing national and international structures and frameworks. It is intended to document what needs to be monitored, with whom, by whom, when, how, and how the M&E data will be used. It also outlines how and when the different types of studies and evaluations will be conducted by the sector. In addition to the above considerations the M&E plan has been developed to address some of the M&E challenges identified during HSSP II. While systems for performance monitoring and 1

16 evaluation were in place during HSSP I & II, there were enormous challenges. Most of the challenges resulted from lack of an M&E plan for the previous strategic plans. The national M&E arrangements were weak and comprised only a few functional systems at program/project level. They were characterized by fragmentation; duplication; weak co-ordination; lack of a clear results chain; poor definitions, tracking and reporting of outcomes and results; use of different formats and approaches with no common guidelines and standards; lack of national ownership; inadequate feedback and poor sharing of results across the sector and other stakeholders. Analysis of information was not carried out in a comprehensive manner, and communication of information was not tailored to the recipients of information this was primarily left in reports, or scientific papers. In addition there was poor use of the data generated; problems related to capacity and resourcing. It is envisaged that this comprehensive M&E plan to which all health partners subscribe shall be the basis for improving the quality of routine information systems and be used to institutionalize mechanisms and tools for measuring quality of both facility and community based services. It should also strengthen dissemination and use of information at both national and sub national levels. In order to achieve the above a lot will have to be done to improve recording and reporting, and use of data at all levels and all stakeholders, public, private and community to effectively monitor and later evaluate the HSSIP 2010/ /15 implementation, including the M&E plan itself. In the absence of an M&E framework, plan and budget, several system have been operating parallel to each other, creating not only an additional burden to health workers at service delivery and Local Government (LG) level, but in some cases bias towards systems that provide incentives. Systems in place include HMIS; Performance Measurement and Management Plan (PMMP) of the UAC; the LOGICS tools of the Ministry of Local Government (MoLG); Output Budgeting Tool (OBT) of the Ministry of Finance, Planning and Economic Development (MoFPED); Office of the Prime Minister (OPM) Report; Joint Assessment Framework (JAF) report; Ministry of Public Service (MoPS) performance report; and many localized programme and project systems. The health sector carries out three types of monitoring which address different stages in the results chain, namely; (i) Financial implementation monitoring addresses whether or not budgets have been released and spent in line with allocations; (ii) Physical implementation monitoring addresses whether activities have taken place in line with targets; and (iii) Outputs, outcome and impact monitoring trace whether or not results are occurring amongst the target population. The purpose of the sector monitoring and evaluation system is to coordinate and support the MoH, related Ministries, Development Partners, LGs and stakeholders to regularly and systematically track progress of implementation of priority interventions of the HSSIP and assess performance of the sector and LGs in accordance with the agreed objectives and performance indicators over the Medium Term (output, outcome and impact monitoring). 2

17 1.1 Goal and Objectives of the HSSIP M&E Plan The HSSIP M&E plan goal is aligned with one of the main objectives of the HSSIP 2010/ /2015 which is; Deepen stewardship of the health agenda, by the MoH. The stewardship function of the MoH focuses around provision of appropriate guidance to implement health programs, but also to other sector actors, to what are the priorities for implementation. In order to do this there is need for an M&E system that provides timely and accurate information to government and partners in order to inform performance reviews, policy discussions and periodic revisions to the national strategic and operational plans Goal The goal of the HSSIP 2010/ /15 M&E plan is to establish a system that is robust, comprehensive, fully integrated, harmonized and well coordinated to guide monitoring of the implementation of the HSSIP and evaluate impact Specific Objectives The specific objectives of the HSSIP 2010/ /15 M&E plan are; 1. To provide a health sector-wide framework for tracking progress and demonstrating results of the HSSIP 2010/ /15 over the medium term. 2. To build capacity of the MoH, semi-autonomous institutions, LGs, PNFP/Private facilities and CSOs to regularly and systematically track progress of implementation of the HSSIP. 3. To facilitate MoH and other stakeholders assess the health sector performance in accordance with the agreed objectives and performance indicators to support management for results (evidence based decision making), 4. To improve compliance with government policies (accountability), and constructive engagement with stakeholders (policy dialogue). 5. To facilitate continuous learning (document and share the challenges and lessons learnt) by MoH, semi-autonomous institutions, LGs and other stakeholders during implementation of the HSSIP 2010/ / To promote the use of locally generated health information Key Outputs The expected key outputs of the M&E framework are: i) A functional sector-wide unified integrated, harmonized and well coordinated M&E system with effective and timely feedback to stakeholders. ii) Performance reports (baseline survey reports, periodic progress reports, annual performance reports, financial audit reports etc.) iii) Basic statistical data on health service delivery, resources, outputs and beneficiaries. iv) Regular updates on core performance indicators. v) National infrastructure for M&E. 3

18 1.1.4 Outcomes This M&E Plan should result in: i) Timely reporting on progress of implementation of the HSSIP; ii) Timely meeting of reporting obligations to government, DPs and International Partners; iii) Objective decision making for performance improvement; planning and resource allocation iv) Accountability to government, DPs and citizens; v) Policy dialogue with stakeholders. vi) Evidence-based policy development and advocacy. vii) Institutional memory on HSSIP 2010/ /15 implementation. 1.2 Process of Development of the HSSIP M&E Plan The development of the HSSIP M&E plan was in conjunction with the development of the HSSIP 2010/ /15, which took into consideration a wide range of policies, the new emerging diseases, the changing climatic conditions and issues of international health. The process also took into consideration the international treaties and conventions to which Uganda is a signatory more especially (i) The Millennium Development Goals (MDGs), three of which are directly related to health and most others address determinants of health; (ii) The International Covenant on Economic, Social and Cultural Rights; (iii) the Convention on All forms of Discrimination Against Women; (iv) the International Health Partnerships and related Initiatives (IHP+) which seek to achieve better health results and provide a framework for increased aid effectiveness; among others. The aim of reviewing policies and plans during the development of the HSSIP M&E plan 2010/ /15 was to harmonize this plan with the other existing sector and inter sectoral M&E plans. A review of a wide range of national and health sector documents was done to provide in-depth analysis and understanding of the sector M&E system; such as HSSP II, NDP 2010/ /15, NHP II, HSSIP 2010/ /15, programme specific M&E plans, the Health Management Information System (HMIS) and the Guidelines for Budget Preparation and Reporting Using the Output Budgeting Tool (OBT), November, The Supervision, Monitoring, Evaluation and Research Technical Working Group (SME&R TWG) comprising of MoH officials, Development Partners (DPs), Civil Society, private sector and academia was responsible for overseeing the development of the HSSIP M&E plan. A Task Force (TF) was formed which worked in consultation with all relevant stakeholders in the development process. The involvement of the different stakeholders was important in order to ensure ownership of the plan. 1.3 The HSSIP 2010/ /15 Conceptual Framework The overall goal for the Health Sector during HSSIP 2010/ /15 is: To attain a good standard of health for all people in Uganda in order to promote a healthy and productive life. The goal, when achieved, shall lead to acceleration in the improvements in the level, and distribution of health in the country, as captured in the health impact indicators. 4

19 To achieve this goal, the health sector shall focus on achieving universal coverage with quality health, and health related services through addressing the following objectives. 1. Scale up critical interventions for health, and health related services, with emphasis on vulnerable populations 2. Improve the levels, and equity in access and demand to defined services needed for health 3. Accelerate quality and safety improvements for health and health services through implementation of identified interventions 4. Improve on the efficiency, and effectiveness of resource management for service delivery in the sector 5. Deepen stewardship of the health agenda, by the MoH. Investments in human resources, health infrastructure, commodities and management support (including planning, supervision and M&E) in this period will primarily aim at achieving better results in the following programme areas: Sexual and Reproductive Health, Child Health, Health Education and Control and Prevention of Communicable Diseases (HIV/AIDS, Malaria and Tuberculosis). It is hoped that these strategic objectives together will result in substantial progress in terms of coverage and demand for services and, ultimately, in an accelerated improvement in the level and equity of the health status of the people of Uganda. Figure 1: HSSIP Conceptual Framework Overall development theme (NDP) Growth, Employment and Socio-Economic Transformation for Prosperity Health Policy Goal (NHP II) Promoting people s health to enhance socio economic development HSSIP Goal: To attain a good standard of health for all people in Uganda in order to promote a healthy and productive life Objective 1: Objective 2: Objective 3: Objective 4: Objective 5: Scale up critical interventions Improve access and demand Accelerate quality & safety improvements Improve efficiency & effectiveness Deepen stewardship health INVESTMENT PRIORITIES Area 1 Area 2 Area 3 Area 4 HUMAN RESOURCES INFRASTRUCTURE MEDICAL PRODUCTS OPERATIONS 5

20 1.4 Implications for Sector Monitoring and Evaluation For more sustained and accelerated improvement in the health impact, it is important for the sector to strategically focus on a comprehensive knowledge management approach. This should guide a comprehensive look at information needs, analysis and use to better guide decision making for health. This requires definition of a comprehensive performance monitoring approach for the sector M&E strategy, which uses input, output, outcome and impact indicators to generate information for analysis and use. The sector M&E component should have a supportive institutional environment, with defined roles and responsibilities for the different stakeholders. There is need for sufficient funding and human resources with adequate technical capacity to manage the various components of an effective M&E system in support of progress and performance reviews. 6

21 Governance Financing 2 Monitoring & Evaluation Mechanism for HSSIP 2010/ /15 This section is comprised of the HSSIP 2010/ /15 M&E mechanism including: the general monitoring framework, the county-led M&E platform, tools for M&E, data management, responsibilities for data management. 2.1 The General M&E Framework The HSSIP 2010/ /15 M&E plan elaborates a Monitoring Framework for the Sector which includes a range of indicators at various levels, sources of information, regularity of various reports, and monitoring structures. The monitoring framework for tracking progress is informed by the need to comprehensively monitor and review sector progress. The framework for the analysis is based on the common steps of the M&E logical framework, which shows the way in which inputs may lead to desirable health impact. The framework is an adaptation of the M&E framework for health systems strengthening (HSS) 1 which was developed by Global Partners and countries. The framework builds upon principles derived from the Paris Declaration on aid harmonization and effectiveness and the International Health Partnership (IHP+). It is intended to ensure that all indicator areas - from inputs to impact - are considered in the analysis, and pathways of influence are clarified (Figure 2). Figure 2: Monitoring & Evaluation Framework Monitoring & Evaluation of health systems reform /strengthening Inputs & processes Outputs Outcomes Impact Indicator domains Infrastructure / ICT Health workforce Supply chain Information Intervention access & services readiness Intervention quality, safety and efficiency Coverage of interventions Prevalence risk behaviours & factors Improved health outcomes & equity Social and financial risk protection Responsiveness Data sources Administrative sources Financial tracking system; NHA Databases and records: HR, infrastructure, medicines etc. Policy data Facility assessments Population-based surveys Coverage, health status, equity, risk protection, responsiveness Clinical reporting systems Service readiness, quality, coverage, health status Vital registration Analysis & synthesis Communication & use Data quality assessment; Estimates and projections; In-depth studies; Use of research results; Assessment of progress and performance of health systems Targeted and comprehensive reporting; Regular country review processes; Global reporting For this, the MoH has identified core indicators to guide analysis of sector progress, at all the indicator domains input / process, output, outcome, and impact. Core indicators are defined, and structured to inform on and compare trends across the different domains. This approach takes into consideration that the respective indicators are not viewed in isolation, but rather are intricately linked to provide information on overall progress. 1 WHO, GAVI, Global Fund and the World Bank, Monitoring and evaluation of health systems strengthening: an operational framework, November 2009, Geneva. 7

22 The HSSIP is implemented through the Long Term Institutional Arrangements (LTIA), with emphasis on one country-led M&E platform for the HSSI - also referred to as the Country Health Systems Surveillance (CHeSS) platform. The country-led platform is expected to bring together the M&E work in disease-specific programmes, such as TB, HIV/AIDS and immunization, with crosscutting efforts such as tracking human resources, logistics and procurement, and health service delivery (Figure 3). The country-led platform aims is to improve the availability, quality and use of the data needed to inform health sector reviews and planning processes, and to monitor health progress and system performance. It is the platform for subnational (regions, districts, projects, CSOs), national and global reporting, aligning partners at country and global levels around a common approach to country support and reporting requirements. Figure 3: Country Health Systems Strengthening Platform 2.2 The HSSIP Monitoring and Evaluation Tools M&E shall be carried out within the National M&E framework using tools that consider outputs and indicators to be drawn from approved work plans and budgets for the HSSIP 2010/ /15. The tools used for monitoring the sector were agreed to consist of; 1. The Country Compact for Implementation of the HSSIP 2010/ /15; 2. HSSIP 2010/ /15 Core indicators (26) harmonized with MDGs, NDP & JAF (See Table 1); 3. Program / Project indicators at various levels 4. A detailed HSSIP 2010/ /15 M&E plan 5. Programme and Project Specific M&E plans Specific indicators from programme M&E plans will be used to supplement the national level indicators in monitoring specific programme performance. 8

23 2.2.1 Country Compact for Implementation of the HSSIP 2010/ /15 The Country Compact for implementation of the HSSIP 2020/ /15 is a memorandum of understanding between the Government of Uganda and its partners in the health sector for the purpose of maintaining policy dialogue, promoting joint planning, and effective implementation and monitoring of the HSSIP 2010/ /15. The partners include Health Development Partners, PNFPs, the Private Health Practitioners (PHP) and CSOs and are collectively referred to as Health Sector Partners. The principles guiding this Compact are: Ownership and leadership by government Alignment of all partner programmes, activities and funding to one national plan (HSSIP) and harmonized annual health plan Use of common management arrangements Value for money; and, One monitoring framework to promote accountability The principles are consistent with national policies, with bilateral and multilateral agreements between the Government of Uganda and its development partners and with international agreements ratified by Uganda including the Paris Declaration on Aid Effectiveness, Accra Agenda for Action, Harmonization for Health in Africa Action Framework and the Global International Health Partnerships and related Initiatives (IHP+) Core Indicators for HSSIP 2010/ /15 Section 8 of HSSIP 2010/ /15 has defined 26 core indicators (Table 1 below) and targets over the five year period for informing progress in critical elements of the health sector framework. The selected core indicators are based on the following critical variables: They reflect all domains presented in the M&E conceptual framework, Having broad information on important components of the indicator domain, and Alignment to existing sector monitoring commitments Programme / Project Specific Indicators During development of the HSSIP 2010/ /15, a total of 262 performance indicators with targets (Annex 7.1) were developed covering all strategic interventions. The 26 HSSIP core indicators were selected from the 262 performance indicators. Performance monitoring at programme level shall be based on the programme specific indicators monitored within the sector M&E plan HSSIP M&E Plan The HSSIP 2010/ /15 M&E plan is another tool for monitoring the health sector. Section 3 of this document provides the details of the HSSIP 2010/ /15 M&E plan. All departments, institutions, LGs, health facilities and other stakeholders will develop individual M&E plans and calendars aligned to the overall sector M&E plan. 9

24 2.2.5 Programme and Project Specific M&E Plans Development of programme and project specific M&E plans is a requirement for most donor supported programmes and projects. Under the country-led platform it is a requirement for these programmes and projects to develop M&E plans aligned to the overall HSSIP M&E plan and utilize the MoH data collection and aggregation system. The HSSIP M&E Plan provides the necessary background for the development of programme specific M&E plans in the sector, e.g. disease specific plans, hospitals, district or (semi) autonomous institutions plans; it is the overall framework for M&E plans in the sector. Table 1: HSSIP 2010/ /15 Core Indicators by Domain INPUT & PROCESS (4) OUTPUT (5) OUTCOME (12) IMPACT (5) Health financing Information, Governance General Government allocated on health as % of total government budget Workforce 6 Annual reduction in absenteeism rate (m/f) % of approved posts filled by trained health workers Service access and readiness % of new TB smear + cases notified compared to expected (TB case detection rate) Per capita OPD utilization rate (m/f) % of health facilities without any stock outs of six tracer medicines % HCs IV with a functioning theatre (providing EMOC) Infrastructure Service quality and safety 14 % of villages/ wards with a functional VHT, by district 9 % clients expressing satisfaction with health services Coverage of interventions % pregnant women attending 4 ANC sessions % of deliveries in public and PNFP (n of deliveries/expected deliveries) % children under one year immunized with 3 rd dose pentavalent vaccine % one year old children immunized against measles % pregnant women who have completed IPT2 % of children exposed to HIV from their mothers accessing HIV testing within 12 months % UFs with fever receiving malaria treatment within 24 hours % eligible persons receiving ARV therapy Risk factors and behaviours Health status Maternal Mortality Ratio (per 100,000 live birth) Neonatal mortality rate (per 1000) Infant Mortality Rate (per 1000) Under 5 mortality rate (per 1000) Financial risk protection % of households experiencing catastrophic payments 18 % of households with a pit latrine 19 % U5 s new visits with height /age above lower line (PR) 20 % children under 5 with weight /age above lower line (PR) 21 Contraceptive Prevalence Rate 10

25 Table 2: HSSIP 2010/ /15 Core Performance Indicators and corresponding reporting commitments Indicator domain Indicator Related reports Source of data Reporting Frequency Target Baseline, (year) 2010/ / / / /15 Health Impact Maternal Mortality Ratio (per 100,000 live birth) MDG, WHO, NDP, JAF, ECSA UDHS Every 5 years 435 (2006) 131 Neonatal Mortality rate (per 1000 live births) WHO, ECSA UDHS Every 5 years 70 (2006) 23 Infant Mortality Rate (per 1000 live births) MDG, WHO, NDP, JAF, ECSA UDHS Every 5 years 76 (2006) 41 Under 5 mortality rate (per 1000 live births) MDG, WHO, NDP,ECSA UDHS Every 5 years 137 (2006) 56 % of households experiencing catastrophic health expenditures - UNH survey Every 2 years 28 (2009) Coverage Health Services for % pregnant women attending 4 ANC sessions WHO, ECSA % deliveries in health facilities WHO, NDP, JAF, MoFPED HMIS Monthly 47 (09/10) HMIS Monthly 33 (09/10) % children under one year immunized with 3 rd dose Pentavalent vaccine (m/f) WHO, NDP, JAF, MoFPED HMIS Monthly 76 (09/10)

26 Indicator domain Indicator Related reports Source of data Reporting Frequency Target Baseline, (year) 2010/ / / / /15 % one year old children immunized against measles (m/f) MDG, WHO, ECSA HMIS Monthly 72 (09/10) % pregnant women who have completed IPT2 - HMIS Monthly 47 (09/10) % of children exposed to HIV from their mothers accessing HIV testing within 12 months (m/f) % U5s with fever receiving malaria treatment within 24 hours from VHT (m/f) - HMIS Monthly 29 (08/09) MDG HMIS Monthly 13.7 (09/10) % eligible persons receiving ARV therapy (m/f) NASP, WHO, UNGAS, ECSA HMIS Monthly 53 (2009) Coverage's for other health determinants % of households with a pit latrine - HMIS, UDHS % U5 children with height /age below lower line (stunting) (m/f) MDG, WHO, ECSA UDHS Every 5 years Annually 69.7 (09/10) (2006) % U5 children with weight /age below lower line (wasting) (m/f) MDG, WHO, ECSA HMIS, UDHS Annually 16 (2006) Coverage's for risk factors Contraceptive Prevalence Rate MDG, WHO, JAF, ECSA UDHS / UNPS Annually 38 (09/10)

27 Indicator domain Indicator Related reports Source of data Reporting Frequency Target Baseline, (year) 2010/ / / / /15 Health System outputs (availability, access, quality, safety % of new TB smear + cases notified compared to expected ( TB case detection rate) (m/f) ECSA NTLP reports/ HMIS Annually 56 (09/10) Per capita OPD utilisation rate (m/f) NDP, OBT HMIS Monthly 0.9 (09/10) % of health facilities without stock outs of any of the six tracer medicines in previous 3 months (1 st line antimalarials, Depoprovera, Suphadoxine/pyrimethamine, measles vaccine, ORS, Cotrimoxazole) NDP, JAF, MoFPED, ECSA Annual Drug availability survey Monthly 41 (09/10) % of functional Health Centre IVs (providing EMOC) - HMIS Annually 23 (09/10) % clients expressing satisfaction with health services (waiting time) ECSA MoH survey Biannual 46 (2008) Health Investments and governance % of approved posts filled by trained health workers NDP, JAF, ECSA HMIS Annually 56 (09/10) % Annual reduction in absenteeism rate JAF UNPS Annually 46 (09/10) % of villages/ wards with trained VHTs, by district - HMIS Annually 31 (09/10) General Government allocation for health as % of total government budget WHO, ECSA MTEF Annually 9.6 (09/10) NB: Not all cells will have annual data, as indicators obtained from studies and surveys (in particular, impact and risk coverage indicators) will not be collected annually. 13

28 2.3 Data Management This section on data management will concentrate on the sources of data, critical gaps and challenges in data management; data collection methods and tools; responsibility for data collection and processing; data analysis and synthesis; data quality assessment; data dissemination; communication and use Sources of Data for Health Sector Monitoring The data needs of the HSSIP 2010/ /15 are based on agreed performance indicators (core and programme specific) to facilitate monitoring, evaluation, reporting and decision-making. The main data sources will include; Facility generated data will be collected by all public and private health service delivery facilities and community. In addition different programs and projects managed at the MoH/national level shall provide reports to the Resource Centre on program-specific activities. Health projects managed by implementing partners (DPs and CSOs) at district or community level shall provide reports through the district health system. This data will be collected routinely using established data collection methods and tools and aggregated at health facility, HSD, district and national level. Administrative data sources will provide information on health inventories, supervision, management meetings, logistics management, human resource, financial resource flows and expenditures at national and sub-national levels. Population based health surveys mainly carried out by Uganda Bureau of Statistics (UBOS) and other institutions that generate data relative to populations (population studies) as a whole. Research Institutions and academia that carry out health systems research, clinical trials and longitudinal community studies will also provide data for interpretation and possible use by the sector. Civil registration and vital statistics system is essential for providing quality data on births, death and causes of death. Efforts will be made to link this system to the Health Information System. This system is under the Ministry of Justice and specifically managed by the Bureau of Births and Deaths Registration, which is mandated to collect data on vital events at the community level, including those related to health. Currently this system is not functioning adequately and most population based data is obtained from the population based surveys conducted by UBOS. Population and Household Census is carried out every ten years and will be the primary source of data on size of the population, its geographic distribution, and the social, demographic and economic characteristics. Annual projections at national and sub-national level will be provided by UBOS. Overall, the sources of M&E information will be guided by different information needs, particularly the Government, Parliament, Development Partners, private sector and the community. The MoH will house the central database for reporting on progress of the HSSIP. The MoH Resource Center will serve as a repository for all service delivery data and information at national level. This implies that all health service delivery data and information should be routed through the 14

29 MoH Resource Center (RC) for validation, analysis & synthesis, and dissemination. Consensus will be generated on the establishment and location of a back-up data storage facility for the central repository of information Critical gaps and challenges in data management The HMIS was noted both during the HSSP I & II (HSSP I & II MTR Reports) to have various weaknesses. The low and declining trends for timelines of monthly reporting by districts (68% end of HSSP II) was worrying. There were mismatches of data between the key points of data collection through the various management levels (HSD, district), programs and the RC. Insufficient funds featured as the major constraint to implementation. The markedly limited government budget funds for HMIS lead to over reliance on donor project resources often associated with piece-meal initiatives. Human resource for implementation remains inadequate at all levels of the structure. At national level, the RC still has approved posts that are not filled since HSSP I; at district level, staff to handle data remain inadequate. The continuous inadequacy (numbers) of HMIS tools has greatly affected expectations of having HMIS as a major source of information for monitoring the sector plans. The HMIS remains manual in most HSDs, which affects quality, timeliness and completeness of reports. The low level of prioritization of the HMIS at all levels including the centre, and the inadequate utilization of data is cause for concern. In addition to the above challenges, there are weak linkages between the various data producers leading to inadequate sharing of information. Analysis, synthesis, effective dissemination and use of information to guide policy dialogue and implementation of health programmes remain a challenge. Following a comprehensive analysis of the HSSP II, a number of recommendations were made for improving data management; Improve the level of prioritization of information management in the sector. Appropriate and strategic advocacy should be carried out for various aspects of sector managers and decisionmakers. Particular efforts to be made for appropriate funding (level, mechanisms) for information management. The human resources issues should be addressed at the various levels: through recruitment for the vacant posts at the MoH, and advocacy for recruitment for available posts at the local government (Biostatisticians) and health facility levels (Health Information Assistants). There should be regular training and updating of skills for health workers. Availability of HMIS tools must be improved. Efforts must be made to establish mechanisms of data sharing by all producers. Use of data should be enhanced through provision of timely analysis and effective dissemination. There is urgent need to improve the timeliness, completeness and quality of facility generated data with the help of information technology and supported by an up-to-date national health facility database that covers all public and private health facilities with data on infrastructure, equipment and commodities, service delivery, and health workforce. 15

30 2.3.3 Data Collection Methods and Tools The methods of data collection will be a combination of quantitative and qualitative methods. As far as possible, standardized data collection tools and techniques will be used. Most data in respect of some indicators will be collected annually, and any survey-based indicators will be collected at baseline, mid-term where possible and in the last year of HSSIP implementation. The specific tools and techniques will among others, include; the HMIS; Human Resource Information System (HRIS); Logistics Management Information System (LMIS); and Output Budgeting Tool (OBT) under the Integrated Financial Management System (IFMS). Specific questionnaires will be designed for surveys (baseline, mid and end term), and socio-economic studies (UDHS, UNPS, NHS, NSDS). Standardized checklist will be used to collect data during ongoing monitoring field visits. Formats shall be applied for case studies, stakeholder meetings, performance review forums and management meetings. Geographical Information System (GIS) shall be used to enhance documentation and accountability where applicable. The main tools and techniques for collection of HSSIP M&E information are explained below. i) Health Management Information System: data collected during health service delivery is critical for tracking performance and trend analysis. It cannot be substituted by any other form of data. It will therefore form an important source of data for measuring progress of the HSSIP implementation. The HMIS 2010 has the following categories of information: data on individual clients, information on curative services, information on preventive services, resource management e.g. inventories (staff list, health facility, equipment), logistics and commodities, finance / user fees and Village Health Teams (Integrated Community Case Management). ii) Human Resource Information System: Is a system for collecting, processing, managing and disseminating data and information on human resource for health (HRH). In Uganda, an open-source HRIS is implemented whereby there is linkage with a variety of independent sources of health workforce data, including data from censuses and other national surveys, MoH administrative records, district level sources, independent research studies, and Health Professional Councils data. The HRIS implemented in Uganda's four Health Professional Councils and at the MoH are valuable sources of information about health workers deployed throughout the country. The system captures health personnel's data by organization unit, cadre, etc., and generate various general and aggregated webenabled reports in different formats including graphical reporting. The HRIS generates reports (twice a year in October and April) that display or aggregate the HRH data in various ways to aid analysis or to answer key HR policy and management questions. It enables health planners to quickly and easily obtain up-to-date information specific to the current Ugandan health workforce. The HRIS is the main source of data on staffing levels in public and PNFP facilities. Data and reports from these systems can be very useful for health planning at different levels. iii) Supply Chain Management System (SCMS): A SCMS will be established to strengthen the information systems for medicines and health supplies. This system should allow facilities to conduct web based ordering as well as the agencies should find it easy to disseminate information about ordering, prices and available quantities through the web to the facilities. iv) Integrated Financial Management System (IFMS): GoU introduced IFMS, a computerised financial management system to promote efficiency, secure management of financial data and comprehensive financial reporting. IFMS is currently operational in central MoH, Referral Hospitals and some LGs. An IFMS is a fiscal and financial 16

31 management information system for government that bundles all financial management functions into one suite of applications, it assists the government and MoH to initiate, spend and monitor the budget, initiate and process payments and manage and report on financial activities. It is a core component of financial management systems reforms which promotes efficiency, security of financial data, management and comprehensive financial reporting. v) Surveys questionnaire will be designed and employed from time to time to collect data from beneficiaries/stakeholders in a structured manner. vi) Report formats will be used for presentation of periodic reports for the various stakeholders fora, sector performance reviews, performance reports, monitoring, supervision, research and evaluations. vii) Case studies will be used to document life states or segments of events experienced by particularly target beneficiaries or particular location. These case studies will animate information generated by the M&E system to allow for in-depth understanding of the context and human factors behind otherwise over summarized or generalized data collected through other methods viii) Field visits using checklists will be used from time to time to obtain information that may be required to improve performance or even for obtaining insights for example the Pre- Joint Review Mission Visit and more in-depth investigations. It will also be an aid in the ix) interpretation of results. Standardized meeting formats will be used to guide stakeholder meetings and management meetings. The purpose of these meetings will be to coordinate efforts with other stakeholders, generate consensus with other stakeholders, and review work plans and progress in order to ensure ownership, accountability and transparency. Minutes of such meetings shall be kept for future reference. x) Geographical Information System: With advancement of technology, GIS enabled photographic and video recordings may be used to track changes of implementation of particular programmes of the HSSIP by geographical location. GIS provides a means of analyzing coverage of general or specific health services in relation to need (e.g. disease prevalence rates) and how these services are related to communities (e.g. income level), one another and the larger health infrastructure. M&E data on key health targets at different levels of health service delivery (e.g. district, health sub district) will be used to generate maps and other graphics (like bar and line graphs) that show which areas are meeting the targets or are lagging behind. Overlays of different indicators and further spatial analysis can also identify the hotspots which would be the basis for prioritization of resource allocation. Maps showing health indicators at district, HSD, sub-county and even health facility level can help health planners and managers to identify disadvantaged areas; examine equity issues; and improve decision making on where to invest. Data collection and processing is carried out at all levels for different purposes however the following activities are necessary for all: i) Performance data collection (i.e. data on inputs-activities-outputs). ii) Processing (aggregation) of the performance data from various service delivery points iii) Ensuring quality of reports Responsibility, frequency and tools for the HSSIP data collection and processing are outlined in Table 14. (Annex 7.6). 17

32 2.3.4 Data Analysis and Synthesis Data analysis and synthesis will be done at various levels of HSSIP M&E (National, sub-national to health facility) to enhance evidence based decision making. The results obtained will be summarized into a consistent assessment of the health situation and trends, using core indicators and targets to assess progress and performance. The focus of analysis will be on comparing planned results with actual ones, understand the reasons for divergences and compare the performance at different levels (Quarterly and Annual Progress Reports, OBT, District League Table, mid and end term evaluations, thematic studies and surveys). In addition health systems research as well as qualitative data gathered through systematic processes of analyzing health systems characteristics and changes will be carried out. Basic indicator information shall be the national average achievement. This is obtained from collating all the available information from all reporting units into the national average figure. This will be complemented by more analysis when information 2 on gender index by district becomes available. This is to provide information on the impact of multi-dimensional poverty on actual coverage, and health status achievements. This shall enable better targeting of strategies to address the multi dimensional poverty issues impacting on the results being sought. The different aspects of analysis to be carried out are shown in the table 3 below. Table 3: Poverty Dimensions and Analysis Poverty dimension Income poverty Illiteracy Gender Poverty of security Data disaggregation to be made Disaggregation of achievements by poverty index Disaggregation of achievements by literacy levels Disaggregation of achievements by gender index Regional analysis of data, to compare secure, with less secure regions Information on indicators will therefore be analyzed in the following lines - Overall national achievement - Disaggregation of achievement, by; o Region and district o Level of poverty o Literacy level o Gender o Level of security This shall primarily apply to coverage information for health services, risk factors, and other health determinants as such, the respective index shall also be disaggregated. The required levels of disaggregation may not be possible on an annual basis. As a proxy, therefore, the sector will use 2 Information on gender index by district is not yet available. District ranking therefore shall be done when this becomes available, through UNDP support. 18

33 district rankings for the different poverty dimensions to separate districts with high, and low attainment of the respective index. The indicator achievements for the top 10 districts (lowest poverty index) will be compared with the achievements of the bottom 10 districts (with the highest poverty index) to illustrate any differences. The districts in the top 10, and bottom 10 will be determined at the beginning of the HSSIP. Information on these indices therefore, will be provided annually to compile the district ranking for various poverty indices, which will be part of the Annual Health Sector Performance Report. Data analysis reports will be validated by key stakeholders to:- i) Obtain stakeholder insight on the information generated; ii) Mitigate bias through discussion of the information generated with key M&E strategy actors and beneficiaries; iii) Generate consensus on the data findings and gaps; and iv) Strengthen ownership and commitment to M&E activities. Particular attention will be paid to strengthen capacity for data analysis and synthesis within LGs, MoH Departments, semi-autonomous Institutions, PNFP & PFP facilities and CSOs. Responsibility, frequency and tools for the HSSIP performance data analysis and synthesis are detailed in Table 15. (Annex 7.6) Data Accuracy and Reliability All reports submitted to the Resource Centre and QAD will be reviewed for accuracy and clarification sought where necessary. Even where there is no need for clarification acknowledgement of receipt of reports will be provided before the due date for the subsequent report. Data quality assurance processes will include periodic Data Quality Audits (DQA) of recorded data by supervisors; regular training of staff, and provision of routine feedback to staff at all levels on completeness, reliability and validity of data; and data quality assessment and adjustment which will be carried out periodically. The objective of data validation is to ensure that the data used by the health sector to make decisions is sound and accurate. Specific efforts will be made to undertake data validity including: application of the computed validation/data accuracy index into district annual reports; specific support for outliers; routine (quarterly) data checks on a sample of districts. Regular data quality assurance for facility based data including regular review and verification for accuracy and completeness will be carried out monthly by the health facility in-charges at all levels. All periodic reports should be checked and endorsed before submission to the relevant stakeholders. Periodic data validation, training and provision of feedback on validity of health facility data will be carried out by the MoH, RC. DQA will be carried out at points of data collection, collation and analysis by the technical staff of the RC for districts and by the District Biostastician within districts. Standardized DQA tools will be developed for application at all levels. The Assistant Commissioner in charge of the RC will be responsible for the overall implementation of the activities on behalf of the sector. The RC will carry out regular training of staff, and provision of routine feedback on completeness, reliability and validity of data. DQA for sector evaluation studies shall be carried out using agreed formats by the MoH M&E unit which is the coordinating entity for the sector evaluation studies. Institutional Review Boards will have the responsibility of data validation for health systems research carried out in the respective institutions as guided by the regulations. 19

34 In addition to the above data checks and validation, the M&E unit under the QAD shall carry out annual Rapid Data Quality Assessment (RDQA) in which a selected number of health facilities will be drawn from the master facility for this assessment. RDQA shall be done together with the facility assessment, conducted 3 months before the JRM. The RDQA will be carried out as a quality assessment of the entire process of data collection, analysis and synthesis. A comprehensive Data Quality Assessment and Adjustment (DQAA) exercise will be carried out at midterm review and end term evaluation. The purpose of carrying out a DQAA is to identify and account for biases due to incomplete reporting, inaccuracies, non-representativeness. It will greatly enhance credibility of the results. DQAA will focus on: Assessment of the completeness of reporting by facilities and districts; Assessment of the accuracy of district population denominators (obtained from UBOS); Accuracy of coverage estimates from reported data; Systematic analysis of facility based and household survey based indicator values. This process could lead to adjustments of the indicator values, using transparent and welldocumented methods. To ensure transparency and overcome bias the MoH, shall engage independent institutions like Universities in the DQAA process. UBOS will be responsible for validating and designating as official health statistical reports generated by the MoH. Responsibility, frequency and tools for performance data quality assessment are outlined in Table 16. (Annex 7.6) Data Dissemination Data need to be translated into information that is relevant for decision-making. Data will be packaged and disseminated in formats that are determined by management at the various levels. Service delivery data shall be packaged and displayed at the various health facilities using the HMIS formats already provided. The timing of information dissemination should fit in the planning cycles and needs of the users. Annual Health Statistical Report This report will be compiled from the periodic statistical reports submitted through the District Health Information System (HMIS). The annual health statistical report provides ample attention to data quality issues, including timeliness, completeness and accuracy of reporting, as well as adjustments and their rationale. The RC will be responsible for compiling and disseminating this report. Detailed data should also be available on the web. Quarterly Performance Review Reports Quarterly sector performance review reports will be presented by the various sector technical working groups and central level institutions during the sector quarterly review meetings. Quarterly district performance (Quarterly District League Table) will also be disseminated and discussed at this forum. At sub-national level (RRHs, LG, projects), quarterly assessment reports will be presented and discussed at the quarterly review meetings attended by the key implementers. 20

35 Annual Health Sector Performance Report (AHSPR) The AHSPR is useful in highlighting areas of progress and challenges in the health sector. The reports will assess progress on the annual work plans and an overall assessment of sector performance against the targets set in the HSSIP 2010/ /15. It will also review progress against the sector priorities set during the preceding National Health Assembly and Joint Review Mission (JRM) with stakeholders. The different levels of health care delivery are expected to compile their reports by the end of July every year, and use them for performance review. The annual district performance reports are then forwarded to the national level for compilation of the AHSPR by the end of August every year. The AHSPR brings together all data from different sources, including the facility reporting system, household surveys, administrative data (minutes, supervision reports, financial reports, SCM reports, HRIS reports, etc) and research studies, to answer the key questions on progress and performance using the HSSIP core indicators and health goals. The AHSPR will present a detailed account of annual performance against the core and programmatic indicators of the sector strategic plan, comparing current results with results of previous years, and formulate challenges and recommendations by cluster and programme. The AHSPR will provide the background and indepth information for annual reviews and disease specific reports. It will be presented by MoH to health stakeholders and discussed at the JRM held October every year. The AHSPR will include an assessment of performance at and within the different levels using league tables. Similar to the analysis of progress in programmes, the report shall presents an analysis of the sector s support systems (health financing, human resources, health infrastructure, essential medicines and health supplies, diagnostic and blood transfusion services, information management & research) against set targets. The current format of the AHSPR shall be reviewed in order to present the sector performance issues in a format that can easily facilitate the knowledge management process. The AHSPR should also be able to reflect attribution of outputs and inputs to the public and private sectors. The aide memoire will summarize the findings and agreed actions of the JRM based on the review of the Pre- JRM filed visits and review of the AHSPR and will provide input in subsequent planning. The compilation of the AHSPR shall be coordinated by the Director Planning and Development in the MoH. The budget of collating annual sector performance data and report writing will be provided for under the sector monitoring budget. Annual sub-national performance reports shall be presented and discussed at the relevant annual stakeholders forum. Dissemination of Survey Findings Feedback on survey findings will be in form of workshops and dissemination of reports which will be circulated to relevant stakeholders in hard copy as well as on the MoH website, 21

36 2.3.7 Data Communication Data communication shall follow the existing MoH coordination structures. In order to ensure routine feedback on performance to sub-national and private providers, it is crucial that all service delivery and administrative structures adhere to the following data flow mechanism. Figure 4: Data Flow Diagram SMC HPAC TMC JRM DSS & Other studies QAD MOH/Resource Centre/Programmes CHD of Regional Referral UBOS and other Partners District Health Service Areas Health Sub-District Health Facilities Communities/VHTs H/H Surveys MIS DHS Household The MoH will use various communication channels in order to ensure public access to data and reports. Quantitative and qualitative data will be made publicly accessible through the MoH database under the RC. The Local Area Network (LAN) installed at the MoH will facilitate inter-departmental communication. accounts were created for all districts and hospitals and should be used for communication with the center, other districts and Continuous Professional Development (CPD) activities. The public will also be able to access health information on the MoH website, 22

37 In addition to the Information and Communication Technology (ICT) facilities at the MoH, institutions and districts, the M&E committee shall collaborate with the Health Education and Promotion Unit in the MoH to translate data and information according to the target audience and utilize various communication channels e.g. radio, T.V, video conferencing, tele-conferencing, newsletters, booklets, etc. Responsibility, frequency and tools for performance data dissemination and communication are outlined in Table 17. (Annex 7.6). 2.4 Responsibilities for Data Management by Level 1. Administrative Data Management At National Level The Secretary to a TWG, Supervisor or Programme M&E Officer is responsible for: Ensuring compilation and processing of administrative data into departmental / Institutional records(minutes, inventory) and reports (supervision, activity); Analyzing the quality of all reports received and ensuring follow-up in case of incompleteness, problems with validity, as well as delays; Compiling all reports from the Technical Officers into a single departmental / institutional report; Preparing an analysis of the data for discussion during the TWG / departmental / institutional / programme/ project implementation committee meetings and sector performance review meetings for decision-making; Forwarding the TWG / Departmental / Institutional / programme/ project report to the QAD. Providing quarterly feed-back to the TWGs / Departments / Institutions / programmes / projects. Disseminating quarterly TWG reports to Senior Management Committee and HPAC At District Level The DHO is responsible for: Ensuring compilation and processing of administrative data (minutes, inventory, supervision and other activity reports). Analyzing the quality of all reports received and ensuring follow-up in case of incompleteness, problems with validity, as well as delays. Compiling all reports from the DHT members into a single District Health Office report. preparing an analysis of the data for discussion during the DHT / DHMT meetings and district forum for decision-making. Forwarding the District Health Office administrative report to the CAO and QAD. Providing quarterly feed-back to the DHT members. 23

38 Disseminating quarterly administrative reports to DHMT, TPC and Health Committee. Dissemination annual administrative report to DHMT, TPC, Health Committee, District forum. At Health Facility Level The Facility In-Charge is responsible for: Ensuring compilation and processing of administrative data (minutes, inventory, supervision and other activity reports). Analyzing the quality of all reports received and ensuring follow-up in case of incompleteness, problems with validity, as well as delays. Compiling all reports from the Technical Officers into a single health facility report. Preparing an analysis of the data for discussion during the staff, HUMC / Board meetings for decision-making. Forwarding the health facility administrative report to the DHO. Providing quarterly feed-back to the health providers. Disseminating quarterly administrative reports to the HUMC or Hospital Board. Disseminating annual administrative report to the HUMC or Hospital Board and Sub County forum. Service Delivery Data Management At National Level The Biostatistician at the Resource Center is responsible for: Receiving all district data (including those from the national referral hospitals. Ensuring entry of all district data (including those from the national referral hospitals) onto the District Health Information System (DHIS) software package. Analyzing the quality of all reports received and ensuring follow-up in case of incompleteness, problems with validity, as well as delays. Compiling all reports from the districts & NRHs into a single national report using the DHIS software. Preparing an analysis of the data for discussion during the SMER TWG meetings and sector performance review meetings for decision-making. Providing quarterly feed-back to the districts / NRHs. Disseminating weekly IDSR reports to all stakeholders and community. Disseminating quarterly district performance reports at sector review meetings. 24

39 At District Level The District Biostatistician, or, where this position in not filled, the HMIS focal person, is responsible for: Receiving all health unit data (including those from the general and referral hospitals). Entering all health unit data (including those from the general and referral hospitals) onto the District Health Information System (DHIS) 2 software package. Analyzing the quality of all HMIS reports received and ensuring follow-up in case of incompleteness, problems with validity, as well as delays. Compiling all reports from the units into a single district report using the DHIS 2 software. Preparing an analysis of the data for discussion by the DHT for decision-making and participating in the DHT discussion. Forwarding the DHIS 2 report electronically to the RC. If not possible, deliver the physical report by the 28 th day of the following month. Providing quarterly feed-back on data management to the health units. Disseminating quarterly district assessment reports to DHMT. Disseminating annual district performance report to District stakeholders forum. The District Planner is responsible: Coordinating the collection, processing, analysis, storage and dissemination of health data/information to stakeholders under LOGICS. Monitoring and evaluation of the performance of the health sector development plans, programmes and projects. At Health Sub District Level The HSD Health Information Assistant is responsible for: Receiving all health unit data (including those from the private providers) in the HSD. Entering all health unit data (including those from the private providers) into the HSD databank. Analyzing the quality of all HMIS reports received and ensuring follow-up in case of incompleteness, problems with validity, as well as delays. Compiling all reports from the units into a single HSD report. Preparing an analysis of the data for discussion by the HSD Team for decision-making and participating in the discussion; Forwarding the HSD report to the DHO by the 7 th day of the following month; Providing quarterly feed-back on data management to the health units. Disseminating quarterly HSD assessment reports to HSD Team. Disseminating annual district performance report to HSD stakeholders forum. 25

40 At Health Facility Level All Health Providers including those from the private and community (VHT) are responsible for: Collecting patient data using relevant patient forms. Compiling relevant patient data from patient forms and entering it into the patient registers on a daily basis. The Health Information Assistant (HIA) or Medical Records Officer, or, where there is no HIA or MRO, the Health Unit In Charge or a designated person is responsible for: Regularly compiling relevant patient data from patient registers including those from the private providers and community (VHT) into the health facility HMIS database. Analyzing the quality of all patient registers and community reports received and ensuring follow-up in case of incompleteness, problems with validity, as well as delays. Compiling all reports from the sections/units / departments into a single health facility report using the health facility HMIS database. Plotting monthly performance on the displayed monitoring graphs. Preparing an analysis of the data for discussion within the health facility for decision-making methods and participating in the discussions. Forwarding or delivering the health facility report to the HSD/DHO by the 7 th day of the following month; In case of IDSR data weekly reports should be forwarded every Monday. Providing quarterly feed-back on data management to the sections / units/ departments and community (VHTs). Disseminating of monthly performance during monthly facility meetings. Dissemination of quarterly facility assessment reports to the HUMC / Hospital Board. Dissemination of annual facility performance reports to the Sub County forum. At Community Level All Community Health Providers including those from the private and community (VHT) are responsible for: Collecting patient/client, or activity data using relevant forms. Compiling data from the relevant forms and entering it into the Community Health / VHT Register on a regular basis. Compiling relevant data from the community health register into the VHT Report. Preparing an analysis of the data for discussion within the VHT for decision-making methods and participating in the discussions. Forwarding or delivering the VHT report to the nearest health center by the 5 th day of the following month; Disseminating quarterly performance report data to Parish Committee. 26

41 3 The HSSIP 2010/ /15 Monitoring and Review Process The framework for reviewing health progress and performance covers the M&E process from routine performance monitoring, quarterly reviews, annual review and evaluation of all the HSSIP indicator domains (Figure 5). Specific questions will have to be answered during the different review processes, especially the annual reviews, but also the performance monitoring. Figure 5: Framework for reviewing health progress and performance Health progress and performance assessment will bring together the different dimensions of quantitative and qualitative analyses and will include analyses on: (i) progress towards the HSSIP goals; (ii) equity (iii) efficiency; (iv) qualitative analyses of contextual changes; and (v) benchmarking. Progress towards HSSIP goals: The monitoring and review process will measure the extent to which the objectives and goals of the HSSIP (core indicators and their targets) have been attained. This will be complemented by a stepwise analysis to assess which policies and programmes were successful; from inputs such as finances and policies to service access and quality, utilization, coverage of interventions, and health outcomes, financial risk protection and responsiveness. Equity: The progress in terms of distribution of health system interventions will involve analyses of differences within and between population groups, among districts, etc. using a series of stratifiers and summary measures. Efficiency: This relates the level of attainment of goals to the inputs used to achieve them. Efficiency measures the extent to which the resources used by the health system achieve the goal that 27

42 people value. Efficiency analyses will be part of the end term evaluation of the HSSIP and health systems strengthening projects. Qualitative assessment and analysis of contextual change: This takes into account non-health system changes, such as socio-economic development that affect both implementation and the outcomes and impact observed. Qualitative information on the leadership, policy environment and context is crucial to understand how well and by whom government policies are translated into practice. A systematic and participatory approach will be used to gather, analyze and communicate qualitative information. This will be combined with quantitative data as basis for a policy dialogue. This will then become a solid basis to inform planning cycles, regular reviews and M&E. Such analysis shall include: - Primary health care reforms and policies stating the importance of assuring service coverage for all communities; - A critical assessment of the health sector or disease programme response and adhering to policy direction to date; - How cultural and political factors impact on health or multi-sectoral programmes and outcomes; - Assessment of organizational context, leadership and accountability mechanisms; - Assessment of the regulatory environment and how it enables or hinders improvements in health systems and programme delivery. Benchmarking: Benchmarking refers to comparisons between and within entities to assess performance. There are different types of benchmarking which may vary according to level of comparison (national or sub-national comparisons), level of assessment (individual service provider facility care organization district region national), measurement focus (process, outcomes, quality, performance), and use of data (public reporting, accountability, internal reporting only, selflearning and improvement). The MoH will utilize the following league tables for this purpose: District League Table to be compiled and disseminated quarterly and annually; HC IV Performance Assessment (Table 19, Annex 7.8) compiled annually as part of the AHSPR; General Hospital Performance Assessment Output (Table 20, Annex 7.8) compiled annually; Regional referral and large PNFP hospital performance assessment (Table 21) compiled annually; League tables for assessing MoH headquarters and departments at National and Regional Referral Hospitals shall be formulated during this HSSIP. 3.1 Country mechanism for review and action The HSSIP monitoring and review process will be interlinked across the different planning entities. Service delivery information to feed the monitoring and review process will be derived bottom up. This process will be based on a national platform approach that: uses the district as the unit of design and analysis; based on continuous monitoring of different levels of indicators; gathers additional data before, during, and after the period to be assessed by multiple methods; uses several analytical techniques to deal with various data gaps and biases; and includes interim and summative evaluation analyses. This implies that information at each level will be provided from the planning entities below 28

43 it. Management support, on the other hand, as well as governance/partnership information will be analyzed at the same level it is to be provided. Systems in place for monitoring and evaluation of the health sector include; monitoring the implementation of periodic work plans, Budget Framework Papers (BFP), Ministerial Policy Statements (MPS) and the sector strategic plan. Regular reporting using the HMIS and other sources of data will be used to assess progress against the agreed indicators and targets. The following reports will be generated; quarterly progress reports, annual performance reports, mid-term and end-term evaluation reports. Quarterly and annual reports will be produced by the different levels and used both for self assessment and by supervisors to determine progress or lack of it. The performance review process will be one of the learning mechanisms in the sector. For proper follow up and learning: i) All performance reviews and evaluations will contain specific, targeted and actionable recommendations. ii) All target institutions will provide a response to the recommendation(s) within a stipulated timeframe, and outlining a) agreement or disagreement with said recommendation(s), b) proposed action(s) to address said recommendation(s), c) timeframe for implementation of said recommendation(s). iii) All institutions will be required to maintain a Recommendation Implementation Tracking Plan which will keep track of review and evaluation recommendations, agreed follow-up actions, and status of these actions. iv) Institutions which have an oversight responsibility on the implementation of public policy will monitor the implementation of agreed actions utilizing the Recommendation Implementation Tracking Plan. These institutions are as follows: Local Councils, LGs, and MoH will monitor the implementation of Local Government actions, and report on these quarterly to MoFPED/OPM through the quarterly reviews and OBT. Sector Working Groups will monitor the implementation of MoH Departments and Institutional actions, and report on these biannually to MoFPED and OPM through the OBT. MoFPED and OPM will report six-monthly to Cabinet on progress on implementing these actions as part of biannual Government performance reports. Respective reviews will be guided by information developed by the Government management structures at each level. These will compile the review information with inputs from the other implementing partners. All reviews will be presented and endorsed at the respective management and partnership structures for the level. Indicator information will be used to inform different sector processes. These are illustrated in Table 4 below. The budget of collating performance data and conducting sector performance reviews will be provided for under the sector monitoring budget. 29

44 Table 4: HSSIP Monitoring and Review process Methodology Frequency Output Focus Level of monitoring and review Performance Assessment Quarterly Quarterly progress reports; transmitted to next higher level of supervision Done by Joint (public + private) Performance Assessment Teams and peers, and planning entity. Inputs, process, output and outcome Technical Review meeting Six months after the JRM Progress report submitted to next higher level of supervision A review of progress against targets and planned activities. Done by Joint (public + private) Performance Assessment Teams and peers, and planning entity. Inputs, process, output and outcome Joint Annual review and planning Annually Annual progress reports, transmitted to next higher level of supervision; A review of progress against targets and planned activities. Done Jointly with development Partner, key stakeholders, and planning entities as from sub district level onwards Input, process, output, and outcome levels National Health Assembly Mid Term Review End Term Evaluation Every 2 years After 2 1 / 2 years At end of HSSIP District and hospital performance league tables Progress report and resolutions for the next 2 years Review progress against set targets outcomes Done by sector. Review progress against resolutions Midterm Review report Done by sector Review progress against planned impact End Term Evaluation report Independent review of progress, against planned impact Input, process, output, and outcome levels Input, process, output, outcome and impact levels Input, output, outcome and impact levels Performance Monitoring and Review at Central Level The HPAC will review the following monitoring reports and recommend action to the Top Management or the Technical Working Groups as appropriate. Area Teams monitoring reports quarterly Annual health sector performance report annual Progress report from Planning Department on implementation of the JRM aide memoires quarterly Technical review meeting reports quarterly The HPAC will use indicators for monitoring the HSSIP for monitoring overall performance of the health sector. Central level institutions and departments will submit their periodic performance reports to the QAD before the 7 th day of the second month of the quarter after discussion with all main 30

45 stakeholders of the institution. They will be reviewed by the M&E committee which, will through the CHS (QA), request for clarifications as necessary. The Quarterly Sector Performance Review will assess progress on the quarterly work plans of MoH departments, semi-autonomous institutions, Health Professional Councils, National Referral Hospitals (NRHs, Mulago and Butabika Hospitals) and LGs. The aim of these reviews is to: Assess progress made on action points/recommendations of previous quarterly reviews. Assess implementation of planned activities against set targets. Highlight budget performance during the quarter (utilization against allocation). Propose strategies to address challenges in subsequent quarters. This shall be carried out using a standardized reporting format during the quarterly performance review workshops, where reports produced by each department/institution and NRHs will be presented and discussed in plenary sessions. The meetings will be attended by representatives of all reporting units. The departmental and institutional reports are to be compiled into quarterly sector performance reports which will feed into the AHSPR. The QAD will be responsible for organizing the quarterly review meetings, compiling and disseminating the quarterly reports. Technical review meetings shall be held six months after the JRMs to review progress against the JRM aide memoire. The technical review meetings shall be organized by the Planning Department. Technical reviews and JRMs shall be organized by the Planning Directorate. National Health Assembly (NHA) The NHA shall constitute members of the Social Services Committee of Parliament, LC V Chairpersons, Secretaries for Health, Chief Administrative Officers, officials from the central level and a few selected DHOs. The objectives of the NHA are; to garner support for mobilizing resources for health interventions at the Local government level; and to enhance support and participation, by stakeholders for health interventions at the implementation level. NHA shall be carried out every two years in conjunction with the JRM Performance Monitoring and Review at Regional Level It is envisaged that with the creation of the proposed regional tier in health services supervision and monitoring, the Community Health Departments shall be responsible for conducting quarterly performance review workshops, where reports produced by each district in the catchment area will be presented and discussed in plenary sessions. The meetings will be attended by representatives of all reporting districts and development partners. Regional stakeholders fora Once agreed and adopted the regional level stakeholders will come together to discuss health and health related issues that affect their region at least once a year based on the district performance 31

46 reports. The fora will use regional data to discuss performance within the region, and agree on priorities to guide districts and RRHs in their respective planning and implementation processes. These stakeholders will include; - Public and non public health providers within the region - Development Partners within the region - CSO groups within the region - Members of Parliament / Local Councils, and other eminent persons - Community, and / or cultural leaders within the region - Representatives of health related sectors Standardized planning and reporting formats, tools and process shall be provided to all the regional coordinators, to guide them in their stakeholders meeting Performance Monitoring and Review in Local Governments/District Level Performance monitoring and review in LGs will be carried out using standard planning and reporting formats. The HMIS remains the routine reporting system for health service delivery in LGs. The District Biostastician shall be responsible for validation and compilation of health facility administrative and service delivery reports into district reports. All weekly, monthly, quarterly and annual reports from HSDs will be compiled into a single district report. The DHO shall be responsible for verification and analysis of the district reports. These reports shall be used for district performance review, planning, supervision and resource mobilization. The District Biostastician is responsible for submission of district reports to the MoH/ Resource Centre. The timeframe for the DHO HMIS reporting is outlined in Table 5 below. Each report should be received at the MoH by the date due. During the monthly DHT meetings performance review should focus on progress of implementation of the workplans, timeliness, completeness and accuracy of the reports. The district quarterly assessment reports will be used for performance review during the quarterly DHMT meetings. The annual district performance report will be presented and discussed during the annual district stakeholders forum. Local Governments (Districts / Municipalities) / Technical Planning Committee: LGs shall be responsible for monitoring frontline service delivery and accountability for results. They shall also be responsible for reporting on progress of implementation and achievement of planned outputs. This involves reporting on the process of implementation focusing on implementation bottlenecks/constraints. Stakeholders in the districts shall come together to discuss health and health related issues affecting their district quarterly. District stakeholders fora These stakeholders include; - Political, administrative and technical leadership within the districts - Public and non-public health providers operating within the district - Development Partners/implementing partners supporting, or facilitating activities within the district - Critical civil society groups operating within the district - Representatives of health related sectors 32

47 The fora shall use data collated at district level to discuss performance within the district, and agree on priorities to guide districts in their respective planning and implementation processes. The District stakeholder s fora shall be coordinated by the DHO. Standardized planning and reporting formats, tools and process shall be provided to all the districts, to guide them in their stakeholders meeting. Table 5: Timeframe for the DHO HMIS reporting Timeframe Report Date Due District Report sent annually District Epidemiological Summary HMIS 033c District Population Report HMIS 122 District Physical Inventory HMIS 200 District Equipment Inventory HMIS 201 District Staff Listing HMIS 203 District Profile (Annual Report) HMIS 128 District Workplan HMIS th August District Report sent weekly District weekly surveillance report (HMIS Every Tuesday the District Report sent monthly HMIS c) Month following Date due week District Monthly Inpatient Report HMIS 124 July August September October November December January February March April May June Month July August September October November December January February March April May June 28 th August 28 th September 28 th October 28 th November 28 th December 28 th January 28 th February 28 th March 28 th April 28 th May 28 th June 28 th July Date due 28 th August 28 th September 28 th October 28 th November 28 th December 28 th January 28 th February 28 th March 28 th April 28 th May 28 th June 28 th July Quarter 1 st Quarter (July-Sep) 2nd Quarter (Oct- 3 rd Quarter (Jan- 4 th Quarter (Apr-June) Date Due 28 th October Dec) 28 th January Mar) 28 th April 28 th July 33

48 3.1.4 Performance Monitoring and Review at Health Sub-District Level Like the district, each HSD shall review their HSD performance based on the HMIS as the routine reporting system. The HSD Health Information Assistant shall be responsible for compilation of the HSDs administrative and service delivery reports. All weekly, monthly, quarterly and annual reports from health facilities will be compiled into a single HSD report. The HSD In-charge shall be responsible for verification and analysis of the HSD reports. The reports shall be used for HSD performance review, planning, performance improvement and resource mobilization. The HSD Health Information Assistant is responsible for submission of the HSD reports to the DHO. The timeframe for the HSD HMIS reporting is outlined in Table 6 below. Each report should be received at the DHO by the due date. During the monthly HSD Team meetings performance review should focus on timeliness, completeness and accuracy of the reports. The HSD quarterly assessment reports will be used for performance review during the quarterly HSD Team meetings, whereas the annual district performance report will be presented and discussed during the annual HSD stakeholders forum. Health Sub District stakeholders fora Stakeholders in the HSDs shall come together to discuss health and health related issues affecting their HSD quarterly. These stakeholders include; - Political, administrative and technical leadership within the HSD - Public and non public health providers operating within the HSD - Development Partners/implementing partners supporting, or facilitating activities within the HSD - Critical civil society groups operating within the HSD - Representatives of health related sectors in the HSD The fora shall use HSD data to discuss performance within the HSD, and agree on priorities to guide facilities and other service providers in their respective planning and implementation processes. The HSD stakeholder s fora shall be coordinated through the facility heading the HSD. Standardized planning and reporting formats, tools and process shall be provided to all the HSDs, to guide them in their stakeholders meeting. Table 6: Timeframe for HSD HMIS Reporting Timeframe Report Date Due HSD Report sent annually HSD Population Report HMIS 109 HSD Physical Inventory HMIS 101 HSD Equipment Inventory HMIS 102 HSD Staff Listing HMIS 103 HSD Profile (Annual Report) th August HSD Report sent weekly HSD weekly surveillance report (HMIS 033c) Every Monday the following week HSD Report sent monthly HMIS 123 Month Date due July August September October 7 th August 7 th September 7 th October 7 th November 34

49 November 7 th December December 7 th January January 7 th February February 7 th March March 7 th April April 7 th May May 7 th June June 7 th July Month Date due July 7 th August August 7 th September September 7 th October October 7 th November November 7 th December HSD Monthly Inpatient Report HMIS 124 December 7 th January January 7 th February February 7 th March March 7 th April April 7 th May May 7 th June June 7 th July HSD Report sent quarterly HSD Quarterly assessment Report HMIS Quarter 1 st Quarter (July nd Quarter (Oct- 3 rd Quarter (Jan- 4 th Quarter (Apr-June) Date Due Sep) 14th October Dec) 14th January Mar) 14th April 14th July Performance Monitoring and Review at Health Facility Level Performance monitoring and review in health facilities (including private health facilities and community (VHT)) will be carried out using standard planning and reporting formats. The Health Information Assistant, or where not available a designated person shall be responsible for compilation of all relevant data from patient registers and compile a health facility report. Health facility In Charges are responsible for verification and analysis of administrative and service delivery reports. The generated reports shall be used for health facility performance review and improvement, planning, and resource mobilization. The Health Information Assistant or designated person is responsible for submission of the health facility reports to the HSD and DHO. Each report should be received at the HSD office by the date due. (Table 7) During the monthly health facility meetings performance review should focus on timeliness, completeness and accuracy of the reports. The health facility quarterly assessment reports will be used for performance review during the quarterly HUMC meetings, whereas the annual district performance report will be presented and discussed during the annual Sub County stakeholders forum. 35

50 Table 7: Timeframe for Health Facility Reporting Timeframe Report Date Due Health Unit Report sent annually HU Physical Inventory (HMIS 101) HU Equipment Inventory (HMIS 102) HU staff Listing (HMIS 103) HU Staffing Summary HU HSSIP Indicators HU Annual Report (HMIS 107) 7 th August 7 th August 7 th August 7 th August 7 th August 7 th August Health Unit Report sent weekly HU Weekly Epidemiological Surveillance Every Monday of the Health Unit Report sent monthly Report Monthly (HMIS Report 033c) HMIS 105 following Date due week July August September October November December January February March April May June 7 th August 7 th September 7 th October 7 th November 7 th December 7 th January 7 th February 7 th March 7 th April 7 th May 7 th June 7 th July Health Unit Inpatient Report sent monthly Monthly Report HMIS 108 Date due July August September October November December January February March April May June 7 th August 7 th September 7 th October 7 th November 7 th December 7 th January 7 th February 7 th March 7 th April 7 th May 7 th June 7 th July Health Unit Report sent quarterly Health Unit Quarterly assessment Quarter 1 st Quarter (July-Sep) Report 2 nd Quarter HMIS 106 (Oct- 3 rd Quarter (Jan- 4 th Quarter (Apr-June) Date Due 14th October 14th January Mar) 14th April 14th July Health facility stakeholders fora Each health facility in the country has a defined catchment area, for which it is responsible for coordinating delivery of services to implement the HSSIP. All stakeholders in the catchment area of the facility shall come together to discuss health and health related issues affecting them quarterly each year. 36

51 These stakeholders include; - Other CSO s operating within the facility catchment area - Development Partners/implementing partners supporting, or facilitating activities within the catchment area. - Representatives of health related sectors in the catchment area of the facility - Community and / or political leaders from the catchment area of the facility. The fora shall use health unit data to discuss performance of health within the catchment area, and agree on priorities to guide the facility and other service providers in their respective planning and implementation processes. The stakeholders meetings shall be coordinated by the head of the Health Facility. Standardized planning and reporting formats, tools and process shall be provided to all the health facilities, to guide them in their stakeholders meeting Performance Monitoring and Review at Community Level Performance monitoring and review in the community will be carried out using standard planning and reporting formats. Sub County (SC) stakeholders fora Stakeholders in the Sub counties shall come together to discuss health and health related issues affecting their SC quarterly. These stakeholders include; - Political, administrative and technical leadership within the SC - Public and non public health providers operating within the SC - Development Partners/implementing partners supporting, or facilitating activities within the SC - Critical civil society groups operating within the SC - Representatives of health related sectors in the SC The fora shall use SC data to discuss performance within the SC, and agree on priorities to guide facilities and other service providers in their respective planning and implementation processes. The SC stakeholder s fora shall be coordinated through the facility heading the SC (HC III). Standardized planning and reporting formats, tools and process shall be provided to all the SCs, to guide them in their stakeholders meeting. In addition, the Subcounties are expected to use the M&E results to sensitize the community and accountability through the Sub-county barazas recently established under OPM Performance Monitoring and Review at VHT Level Each VHT is required to compile monthly HMIS reports, for use as well as submission to the nearest health center by the 5 th day of the following month. All relevant data from community registers will be compiled into a VHT report. Performance review and feedback will be carried out quarterly by the Village Health Committee. 37

52 3.1.8 The National Health Accounts The National Health Accounts will be institutionalized as a process of generating routine and standardized health expenditure data to inform HSSIP implementation and policy decisions. Resources flowing in the health system from all sources managed by all agents and used to provide services whose primary intention is health will be tracked. Among the objectives to be achieved is, benchmarking performance against established targets; allocating scarce resources according to needs; improve accountability and efficiency; planning for future and raising additional funds (based on gaps and needs) and; ensuring sustainability. A comprehensive NHA with sub accounts (TB, Malaria, HIV, RH, CH), as deemed necessary by stakeholders will be produced every two years. All actors in the sector (service delivery levels, MDAs, central level institutions and MoH, Districts, private providers, PNFPS, CSOs and Development partners) will be obliged to compile, analyze, utilize health expenditure data and report to appropriate levels. The budget and finance division in the MoH, under the leadership of Commissioner Planning, will be responsible for coordinating the whole process Joint Review Mission The JRM is a national forum for reviewing sector performance annually. The annual reviews will focus on assessing performance during the previous fiscal year, and determining actions and spending plans for the year ahead (current year+1). These actions and spending should be addressed in amendments to the Sector Investment Plan operations, and the budget framework papers and Ministerial Policy Statements of the Ministry. Annual Sector Reviews should be completed by the 30 th September each year, to ensure that the findings feed into the planning and budget process of the coming year, and annual reporting to Cabinet and Parliament. (National Policy on Public Sector Monitoring and Evaluation, October 2010). The main purpose is to; (a) Receive and discuss the previous financial year sector annual performance report against priorities. (b) Raise and respond to issues arising from the AHSPR. (c) Identify and agree on sector priorities for following financial year. The annual review shall be organized by the MoH in collaboration with Health Development Partners. The participants include; key Policy makers (Parliamentary Social Services Committee), Health Service Commission, key MoH staff (SMC) and departments, Professional Councils, representatives of local governments, other Health Related Ministries (e.g. Ministry of Finance, Planning and Economic Development, Ministry of Local Government, Ministry of Education and Sports, Ministry of Lands, Water and Environment, Ministry of Agriculture, Animal Industries and Fisheries, Ministry of Gender, Labour and Social Development, Ministry of Public Service), UN agencies, DPs, non-state implementing partners (PNFPs, CSOs, private providers), representatives of Health Consumers Organizations and CSO advocacy organizations. The proceedings of the JRM will be documented in the Aide Memoire to be signed by the MoH and DPs. The budget for conducting annual sector reviews will be provided for under the sector monitoring budget. 38

53 3.2 Performance Monitoring by the MoFPED and OPM In order to meet obligations in monitoring the NDP, the MDAs are required to report quarterly, semi-annually and annually, to the MoFPED and OPM on a limited number of indicators that reflect performance in terms of service delivery outcomes (health service coverage), contributory outputs and main actions. The Output Budgeting Tool (OBT) will be the single instrument for monitoring and reporting to MoFPED and OPM. The OBT enables reporting on the sector outcomes, performance indicators and is linked down to Vote function outputs and related indicators. The activities and outputs are presented by vote and vote function and selected from the annual work plan, and will be reviewed each financial year by the SMER TWG during the development of the BFP. Of specific importance to this M&E plan is the need to ensure that, output targets and corresponding indicators, are coherent with those in the HSSIP Performance Monitoring by the MoFPED and OPM at Central Level Performance reporting is directly related to the Performance Contracts which Accounting Officers at central and local government level now sign (with the PS/TS and PS/MoLG) agreeing to deliver the outputs approved by Parliament or local council. The OBT has been developed linking the procurement and requisitions action to budget and work Plan. Detailed forms are available in MS ACCESS format for both quarterly and cumulative reporting. Votes must prepare and submit a draft performance report for each quarter within one month after the close of that quarter. The submission will be reviewed, and revisions requested from MoFPED (and line ministries for LGs) which will be provided by the end of the second month of the quarter following the reporting period. (MoFPED, 2010) i. Votes will then revise their reports accordingly before the 15 th day of the last month of the quarter following the reporting period. ii. The release for any given quarter will be conditional on a final and acceptable report being submitted for the previous quarter but one. In the health sector, the vote holders are: the MoH, The Uganda Cancer Institute, the Uganda Heart Institute, the NRHs, the RRHs, the Uganda Blood Transfusion Services, National Medical Stores (NMS), the Health Service Commission and each local government. Vote Holders are responsible for ensuring that outputs and actions set out in the Performance Contract are achieved and that reporting requirements are fulfilled in an accurate and timely manner. The Semi-Annual Report aims at assisting Cabinet and Parliament in ensuring that agreed targets and actions are met by the end of the Financial Year. The Government Annual Performance Report also represents one part of the dual annual review of progress in implementing the NDP which has been agreed with the Joint Budget Support Framework (JBSF) Development Partner. The Government Annual Performance Report will be shared with the JBSF DPs, and includes the JAF indicators. The Annual JAF appraisal by the JBSF DPs in principle will constitute a response to the Annual Performance Review and will be used to validate the findings of Government. Both reports are extremely important, as they are the basis for discussions between the Policy Coordination Committee and the Heads of Mission of the JBSF DPs on continued sector and general budget support. A mechanism shall be developed to ensure that Financial Performance Reports from the districts are reviewed in a rational manner at the MoH and that feed-back is provided timely. 39

54 Timeframes for Submission of Central Government Quarterly Performance Reports In order to give Votes sufficient time to produce and compile credible performance data, a lag of one quarter has been incorporated into the cycle. For example, the cash release for Q2 of FY 2010/11 will be conditional on the submission and approval of the Performance Report from Q4 of FY 2009/10. Therefore, instead of Votes rushing to compile accurate performance data by 15 th October (Q2) following the close of the Q1, there will be a lag of one quarter to Q3 when they will be required to submit Q1 s performance report to guarantee Q3 s release. Table 9 below shows the timeframe for central government reporting and release throughout the financial year. Table 8: Timeframes for Central Government Quarterly Performance Report Submission Quarter Date Release Conditions for Release Q4 (Previous FY) 30 th June Q1 Vote Cash Limits from N/A MoFPED Q1 30 th September Q2 Vote Cash Limits from Q1 Release conditional on approved MoFPED Q3 (previous FY) performance report Q2 31 st December Q3 Vote Cash Limits from MoFPED Q3 31 st March Q4 Vote Cash Limits from MoFPED Q2 Release conditional approved Q4 (previous FY) performance report Q3 Release conditional approved Q1 performance report ** NOTE! Reporting is a continuous process! Votes should begin the reporting process immediately when the quarter closes, not wait until the following quarter to begin reporting for submission** Reporting in Q4: Quarter 4 marks the close of the financial year. Annual performance is compiled in this quarter, alongside the preparation of detailed budget documentation and quarterly work plans for the next financial year. However, the table above illustrates that the requirements for quarterly performance reporting is the same in Q4 as it is in all of the other quarters of the year. The following will be prepared and submitted to MoFPED. Date Report 15 th June Submission of MPS and Performance Contract Form A with provisional yearend reporting and annual Cash Plan 31 st July Submission of Q4 performance report including actual Q4 outputs and cumulative performance to year-end 31 st July Submission of revised Q1 work plan ** NOTE! The preparation of the Ministerial Policy Statement for submission in June does not supersede the requirement for a Q4 performance report submission. This is still required to trigger Q2 s release in the next financial year** 40

55 3.2.2 Performance Monitoring by the MoFPED at LG Level In addition to the HMIS reports LGs are required to submit quarterly performance reports (OBT) to MoFPED and copy to MoH. The MoFPED guidelines for budget preparation and reporting using the OBT (MoFPED, November 2010) provide details in key operations and functions of various offices in the preparation of the OBT for planning and reporting at central and LG level. Key performance indicators (Table 10) are determined annually based on the core vote functions and key outputs during the development of the BFP. Also, mechanisms have to be developed to ensure that performance reports from districts are reviewed in a rational manner at the MoH and that feedback is provided timely if required. Table 9: Local Government OBT indicators Output Name Indicator Name Indicator Measure Indicator Type Method of Measurement Source of Data Frequency of Collection Medical Supplies for Health Facilities Medical Supplies for Health Facilities Medical Supplies for Health Facilities Medical Supplies for Health Facilities District/General Hospital Services (LLS) District/General Hospital Services (LLS) District/General Hospital Services (LLS) District/General Hospital Services (LLS) CSO Hospital Services (LLS) CSO Hospital Services (LLS) CSO Hospital Services (LLS) CSO Basic Healthcare Services (LLS) Value and Number of drug orders processed against procurement plans Value and Number of drug deliveries for medicines and supplies by NMS as per the published schedule Number of deliveries for essential medicines and supplies to health facilities by the District. Number of health facilities reporting no stock out of the 6 tracer drugs. No. of qualified staff and % of approved posts filled with qualified health workers in the District/General Hospital(s). Number of inpatients that visited the District/General Hospital(s). No. and proportion of deliveries in the District/General Hospital(s). Number of total outpatients that visited the District/General Hospital(s). Number of inpatients that visited the CSO Hospital Facilities. No. and proportion of deliveries conducted in CSO Hospital Facilities. Number of outpatients that visited the CSO Hospital Facilities. Number of outpatients that visited the CSO Health Facilities. Number Quantity Count Records Quarterly Number Quantity Count Records Quarterly Number Quantity Count Records Quarterly Number Quantity Count Records Quarterly Number and Percentage Quantity Count Records Quarterly Number Quantity Count Records Monthly Number and percentage Quantity Count Records Monthly Number Quantity Count Records Monthly Number Quantity Count Records Monthly Number and percentage Quantity Count Records Monthly Number Quantity Count Records Monthly Number Quantity Count Records Monthly 41

56 Output Name Indicator Name Indicator Measure Indicator Type Method of Measurement Source of Data Frequency of Collection CSO Basic Healthcare Services (LLS) CSO Basic Healthcare Services (LLS) CSO Basic Healthcare Services (LLS) Basic Healthcare Services (HC IV-HC II-LLS) Basic Healthcare Services (HC IV-HC II-LLS) Basic Healthcare Services (HC IV-HC II-LLS) Basic Healthcare Services (HC IV-HC II-LLS) Basic Healthcare Services (HC IV-HC II-LLS) Basic Healthcare Services (HC IV-HC II-LLS) Buildings & Other Structures Basic Healthcare Services (HC IV-HC II-LLS) Basic Healthcare Services (HCIV-HCII- LLS) Basic Healthcare Services (HC IV-HC II-LLS) Basic Healthcare Services (HC IV-HC II-LLS) Basic Healthcare Services (HCIV-HCII- LLS) Basic Healthcare Services (HC IV-HC II-LLS) Basic Healthcare Services (HC IV-HC II-LLS) Basic Healthcare Services (HCIV-HCII- LLS) Basic Healthcare Services (HC IV-HC Number of inpatients that visited the CSO Health Facilities. No. and proportion of deliveries conducted in the CSO Health Facilities. Number of children immunized with Pentavalent vaccine in the CSO Health Facilities. No. of qualified health workers and % of approved posts filled with qualified health workers Number of outpatients that visited the Govt. health facilities. Number of inpatients that visited the Govt. health facilities. No. and proportion of deliveries conducted in Govt. health facilities. Number of children immunized with Pentavalent vaccine. % of Villages with functional (existing, trained, and reporting quarterly) VHTs. Number of planned health infrastructure projects completed. Number of outreach visits conducted Number of support supervision visits conducted Number of health workers trained Number of health related training sessions held Number of HCT activities conducted Number of management meetings held(humc,dht,dhmt) Number of contacts related to health promotion and health education Number of EPI child days conducted Number of HMIS reports submitted in time Number Quantity Count Records Monthly Number and percentage Quantity Count Records Monthly Number Quantity Count Records Monthly Number and Percentage Quality Count Records Quarterly Number Quantity Count Records Monthly Number Quantity Count Records Monthly Number and percentage Quantity Count Records Monthly Number Quantity Count Records Monthly Percentage Quantity Count Records Quarterly Number Quantity Counts Records Quarterly Number Quantity Counts Records Quarterly Number Quantity Counts Records Quarterly Number Quantity Counts Records Quarterly Number Quantity Counts Records Quarterly Number Quantity Counts Records Quarterly Number Quantity Counts Records Quarterly Number Quantity Counts Records Quarterly Number Quantity Counts Records Quarterly Number Quantity Counts Records Quarterly 42

57 Output Name Indicator Name Indicator Measure Indicator Type Method of Measurement Source of Data Frequency of Collection II-LLS) Machinery & Equipment Value of basic medical equipment procured during the FY Number Quantity Funds spent Records Quarterly Timeframes for Submission of Local Government Quarterly Performance Reports As with the Central Government reporting timeframe described above, the LG Performance Reports are similarly submitted within one month of the close of a quarter, and report approval is required to trigger the release of funds for the next quarter (Table 11). The Quarterly Performance Reports are also used by a number of different audiences to monitor and analyze performance of the budget and institutions at the LG level. Table 10: Timeframes for Local Government Quarterly Performance Report Submission Quarter Date Release / Submission Q1 31 st July Draft Q4 Performance Report (previous FY) and draft Budget Performance Contract 1 st to 31 st August Review of Q4 (Previous FY) Performance Reports 31 st August MoFPED Requests Amendments from Votes 15 th September Final Q4 Performance Report Q2 31 st October Draft Q1 Performance Report and final Budget Performance Contract Form B 1 st to 30 th November Review of Q1 Performance Reports 30 th November MoFPED Requests Amendments from Votes 15 th December Final Q1 Performance Report Q3 31 st January Draft Q2 Performance Report 1 st to 22th February Review of Q2 Performance Reports 28 th February MoFPED Requests Amendments from Votes 15 th March Final Q2 Performance Report Q4 30 th April Draft Q3 Performance Report and Q4 work plan 1 st to 31 st May Review of Q3 Performance Reports 31 st May MoFPED Requests Amendments from Votes 15 th June Final Q3 Performance Report 43

58 3.2.3 Monitoring Performance of the Supply Chain Management System The SCMS for medicines and health supplies is critical in ensuring availability of supplies. There are various stakeholders involved and therefore a Memorandum of Understanding was signed between GOU represented by the MoH, MoLG and NMS. Performance monitoring will be carried out by the MoH Pharmacy Division on a quarterly basis. Table 11: Medicines Management Performance Summary Matrix Medicines Management Quarter Performance Year Performance Summary Matrix Quarter (HC II IV & General Hospitals) Resp. Key Performance Indicator MoFPED 1. Not less than 100% of approved medicines budget released to NMS before the end of a FY 2. Timely disbursement of funds under Vote 116 in two batches per FY (Front loading) Threshold Pass Fail Sanction 99% Under performance to be communicated to Parliament and Health Development Partners 95% of budget target amount by quarter. L/Govt 3. Ensure that Medical Directors (MDs), Medical Superintendants (MS s) and DHOs submit timely and complete orders to NMS. Ensure 75% orders reach NMS not later than 7 working days before scheduled district delivery date. For districts and hospitals failing to submit timely / within quarter target allocation orders, the CAO and MD should be summoned to explain. 4. Ensure medicines delivered by NMS to District HQ is delivered by DHO to lowest HF within 72 hours. Ensure 75% is delivered by DHO to lowest HF within 72 hours 5. Ensure approved list of public health facilities is submitted to NMS by 30 th May each year. Ensure 100% districts have approved health facility inventory 44

59 MoH 6. To maintain standard list of essential medicines to be stocked at the various levels in line with EMLU Communicate Policy changes on treatment at least 12 months before implementation 8. Communicate all Policy changes on ordering and delivery of medicines / health supplies 9. Ensure that range, quantity and delivery schedules are agreed upon between NMS and MoH for all third party items. List available There should be sanctions for the MoH 12 months notice given before order of any new item Communication copies of all communications 100% 10. Ensure sharing of pipeline information on 3 rd party items at least 3 months in advance NMS 11. NMS adherence to agreed published delivery schedule. 12. Deliver agreed upon indicator range and quantity of essential medicines and health supplies in line with MoH Policies. % districts deliveries on scheduled date. 85% range 85% quantity Sanction the General Manager in line with Public Service regulations. 13. Ensure utilization of GOU funds made available to NMS is spent / committed by FY end 14. NMS adherence to agreed prices by 4 month cycle 100% Subsequent allocation to be in line with absorption capacity 100% district accounts debited with correct prices 45

60 3.2.4 Monitoring by the Health Development Partners The Joint Assessment Framework (JAF) is the approach agreed by the Government of Uganda and the DPs engaged in budget support (World Bank, Delegation of the European Union, African Development Bank, and Governments of Belgium, Denmark, Germany, Ireland, Netherlands, Norway, Sweden, and the United Kingdom). It is a framework that tracks overall government performance as well as performance in five sectors: health; education; water and sanitation; transportation and roads; agriculture. Based on the annual assessments, the donor governments make decisions on sector budget support to Uganda. Sector performance is assessed using two sets of indicators. Health sector-specific performance indicators (JAF Indicators) are at impact, outcome and output levels. The MoH reviews and sets annual targets and actions. All the 8 JAF indicators at the beginning of HSSIP 2010/ /15 are among the 26 HSSIP 2010/1 2014/15 core indicators and include: Impact level indicators 1) Infant Mortality Rate 2) Maternal Mortality Ratio Sector result level indicators 3) Number and proportion of deliveries in health facilities 4) Contraceptive Prevalence Rate 5) Number and proportion of children immunized with 3 rd dose pentavalent vaccine Performance level indicators 6) Proportion of approved posts filled by qualified health workers 7) Absenteeism rate 8) Proportion of health facilities without drug stock outs for 6 tracer drugs The JAF Quarterly Performance Report is compiled by the QAD, in conjunction with officers from various departments (both the Resource Centre and user departments) who have the required information and submitted to the MoFPED. 3.3 Linkage between the health sector reviews, disease / programme / project specific reviews and global reporting Detailed program-specific reviews shall be linked to the overall health sector review and contribute to it. Program-specific review should be conducted prior to the overall health sector review, and help inform the content of the health sector review in relation to that specific programme area. It is important that the specific programme reviews involve staff and researchers not involved in the programme itself to obtain an objective view of progress. All disease / programme / project reviews shall be carried by the specific M&E Officers and managers and progress review reports submitted to the MoH M&E unit in the Quality Assurance Department before the quarterly and annual sector reviews. 46

61 Global reporting requirements shall be based on ongoing country processes of data generation, compilation, analysis and synthesis, communication and use for decision making. This has been clearly spelt out in the Country Compact for implementation of the HSSIP. The HSSIP M&E plan serves as the basis for all M&E related processes such as the health sector component of the Sector Wide Approaches (SWAP) and IHP+, for programme activities supported by GAVI, the Global Fund and other DPs, and for disease- and programme-specific needs. The figure below shows the common M&E platform for national health strategies with country data generation and use processes in the centre. Figure 6: Country-led platform for monitoring & review of the national health strategy Source: WHO Performance Monitoring and Review for Global Health Grants The HSSIP 2010/ /15 has both core and programme specific indicators. Whereas the main purpose of this M&E plan is to provide a framework for monitoring the HSSIP core indicators, program managers and other stakeholders are encouraged to develop and implement program/project specific M&E plans for monitoring their indicators and performance reviews with linkage to the general health sector review outlined in this plan. Under the Global Health Initiatives, the health sector is supported through initiatives like the Global Fund for Tuberculosis, HIV/AIDS and Malaria (GFTAM) and Global Alliance for Vaccines and Immunization (GAVI) which provide funds based on performance. There are other sector support projects which also disburse funds based on performance and CSOs which contribute to the overall sector performance. All these require M&E Plans at the time of Grant/Loan signature and ongoing disbursements are linked to the achievement of clear and measurable programmatic results. The Long Term Institutional Arrangements (LTIA) for management of Global Health Grants in Uganda, (Abridged Version, 2009) outlines the processes and mechanisms for aligning the Global Health Grants with the overall Government of Uganda s national development framework and NDP. For programme monitoring and evaluation the LTIAs spell out that; 47

62 M&E shall be carried out within the National M&E framework using tools that consider outputs and indicators to be drawn from approved workplans and budgets for the HSSIP. Programme specific indicators from programme strategic and or M&E plans will be used to supplement the national level indicators in monitoring specific programme performance. There shall be strengthening of the M&E systems to include an agreed upon and costed M&E plan developed by the TWGs and coordinated by the Directorate of Planning and Development. The QAD shall be the secretariat for the SMER TWG and; will be responsible for implementation of the M&E plan. The Policy Analysis Unit shall be supported to undertake the role of evaluation. A mechanism shall be developed to ensure follow up action as needed from reviews and reports that indicate discrepancies or short comings with observed program results. This mechanism shall involve spelling clear timelines, responsibility centers and indicators. There shall be a transparent and documented process to ensure input of a broad range of stakeholders in the programme monitoring and evaluation. This shall involve joint monitoring visits. Programme managers will provide oversight for monitoring implementation of work plans and preparation of quarterly and annual performance reports. The M&E specialists will work under the overall stewardship of the SMER TWG. They will be responsible for analyzing data and assembling reports that will be reviewed and verified by programme managers before submission to the working group. They will use information generated by the Resource Center and monitoring reports form Area Team supervision visits. Specifically for the GFTAM; The existing framework for performance M&E will be adopted using agreed performance indicators. Implementers at national and local government levels will prepare technical reports on a quarterly basis. Annual performance reports presented at the JRM and Joint AIDS Review shall capture overall performance (including GFTAM) and constitute the performance report for the sector for use by Development Partners and other stakeholders. 3.5 Performance Monitoring and Review for Civil Society Organisation and Private Sector CSOs and the private sector contribute significantly to health service delivery in the country. Most times their input is not captured in the sector performance reports. In order to measure their contribution to the overall sector performance they will be required to report to the relevant sector entities using the existing monitoring and review structures. The CSO and Private Providers umbrella bodies will play a coordination role in monitoring all CSOs and private sector providers to ensure alignment with national priorities. Government will support this process through providing the necessary M&E tools. 48

63 3.6 Performance Monitoring and Review for Implementing Development Partners Sector Implementing developing partners contribute significantly to health service delivery in the country. Most times their input and attribution to health outcomes is not captured in the sector performance reports. In order to measure their contribution to the overall sector performance they will be required to report to the relevant sector entities using the existing monitoring and review structures. The MoH M&E Unit will play a coordination role in monitoring all national level Implementing DPs to ensure alignment with national priorities. DHO will coordinate monitoring and reporting of district based Implementing DPs. Statistical outputs from these partners should be routed through the HMIS. Government will support this process through providing the necessary M&E tools and reporting format. 3.7 Routine Feedback to Sub-national and Key Stakeholders The advantages of routine performance feedback include; helping local managers, supervisors and implementers to consider what their own strengths and weaknesses are, and where they need to be making more of an effort. Secondly, for those collecting the information, seeing how that data is used, and how it can assist their own work and the cork of their colleagues, helps to motivate them to improve the quality of information they provide. Feedback on performance shall be provided for national, regional and institutional level performance on quarterly basis through performance review meetings and reports and league tables. 3.8 Reporting Progress against the NDP 2010/ /15 The health sector M&E plan is informed by the priorities of the NDP and is aligned with the M&E plan of the NDP. The Key Results Area matrix which requires the sector to report on 22 health sector indicators (Annex 7.2) provides the annualized expected results (outcomes) for each indicator. (Monitoring and evaluation strategy for the National Development Plan (2010/ /15)). Of these indicators, those that are also JAF indicators are currently included in the performance reports submitted to the MoFPED/OPM. A system will be developed to ensure appropriate reporting for the 22 NDP health sector performance indicators. 3.9 Reporting Requirements on International Commitments and Resolutions The NDP reflects and spells out the international initiatives to which Uganda is a signatory and these include the Millennium Development Goals (MDGs), the International Conference on Nutrition, the Convention on the Rights of the Child, the UN Convention on the Rights of PWDs, the International Conference on Population and Development, the New Partnership for Africa s Development (NEPAD), the Paris Declaration on Harmonization and Alignment, the International Health Partnerships and related initiatives (IHP+), World Health Organization member states and the Abuja Declaration among many initiatives. 49

64 Currently in the NDP M&E strategy, it is indicated that there is no M&E framework for implementation of regional and international agreements and treaties. There is therefore need to develop and operationalise such a framework. Millennium Development Goals (MDGs): Uganda is committed to achieving the MDGs by The MDGs directly relevant to the health sector are: MDG 4: Child Health; MDG 5: Maternal Health; MDG 6: HIV and AIDS. The health sector also bears indirect responsibility in the achievement of MDG 1: End poverty and hunger; MDG 3: Gender equality; and MDG 7: Environmental sustainability. The HSSIP follows the principles of the Paris Declaration and the Accra Agenda for Action through the IHP+ in the interaction and collaboration with national and international development partners. World Health Statistical Report The World Health Statistics series is WHO s annual compilation of health-related data for its 193 Member States, and includes a summary of the progress made towards achieving the health-related Millennium Development Goals (MDGs) and associated targets. It s compiled using publications and databases produced and maintained by the technical programmes and regional offices of WHO. Indicators are included on the basis of their relevance to global public health; the availability and quality of the data; and the reliability and comparability of the resulting estimates. Taken together, these indicators provide a comprehensive summary of the current status of national health and health systems in the following nine areas: mortality and burden of disease; cause-specific mortality and morbidity; selected infectious diseases; health service coverage; risk factors; health workforce, infrastructure and essential medicines; health expenditure; health inequities; and demographic and socioeconomic statistics. The estimates in these publications are derived from multiple sources, depending on each indicator and the availability and quality of data. Every effort is made to ensure the best use of countryreported data adjusted where necessary to deal with missing values, to correct for known biases, and to maximize the comparability of the statistics across countries and over time. In addition, statistical techniques and modeling are used to fill data gaps. As a member of the World Health Organisation member states, the MoH is required to submit an annual statistical report on selected indicators. (Annex 7.3). The MoH Global Desk Officer will compile this report in collaboration with the RC and relevant programme managers. The report will be submitted by 28 th February the following year. United Nations General Assembly Special Session (UNGASS) As a signatory to the UNGASS Declaration of Commitment on HIV/AIDS IN 2001, Uganda continues monitoring progress in combating HIV/AIDS. The Uganda Aids Control Program (UACP) has developed appropriate M&E mechanisms to assist with follow-up in measuring and assessing progress, developed appropriate monitoring and evaluation instruments, with adequate epidemiological data. UACP conducts national periodic reviews involving the participation of civil society, particularly people living with HIV/AIDS, vulnerable groups and caregivers, of progress achieved in realizing these commitments and identify problems and obstacles to achieving progress 50

65 and ensure wide dissemination of the results of these reviews. Uganda submits annual reports based on the UNGASS indicators (Annex 7.4) East African Community and the Common Market for Eastern and Southern Africa (ECSA) As a partner state of the ECSA, Uganda s health policy has been aligned with existing and new regional health sector frameworks. The MoH will report progress of the Resolutions to the ECSA- HC Secretariat against a standard set of indicators, using a uniform format for all future Health Ministers Conferences (HMC). This is to enable comparison between member states. The reporting tool has two sets of indicators; the core regional indicators and the resolution-specific indicators. The core indicators (Annex 7.5) will measure performance in the socio-economic and demographic status of each member state (10 indicators), the nutritional status (9 indicators), the health system (22 indicators) and the epidemiological status and performance of the health system (46 indicators, including MDG indicators). The resolution-specific indicators aim at assessing progress on each of the 10 resolutions made at the previous HMC. Reporting is expected to be done by 30 th June every year for set one, and 30 th of September for set 2 (resolutions), ahead of the October HMCs. The Ministry of Health Global Desk Officer shall be responsible for developing an M&E framework for implementation of regional and international agreements and treaties as well as compiling and submitting the reporting requirements on international commitments and resolutions through the Permanent Secretary. 51

66 3.10 HSSIP Evaluation Evaluation is a process of measuring the outcomes, impacts and effectiveness of a programmes / projects as well as documenting experiences and lessons learnt. Evaluations are separately scheduled activities performed at specific intervals (for example baseline, midterm or at the end of a programme / project) Programme / Project Evaluation A number of health sector investment and intervention projects will be undertaken during the period 2010/ /15. All projects will be subjected to rigorous evaluation. The type of evaluation to be planned for and conducted should reflect the nature and scope of the public investment. For example, pilot projects that are being conducted amongst a random group of participants shall be selected for impact evaluation to determine whether or not the investment should be scaled up. As a minimum requirement, each project in this category will be required to conduct the following: (i) A Baseline study during the preparatory design phase of the project (ii) A Mid-term review at the mid-point in the project to assess progress against objectives and provide recommendations for corrective measures (iii) A Final evaluation or value-for-money (VFM) audit at the end of the project. A VFM audit will be carried out for key front-line service delivery projects where value for money is identified as a primary criterion. All other projects will be subjected to standard rigorous final evaluation. The MoH through the specific programme / project managers will be responsible for the design, management and follow-up of the programme and project evaluations (including baseline and midterm reviews). All projects are required to budget for periodic project evaluations. All project evaluations will be conducted by external evaluators to ensure independence. VFM will be undertaken by the Office of the Auditor General. Programme / Project evaluation reports shall be disseminated during the sector quarterly and annual review meetings Mid Term Review A Mid-Term review of the HSSIP 2010/ /15 will be done after two and half years (January June 2013) of implementation and report finalized by September 2013, for presentation at the JRM. The purpose of the MTR is to review the progress of implementation; identify and propose adjustments to the HSSIP and other government policies as required. The specific objectives of the MTR are to: 2. Assess progress in meeting HSSIP 2010/ /15 targets and to make recommendations for their adjustment if found necessary; 3. Review the appropriateness of outputs in terms of inputs, processes and desired outcomes; 4. Review the costing and financing mechanisms of the HSSIP 2010/ /15; and 5. Coordinate the MTR process with the NDP review. The MTR shall entail extensive review of documents including routine reports and recent studies in the sector; special in-depth studies may also be commissioned as part of the MTR; and interviews with selected key stakeholders. The MTR is undertaken in a participatory manner involving 52

67 government line ministries (MoH and related line ministries), national level institutions, service delivery levels, DPs, civil society, private sector and academia. The analysis will focus on progress of the entire sector against planned impact, but will also include an assessment of inputs, processes, outputs and outcomes, using the HSSIP indicators (core and others). The main result will be a list of recommendations for the remaining HSSIP years. This will be an internal, joint exercise involving all stakeholders. The overall responsibility of the process will be with the Director General of Health Services. A secretariat shall be constituted to support and co-ordinate the MTR process. The secretariat is to be located in the Health Planning Department of the MoH working closely with the Quality Assurance Department. A Technical Assistant may be required to support the MTR secretariat. The review shall be carried out by the Technical Working Groups. Each TWG will be responsible to undertake a review according their specific Terms of Reference. Issues not covered by any specific working group will be a responsibility of the MTR Secretariat. External facilitation may be required to address critical issues identified by the Working Groups. The outputs of the Working Groups shall be presented to the Health Sector Review Committee. Specific Terms of Reference will be developed for the Lead Facilitator, TWGs and MTR Task Force HSSIP End Term Evaluation The End Term evaluation will be conducted in the second half of 2014 (four and half years of the HSSIP implementation) in order to enable the sector to make use of its findings and recommendations for the formulation of the next strategic plan. Like the mid-term review, the analysis will focus on progress of the entire sector against planned impact, but will also include an assessment of inputs, processes, outputs and outcomes, using the HSSIP indicators (core and others). It will focus on expected and achieved accomplishments, examining the results chain, processes, contextual factors and causality, in order to understand achievements or the lack thereof. The evaluation will have to answer questions of attribution (what made the difference?) and counterfactual (what would have happened if we had not done A or B?) and take into account contextual changes (economic growth, social changes, environmental factors etc.), as well as policies and resource flows. a) Relevance: (i) Did the HSSIP address priority problems faced by the target areas and communities? (ii) Was the HSSIP consistent with policies of both Health Development Partners (HDP) and government? b) Economy: Have the HSSIP inputs (financial, human, Assets etc) been applied optimally in the implementation process? c) Efficiency: Were inputs (staff, time, money, equipment) used in the best possible way to maximize the ratio of input/outputs in HSSIP implementation and achieve enhanced outputs; or could implementation have been improved / was there a better way of doing things? d) Effectiveness: Have planned HSSIP outputs and outcomes been achieved? e) Efficacy: To what extent have been the achievements of the HSSIP objectives and goal? f) Impact: What has been the contribution of the HSSIP to the higher level development goals, in respect of national development goals; did the HSSIP have any negative or unforeseen consequences? 53

68 Trends in performance will be based on the HSSIP 2010/ /15 baseline indicators. Districts shall be the primary units of statistical analysis for the national evaluation platform approach. A dose response analysis that examines (for instance) the association between funding levels and service coverage shall be carried out to further contribute to the evaluation. The evaluation will be conducted by a team of independent in-country institutions in close collaboration with international consultants. The purpose of conducting the evaluation prior to the conclusion of the HSSIP is to generate lessons and recommendations to inform the next HSSIP. 54

69 4 Surveys Surveys will be conducted where there are information gaps or outdated information throughout the HSSIP period as a basis to confirm the occurrence of change. Surveys shall be carried out at all levels to provide basis for a 'before and after' assessment of the HSSIP 2010/ /15 progress. The results of these studies are supposed to inform decision making hence contribute to improving delivery of and access to health care and nutrition services. Several institutions conduct health and nutrition research in Uganda e.g. universities, semi-autonomous institutions and other public institutions with diverse affiliations and districts. The UNHRO is the Secretariat for health and related research and therefore has the mandate to coordinate health research activities. UNHRO will provide guidance to ensure that research data is disaggregated by sex, residence and wealth quintile among other variables. A gender analysis of the research findings will also be helpful in terms of contributing to development of policies and interventions. 4.1 Surveys Commissioned by the MoH These may be carried out directly by programmes under the MoH or contracted out. These will include; the Antenatal HIV Sentinel surveillance, Uganda HIV/AIDS sero-behavioral survey, Malaria indicator survey, availability of the six tracer medicines study, client satisfaction surveys, facility assessment for service delivery, health and human rights, and gender survey, and other operational surveys Service Availability and Readiness Assessment The service availability and readiness assessment shall be conducted as a rapid survey, just ahead of the health sector review, to assess/monitor readiness of facilities to deliver services (i.e. to assess if the necessary medicines, supplies, equipment, trained staff, diagnostic capacity, infection control precautions etc. are available to provide services such as family planning, MNCH, HIV, TB, malaria, NCDS). The readiness assessment is conducted using a standard instrument that has been jointly developed by WHO, USAID and other partners. A record review will also be conducted during the facility assessment to allow for verification of the quality of routinely reported facility data (RDQA). The approach serves to; Fill the data gaps that many countries face in terms of service delivery, generating standard tracer indicators that are required to plan and monitor scale-up of interventions for key health services and assess /measure progress in health system strengthening within the broader context of M&E of the national strategy; and Systematically assess and verify the quality of routinely reported facility information that goes into progress and performance reports. The facility readiness assessment shall be conducted on an annual basis to inform the analytical reviews. The methodology is based on a random sample of facilities that is nationally representative and stratified by region, facility type and managing authority. A complete list of public and private list of facilities is required for the sampling frame. The whole process shall be conducted prior to the annual JRM (September) every year. 55

70 4.1.2 HIV/AIDS Epidemiological Surveillance Active surveillance of HIV infection based on annual ANC-based HIV sero-prevalence surveys conducted in sentinel clinics distributed in all geographic areas of the country. The ANC-based sentinel surveillance system has evolved ever since it was set up initially in six urban sites in The system has expanded over the years to include more rural sites. In 2005, 2006, 2007 and 2010 ANC sero-prevalence data were obtained from 25, 26, 29 and 30 ANC sentinel surveillance sites respectively. The methodology of ANC HIV surveillance involves anonymous and unlinked HIV serological testing of residual blood samples after performing routine ANC serological testing for syphilis at the clinics. Blood specimens are collected from mothers attending their first visit for ANC during a defined survey period. The sampling frame is consecutive and therefore all eligible women who present at the sentinel sites in the sampling period are sampled. A minimum of 300 samples are collected from each sites. However, in high volume clinics in major urban areas, deliberate over sampling is conducted to obtain at least 500 samples from each clinic to permit stratified analyses. A sampling period of ten weeks is observed simultaneously in all clinics. HIV serological tested are conducted at the Uganda Virus Research Institute HIV Reference Laboratory in Entebbe after all personal identifiers have been irreversibly removed. Serological tests for HIV are based on a serial testing algorithm. HIV/AIDS epidemiological data will be collected from multiple sources. These include the national ANC HIV sentinel surveillance system, STD sentinel surveillance survey, periodic population based HIV/AIDS surveys, M&E of HCT, PMTCT and ART programs. The epidemiological data will be collected annually and presented in an annual report based on calendar year The Uganda HIV/AIDS Indicator Survey The Uganda HIV/AIDS Indicator survey 2011 will be a cross-sectional population based survey that will be conducted based on a nationally representative sample of adults aged years and children aged less than 5 years. This survey will provide data on the prevalence of HIV infections, Herpes simplex virus type-2 (HSV-2), syphilis and hepatitis B virus (HBV); the risk factors for HIV infection, programme indicators of knowledge, attitudes, sexual behaviour as well as the HIV/AIDS programme coverage indicators. The survey will provide national estimates and sub-national estimates for nine survey regions including the capital city, Kampala. The detailed report of UHSBS 2011 will be disseminated in print and on the website. This report will present summaries of the data on the magnitude of HIV and STI prevalence. In addition, data on selected behavioural indicators and some indicators of programme coverage will be presented. The process of data and sample collection will be conducted from February 2011 to June 2011 and the final report presented by December Availability of the six tracer medicines study Availability of the six tracer medicines is one of the core indicators of HSSIP to be monitored at facility level monthly through the HMIS. At national level availability of the six tracer medicines will also be monitored annually through the six tracer medicine availability study conducted in September October in selected districts by the MoH Pharmacy Division. The findings of the six tracer medicines availability study will be analyzed and reported in the AHSPRs. 56

71 As the MoH works towards increasing access and availability of the essential medicines and health supplies it is shall use appropriate innovative approaches to improve monitoring and accountability for medicines and specified items. The information generated shall be used for improving the supply chain management and feedback will be provided through SMS, printed reports, newsletters and bulletins to the districts, health facilities and villages Uganda Malaria Indicator Survey The Uganda Malaria Indicator Survey (UMIS) was designed to determine the progress made in malaria control and prevention in Uganda. It provides data on key malaria indicators, including mosquito net ownership and use, as well as prompt treatment using ACT. The key objectives of the UMIS are to: Measure the extent of ownership and use of mosquito bed nets Assess coverage of the intermittent preventive treatment programme for pregnant women Identify practices used to treat malaria among children under age 5 and the use of specific antimalarial medications Measure the prevalence of malaria and anaemia among children age 0-59 months Determine the species of plasmodium parasite most prevalent in Uganda Assess knowledge, attitudes, and practices regarding malaria in the general population The UMIS will be carried out every two to three years by the Malaria Control Programme during the months of November and December, using a nationally representative sample of households. The Survey will provide some of the core HSSIP indicators as well as national and regional estimates of a range of malaria indicators and thus provides a robust and comprehensive picture of malaria control in Uganda. It captures both biological and behavioural information relevant to malaria and will provide a useful reference tool and evidence base for national policy decision making National Tuberculosis prevalence survey This population-based TB disease prevalence survey aimed at establishing an accurate estimate of the burden of TB disease; estimating the age-sex distribution of prevalent PTB cases; establishing the proportion of prevalent cases found by the health system; and comparing the health seeking behavior of prevalent cases that were not being treated to those being treated by the health system. The survey is planned to take place in This will be a cross sectional descriptive study targeting adults aged 15 years and above from all over the whole country. Preliminary results will be discussed 4-6 months after the completion of the survey using standardized reporting formats provided by WHO Task Force. The prevalence survey is estimated to cost US$ 2,487,649 of which US$ 2 million is budgeted under GFATM Round Client Satisfaction Surveys a) Client satisfaction surveys will be carried out at all levels of service delivery to determine the quality of services offered in the client perspective. A client satisfaction survey tool shall be developed by the Quality Assurance Department for incorporation into the HMIS. Facility client satisfaction surveys will be carried out biannually (December and June every year) and findings utilized for quality improvement. 57

72 b) The national client satisfaction surveys will be needed to provide the baseline, mid-term and end of HSSIP 2010/ /15 indicators. These national surveys will be carried out in June - July 2011, January March 2013 and January March 2015 respectively. A baseline client satisfaction survey shall be carried out in The MoH shall collaborate with UBOS and other stakeholders to identify modalities of conducting more regular population based client satisfaction surveys Non-Communicable Diseases (NCD) Survey During HSSP II the MoH initiated the process of conducting a national NCD baseline survey on risk factors and magnitude of non-communicable diseases in the country. The survey is expected to be completed by The NCD baseline survey will provide baseline data on the prevalence of NCDs and their risk factors. This should result in the formulation of evidence based national NCD policies and strategies as well as the development of a comprehensive and integrated action plan against NCDs in our population. Other relevant surveys may be initiated by the MoH during the course of implementation of the HSSIP. 4.2 Health related surveys by other institutions The Uganda Bureau of Statistics and the office of Births and Deaths Registration (for Vital Registration) are invaluable stakeholders that will provide data for sector monitoring, especially for health outcome and impact indicators. The MoH shall be actively involved in design of population surveys such as the Uganda Demographic and Health Surveys, Household surveys e.g. Uganda National Panel Surveys (UNPS), Uganda National Household Survey and National Service Delivery Survey. Other research institutions and academia may conduct population surveys and share findings with the MoH Uganda Demographic and Health Survey The UDHS is a national survey carried out by UBOS. The primary objective of this survey is to provide up-to-date information for policy makers, planners, researchers and program managers, to use in the planning, implementation, monitoring and evaluation of population and health programs in the country. The UDHS provides detailed information on fertility; family planning; infant, child, adult, and maternal mortality; maternal and child health; nutrition; and knowledge of HIV/AIDS and other sexually transmitted infections. Additional information needed by the sector can be provided in collaboration with UBOS. The last survey was conducted in 2006 and the findings were used for setting targets for HSSIP health status impact indicators and coverage for risk factors. The next UDHS - 5 shall be carried out in Data collection process to commence May The specific objectives of the UDHS - 5 are as follows: 58

73 To collect data at the national level that will allow the calculation of demographic rates, particularly the fertility and infant mortality rates To analyze the direct and indirect factors that determine the level and trends in fertility and mortality To measure the level of contraceptive knowledge and practice of women and men by method, by urban-rural residence, and by region To collect data on knowledge and attitudes of women and men about sexually transmitted infections and HIV/AIDS, and to evaluate patterns of recent behaviour regarding condom use To assess the nutritional status of children under age five and women by means of anthropometric measurements (weight and height), and to assess child feeding practices To collect data on family health, including immunizations, prevalence and treatment of diarrhoea and other diseases among children under five, antenatal visits, assistance at delivery, and breastfeeding To measure vitamin A deficiency in women and children, and to measure anaemia in women, men, and children To measure key education indicators including school attendance ratios and primary school grade repetition and dropout rates To collect information on the extent of disability To collect information on the extent of gender-based violence. A Technical Working Committee with membership from the MoH, UBOS Directorate of Population and Statistics, POPSEC, MoGLSD Probation and Social Welfare Department, MoES, WHO, UNICEF, UNFPA, USAID shall be responsible for development of the general scope, methodology and workplan for the UDHS-5. It will also provide technical guidance on development of the survey instruments; oversee data collection, analysis, report writing, quality assurance, dissemination and presentation of the survey results. The main survey (field operations) i.e. data collection entry, analysis and report writing shall take place from May 2011 to May To ensure better understanding and use of these data, the results of this survey shall be widely disseminated at different planning levels The Uganda National Panel Survey The Uganda National Panel survey (UNPS), which is carried out annually over a period of 12 months, is one of the household surveys that shall be carried out by UBOS. The UNPS aims at producing annual estimates of outcomes and outputs in the key policy areas as well as providing a platform for the experimentation and assessment of national policies and programs. The survey collects information on Socio-economic characteristics at household, individual and community levels. The survey objectives are; i) To provide information required for monitoring the National Development Strategy of major programs. ii) To provide high quality nationally representative information on income dynamics at the household level. iii) To provide a framework for policy oriented analysis and capacity building. 59

74 This survey provides data on two core HSSIP indicators; the contraceptive prevalence rate and health worker absenteeism. The survey covers all districts in the country and the sample provides estimates at national level; rural-urban level; and regional level. Analysis and dissemination of the findings will be carried out by UBOS and report availed to the MoH by October every year The Uganda National Household Survey The UBOS carries out an integrated household survey, popularly known as Uganda National Household Survey (UNHS) every other year (May to April) since the late 1980s. The UNHS is the main source of statistical information for monitoring poverty levels, trends and related welfare issues. The objectives of the survey are; i) To provide information on selected socio-economic characteristics of the population ii) To meet data needs of users for MDAs and other collaborating Institutions, donors as well as the NGO community so as to monitor the progress of their activities and interventions. iii) Generate and build social and economic indicators to monitor the progress made towards social and economic development goals of the country. The survey covers all districts in Uganda and the sample provides estimates at national level, rural and urban levels, regional level and Kampala district. The survey provides findings on prevalence of illness, type of illness suffered, days lost due to illness, type of treatment sought, distance to the health facilities; usage of mosquito nets and prevalence of Non-Communicable Diseases (NCDs) among others. The socio-economic data collected also provides indicators on household expenditures include consumption and non-consumption expenditures. The health sector is thus able to estimate the % of household experiencing catastrophic health expenditures from the UNHS. The last survey was carried out in 2009/10 and subsequent surveys will be carried out every two years National Service Delivery Survey (NSDS) The NSDS surveys are designed by the Ministry of Public Service of the Republic of Uganda and implemented by the UBOS as the primary investigator. Other producers are: the Inter-Ministerial Steering Committee, Cabinet of Uganda, Oversight policy direction and the Inter-Ministerial Technical Committee, Cabinet of Uganda, Technical Input and coordination. The NSDS is carried out every four years (2000, 2004 & 2008) to monitor and evaluate the delivery of public services and to obtain feedback from service recipients, regarding their efficiency and effectiveness. The NSDS has been institutionalized by Government as a key instrument to that effect. The overall objective of this survey is to provide a comprehensive assessment of the trends in service delivery in the areas of Health, Education, Justice, Law and Order, Agricultural services, Transport services (Road Infrastructure and Water transport), Energy use, Water and Sanitation, Public Sector Management and Accountability. The specific objectives are to: provide up to date information about the performance and impact of selected public services at national and local governments levels; measure changes in service delivery in selected sectors; identify constraints and gaps in the provision of selected public services by sectors; provide recommendations for improvement in service delivery; generate and disseminate information about the services offered by selected government sectors. The study is conducted in all regions of Uganda. The scope of the NSDS includes; - HOUSEHOLD: Household and housing characteristics, household listing, immunization for children aged 5 years and above, water and sanitation, household use of health services, education, and other service delivery issues. 60

75 - DISTRICT: Service delivery by institutions and accountability in institutions. - SUB COUNTY: Children's characteristics, birth registration and early learning, vitamin A, breastfeeding, care of illness, malaria, immunization, and anthropometry, with an optional module for child development. 4.3 Health Research and Evidence Generation The HSSP II MTR acknowledges that a lot of research is conducted in Uganda. The results of these studies are intended to inform decision making hence contribute to improving delivery of and access to health care and nutrition services. Operational surveys shall be carried as planned on approval by the relevant institutions and findings disseminated through the existing structures. Operational research shall encompass a wide range of problem-solving techniques and methods applied in the pursuit of improved decision-making and efficiency. The UNHRO shall be responsible for coordinating all the health related research in Uganda. The Secretariat will be responsible for mobilizing resources, setting, updating and disseminating health services research agenda, commissioning and organizing health research in collaboration with other research and academic institutions and NGOs. The QAD shall coordinate the sharing of research findings in the MoH by liaising with research institutions, universities and UNHRO which is a member of the SMER TWG. Local Governments and Institutions will be responsible for identifying research focal persons in each department / programme / Regional Referral Hospitals. The DHO will promote and coordinate research at district and lower levels. 4.4 Knowledge Management One of the recommendations that arose from the comprehensive analysis of the HSSP II was to undertake a comprehensive knowledge management approach in the sector. This should guide a comprehensive look at information needs, analysis and use to better guide decision making for health. The definition of a comprehensive performance monitoring approach for the sector, using input, output, outcome and impact indicators as outlined in the HSSIP M&E Plan should be able to generate adequate information for analysis and use. Data and information generated at all levels of the sector and from different sources will be shared, translated and applied for decision making during routine monitoring, periodic sector performance review, planning, resource mobilisation and allocation, accountability, designing disease specific interventions, policy dialogue, review and development. Effective knowledge management will be based on the following assumptions; first all relevant data will be aggregated, synthesized and analyzed for use at various levels of the sector. Second is that all reports produced through M&E activities, once approved, will be made easily accessible and in a timely manner to all stakeholders, including OPM, Parliament and citizens, in respect of the Access to Information Act. Thirdly all M&E results users should be able to translate and use the data/information for decision making, policy dialogue, review and development. Figure 7: The knowledge management process 61

76 4.4.1 Use of Knowledge / Translation and Application The objective of establishing M&E systems is to produce evidence of performance and results which can inform public policy and ensure the good stewardship of resources. Knowledge needs to be contextualized to be meaningful, which is why identifying and prioritizing needs of the decision makers and communities is essential. Knowledge mapping (assets, flows and gaps) at all levels will be used as a means to guide knowledge management work. The MoH will develop tools and methods for scaling up knowledge translation efforts. The main users within the health sector comprise MoH management, programs, LGs, health facilities, local and international partners and agencies. Those outside the health sector include Cabinet, Parliament, other Ministries/departments such as MoFPED, UBOS, MoLG, MoES, Health training institutions, individual researchers/students and the general public among others. Sector Performance Review Performance review at all levels of the sector will utilize data generated under the sector M&E framework. This shall include administrative reports, HMIS reports, OBT reports, survey reports among others. Planning On annual basis achievements, best practices and challenges in previous year will be used to identify priorities for the subsequent year, set new targets for the BFP, Annual Workplan, OBT key performance indicators and annual JAF indicators and actions. The MTR performance shall be used to review the HSSIP core indicators for the remaining HSSIP 2010/ /15 implementation period. The End Term Evaluation findings shall guide the development of the subsequent HSSIP. Resource Mobilization and Allocation The MoH will move towards evidence based priority setting to guide more effective and efficient resource allocation levels, rather than proportionate allocation of resources. Currently the MoH uses a formula that that takes into consideration the population size, infant mortality rate, poverty index and geographical location (such as border Districts, islands and mountainous terrain) when allocating resources within the health sector. 62

77 Accountability i) All departments, institutions and LGs will be held accountable for the achievement of targets set and agreed upon annually as documented in policy statements and framework papers. These will, where appropriate, include targets linked to client charters and service delivery standards. Performance information for departments, institutions and LGs against set targets will be scored, and institutions will be benchmarked. Success and failure to achieve set targets, upon review, will impact upon the resources provided to the accountable institution in future budget rounds. ii) All accounting officers will be held accountable for the use of resources set out in their Performance Contracts with MoFPED. Failure to account adequately for such resources will result in sanctions in accordance with Public Finance and Accountability Act and other laws. iii) All Hospital Directors will be held accountable for the achievement of targets set and agreed upon in their Performance Contracts with the MoPS. These Contracts will pertain to targets reflecting adherence to Public Service Code of Conduct and Ethics, and to the contribution to institutional results linked where appropriate to Client Charters and service delivery standards. Success and failure to achieve set targets, upon performance appraisal, will impact upon the individual through the reward and recognition scheme, and in professional career advancement. Policy Dialogue, Review and Development Midterm review of the NHP II will be guided by the HSSIP 2010/ /15 end term evaluation report. Other programme specific policy reviews and development will be guided by M&E findings during periodic reviews, programme evaluations and research findings during the HSSIP implementation period. Figure 8: Translating knowledge into policy and action The HPAC is responsible for discussing health policy and advise on the implementation of the HSSIP. At national level policy issues / problems are raised through reports presented by SMC to HPAC which meets every two months. At sub-national level technical committees will be responsible for generating policy issues / problems for discussion during the TPC meetings and forwarding to the respective Local Council Committees. The Local Council Committees will be responsible for analyzing and making policy recommendations to the District / Sub-county Council. 63