Assessment of the Zambian Health Management Information System

Assessment of the Zambian Health Management Information System Project SSG/9 ACP ZA 9/5 Arthur Heywood Erik Nielsen Stanislaw Orzeszyna September 2005 c o n s u l t a n t s EURO HEALTH GROUP Tinghøjvej 77 DK 2860 Søborg Denmark Phone: (+45) 3969 6888 Fax: (+45) 3969 5888 email: eurohealth@ehg.dk internet: www.ehg.dk

Content Content... 2 Abbreviations... 3 Executive summary... 5 Background... 6 Background... 7 Poverty reduction strategy... 7 National Health Strategic Plan 2006-2011... 10 Health Sector Reform... 11 Health Management Information System... 11 Objectives of the Assignment... 12 Methodology... 13 Activities... 17 Findings... 25 Millennium development Goals... 25 National MDG progress report... 27 Health system Indicators... 28 Population Data... 28 Vital Registration... 28 Data sources... 29 Surveys... 29 Epidemiological surveillance... 29 Routine HMIS... 29 Human Resources... 33 Information and Communication Technology (ICT)... 34 HMIS Software... 34 Administrative and Management Systems... 35 SWOT Analysis... 38 Strengths... 38 Weaknesses... 38 Opportunities... 40 Threats... 41 Key strategic issues... 43 1. Capacity development... 43 2. Implementation of DART principles... 44 3. Information and communication Technology strengthening... 46 4. Effective use of information... 46 5. HMIS staff retention... 48 The Way Forward... 49 1. Revised HMIS... 49 2. Capacity development... 50 3. Improved ICT... 51 4. Increased use of information... 51 Annex 1 Terms of Reference... 51 Annex 1 Terms of Reference... 52 Comments on the Terms of Reference and Key Issues... 60 Annex 2 Persons met... 63 Annex 3 Timetable...70 Annex 4 Questionnaires... 71 Annex 5 Database... 89 Annex 6 - Millennium Development Goals... 96 Annex 7 Review of the implementation of the 1996 HMIS... 97 Annex 8 HMIS Forms... 109 Annex 9 Presentations of results... 118

Abbreviations ACP ADB AFP AIDS ARI ART ARVs AU AWP CARITAS CBHMIS CBHG CBoH CBHIS CHAZ CHW CMAZ CP CRS CSO CSP DANIDA DART DFID DHB DHIO DHMT DHO DHS DPT EC ECHO EDF EPI EU FAMS GIS GRZ HAST HBC HC HIS HIU HIV HMB HMIS HRD HSR HSSP IDA IFMIS African, Caribbean, Pacific Group of States African Development Bank Acute Flaccid Paralysis Acquired Immuno Deficiency Syndrome Acute Respiratory Infection Anti-Retroviral Treatment Anti-Retroviral Drugs African Union Annual Work Plan Catholic Health Initiatives Community Based Health Management Information System Community Based Health Groups Central Board of Health Community Based Health Information System Christian Health Association of Zambia Community Health Worker Churches Medical Association of Zambia Cooperating Partners Catholic Relief Service Central Statistical Office Country Support Paper Danish International Development Agency Decentralised, Action-oriented, Responsive and Transparent Department for International Development District Health Board District Health Information Officer District Health Management Team District Health Office Demographic and Health Survey Diphtheria, Pertussis, Tetanus European Commission European Commission Humanitarian Aid Office European Development Fund Expanded Program on Immunization European Union Financial & Administrative Management Systems Geographic Information System Government of the Republic of Zambia Integrated HIV, AIDS, STI and TB Program Home Based Care Health Centre Health Information System Health Information Unit Human Immunodeficiency Virus Hospital Management Board Health Management Information System Human Resources Development Health Systems Research Health Sector Support Program International Development Agency Integrated Financial Management Information System Page 3 of 118

IP ICT KISS LAN LDC LIMS M&E MCH MDG MoFED MoH MoU MTR NDP NGO OPD ORT PEMFAR PEPFAR PLWHA PPAZ PRSP STI SWAP TA TB TBA UCI UN UNCTAD UNDP UNICEF WHO ZMK ZNBTS In-Patient Information and Communication Technology Keep it Simple and Sustainable Local Area Network Least Developed Countries Logistic Information Management System Monitoring and Evaluation Maternal and Child Health Millennium Development Goals Ministry of Finance and Economic Development Ministry of Health Memorandum of Understanding Mid Term Review National Development Plan Non-Governmental Organisation Out-Patient Department Oral Rehydration Therapy Public Expenditure Management And Financial Accountability Review Presidents Emergency Plan for AIDS Relief People Living With HIV/AIDS Planned Parenthood Association of Zambia Poverty Reduction Strategy Paper Sexually Transmitted Infection Sector Wide Approach to Planning Technical Assistance Tuberculosis Traditional Birth Attendant Universal Childhood Immunization United Nations United Nations Conference on Trade and Development United Nations Development Programme United Nations Children s Fund World Health Organization Zambian Kwacha Zambia National Blood Transfusion Service Page 4 of 118

Executive summary Under the Poverty Reduction Strategy Support (PRBS), the European Union provides substantial funds to Zambia for implementation of poverty reduction policies. The PRSP emphasises health and has a number of indicators to monitor the health sector in line with the National Health Strategic Plan 2006-2011 strategies and objectives. Strengthening of the National Health Management Information System (HMIS) will lead to improved poverty reduction monitoring. This document contains the results of an assessment of the Health Management Information System (HMIS) conducted by the Euro Health Group (EHG) for the European Union in July and August 2005. The second part of the assignment was to formulate a detailed, quantified plan of action to strengthen the HMIS in the country. This plan of action is contained in a separate document. The HMIS assessment was carried out in close collaboration with the Ministry of Health (MoH) and the Central Board of Health (CBoH) during all phases of preparation, field work, data analysis and report preparation. Preliminary findings of the consultation were presented and discussed at two meetings (13 July and 3 August 2005) of the Monitoring and Evaluation Subcommittee of MoH and at two meetings (29 July and 5 August 2005) of the Implementation Review Subcommittee of MoH. The methodology and the results of the mission were also discussed with developmental partners. To assess the HMIS, a framework of the WHO Health Metrics Network was adapted. Particular emphasis was made on the Millennium Development Goals indicators. Other major reference points included the Poverty Reduction Strategy Paper, draft National Health Strategic Plan 2006-2011 and National Development Plan. The assessment found that the Zambian HMIS is basically functional at all levels of the health system and compares favorably with most African HMISs. There is a defined indicator set and regularly collected data elements. Data collection and reporting tools are available at all facilities and district offices and the flow of information are clearly set out using the one channel principle. There is regular quarterly analysis of routine data with reasonably good coverage for service-based indicators of underweight children, measles immunization, supervised deliveries, malaria and TB incidence and TB DOTS coverage. However, many MDG indicators are not monitored, there is poor integration of vertical programs and administrative information into the routine HMIS, in general quality of data is not checked regularly and coherently and the system of vital registration is weak. Most staff are inadequately trained in HMIS procedures and there is not much faith in the results coming out of the HMIS. Many of the other subsystems need attention and it is proposed that specific plans are made for HMIS for Hospitals, Finances, Human Resources and community based activities. In addition, the MoH should work closely with the CSO to ensure that the results from the census and household surveys are widely available and that vital registration is strengthened. Five key strategic issues have been identified to strengthen the HMIS: 1. Capacity development of all cadres of staff, including intensive skills development through in-service and pre-service training programs, upgrading of manuals and study of best practice sites. Page 5 of 118

2. Return to the 1996 HMIS principles of Decentralization, Action oriented, Responsive and Transparent health information system, and introduction of the information pyramid. 3. Information and communication technology strengthening, through making the database more flexible and strengthening of decentralised information centres that are linked by internet to a central data warehouse. 4. Effective use of information through integration of vertical systems, with improved central coordination between stakeholders and sectors so that the information from HMIS can be used to assess output-oriented performance. Improved action research capacity is needed to improve feedback and dissemination and reduce overlap and duplication. 5. HMIS staff retention, particularly District Information Officers, is needed by improving skills and status and ensuring sustainability of systems, procedures and staff. Major ideas for the Way Forward include: A review of the present system, with adoption of a few key performance indicators that ensure that MDG and NHSP indicators are regularly analysed by all levels and local self-assessment is routinely done to satisfy the needs of health providers, vertical programs and service managers An intensive capacity development program targeted at all users as well as managers, planners and policy makers Strengthening of the ICT, focusing on a revised database and data warehouse, with improved equipment and capacity at provincial and district level information centres. Improved use of information for local action that responds to local needs as well as the needs of program monitoring and management improvement. Promote local action research skills to increase local in-depth analysis of existing records and increased use of Sentinel sites to get quality routine data and up- to- date analysis at local level and improved dissemination and feedback of surveys A MOH project will provide Zambian leadership to the HMIS through use of international standards and regular monitoring and evaluation Page 6 of 118

Background Poverty reduction strategy In July 2002, the Zambian Government officially launched its first Poverty Reduction Strategy Paper (PRSP) for the period 2002-2004. The main goal is poverty reduction through sustained economic growth and employment creation. In the health sector, in line with existing National Health Strategic Plan, the PRSP has the following programmatic priorities 1 : Provision of the basic health care package, to reduce morbidity and mortality and contribute to poverty reduction. An important component is cost sharing through fee paying; user fees should not, however, constitute a barrier to the poor accessing public health services; New approaches to allocation of financial and human resources to districts to give more weight to poverty issues; community participation and accountability of resources will be promoted; the Sector Wide Approach will be retained; Restructuring of the procurement system, to ensure that purchasing of drugs is done more efficiently and on a need basis; Improving access to health care in hard to reach and under-served areas as well as for vulnerable groups; The following areas are public health priorities: o o o o o o o Malaria - within the framework of the Roll Back Malaria initiative HIV/AIDS, TB, and STI (HAST) - through the National HIV/AIDS Strategic Framework, Integrated Reproductive Health - which includes family planning, safe motherhood, Adolescent health, abortion and post-abortion care, infertility, and (sexual) violence against women, Child health - which will aim at reducing morbidity and mortality due to childhood diseases, Epidemics - improved public health surveillance and control of epidemics, Hygiene, sanitation, and safe water The areas of focus of support services are following: o Development of physical infrastructure and provision of medical equipment o Human resource development o Strengthening of existing management systems The overall goal of interventions in nutrition is to achieve sustainable food and nutrition security among the poor and to eliminate all forms of malnutrition in order to have a well-nourished and healthy population that can contribute to national economic development. The strategies are: 1 Zambia Poverty Reduction Strategy Paper 2002-204, Ministry of Finance and National Planning, Lusaka, March 2002, http://www.cboh.gov.zm/documents/prsp%20final%20document.pdf Page 7 of 118

o To incorporate nutrition objectives into development policies and programmes; improve human resources and institutional finance; and establish strong nutritional networks, o To strengthen nutrition care practices for the poor, the HIV/AIDS infected, and vulnerable groups, o To prevent and control specific macro and micronutrient deficiencies and promote appropriate diets and lifestyles throughout the life cycle. PRSP Cross-Cutting Issues HIV/AIDS - The prevalence and incidence of HIV/AIDS has reached alarming levels in Zambia. The human toll of AIDS is a tragic reality being experienced by families, communities, and the nation at large. There is no aspect of life that has not directly or indirectly been negatively influenced by the AIDS epidemic. AIDS has become the major cause of illness and death among young and middle aged adults, depriving households and society of a critical human resource base and thereby reversing the social and economic gains made since independence. Interventions for HIV/AIDS under PRSP Reduce New HIV/AIDS Infections: Reduce Socio-Economic Impact of HIV/AIDS: Expansion of Access to Quality VCT Plus Services: Community Home Based Care: Anti-Retroviral Viral Treatment (ART): Improve the Quality of Life of Orphans and Vulnerable Children (OVC): Improving STI Management and Treatment in Zambia to Reduce STI infection: Expansion of Access to Quality Prevention of Mother to Child Transmission: Prophylaxis against TB: Drugs for Opportunistic Infections: Evaluation - A number of clinical, epidemiological, behavioural, and impact studies related to HIV/AIDS will be carried out. A sentinel surveillance system for HIV and populationbased studies will continue to be used to monitor the trend of the HIV epidemic. A system of collecting information from health facilities that is already in place to capture cases of AIDS, TB, and other STIs will be used. Lastly, data from various programmes and ministries will be collated and analysed at the national level. Gender - Issues of gender play a very important role in developing a sustainable strategy for poverty reduction. In addition to economic factors, the socially and culturally ascribed gender roles have contributed to high poverty levels among women. Women are usually not part of decisions made on resource allocation at household and other levels. PRSP Evaluation Strategy Poverty monitoring involves tracking key indicators over time and space with a view to seeing what changes have taken place to the indicators following the implementation of the Page 8 of 118

PRSP. The central objective of the PRSP is to reduce poverty and evaluations will enable the assessment of the impact on poverty of PRSP interventions. There are three main aspects of the PRSP evaluation strategy: evaluation of the implementation process: to attempt to establish whether programme implementation is done according to design and whether things are working or not; evaluation of outcomes to establish what the results are in relation to the original or revised programme goals and whether particular outcomes are the result of the interventions under the PRSP or a component of it. PRSP monitoring indicators for the health sector Objectives Objectively verifiable indicators (unit) Overall: To improve the health status of all people in Zambia, especially the poor To increase the life expectancy of the population To encourage lifestyles that support health To create environments that support health Life expectancy % Infant mortality rate Maternal mortality rate Under-5 mortality rate Infants aged 12-23 months fully immunised by 12 months Sexually active adults practising safe sex Children under-5 with low weight for age Reduce by 8% Children who are stunted % % % % To achieve equity in access to health opportunities To provide assured quality health services To provide quality policy and technical guidance to service providers Rural households within 5 km of a health facility Health workers per 1,000 population Cots and beds allocated per 1,000 population Health facilities with 80% of established posts filled Health facilities with essential drugs always available Health facilities rehabilitated New health posts constructed (according to health sector plan) Policies developed Policies reviewed % Page 9 of 118

National Health Strategic Plan 2006-2011 A 5-year National Health Strategic Plan (NHSP), for the period 2001-2006 is coming to an end. The preparation of NHSP 2006-2011 takes into account the PRSP/NDP process, the MDGs and new challenges to the basic health care package such as treatment for People Living with HIV/AIDS (PLWHs). Measure Infant Mortality Rate Maternal Mortality Ratio HIV/AIDS Prevalence Summary of Key Performance Indicators Indicator 92 per 1,000 live births 729 per 100,000 live births 16% (16-49 years) Malaria Incidence 396 per 1,000 TB Incidence Life Expectancy at Birth (Yrs) Health Annual Budget Proportion Health Expenditure per Capita Per Capita Income 545 per 100,000 population 50 (CSO) 10% (average) 10.80 USD 280 USD The NHSP directly addresses the Health Management Information System. It states that current weaknesses of HMIS are, among others: Lack of evidence based decisions Undue influence of sectional interests Weak demand for data Difficulty of monitoring progress Duplication of effort Underuse of data with absence of key data elements Lack of reliable data on inequalities Weak capacity to analyze existing data The NHSP identifies the following strategies and objectives for HMIS: Strategy 1: Create a harmonized framework for the national health management information system Objective 1.1 Define key health indicator standards, along with stratifiers (gender, rural/urban, socioeconomic, etc.) Objective 1.2 Develop platforms for providing essential health information Objective 1.3 Determine data and analytic capacities Objective 1.4 Develop frameworks or guidelines for information use Strategy 2: Strengthen country health information systems Objective 2.1 Establish collaboration with academic and research Institutions for analytic rigor to make information supplied reliable and acceptable Strategy 3: Improve access to and use of health information Page 10 of 118

Health Sector Reform Health reforms have been implemented in Zambia since 1992 under the framework of the Sector Wide Approach (SWAP). Resources from government and other stakeholders are pooled so they can be used efficiently. The main success of the health reform processes in Zambia is decentralization down to the district level. In 1996, the Central Board of Health (CBoH) was established as the implementing body. However, failure to implement the delinkage of public health workers from the civil service led to the end of the dualism of the Ministry of Health and Central Board of Health. A new structure of the Ministry of Health is in its final stages of preparation. It will comprise a monitoring and evaluation unit addressing in a comprehensive way the Health Management Information System. There was however a change in attitude generated by the new health reforms whereby a number of initiatives (mainly in the private and NGO sector) were seen as being effective. This showed the possibility of implementing a new national HMIS system which was flexible and more responsive to the needs of all levels of the health system. Health Management Information System Before 1994 the Health Management Information System (HMIS) in Zambia was overly centralized and fragmented, and resulted in the collection of large amounts of data which was not useful for decision making and action. A number of problems were identified with the previous system that included fragmentation, duplication, centralization, delay, unreliability. The system was not producing results and was donor driven, rather than being used for local decision making. Improvement of the HMIS was therefore an important part of overall health reforms and a comprehensive plan was drawn up in 1996 to ensure a functional, district-based HMIS. A review of the development and implementation of the 1996-HMIS is presented in Annex 7. Page 11 of 118

Objectives of the Assignment The objective of the consultancy is to make an assessment of the existing health management information system in Zambia and to prepare a work plan for its strengthening. The global objective is to improve the efficiency and effectiveness of health care delivery through the strengthening of HMIS. Support to HMIS is assumed to strengthen poverty reduction monitoring. The specific objective of the consultancy is to develop a detailed, quantified action plan for a strengthened, revised and comprehensive Health Management Information System, focused on coverage of essential health services and on the health status, but open to integration with other components that may be developed at a later stage. The assessment follows the following broad areas: 1. The relevance of the currently operational HMIS to the general M&E framework of the health sector. 2. NDP and HSSP information needs from the HMIS 3. Critical information gaps to the NDP monitoring of the Health sector 4. HMIS capabilities in generating gender and age responsive information on key health indicators and the reporting formats and frequency 5. Investment needs for strengthening the HMIS on: infrastructure (hardware and software), and personnel (training, conditions of service, retention ) to operate the HMIS 6. Mechanisms for quality assurance and control of the HMIS. This report is a product of an assessment of the health management information system carried out by EHG consultants in close collaboration with the Ministry of Health, Central Board of Health and development partners in July-August 2005. The second product of the consultancy, a Plan of Action for HMIS strengthening constitutes a separate report. Page 12 of 118

Methodology As stated in the Terms of Reference (TOR) the assignment should result in two products, (i) an assessment of the state of the country health information system, its ability to collect, analyse and generate reliable data on specific indicators, and the extent and timing of data dissemination and use, (ii) a detailed, quantified action plan that will build the rationale for HMIS reform, the vision to which Zambia should aspire and options and strategies to help Zambia turn that vision into a reality. It should be quite clear that the content and quality of the action plan, must depend on the results from the assessment of the current HMIS. The short time allotted to the assessment allowed us only to do a thorough assessment of the HMIS ability to collect, analyse and generate data on specific indicators. Although we do have some impressions of the quality and the reliability of data, we cannot substantiate or justify our impressions by exact findings. However it is recognized that the quality of the data is an important issue that has to be addressed by the project for strengthening the HMIS. What follows below is a brief outline of the methodology applied for the assessment of the HMIS, the underlying principles for the exercise, and the key areas of concern. Consultative and participatory One of the basic principles that has guided our work is that it must be a consultative and participatory process, involving as many users of the HMIS as possible, from the facility health workers at ground level, through managers at district, provincial, and national level, to external donors and other cooperative partners. The current HMIS was designed in 1996 and built on the DART-formula, i.e. Decentralised, Action-oriented, Responsive, and Transparent. We have tried to build our assessment using the same principles. The current HMIS has not been revised since its inception. As the ultimate goal of the assignment is to build the rationale for a reform of the existing HMIS we have seen it as an important outcome of the assessment that awareness was raised among the different stakeholders about the need for revising the system. We consulted as many stakeholders, within and outside the ministry, as practically possible within the timeframe given. We interviewed health staff in all 9 provinces, and in 17 of the 72 districts in the country, and a considerable number of health institutions. We visited rural and urban districts, public, private, semi-public, small health centres with very few staff and the biggest hospital in the country, UTH, with more than 1.600 beds and around 3.000 employees. During our field visits we were accompanied by staff from the MoH and/or CBoH. When visiting districts and health facilities we were accompanied by the Data Management Specialist from the relevant Provincial and/or District Health Management Board. In that respect the assessment process was also used to facilitate capacity building with in-depth discussions with local staff about findings and the way forward. Page 13 of 118

Finally, we were aware during all our work that we should be working with and not for the Ministry of Health. We emphasized direct and active participation of the ministry in the exercise, enabling the ministry to take responsibility and ownership of the plan of action that is envisaged as the offshoot of the assessment. During our work in Lusaka we had daily meetings with the MOH official responsible for the HMIS and a number of officials of MoH and CBoH. We presented our interim reports at the following meetings: 1. Presentation of Inception Report M&E Meeting 13 th July 2. Presentation of 1 st Draft of Assessment Report IRS Meeting 29 th July 3. Presentation of 2 nd Draft of Assessment Report M&E Meeting 03 rd Aug. 4. Presentation of Final Draft of Assessment Report IRS Meeting 05 th Aug. Acceptability The involvement and participation of the ministry in the process of compilation of the report on the assessment was imperative to ensure acceptability and to achieve a realistic plan for strengthening the HMIS. An effective, reliable and timely HMIS is crucial for ensuring optimal health care delivery and as a tool to strengthen poverty reduction monitoring. Its acceptability depends on its ability to meet local needs and be an effective management and planning tool. UN Millennium Development Goals (MDG) Another guiding principle for our assessment has been that the HMIS should monitor the Zambia health sector s commitment to and achievements in fulfilling the Millennium Development Goals (MDGs). The United Nations Millennium Declaration is the commitment to making the right to development a reality for everyone and to freeing the entire human race from want. It was signed by 147 heads of state in September 2000. They acknowledged that progress is based on sustainable economic growth, which must focus on the poor, with human rights at the centre. The objective of the Declaration is to promote "a comprehensive approach and a coordinated strategy, tackling many problems simultaneously across a broad front." The declaration calls for halving by the year 2015, the number of people who live on less than one dollar a day. This effort also involves finding solutions to hunger, malnutrition and Page 14 of 118

disease, promoting gender equality and the empowerment of women, guaranteeing a basic education for everyone, and supporting the Agenda 21 principles of sustainable development. To help track progress, the United Nations Secretariat and the specialized agencies of the UN system, as well as IMF, the World Bank and OECD defined a set of time-bound and measurable goals and targets for combating poverty, hunger, disease, illiteracy, environmental degradation and discrimination against women. Each year, the Secretary-General will prepare a report on progress achieved towards implementing the Declaration, based on data on the 48 selected indicators. Of the 48 selected indicators 15 (i.e. 31,25%) are directly linked to health services (see list of health MDG indicators in Annex 6). Consequently, monitoring and evaluation of health services is a very important part of the monitoring of the achievement in fulfilling the MDG. HMIS Assessment tool A tool was developed, based on the WHO HMN assessment framework for assessing the Zambian health information systems (HIS) 1. The aim of this tool is: 1. To provide a standardised baseline assessment of the country s health information system 2. To provide a basis for a country HIS development plan, 3. To enable monitoring of progress of the country over time. Objectives of the tool are: Determine performance of sub-systems and overall HIS, and the level of functional integration. To assess performance of HIS with respect to data quality, collection, processing, analysis and use of information Identify on-going development processes and major problems/shortcomings as well as identifying options for addressing such issues. Develop local skills to perform similar assessments in the future Provide clear criteria by which country information systems can be judged This tool provides a standardised baseline assessment for Zambia s monitoring of poverty reduction as defined in the MDGs. However, the tool also incorporates other important components of health system like for instance legal aspects, infrastructure, data sources and utilisation of available data as illustrated. 1 Unpublished paper: Jorn Braa, Arthur Heywood, Sundeep Sahay, Calle Hedberg: Health Metrics Network. A tool for assessing country health information systems (HIS). Page 15 of 118

Assessment palette Health System indicators HIS Context Monitoring of MDGs Data Sources Human Resources Legal Framework Information Processes The team focused the assessment on indicators for monitoring poverty reduction, and particularly on the MDG indicators. The assessment tool may serve for developing a standardised diagnostic method for monitoring of progress of the Health Management Information System development. To summarise the methodology, the guiding principles for the assessment have been o o o o o o Consultative and participatory, Decentralised, Action-oriented, Responsive, and Transparent, Acceptable, Awareness raising, Capacity building, Mainly focusing on poverty reduction and MDG indicators Page 16 of 118

Activities As illustrated the list of activities in the technical proposal approved by the MoH for the assessment contained 8 steps. What follows is a brief description of the activities carried out during the assessment. Inception 1. Inception and Document Review During the inception and document review, the following activities were carried out: 1. The Terms of Reference (ToR) were clarified and agreed upon with the project steering committee (the MOH Monitoring and Evaluation Subcommittee). The ToR including comments are found in Annex 1. 2. Focus and priority areas for the assessment were defined as the ability of the current HMIS to monitor poverty reduction within the framework of the MDGs. 3. Existing data, reports, studies etc. relating to the HMIS were reviewed and key areas and focus points defined for inclusion in the field assessment (see Annex 7 for details). 4. Instruments and logistics for data collection were agreed on. The general outline of the Health Metrics Network tool was adapted to the realities of Zambia s health sector. Three different questionnaires were developed, one to be used at national and at provincial level, the second one for district level and the last one for health facilities. The questionnaires are found in Annex 4. 5. Geographic areas for data collection were agreed upon and the team members allocated to these. One staff member from either MoH or CBoH was appointed to accompany each of the team members on the field work. The composition of the teams and the areas they visited, are found in Annex 8. 6. Finally a plan was elaborated for the logistics of the field visits and the provinces informed about the visits. Presentation of inception report Field visits Lusaka assessment Draft assessment report Draft plan of action National workshop Final report 2. Presentation of Inception Report Concurrently with the preparations for the field visits the team was reviewing existing documentation on the design and implementation of the 1996 HMIS plan. The review was presented in a steering committee, held on Wednesday 13 th PowerPoint used as the frame for the presentation is included in Annex 9. July. The Page 17 of 118

3. Field visits The field visits took place between 14 th and 23 rd of July visiting 8 provinces and 16 districts. Lusaka Provincial and District Health Management boards were visited on 25 th July. Thus, a total of 17 of the 72 districts in the country were visited, which gives a coverage rate of 23.6%. Of the 17 districts visited, 7 were urban, 8 rural, and 2 a mixture of urban and rural. The sample is seen as representative for the country as a whole. Type of district No. % Urban 7 41,2% Rural 8 47,1% Mixed urban/rural 2 11,8% Total districts included in the survey 17 100,0% Total no. of districts in the country 72 Country coverage rate 23,6% A total of 33 health facilities were interviewed and included, including public, private, mission, urban and rural health centres, 1 st, 2 nd and 3 rd level hospitals. The table below gives the details. Type Public % Mission % Private % 1st level hospitals 1 3,0% 1 3,0% 0 0,0% 2nd level hospitals 10 30,3% 5 15,2% 0 0,0% 3rd level hospitals 1 3,0% 0 0,0% 0 0,0% Urban Health Centres 4 12,1% 0 0,0% 2 6,1% Rural Health Centres 8 24,2% 0 0,0% 0 0,0% Hospital Affiliated Health Centres 1 3,0% 0 0,0% 0 0,0% Total HCs included in the survey 25 75,8% 6 18,2% 2 6,1% The field visits were spread all over the country, from the densely populated Copperbelt and Lusaka provinces to the remote and sparsely populated provinces in the north, northwest and west of the country. Results of field visits As each of the provincial and district questionnaires contained 124 records and the facility level questionnaire 150+ records, the amount of data collected during the field visit was huge. As it was impossible to analyse this amount of data manually, a database was developed in MS-Access. The database can in the future be utilised to assess the performance of the Zambia HMIS, or other assessments using a similar tool. The outputs from the database include a number of reports on performance as well as analytical tools that can be used to compare the capacity of the different sub-national users in collection and use of data. Page 18 of 118

With the database the Zambia health sector (and all other users of the tool) has a tool that can be utilised to continuously assess and evaluate the performance of the HMIS at all levels. A selected assortment of analytical reports is contained in Annex 5 and the database program engraved on an enclosed CD disc. The structure of the database follows the HMN framework as illustrated below (see next page and Annex 5). It should be noted that the assessment tool neither assess health services nor the health information system, but only the health management information system. We are aware that even if the health management information system is running perfectly it does not imply that the health information system is perfect. However, we believe that a well functioning health management system can add value to the health information system and the performance of the health sector. Database results of the interviews The database reports on the performance of the health information system for the whole country, for the ministry of health, by province, by district and by facility. Further, the database breaks the data down into 4 levels and in addition reports on all the micro-data collected from the interviews with the health service authorities and providers. At Level 0 the overall performance of Zambia HMIS by province and MoH is illustrated in the table and graph below. On average the provinces score is 40.7% of the total score, with relative high variations, with Copperbelt as the best province at 55.6%. Entity Score CEN 40,7% COP 55,6% EAS 47,3% LUA 26,5% Overall performance of Provincial HMB and MoH CEN MOH 60,0% 50,0% 40,0% COP LUS 44,2% NOR 26,3% Prov. Av. 30,0% 20,0% EAS NOW 39,1% SOU 43,3% WES 43,0% WES 10,0 % 0,0% LUA Prov. Av. 40,7% MOH 48,7% SOU LUS NOW NOR Zambia HMIS performance ratio Page 19 of 118

Tool for assessment of health management information system LEVEL 0 LEVEL 1 LEVEL 2 LEVEL 3 Prevalence of underweight children (under five years of age) Under five mortality rate Infant mortality rate Proportion of 1 year old children immunized against measles Maternal mortality ratio Monitoring of MDG indic ators MDG Indicators Proportion of births attended by skilled health personnel HIV prevalenc e among 15-24 year old pregnant women Contraceptive prevalence rate Prevalence of malaria Prevalence of tuberculosis Proportion of TB cases cured under DOTS Zambia HMIS Proportion of population with access to affordable essential drugs on a sustainable Health System Indicators Total health expenditure per capita Health workers per 1000 population Census Vital registration Data Sources Household surveys conducted Surveillance system for acute diseases Monitoring of other indic ators Surveillance system for chronic diseases - HIV/AIDS/ STI/ TB Health service statistics / HMIS Health systems information Legal Framework Legal framework and context HIS context Infrastructure Data availability and management Policy advocacy and dissemination Human Resources Capacity Building Implementation and action Information Proc esses Planning and priority setting Page 20 of 118