EVALUATION OF THE COMPUTER ASSISTED HOSPITAL DATA MANAGEMENT SYSTEM IN TANGA

Transcription

1 EVALUATION OF THE COMPUTER ASSISTED HOSPITAL DATA MANAGEMENT SYSTEM IN TANGA Consultant Dr. Beatus K. Leon October,

2 ACKNOWLEDGEMENT I thank the GIZ-TGPSH for trusting me and assigning to me this very valuable task. I am grateful to the Tanga Regional Health authorities, especially the RMO Dr Alli Uledi for his commitment in ensuring that this evaluation succeeded as planned. The GIZ staffs at Tanga were also very supportive; I particularly thank Merriment Hiza for her facilitation and logistical support during the data collection days. Data collection in the six hospitals was facilitated through the kind support of the hospital secretaries: Mr Adam Lyatuu (Bombo), Upendo Nkala (Handeni), Mwabutu Hosseni (Korogwe), Sarah Lupatu (Lushoto), Clare Chizazi (Muheza), and Milley Miduka (Pangani). Sincere gratitude is also expressed to Mr William Masika who played a very useful role as a liaison officer between the consultant and the persons from which data were to be collected. I thank my partners Sunday Morabu and Manase Nkya for the software assessment, and field assistants Irene Haule and Benedicta Msaki for collecting data on service observations and client exit interviews respectively. Dr Balthazar Ngoli, the Senior Technical Advisor at GIZ responsible for this evaluation is credited for his commitment, support, patience and tolerance. Last but not least, I thank everyone who answered questions in relation to this evaluation, especially the health workers who took time off their busy schedules and the patients who volunteered to participate in the exit interviews at a time they should have been rushing home. 2

3 TABLE OF CONTENTS ACKNOWLEDGEMENT... 2 TABLE OF CONTENTS... 3 LIST OF TABLES... 5 LIST OF FIGURES... 6 ABBREVIATIONS... 7 EXECUTIVE SUMMARY... 8 INTRODUCTION OBJECTIVES Overall Objective Specific Objectives METHODOLOGY Study sites Study population or data sources Sampling Data collection procedures and tools Types of data collected: Data collection procedure: Data collection tools: Data analysis Ethical issues Preparation for data collection Consensus meeting: The data collection process The process at each hospital FINDINGS Validity of the Aims and Objectives of the CAHMA The original aims of the CAHMA Current needs related to data management in the hospitals Activities and outputs of the project in relation to project s objectives

4 4.3. Extent to which CAHMA aims and objectives have been achieved Revenue trends from the billing department Trends in outpatient attendances before and after Client service time observations Cost efficiency of the CAHMA Most significant changes related to the CAHMA Changes as Experienced by Clients Significant changes due to CAHMA from health workers perspective Major factors behind the results and outcomes Main results Factors behind these results: More efficient alternatives in the implementation of a CAHMA CONCLUSIONS AND RECOMMENDATIONS ANNEXES Annex 1: Hospital Client Flow Charts Annex 2: Names of interviewed individuals Annex 3: Tools used in the evaluation Tool 1: Interview with Hospital Management Team Tool 2: Interview with Head of Sections Tool 3: Interview with Health Care Providers Tool 4: Observation Checklist for project activities Tool 5: Observation checklist for service delivery time Tool 6: Client Exit Interviews Exit interviews with clients (Swahili version) Tool 7: Checklist for technical assessment of the Afyapro software Tool 8: Checklist for data extraction from ledgers, cashbooks, annual reports, and registers

5 LIST OF TABLES Table 1: Harmonization of Specific Objectives, data required sources and methods Table 2: Hospitals and sections with Afyapro installation Table 3:Revenue trends for three hospitals in Tanga before and after installation of the CAHMA Table 4:Monthly Revenue Trends for Bombo Hospital for the years 2010 and Table 5:Monthly revenue trends for Lushoto Hospital for the years 2010 and Table 6: Monthly revenue trends for Muheza Hospital for the years 2007 and Table 7: Changes in Monthly revenue trends for three hospitals before and after introduction of the CAHMA Table 8: Trends in proportions of Bombo Hospital revenue that was recorded in the Afyapro Table 9: Trends in percentages of Lushoto hospital revenue that was recorded in the Afyapro system Table 10: Correlation between recording revenue in Afyapro and changes in revenue collection in two hospitals in Tanga Table 11: Trends in outpatient attendances before and after Afyapro installation Table 12: Correlation between percentage changes in OPD attendances and changes in revenue collection Table 13: Changes in inpatient bed occupancy before and after Afyapro installation Table 14:Correlation between percentage changes in bed occupancy and changes in revenue collection Table 15: Distribution of the 193 interviewed clients by hospital Table 16: Clients who had witnesssed computers in use in the 6 hospitals Table 17: Sections of the six hospitals where clients obtained services before the interview Table 18: Last time clients visited the hospital for service Table 19: Reasons for clients' perception of service quality in the six hospitals Table 20: Clients' ranking of their hospitals' record keeping system as compared to other hospitals in Tanzania Table 21: Clients' ranking of their hospitals' record keeping system according to the hospital they attended Table 22: Clients' experiences with data loss by year Table 23: Clients' experience with hospitals' responses to patient data losses

6 LIST OF FIGURES Figure 1: Registration desk at Bombo Hospital with a computer installed with the Afyapro software. 28 Figure 2: Computer facilities at Bombo hospital laboratory Figure 3: The server room at Bombo Hospital Figure 4: Evaluation meeting at Pangani hospital Figure 5: Percentage changes in revenue collection before and after installation of Afyapro Figure 6: The registration desk at Handeni hospital with the unused computer neatly covered in hospital sheets Figure 7: Average client service time at registration in six Tanga hospitals Figure 8: Average client service time at Pharmacy in 6 Tanga hospitals Figure 9: Average Client Service time at OPD in six Tanga Hospitals Figure 10: Average client service time at laboratory in two Hospitals Figure 11: Average client service time at billing sections of three hospitals using the Afyapro system 43 Figure 12: Average client service time at Bima sections of three hospitals handling data manually Figure 13: Client service time at different sections of Lushoto hospital by data management system. 45 Figure 14: Average client service time at different sections of Handeni hospital handling data manually Figure 15: Average client service time in sections of Korogwe hospital by data management system used Figure 16: Average client servicet time in sections of Muheza hospital by data management system used Figure 17: Average client service time in sections of Pangani hospital by data management system used Figure 18: Average client service time in sections of Bombo hospital by the data management system used Figure 19: Evidence of computer use in the six hospitals as witnessed by clients in Tanga Figure 20: Clients' perception of the quality of services in six hospitals on the day of interviewas compared to their prior experience with the same hospital Figure 21: Clients queue up for service at the reception desk of Handeni Hospital. Each client carries a primary-school-style exercise book for keeping his/her records

7 ABBREVIATIONS AMO Assistant Medical Officer ANO Assistant Nursing Officer CAHMA Computer Assisted Hospital data Management CHMT Council Health Management Team CTC Care and Treatment Centre DANIDA Danish International Development Agency DDH District Designated Hospital DMO District Medical Officer GIZ-TGPSH Gesellschaft für Internationale Zusammenarbeit Tanzania German Program to Support Health HMT Hospital Management Team ICT Information and communication Technology MoHSW Ministry of Health and Social Welfare MTUHA Mfumo wa Taarifa za Utoaji wa Huduma za Afya NACP National AIDS Control Programme NHIF National Health Insurance Fund NTLP National Tuberculosis and Leprosy Programme OPD Outpatient Department OS Out of Stock PFA Patient Flow Analysis RCH Reproductive and Child Health RCHC Reproductive and Child Health Clinic RHMT Regional Health Management Team RMO Regional Medical Officer TB Tuberculosis ToR Terms of Reference TSh Tanzanian Shillings 7

8 EXECUTIVE SUMMARY Since 2006, the TGPSH collaborated with the Tanga Regional Health management Team (RHMT) to pilot a computer assisted hospital data management (CAHMA) project in 6 hospitals in the region. The aims of CAHMA included: to increase income out of cost sharing (NHIF claiming and user fees), to improve business processes such as patient registration, logistics and drugs management, and financial management especially accounting. The system intended to reduce work load, smoothen the processes, avoid double work and improve efficiency. This evaluation was carried out in six hospitals in Tanga region to assess the relevance, effectiveness, efficiency, impact and sustainability of the computer assisted hospital management system in the 6 Tanga hospitals. A triangulation of methods was used to obtain data through document reviews, observations, interviews and assessment of the Afyapro software used in the project. Some of the major findings are as follows: The aims and objectives of the CAHMA are valid and address real problems experienced by health care managers, providers and clients. However, the objectives are not in good fit with current needs and health care management challenges in the hospitals, especially those relating to integrated reporting and utilization of data in informing district, regional and national health plans. There were improvements in revenue trends of Bombo compared to two other hospitals, despite Bombo s decline in inpatient bed occupancy rate and outpatient attendances, although a causal link between CAHMA and this change could not be established. Patients generally spent relatively less time in hospital sections which had the CAHMA compared to similar sections which handled data manually. Only two district hospitals fitted the comparison because they were using the system, the regional hospital had no comparable entity. A before-after analysis was impossible due to lack of baseline data There was no evidence linking staff movement within hospitals to activities or outputs of the project Three district hospitals recorded a decline in outpatient attendances and an increase in inpatient bed occupancy, while Bombo hospital experienced a decline in both parameters. There was however no evidence to associate this change with Afyapro due to the multiple confounders. Although patients experience with file losses apparently increased, the chances of recovering the same records also increased after introduction of CAHMA than before. 8

9 Bombo hospital was the only site where the Afyapro was utilized in the sections where it was installed with 64% of the hospital s revenue recorded in the Afyapro during its first year of implementation, which however dropped to 58% in the second year. The other hospitals did not make efficient use of the system. In all the six hospitals, manual data handling continued despite the availability of the electronic system Although the project was perceived as beneficial by service providers, most clients did not notice any change in the quality of services as a result of the project It is concluded that some of the aims and objectives of CAHMA, especially those addressing improved business processes (patient management including registration and medical records, logistics and drugs management, and financial management in particular accounting) have not been achieved since there are still lots of redundancies and double-work in all these areas. The changes in client volumes and revenue cannot be directly linked to the CAHMA. It is recommended that the CAHMA project should be re-designed, starting with a needs assessment and a situational analysis to determine respectively the entire health system s datarelated expectations from the hospital and gaps in fulfilling those expectations. The findings from the needs assessment and situational analysis should form variables for monitoring indicators of the reformed project. A comprehensive project plan should be developed, fully involving all stakeholders as participants in the processes, and with an implementation plan and a logical framework. Roles of each participating partner should be clearly spelt out and with a timeline. 9

10 INTRODUCTION GIZ-TGPSH has a long history of development cooperation in health with Tanga Regional and council Health authorities as well as hospitals and Primary Health Facilities. The partnership covers areas of sexual and reproductive Health, Health financing and social insurance, human resource for Health and support to district health services. In 2006, TGPSH and Tanga Regional Health management Team (RHMT) agreed to pilot computer assisted hospital management (CAHMA) in 6 hospitals in the region. The aims of CAHMA were among others to increase income out of cost sharing (NHIF claiming and user fees), to improved business processes (patient management including registration and medical records, logistics and drugs management, and financial management in particular accounting). The system is intended to reduce work load as means of addressing the shortage of staff in the involved hospitals. The system will serve to smooth the processes, avoid double work and use its automatic features to improve efficiency. To achieve the above, there were major activities which had to be undertaken. The first and most important was to conduct workshops and interviews with heads of departments and personnel. The interviews focused on identification of the needs of the management and the expected users of the system. This was followed by identification of server room and renovation, installation of the server, network and computer work stations and finally user training. The life history of using the system differs from hospital to hospital. Lushoto, Korogwe and Pangani started trial runs of the systems in 2007, followed by Handeni and Muheza in January 2009 and finally the Regional hospital in October All the hospitals are using Afyapro software. The RHMT of Tanga in collaboration with GIZ-TGPSH is determined to learn from the experience of implementing CAHMA in those hospitals. Information that will result from this study will provide shared memory for policy makers, health authorities and hospital managers in designing and improving computer based hospital data management in their respective areas. A baseline evaluation would have been ideal before the project was implemented to establish reference data upon which future evaluations would be based. Unfortunately there is no baseline data available so this evaluation is based on the situation found at the time of data collection, with some attempts at reconstruction of information where applicable. 10

11 2.0. OBJECTIVES 2.1. Overall Objective To assess the relevance, effectiveness, efficiency, impact and sustainability of the computer assisted hospital management system in 6 Tanga hospitals 2.2. Specific Objectives The specific objectives as derived from the ToR and scope of the evaluation are as follows To assess the extent to which the aims and objectives of the CAHMA are still valid To assess the activities and outputs of the project and compare consistency with the project objectives To assess the activities and outputs of the project and compare consistency with the expected results and outcome To define the extent to which the aims and objectives of the CAHMA have been achieved To document the major factors behind the results and outcomes (negative or positive) To determine the cost-efficiency of CAHMA and the timing of objectives achievement (money spent, human recourses for implementation and running costs, maintenance and re-trainings in each hospital in comparison to results and outcomes) To explore possibility of more efficient alternatives in the implementation To list most significant changes directly or indirectly linked to CAHMA implementation To identify evidence, opportunities and threats to sustainability To document the major factors behind the sustainability (negative or positive) 11

12 3.0. METHODOLOGY This methodology was specifically designed to address each item on the terms of reference (annexed) Study sites The geographical coverage of the evaluation was primarily limited to the project s coverage area, specifically the six hospitals namely Bombo, Korogwe, Lushoto, Pangani, Muheza and Handeni. For some of the objectives, specifically objectives 8 to 10above, additional information was sought from sources outside the hospital premises, targeting secondary populations that are indirect beneficiaries to the project Study population or data sources This evaluation employed a triangulation of data sources and populations such as documents and individual persons with information related to each specific objective. Data was obtained from the following categories of individuals: Members of hospital management teams Health care service providers in clinical areas Health workers who work with or manage data in the hospital (registration, billing, administration and finance) clients 3.4. Sampling The sample size for each population addressed and the sampling procedure depended on the evaluation objective being addressed, and each was considered individually. In the case of information sought from clients, efforts were made to include at least 10% of the day s outpatient load in the hospital, hence the targeted sample was 20 patients per day in Bombo hospital (40 in two days of data collection)and 15 patients per day (30 in 2 days) in the remaining 5 district hospitals, making a total targeted sample of 190 patients. For the hospital management teams, the intention was to interview the entire team in a focus group discussion, but in situations where assembling the entire team failed, an interview with up to three of the core members was sufficient to achieve information saturation. Head of sections were sampled from each hospital to represent equal numbers of sections that had the CAHMA system installed and those which did not have the system. The same was done for service providers. 12

13 3.5. Data collection procedures and tools A summary of the procedures and accompanying tools for obtaining data to answer each objective of the evaluation is provided in the table below: 13

14 Table 1: Harmonization of Specific Objectives, data required sources and methods Specific objective Data required Source of the data Data collection method To assess the extent to which the aims and objectives of the CAHMA are still valid Aims of CAHMA Justification of the project The specific needs that the project sought to address at its inception Current needs related to hospital data management CAHMA project document and logical framework Report of the needs assessment conducted before the project started Hospital workers involved in data collection, management and utilization Document review Interview with at least one worker in each of the hospital s section that collects data, manages the data and utilizes the data To assess the activities and outputs of the project and compare consistency with the project objectives Project objectives Reported activities and outputs of the project Observed activities of the project Project document and logical framework Project implementation reports since 2006 Actual on-going activities as observed during the evaluation Document reviews Direct on-site observation of activities To assess the activities and outputs of the project and compare consistency with the expected results and outcome Expected project results and outcomes Actual project results and outcomes (as reported and as observed by stakeholders) Project document and logical framework Project implementation reports since 2006 Stakeholders in the project (Hospital management, workers and clients) Document reviews In-depth interviews with management and staff Exit interviews with clients 14

15 To define the extent to which the aims and objectives of the CAHMA have been achieved Objectives of the project and indicators of their achievement Revenue trend from billing department and pharmacy before and after the project Time spent per client before and after the project Staff retention in the hospitals before and after the project Trends of outpatient attendances before and after Average waiting time per patient before and after Trends of inpatient hospital stays before and after The ToR for this consultancy assignment, pages 3 to 7 All revenue collection points in each hospital All major service delivery points in the hospital (especially reception, consultation, pharmacy, laboratory and cash/payment points) Human resource unit of each hospital HMIS/MTUHA data, trends from 2004 to present Client and staff experiences of patient waiting time Hospital in-patient records since 2004 Staff experiences with patient information sharing Document reviews Direct on-site Observations in the main service delivery points Interviews with staff in the main service delivery points of each hospital Interview with HR officer of each hospital Analysis of MTUHA/HMIS data for the mentioned indicators Frequency and ease of patient information sharing before and after Patient information retrieval time before and after Frequency of patient data losses before and after Client and staff experiences with data and file retrieval Client and staff experiences with files and data losses Management and staff experience and perceived benefits of the project Interviews with staff on patient information sharing Client exit interviews Patient flow analysis (to 15

16 Benefits of the project as experienced by management and staff measure patient time in the hospital) To document the major factors behind the results and outcomes (negative or positive) Reasons for outcomes Technical efficiency/challenges Desired technical standards of the system as compared to current system standards Users of the system Technical expert in medical informatics Interview with users Technical assessment of the software in relation to standard requirements of a hospital data management system To determine the costefficiency of CAHMA and the timing of objectives achievement (money spent, human recourses for implementation and running costs, maintenance and retrainings in each hospital in comparison to results and outcomes) Cost of the CAHMA (overall costs per site) Itemised prices of the results/outcomes (where monetary value applies) Hospital management Income-expenditure reports, audited reports since 2006 Document reviews Focus group discussions (with management for consensus on the prices of the results) To explore possibility of more efficient alternatives in the implementation Missing elements in the current system Alternative systems Technical expert in medical informatics Discussions Review of catalogues with 16

17 To list most significant changes directly or indirectly linked to CAHMA implementation addressing those missing elements Cost of the alternative systems All data above system specifications and pricelists Analysis and synthesis To identify evidence, opportunities and threats to sustainability All data above Hospitals potential to sustain the program from own income Analysis and synthesis Focus group discussions To document the major factors behind the sustainability (negative or positive) Minimum requirement for a hospital to sustain the program Factors favouring/hindering achievement of the minimum requirement Technical expert in medical informatics Stakeholders in the project Discussions, synthesis analysis, 17

18 3.6. Types of data collected: The primary and secondary data required for this assignment are as indicated on the table above Data collection procedure: The consultant liaised with the CAHMA project coordinator and the management of the six hospitals to inform participants in the selected sites of the evaluation schedule. For both the qualitative and quantitative aspects, contact was established with potential participants and an appointment was set for a data collection event (interview, observations, document review etc). Data was collected at the convenience of the participant, to avoid interference into their usual official duties. The data collection phase of the assignment comprised the following activities Review of documents Individual and group interviews with management staff Interviews with health care providers Observation of the time taken to serve a client at various service delivery points Exit interviews for clients Patient flow analysis Technical assessment of the Afyapro software against a standard checklist of specified technical requirements of a hospital data management system Focus group discussions for issues that required a consensus, as indicated and where feasible 3.8. Data collection tools: A separate instrument was designed for each data collection method. There was a total of 7 tools as listed below: Tool1: Interview guide for hospital management team Tool 2: Interview guide for head of sections in the hospital Tool 3:Interview guide for health care providers Tool 4: Observation checklist for CAHMA project activities in the hospital Tool 5: Observation checklist for service delivery time Tool 6: Questionnaire for client exit interviews 18

19 Tool 7: Checklist for technical assessment of the Afyapro software Tool 8: Checklist for data extraction from documents 3.9. Data analysis A thematic approach was used for analyzing the qualitative information, focusing on the predetermined themes derived from the ToR. The remaining quantitative data was managed using MS Excel and Epi info 7 data management software Ethical issues This evaluation collected and utilized data that participants in the exercise may have considered sensitive. Handling of patient information requires strict adherence to ethical principles as stipulated in regulations governing such issues as privacy and data protection in Tanzania. All ethical concerns were thoroughly addressed. For the client exit interviews, care was taken to ensure patients privacy was respected and only consenting clients were interviewed. All information obtained, whether primary data from participants or secondary data from records and documents, was treated in the strictest confidential manner, dully respecting individuals right to privacy Preparation for data collection Consensus meeting: The consultant met the GIZ-TGPSH management, some key staff and representatives of project beneficiaries before the assignment started. The brief meeting was convened and chaired by the CAHMA project coordinator in Tanga, with a purpose of ensuring the evaluation was participatory, and satisfy all parties that whole assignment was planned according to the objectives of the evaluation as determined primarily by the stakeholders. The agenda for the first meeting with the consultant were: a. Formal introductions of the assignment, its objectives and persons involved b. Discussing the approach to the assignment (basically the methods including the tools) c. Issues for clarification, improvement etc d. Identify persons from the GIZ/stakeholders/beneficiaries who will join the consultant in the assignment. Identify also a liaison officer for the assignment. e. Ratifying team targets and due dates for each component of the assignment f. Revise the assignmentcontract to accommodate the consensus reached in this meeting, and sign it. The meeting appointed a liaison person, allocated resources for the work and identified contact persons at each hospital to support the data collection exercise. 19

20 The data collection process The CAHMA project management was requested to avail the following information and documents: Information for establishing sampling frames: 1. Organizational Chart of the project with names and number of persons at each level 2. The number and names of health facilities where the CAHMA is operating 3. The geographical locations of the named facilities and their distance from a central reference point (eg the city centre) 4. The numbers (or estimates) of health care workers using the CAHMA in each health facility 5. The numbers (or estimates) of daily outpatient load in each facility Documents for desk review: 1. The project document (or its implementation logical framework) 2. Report of the needs assessment conducted before the project started (or any document used in the initial proposal to justify the project) 3. Project implementation annual reports since its inception in Audited reports of the expenditure on the project since its inception in 2006 The actual processes of data collection: This included the following methods: 1. Review of relevant documents 2. Individual and group interviews with management staff 3. Interviews with health care providers 4. Observation of service delivery time at various points 5. Exit interviews for clients 6. Patient flow analysis 7. Technical assessment of the Afyapro software against a standard checklist of specified technical requirements of a hospital data management system 8. Focus group discussions for issues that require a consensus, where feasible The detailed process for each are as follows: 20

21 Desk reviews: The desk reviews continued in tandem with the other data collection methods. The consultant did the desk reviews while assistants continued to collect other data for the assignment. The review was focused around the themes isolated from the ToR for the assignment. Individual and group interviews Individual interviews were conducted with: 1. CAHMA project staff 2. Senior regional and district level health managers 3. Health care providers in selected delivery points 4. Clients in the health care facilities 5. Primary Users of the CAHMA Group interviews or Focus Group Discussions These were conducted with management staff of each hospital (especially heads of sections in which the CAHMA is expected to bring impact) Observations These were done in the service delivery points where the CAHMA is installed. The observer recorded information about the person being observed, the working space, the working equipment available (and condition of the equipment), and the average time it took to serve one client. Technical assessment of the Afyapro software This was the last item on data collection because some of the assessed features were picked from the preceding data (from interviews, observations, PFA etc). An expert in the field of medical informatics assessed the software against a checklist of desired features of an HMIS system The process at each hospital The process that was followed in each hospital during data collection was as follows: The liaison officer (identified from the GIZ during the consensus meeting) set appointments for the team to visit each of the selected hospitals. In the communiqué, the officer requested the hospital administration to make the following information available to the consultant: 1. Reports containing trends of outpatient attendance since year If such summarized reports are unavailable, the hospital should present the respective MTUHA books. 21

22 2. Hospital s annual reports since Reports containing trends in the hospital s revenue collection since year Inpatient hospital records since year 2004, showing trends in average inpatient days 5. Audited financial reports indicating income and expenditure since 2004 The liaison officer also made an appointment for a meeting between the consultant and the hospital management, which was planned to be scheduled during the two days that the consultant was in the hospital. The data collection protocol at each hospital was as follows: Team arrives at the hospital in the morning; consultant is introduced to hospital management and receives the requested documents listed above. Consultant agrees with hospital management about the data collection schedule in the hospital, sets appointments with potential respondents and identifies locations for PFA and client exit interviews. On the same day, after a brief introductory meeting with management, the evaluation team immediately starts data collection. Data collection in each hospital should complete in the evening of second day of arrival at the hospital. The liaison officer should ensure that there is a timetable covering the six hospitals in twelve working days, the timetable should indicate exactly the date and time of visiting each hospital and the contact details of a named officer of the hospital with whom the team should communicate before arrival at the hospital. The records and documents obtained from each hospital shall be returned to the hospital management on the evening of the second day, before the team departs. The consultant shall not carry away any reports from the hospital, these shall strictly be used for the purpose of enriching the evaluation and will be left at the hospital on departure. The consultant shall summarize all the collected data on the same day the information is collected. 22

23 4.0. FINDINGS This chapter addresses the detailed findings of the evaluation according to the Terms of Reference and the Specific Objective Validity of the Aims and Objectives of the CAHMA The first objective sought to establish whether the aims and objectives of the CAHMA were still valid. To answer this, it was important to explore into the aims of the CAHMA as they were when the project was established. Incidentally, The Terms of Reference for this Evaluation became the only document on which aims and objectives of the CAHMA were listed. There was no other document availed to the consultant for this purpose, not even a proposal document for the CAHMA project The original aims of the CAHMA When interviewed, the project staff listed the following as objectives of the project: To improve business process Reduce workload and simplify work Increase income from cost sharing collections Helping management in decision making To improve quality of data Monitoring patient movement in the hospital Improve resource management practices in the hospital Members of hospital management teams mentioned the following as the aims of the project Bombo: To simplify work in the hospital To improve revenue collection To improve record keeping Improve quality of services provided to clients Muheza: Simplify data handling in the hospital Lushoto: To control patient movement and avoid bypassing of the formal registration system 23

24 To enable the accounting section to compare revenue recorded in the computerized system with the actual collection in the books These aims were found to relate well with those listed in the ToR for this evaluation, although in some hospitals the staff didn t appear to internalize and own these aims. To establish whether these aims were valid, it was important then to compare them with the current needs related to data management in the hospitals, which are detailed below Current needs related to data management in the hospitals These needs were sought from members of hospital management teams, head of sections and service providers in the various sections. All these individuals described what they desired to have as a data management system, which was summarized as a system that could: A system that could automatically generate reports A comprehensive system capturing all the data in the hospital (implying that there was some required data that Afyapro couldn t pick, for example vaccination data required in preparation of annual reports) A system that would very much reduce the need for manual data handling, or allow the phasing out of data keeping in hard copies A system that would incorporate all other reports that are required by other stakeholders and partners in the health system; such as the MoHSW, the NACP, other vertical programs such as NTLP, malaria, and donor partners Direct connection of each data source to the system; eg connection of the lab machines such as those of full blood count to the system so that a patient s lab results are directly loaded to the system after being generated by the machine. This will help in retention of patients lab results even in the case of technical faults with the machine after a lab test. A system that would be broadly networked to cover all the units and sections of the hospital A system that would capture all morbidity and mortality data to simplify the generation of reports in the same format required from the MoHSW Capture disease prevention data (eg immunizations) Link with all equipment that perform automated tests so results can be easily stored and accessed Comprehensive training on ICT to all staff of the hospital so they could make better and more efficient use of the system. There was a general agreement among interviewed staff and management that their hospitals still needed to: 24

25 Institute better controls for revenue collection Collect good quality data that would be useful in informing the hospital s plan Simplify data management to the extent that it was handy and not consuming too much of staff time Improve the quality of patient care Contribute better towards evidence based national health plans. From the cited points, a computerized hospital data management system was and is still a valid intervention that addresses objectives that are shared by the management and staff of the hospitals. However, the objectives of the data management system need to be updated to match the current needs Activities and outputs of the project in relation to project s objectives The activities of the project were reported first by project staff as Installation of local area network in all the 6 Hospitals Supply of computer hardware in which more than 30 computers were supplied to the hospitals. Although this was reported in a presentation by the ICT technician stationed at Bombo hospital, there was no documentation indicating the exact number of computers procured for the CAHMA project and featuring in the respective hospitals inventories. Installation of servers: There were reports that servers were installed in each hospital where the project was operating, and these were observed during the visits. Installation of software and updating of windows Training staff from involved units on computer basics and use of the Afya pro software. The evaluation team was unable to find training reports indicating the details of hospital personnel trained on Afyapro. During the interviews, there were lots of complaints about the selection of hospital workers for training on Afyapro. Citing specifically the cases of Lushoto and Muheza hospitals, there were concerns that the senior management staff in these hospitals were not trained on the use of the software, and were therefore unable to use the information from it to inform management decisions. Muheza hospital director and administrator reported that they were informed that the training would take place in phases, starting with those involved in data entry and then would scale up to senior management, but the training ceased after the initial phase of the training so the senior management (including hospital secretary, medical officer in charge, accountants) remained untrained. None of the mentioned activities was ongoing at the time of the evaluation so they were not observed. However, during the interviews some workers reported to have attended training 25

26 on the use of Afyapro. Servers and computers with installed programs were observed as shall be seen later in the report on technical assessment of the software. Training on the use of the Afyapro software in the management of hospital data was reported by staff in Bombo,Lushoto, Handeni, Korogwe, Muheza and Pangani. However, when the evaluation team visited Handeni they found that the system was not working, and received information that it operated for only a few months after its installation. The evaluation team met and worked with a computer systems analyst stationed at the Bombo Hospital. The person and the position are part of the project s initiative to ensure there is technical support to sustain the project s activities in the region. No other hospital had the privilege of a computer technician readily available for technical support, and this was described by some as one of the reasons for their failure to utilize the data system in their hospital as it emerged during the interview with the Hospital management team of Handeni where one member said the project was handed over to us when the system had a lot of unresolved technical problems! The reported activities of the project were noted to be quite in line with the project s objectives. It was difficult however, to judge on the timeliness of those activities because a logical framework for the project was not available. An implementation plan would be of use in deciding whether the reported premature handing over in Handeni was actually to blame for the hospital s failure to use the system. Formal implementation reports (quarterly, semiannual or annual reports) would have helped as monitoring tools during the implementation, and as sources of evaluation data during the evaluation phase. Unfortunately no such reports were available to the evaluation team. The only sure means of tracking all activities of the project in the absence of implementation reports would be to track the expenditure details from the documents that indicate what the payment was for (eg the payment vouchers). This might take far longer than initially scheduled for this assignment. A conclusion about the fit between project activities and project objectives in this report is therefore not based on comprehensive analysis of all activities undertaken by the project since its inception, and this has been due to lack of supporting documentation. Some of the project s outputs were vividly tangible. Computers with the Afyapro software were seen in the following hospitals and sections: 26

27 Table 2: Hospitals and sections with Afyapro installation Hospital Bombo Sections with computers and the Afya pro software installed Registration Billing Laboratory Pharmacy Lushoto Registration Billing Pharmacy (not used) RCH (not used, building was under renovation) Pangani Registration and Billing as one section Bima (not used) Handeni (not using the system). Not even have a filing system, clients keep their own records in a daftari Registration Bima Pharmacy RCH Korogwe Registration and billing Pharmacy Muheza Registration Billing (not used) Pharmacy (not used) RCH 27

28 Figure 1: Registration desk at Bombo Hospital with a computer installed with the Afyapro software Figure 2: Computer facilities at Bombo hospital laboratory The other outputs are the numbers of staff trained on the use of computer assisted hospital data management which were not known to the evaluation team due to the lack of implementation reports. It was impossible to verify the details of staff trained on Afyapro 28

29 because training reports were not available. If such reports were available we would be able to determine the specific activities for which the staff members were trained, the content and duration of the training, and the numbers and distribution of the trained staff. There were verbal reports of reallocation of Afyapro-trained staff to other sections of the hospital where the Afyapro was not in use. These outputs could have been objectively evaluated by comparing them to the targeted outputs. Although the outputs relate in context to the objectives of the project, their measurement remain incomplete due to lack of a reference as would have been shown in the statement of intended or expected outputs. The lack of a project document showing the project s logical framework with monitoring indicators was the most significant obstacle in attaining objectivity in this evaluation. Figure 3: The server room at Bombo Hospital 4.3. Extent to which CAHMA aims and objectives have been achieved The extent of achievement of the project s aims and objectives was supposed to be measured, according to the ToR for this evaluation, through: 1. Revenue trends from billing department and pharmacy before and after the project: The only reliably available data here was those for revenue from user fees and drug revolving funds. It was only possible to compare data from three hospitals, namely 29

30 Bombo, Lushoto and Muheza which managed to supply the required information within the time frame of this evaluation. 2. Time spent per client before and after the project: It was only possible to measure time spent per client after the project. There was no data for this indicator before the project. This deficiency was compensated for by comparing similar hospital sections which have the Afyapro to those that lack the Afyapro 3. Staff retention in the hospitals before and after the project: The measurement for this was largely subjective and was practically impossible to link causally to the presence of Afyapro 4. Trends of outpatient attendances before and after: These were extracted from hospital s annual reports 5. Trends of inpatient hospital stays before and after: These were extracted from hospitals annual reports 6. Frequency and ease of patient information sharing before and after: These were derived from interviews on staff who shared their experiences on the subject. It was impossible to quantitatively determine the frequency of information sharing, this was very much subject to recall bias as none of the clinical staff kept record of this. It was sufficient in this regard to establish that at least information was shared among clinical staff and that there had been some changes after the introduction of Afyapro especially at Bombo hospital as shall seen later in this report. 7. Patient information retrieval time before and after: This was also measured qualitatively during interviews with clinical staff. The actual retrieval time for specific patient information was not recorded before and even during the implementation of the project and hence it would be impossible to make a comparison. It was possible though to establish from the clinical staff whether the retrieval time had been faster or slower after the introduction of the Afyapro. 8. Frequency of patient data losses before and after: It was only possible to measure this qualitatively through the client exit interviews because such events were not recorded and hence were impossible to quantify. 9. Benefits of the project as experienced by management and staff: These were obtained in the interviews with staff. 30

31 Figure 4: Evaluation meeting at Pangani hospital Revenue trends from the billing department Revenue trends for each hospital were analyzed after extracting the data from the books of accounts. To maintain consistency, data was extracted covering two years before and two years after the inception of the project. A summary for the three hospitals whose data was available is shown in the table below: Table 3:Revenue trends for three hospitals in Tanga before and after installation of the CAHMA Hospital name Year CAHMA introduced Revenue 1 year before Revenue 1 year after Difference % change Bombo ,121, ,276,680-17,844, Lushoto ,858, ,770, ,911, Muheza ,140, ,704,000 56,564, All three hospitals 328,120, ,750,797 50,630, The table above shows a decline in revenue for Bombo hospital after the introduction of Afyapro, while for the other two hospitals the revenue increased. The administration at Bombo hospital attributed the decline in revenue to shortages in medication and reduced outpatient attendances which could be related to doctors strike which happened at the time. Overall, the three hospitals together experienced an increase in revenue by 15% with Muheza recording the largest increase. 1 CAHMA was introduced in Bombo hospital towards the end of 2010, so it became operational in The revenue before and after Afyapro is therefore the money collected during the years 2010 and 2011 respectively 31

32 Monthly revenue trends in the six hospitals before and after the project Table 4:Monthly Revenue Trends for Bombo Hospital for the years 2010 and Monthly revenue collections in 000 TSh Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Total Year Year 2011 Differ ence % chan ge Source: Bombo Hospital Annual Reports for the years 2010 and 2011 There was no explanation for the alarmingly high income for Bombo hospital in February. 2 This revenue is only from two sources namely User fees and Drug Revolving Fund. Data from NHIF claims paid was not available. 32

33 Table 5:Monthly revenue trends for Lushoto Hospital for the years 2010 and 2011 Monthly revenue collections in 000 TSh Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Total Year Year Difference % change From the above table, Lushoto hospital recorded an annual revenue improvement of 12% after a year of CAHMA implementation. In subsequent years however (not shown on the table), the hospital stopped using the Afyapro due to technical problems. Table 6: Monthly revenue trends for Muheza Hospital for the years 2007 and 2011 Monthly revenue collections in 000 TSh Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Total Year Year Diffe renc e % chan ge From the above table, Muheza hospital had an overall increase in revenue by 44.5%, with the highest increase recorded in the month of June. 33

34 Table 7: Changes in Monthly revenue trends for three hospitals before and after introduction of the CAHMA % change in Monthly revenue collections Hospital name Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Total Bombo Lushoto Muheza Overall, the 3 hospitals whose data were available recorded an improvement in revenue collection after introduction of the Afyapro. The February increase for Bombo was so alarming that it raised suspicion of error in recording. The figure below shows a line graph representing the same data: Figure 5: Percentage changes in revenue collection before and after installation of Afyapro 34

35 The figure above shows that Bombo hospital had the largest overall increase in revenue collection after installation of Afyapro in the hospital. To establish whether Afyapro contributed to the changes seen in revenue collection, it was important to appraise the extent of Afyapro utilization in the revenue collection process. The next tables show the proportion of collected revenue that was recorded in the Afyapro system. Table 8: Trends in proportions of Bombo Hospital revenue that was recorded in the Afyapro total Total Revenue recorded in books of accounts 143,276, ,005, ,401,018 Revenue recorded in Afyapro 91,671,819 45,729, ,401,018 Revenue recorded in Afyapro as % of total revenue Source: the Afyapro database in Bombo and Revenue books from Accounts section of the hospital The table above shows that 64% of hospital revenue (user fees and drug revolving fund only) was recorded in Afyapro during the first year of implementation, but this dropped to about 58% during the second year. The administration attributed this to technical problems related mainly to power interruptions when the electric power supply to the hospital was frequently cut. Table 9: Trends in percentages of Lushoto hospital revenue that was recorded in the Afyapro system Total Total Revenue recorded in books of accounts 39,585, ,770, ,641, ,870, ,149, ,239, ,256, 797 Revenue recorded in Afyapro 5,625, ,308, ,326, ,161,8 61 9,501,65 8 2,727, ,651,6 38 Revenue recorded in Afyapro as % of total revenue Table 9 above shows that overall, 24.4% of Lushoto hospital revenue was recorded in the Afyapro system. The highest proportion of the hospitals income was captured by the system in the year 2009 when 34% was recorded, but this kept declining in subsequent years. 3 Up to August Up to June

36 Handeni hospital did not record any data in the Afyapro system because it was not in use. Although Lushoto and Muheza hospitals used the Afyapro during the first few months of its installation, they did not record revenue trends in the system, so these could not be traced. Figure 6: The registration desk at Handeni hospital with the unused computer neatly covered in hospital sheets Correlation between recording revenue in Afyapro and changes in revenue collections in six Tanga hospitals Table 10: Correlation between recording revenue in Afyapro and changes in revenue collection in two hospitals in Tanga Hospital name %total revenue recorded in Afyapro % change in revenue collection Bombo Lushoto Table 10 above shows that the increase in the percentage of revenue recorded in the Afyapro system also led to an increase in percentage improvement in revenue collection for Bombo only. The correlation was negative for Lushoto hospital, where electronic recording of 24% revenue led to just 12% improvement in total revenue collection. 36

37 Trends in outpatient attendances before and after Four hospitals were able to provide data on outpatient attendance before and after implementation of CAHMA in their hospitals and the following table shows a summary of the findings. Trends in outpatient attendances in four hospitals before and after introduction of Afyapro as a data management system in their hospitals Table 11: Trends in outpatient attendances before and after Afyapro installation Hospital name Year CAHMA introduced OPD attendances 1 year before OPD attendances 1year after Difference % change Bombo , ,375-8, Lushoto ,670 23,650-1, Muheza ,528 61,212-18, Pangani 81,520 79,410-2, All 4 hospitals 409, ,647-30, Sources: Bombo Hospital Annual reports 2010 and 2011, Lushoto Hospital Outpatient Registers, Muheza Hospital Annual Report, and Pangani Hospital Outpatient Register. The table above shows an overall decline in outpatient attendances for the four hospitals, with Muheza recording the largest decline of 23%. Although this decline happened in all the four hospitals, it is still impossible to relate it to Afyapro use because there are multiple other factors which could have caused it. Administration at Bombo hospital for example attributed it to the doctors strike at the time. A correlation between the outpatient trends and the percentage changes in revenue collection in each hospital can provide a further insight into the effect of better revenue controls (which is expected to result from optimal utilization of Afyapro).The following table shows such a correlation: Correlation between percentage changes in OPD attendance and changes in revenue collection Table 12: Correlation between percentage changes in OPD attendances and changes in revenue collection Hospital name % change in OPD attendance % change in revenue collection 37

38 Bombo Lushoto Muheza The table above shows that despite the decline in outpatient attendance, the hospitals still recorded an improvement in revenue collection. Bombo hospital had the largest almost twofold rise in revenue despite a similar decline in OPD attendance to Lushoto. In a previous finding, we noted also that Bombo was the only hospital whose revenue could be tracked through the Afyapro system. Trends in inpatient bed occupancy before and after Afyapro installation Table 13: Changes in inpatient bed occupancy before and after Afyapro installation Hospital name Year CAHMA introduced Bed occupancy 1 year before Bed occupancy 1 year after Difference % change Bombo % 69% -3% -4.2 Lushoto 78.20% 99.80% 22% 27.6 Muheza % % 55% 46.8 Sources: Bombo Hospital Annual Reports 2010 and 2011, Lushoto Hospital inpatient registers for 2006 and 2008, and Muheza Hospital Annual Reports for 2007 and The table above shows that Bombo hospital experienced a 4.2% decline in bed occupancy while the two other hospitals had an increase, with Muheza registering the highest increase of about 47%. A correlation between the percentage of beds occupied and the percentage change in revenue collection still shows a more positive change for Bombo hospital in comparison to the other hospitals as shown in the table below: Correlation between percentage changes in inpatient bed occupancy and changes in revenue collection Table 14:Correlation between percentage changes in bed occupancy and changes in revenue collection Hospital name % change in bed occupancy % change in revenue collection Bombo Lushoto Muheza

39 The table above shows that although Bombo hospital had a decline in its annual percentage of beds occupied, it still recorded the largest improvement in revenue collection compared to the other hospitals which had increases in annual percentage bed occupation. This can be attributed to the previous finding that Bombo hospital utilized the Afyapro system in tracking hospital revenue while the other hospitals didn t Client service time observations Clients were observed in different sections the six hospitals for the time it took them to receive the services they sought in the hospitals. Sections that performed similar tasks across hospitals were compared, and the following are the results. 39

40 Client Service Time at Registration Figure 7: Average client service time at registration in six Tanga hospitals The figure above shows that the shortest time spent by clients at the registration section was observed at Bombo hospital. Handeni hospital reported they were not using the Afyapro system at all. Although the Afyapro system was observed the other hospitals as seen in the blue bars in the figure, it was evidently seen constantly in use at Bombo hospital alone Average client service time at Pharmacy Figure 8: Average client service time at Pharmacy in 6 Tanga hospitals 40

41 The chart above shows that the longest time at pharmacy was spent by clients receiving dispensing services at Handeni hospital which did not use the Afyapro system at pharmacy. Muheza clients spent the shortest time at pharmacy. Although Pangani did not have the Afyapro installed at Pharmacy, they had a shorter service time for clients than Korogwe, Lushoto and Bombo which had Afyapro at Pharmacy. From this chart, availability of Afyapro could only be thought to influence the speed at which clients received services at the pharmacies of the six hospitals if an alternative explanation was available for the Pangani observation. Perhaps a detailed analysis of the actual quality of care could explain the difference. It takes much shorter to scribble the letters OS (out of stock) on a prescription than to actually locate, pack, label, issue and provide dosage and other information to a pharmacy client Average client service time at the OPD The OPD in this case referred to the actual consultation service rooms for outpatients. These were the rooms where patients went to receive services after completing registration formalities at the hospital s reception or registry.none of the OPD consulting rooms (or Doctor s Rooms as they were called in those hospitals) had computers with Afyapro installed. Figure 9: Average Client Service time at OPD in six Tanga Hospitals The chart above shows that the longest service time at OPD was spent by clients at Bombo hospital and the shortest was at Lushoto hospital. It was expected that Bombo hospital would be providing services at a more complex level than the other five hospitals because Bombo is a regional referral hospital with more specialized services some of which might require more time. The five district hospitals would be expected to be similar on account of the level of services offered, and this is what appears to be the case for Handeni, Korogwe, and Muhezaas shown on the chart above. However, due to lack of a comparison, it was impossible to establish whether Afyapro installation would have any direct effect on client service time at OPD. 41

42 Client service time at Laboratory Only two laboratories were sampled to study the effect of Afyapro installation on service time, and this was mainly because none of the district hospitals had Afyapro installed in their laboratories. It would have been more appropriate to compare laboratories of the same level (such as two district hospitals) but this was not possible. An observation was made and recorded of the time lapse between a client s arrival at the laboratory s waiting lounge and the taking of specimen for which examination was requested. It did not include specimen processing and release of findings. Figure 10: Average client service time at laboratory in two Hospitals The chart above shows that clients spent an average of 17 minutes at Bombo hospital which had Afyapro compared to Lushoto hospital which did not have the Afyapro system. A regional referral hospital like Bombo would be expected to offer more complex laboratory services than a district hospital, and hence the spending of more time would be justifiable. An effect of Afyapro on the service duration could be better studied by comparing laboratories of the same level, or studying the same level before and after installation of the system. In the interviews with laboratory staff at Bombo hospital, the interviewed workers were of the opinion that the Afyapro installation did not make a difference in client service time because it didn t reduce the number of routine procedural tasks per client. They reported that despite Afyapro installation, they were still required to record the clients in the laboratory register book, which literally means Afyapro was an added task Client service time at Billing Sections Only three hospitals had sections designated specifically for billing. In the other hospitals, billing was merged with registration, hence these would not be comparable. All the specially designated billing sections had Afyapro installed. The service time at billing sections ranged from an average of three minutes at Lushoto Hospital to six minutes at Bombo hospital as shown in the figure below. The effect of Afyapro installation in these sections was impossible to determine quantitatively. From the interviews however, clients especially of Bombo 42

43 hospital acknowledged that they spent less time receiving their bills this year compared to three years ago. Cashiers at Bombo were happy with the way the Afyapro system generated bills and receipts automatically. The cashiers were however unhappy with the requirement at the hospital that clients should be issued with other hand written receipts in addition to those generated by the system, as this resulted to double work. In this case, it appeared that the time saved by the Afyapro system in generating those automatic receipts was spent again by cashiers in re-writing manual receipts! Figure 11: Average client service time at billing sections of three hospitals using the Afyapro system The table above shows Bombo clients spent more time at billing sections compared to the two other hospitals. Workers were of the opinion that it could take shorter than that if they did not have to re-write receipts manually Client Service time at Bima sections The six hospitals had specially designated sections for delivering services to clients of The National Health Insurance Fund, popularly known as Bima. Once a client declared at registration of the hospital that she was a Bima client, she would be directed to this section for services. None of the Bima sections had Afyapro installed. 43

44 Figure 12: Average client service time at Bima sections of three hospitals handling data manually Figure 12 above shows that clients spent the longest time in the Bima section of Handeni hospital. It was later reported that on the observation day there was no doctor available for the Bima section of Handeni, hence the very long waiting time observed. Due to lack of a comparison again, it was impossible to establish whether Afyapro installation would make a difference. Service providers at Bima section of Bombo hospital expressed a desire for a computerized data management system during the interviews, and they specifically wanted it to capture all the variables in the Bima forms, including the claim forms. 44

45 Client Service Time in all Sections in each Hospital Lushoto Figure 13: Client service time at different sections of Lushoto hospital by data management system Handeni Figure 14: Average client service time at different sections of Handeni hospital handling data manually 45

46 Korogwe Figure 15: Average client service time in sections of Korogwe hospital by data management system used Muheza Figure 16: Average client servicet time in sections of Muheza hospital by data management system used 46

47 Pangani Figure 17: Average client service time in sections of Pangani hospital by data management system used 47

48 Bombo Figure 18: Average client service time in sections of Bombo hospital by the data management system used Clients at the CTC of Bombo Hospital spent the longest time in the hospital, exceeding two hours. The CTC workers attributed to too much paper work due to the multiple and often repetitive data collection procedures for the clients visiting the CTC for services. 48

49 4.4. Cost efficiency of the CAHMA The cost efficiency of the CAHMA would have been measured by comparing the costs of investment per site to the outcomes. Documents detailing the amount of finances the GIZ had invested in the project since its inception were not available. In a presentation during the start of this evaluation exercise, an employee in the Tanga Regional Health Department mentioned that the CAHMA project had spent over TSh 150 million 5, but details to substantiate this expenditure were later not available. There was information also from the interviews that the initial investment for the CAHMA had been contributed through a DANIDA funded program some years before the GIZ took over, but it was not possible to quantify this investment because documents were lacking. It is also mentioned earlier in this report that the precise quantities of the project s outputs are not recorded (such as numbers of computers purchased, number of staff trained and the duration of the training etc). Since measurement of efficiency would require quantifying the values of both the inputs to the projects and the outputs 6, and since evidence was lacking in quantifying these, the efficiency was then impossible to measure Most significant changes related to the CAHMA An assessment of the most significant changes related to the CAHMA was approached from both clients and providers perspectives Changes as Experienced by Clients In order to establish whether clients had experienced any changes as a result of the project, exit interviews were conducted with an intent of determining clients experiences in the six hospitals before and after the CAHMA intervention. Six hospitals were visited and interviews were conducted with clients in each hospital. All interviewed clients had received services at the hospital and were met when they were about to exit from the hospital. In total, 193 clients were interviewed, of whom 113 (58.5%) were female. Their age ranged from 16 to 82 years, with a mean, median and mode of 38, 33 and 28 years respectively. Most of the respondents were therefore young. Table 1 below shows the numbers of interviewed clients for each hospital. 5 William Masika in a presentation entitled Computer Assisted Hospital Data Management System in Tanga during the pre-evaluation meeting conducted at Bombo hospital in Tanga on 17 th July Ovretveit J (1998): Evaluating Health Interventions. Philadelphia: Open University Press 49

50 Table 15: Distribution of the 193 interviewed clients by hospital Hospital Number of clients Percent Bombo % Handeni % Korogwe % Lushoto % Muheza % Pangani % Total % In responding to the question about whether they had seen a computer in use anywhere in the hospital, the clients gave answers as summarized in Table 2 below: Table 16: Clients who had witnesssed computers in use in the 6 hospitals SITE Client computer saw Client didn t computer see Client doesn t know computer Total Bombo Handeni Korogwe Lushoto Muheza Pangani

51 TOTAL Figure 19: Evidence of computer use in the six hospitals as witnessed by clients in Tanga Clients witnessed computer use mostly in Bombo and least in Handeni. The hospital sections which the clients had visited for service prior to the interview are summarized below. In many cases, clients were attended in more than one section of the hospital so a column total is not applicable in the table shown. Sections of the six hospitals where clients obtained services before the interview (n=193) Table 17: Sections of the six hospitals where clients obtained services before the interview Section of the hospital Number of clients Percent Registry

52 OPD Laboratory Pharmacy Most clients (about 96%) who visited the hospitals for service also received services from the pharmacy. It would be expected that all clients would pass through the Registry as an entry point to the hospitals but apparently some didn t (almost 11%). A total of 170 clients (88%) had prior experience with the hospitals in which they were interviewed in that they had visited the same hospital for service before. The table below shows details of clients prior experience with the hospitals. Last time client visited the hospital for service Table 18: Last time clients visited the hospital for service When was your last visit to this hospital Number of clients Percent About a week ago % About a month ago % About a year ago % More than a year ago % Total % Most clients had been to the hospital about a month previously. Nearly 54% of the clients had been attended at the hospital 5 years previously, hence would have experiences dating back to the times before the establishment of the CAHMA in these hospitals. Most of those clients who had a previous experience with the hospitals services did not notice any difference in the quality of services received on the day of the interview as compared to services they had received previously as shown in the figure below. 52

53 Figure 20: Clients' perception of the quality of services in six hospitals on the day of interviewas compared to their prior experience with the same hospital The figure above shows that two thirds of clients who had a previous experience with the hospitals felt that there was no change in the quality of services. Most clients in this group however, did not give any reason for their position, they just maintained that from their experience the situation was just the same as it had been for years. Only five clients in this group mentioned specifically that it had always taken them too long to receive services in the hospital. Most of those who said the services were better provided reasons for their perception as shown in the table below. 53

54 Reasons for clients perception of service quality in the six hospitals are summarized in the table below. Table 19: Reasons for clients' perception of service quality in the six hospitals Reason for the perception Worse today No difference Better today Total Not so much queuing Services are faster Services are generally improved Services have deteriorated Better record keeping Takes too long to receive service Service providers not committed to their work Too many patients TOTAL Most clients did not provide a reason for their perception of service quality. Among the 61 that had a reason to give, 46 (75%) said the services were better mainly because they were faster. Clients also used their previous experience with the hospital to judge the performance they noticed on the day of the interview, in which 99 clients (58%) reported they didn t notice any improvement in performance. Among the 71 who reported that they had noted an improvement, 58 clients attributed this improvement to better handling of patients data, especially the speed at which information and files are retrieved when needed. Comparing their hospital s record keeping system to other hospitals they had been to in Tanzania, most clients who had been to other hospitals felt that their hospital s system was just similar to other hospitals. However, almost half (48%) of the interviewed clients had never visited any other hospital in the country. Clients ranking of their hospitals record keeping system as compared to other hospitals intanzania 54

55 Table 20: Clients' ranking of their hospitals' record keeping system as compared to other hospitals in Tanzania Ranking Number of clients Percent Worse than other hospitals % Just like other hospitals % Better than other hospitals % Never been to any other hospital % Total % Figure 21: Clients queue up for service at the reception desk of Handeni Hospital. Each client carries a primary-school-style exercise book for keeping his/her records 55

56 56

57 Clients ranking of their hospitals record keeping system varied with each hospital as shown below: Table 21: Clients' ranking of their hospitals' record keeping system according to the hospital they attended Clients ranking of the hospitals record keeping system Hospital Worse than other hospitals Just like other hospitals Better than other hospitals Never been to other hospitals Total Bombo 3 (7.9%) 12 (31.6%) 16 (42.1%) 7 (18.4%) 38 (100%) Handeni 3 (9.7%) 5 (16.1%) 2 (6.5%) 21 (67.7%) 31 (100%) Korogwe 3 (10.0%) 13 (43.3%) 3 (10.0%) 11 (36.7%) 30 (100%) Lushoto 0 (0.0%) 8 (25.8%) 1 (3.2%) 22 (71.0%) 31 (100%) Muheza 3 (9.4%) 13 (40.6%) 0 (0.0%) 16 (50.0%) 32 (100%) Pangani 3 (10.0%) 9 (30.0%) 3 (10.0%) 15 (50.0%) 30 (100%) TOTAL 15 (7.8%) 60 (31.3%) 25 (13.0%) 92 (47.9%) 193 (100%) Bombo hospital had the largest proportion of clients (42.1%) who said the hospital s record keeping system was better than other hospitals, and the smallest proportion of patients (18.4%) who had never been to other hospitals. A total of 33 clients (17%) had witnessed or experienced at least one incidence of data loss during their encounter with the hospital. The years in which the incidences occurred are listed in the table below: 57

58 Clients experiences with data loss by year Table 22: Clients' experiences with data loss by year Year Number of clients Percent % % % % % % % Total % The table shows that most clients experienced file losses in the year Five clients did not remember when the data losses occurred. 58

59 Clients experience with hospital s responses to patients data loss, by year Table 23: Clients' experience with hospitals' responses to patient data losses Year Hospitals response to data loss Total Same file retrieved New file opened TOTAL The table above shows that the trend of file retrievals improved with time. In the year 2012, 40% of patients lost files were retrieved compared to the years 1995, 2004, 2005 and 2007 when the data for the lost files were not retrieved. Overall, 94% of clients said that they were generally satisfied with the manner the hospitals kept and managed patient records. 59

60 Significant changes due to CAHMA from health workers perspective Health workers of different levels (management, administration and clinical cadres) shared their experiences of the changes they had observed as a result of the CAHMA in their hospitals. The changes mentioned by most workers at Bombo hospital were: Reduced incidents of clients by-passing the established client flow pattern Improvements in revenue collection Better record keeping Easy retrieval of patient records Simplified report writing Most significant change in Pangani: Record keeping was very challenging before the establishment of CAHMA, where clients were allowed to take their records home. After the establishment of CAHMA, all patient information is kept in the hospital and is easily retrieved. Most significant change in Lushoto: No more replication of patient records, it is now possible to have only one file per patient It takes shorter to prepare reports No more clients by-pass the registration section of the hospital Handeni staff did not report any change due to the CAHMA because they have largely not been using the system in any section of the hospital. Most significant changes in Korogwe: Easy access to patient records when needed 60

61 Reduced loss of patient records Improved stock management, reports and ordering at Pharmacy The change in Muheza was mainly reported at the Registration section with easy retrieval of patient records and billing, although they used both the computerized and manual systems. Bombo hospital was noted as the only site where the CAHMA was utilized more regularly despite its late introduction. It was therefore justifiable that more changes resulting from the project were noticed in Bombo than in the other hospitals. One important impact of the CAHMA, though may not be one of the intended, was an increase in computer literacy among health workers in the implementing hospitals. Quite a good number of health workers especially those working in registry sections admitted that they were introduced to ICT, some of them for the first time. Those who had a faint idea of ICT had their skills improved. With the introduction of CAHMA (with Afyapro) in the hospital, Bombo hospital administration for instance noted that there were increases in numbers of staff expressing demands for ICT training not only for use in the system but also for general ICT. The project has therefore sparked ICT awareness among staff and given it more importance than initially held Major factors behind the results and outcomes At this point it is important to summarize the results before analyzing the factors behind them Main results In summary, the main findings of this evaluation are so far the following: The aims and objectives of the CAHMA are still very valid due to the persistently expressed need for a data management system and the commitment by some hospitals to initiate their own computerized data management system even before the establishment of the project. The original aims however are not in good fit with current needs and health care management challenges in the hospitals. It was impossible to track the activities and outputs of the project and compare them to project objectives due to lack of a project framework and monitoring indicators There were improvements in revenue trends of Bombo compared to two other hospitals, despite Bombo s decline in inpatient bed occupation and outpatient attendances, although a causal link between CAHMA and this change could not be established due to presence of confounders 61

62 Patients generally spent relatively less time in hospital sections which had the CAHMA compared to similar sections which handled data manually. Only two district hospitals fitted the comparison because they were using the system, the regional hospital had no comparable entity. A before-after analysis was impossible due to lack of baseline data There was no evidence linking staff retention in the hospitals to activities or outputs of the project Three district hospitals recorded a decline in outpatient attendances and an increase in inpatient bed occupancy, while Bombo hospital experienced a decline in both parameters. There was however no evidence to associate this change with Afyapro due to the multiple confounders. Although patients experience with file losses apparently increased, the chances of recovering the same records also increased. Bombo hospital was the only site where the Afyapro was utilized in the sections where it was installed with 64% of the hospital s revenue recorded in the Afyapro during its first year of implementation, which however dropped to 58% in the second year. The other hospitals did not make efficient use of the system. Although the project was perceived as beneficial by service providers, most clients did not notice any change in the quality of services as a result of the project Factors behind these results: Ownership of the objectives Although the objectives addressed some real challenges that hospitals experienced in handling patient data, they didn t integrate and consolidate the important issues related to hospital data collection and utilization as actually experienced in the hospitals. As a result of this lack of integration, hospitals continued to collect data using different systems and for different purposes such as the MTUHA data required by the Ministry of Health and Social Welfare, the District Health Information System data, Human Resources Information System, and the SONPAN used in Pharmacy. There are also other systems used by vertical programs such as the National Tuberculosis and Leprosy Programme, National AIDS Control Program and the Essential Program on Immunization. The implementation of the objectives as they were augmented redundancies in data handling and in some cases did not improve efficiency. The preparation of the hospitals annual reports for instance would require some data such as top ten diseases and immunization coverage which are not captured by the Afyapro system. Data users in the hospital continued to use multiple data sources in planning and decision making despite the presence of the CAHMA. It seems a thorough needs assessment was not done before the CAHMA intervention was incepted; there was no report on data management needs identified for the hospitals. It is 62

63 therefore difficult to tell whether the needs addressed by the CAHMA intervention are actually and specifically the most pressing issues related to data in those hospitals. Eventually, some of the project s objectives may have appeared as someone else s rather than the hospital s. A thorough needs assessment would have identified the specific areas to intervene as far as data management is concerned, and would have tailored the CAHMA intervention to suit the purposes for which hospital data is collected Absence of project logical framework and monitoring indicators The absence of the project s logical framework indicating the implementation plan and monitoring indicators for each objective suggests that the project was not properly planned. This has given an impression that a lot if not most of the project s activities were carried out on an ad-hoc basis, hence failing to maintain consistency between the activities and intended outcomes. There was apparently no quantification of resources required to achieve the desired level of efficiency in hospital operations, therefore CAHMA project s contribution was impossible to assess. A thorough situational analysis would have provided important clues on the state of hospital data management in Tanga as compared to an ideal, desired standard system that would address district, regional and national needs. Gaps identified in the situational analysis would then be used in designing an intervention. The intervention should have specific objectives derived from identified needs, and these should be in an implementation framework that has measurable indicators and includes stakeholders as implementers, not spectators. In most of the sites visited for this evaluation, it was common to hear providers and managers make such statements like GIZ is implementing this project in this hospital rather than we are implementing this project. Impliedly, there was no explicitly detailed plan for shared responsibility in the implementation of the project. Monitoring indicators would be used to track the extent to which each party in the implementation plan fulfilled their obligation. The absence of a situational analysis resulted to poor designing of the intervention (which I hesitate to refer to as project ) with no implementation framework nor monitoring indicators, resulting to misalignment of context and inputs to processes, with eventual exclusion of the same people whose problems the intervention purported to solve Project s influence on hospital efficiency The apparent absence of direct relationship between CAHMA and the hospitals efficiency resulted from the difficulty in controlling for the confounders that go with the indicators used in this assessment. It was assumed for instance that use of CAHMA would lead to increased revenue collections in the hospitals, reduce client service time, and reduce the frequency of patient data loss among others. However, there are many other factors that affect the same outcomes but are not addressed through the use of CAHMA. Hospital revenue results from the services it offers; hence it is a function of the price and quantities of services that clients purchase from the hospital. One of the characteristics of the health care market which distinguishes it from the ideal market is its unpredictability and 63

64 inelasticity of demand 78.Clients may continue to purchase a service as a result of perceived necessity, which means an added value to the service may not necessarily convince more clients to purchase it, or convince one client to purchase more of it. If a diabetes patient is supposed to attend the diabetes clinic once a month, an improvement in the quality of care in the clinic will not make him want more visits per month! In such a situation, revenue collection may not improve. An increase in hospital revenue collection may result from an increase in the number of people perceiving the necessity to seek services in the hospital, not because they want to, but because they have to. Revenue may therefore not be a very reliable outcome indicator for a single intervention like the CAHMA. Client service time depends on the totality of tasks that a health worker performs on or for a client in the specific service point. Some of these tasks may benefit from computerization, and many others may not. In some situations, the gains from computerization may be lost through impediments to performance, and examples of these were seen during this evaluation of the CAHMA project. A cashier would spend a minute or two generating a bill for a client, receiving the payment and issuing an electronic receipt, but government accounting procedures operating at the hospital would still require issuance of a handwritten receipt which would eventually neutralize the time gained and imply no time was gained. There were many examples like this in all the sites where the CAHMA was operating, of repeating manual recording of what was already electronically recorded, and in all cases the two systems (manual and electronic) were irreconcilable. Power failures and other technical challenges were often cited as reasons for frequently reverting back to the manual system Project s influence on quality of care Quality of care is one of the most challenging outcomes to measure. As McGlynn (1997) puts it, Quality is in the eyes of the beholder. This means balancing the competing views and needs of purchasers, patients and health care professionals 9. The fact that most clients in the CAHMA project implementation sites expressed an overall satisfaction with the quality of services offered despite the obvious shortages in equipment, supplies and staff is a further testimony to the difficulty of measuring quality especially from clients perspective. Almost the same levels of client satisfaction were expressed in sites with implementation of CAHMA as in those in which the system was at minimal or no use, implying that the CAHMA did not result in changes in client satisfaction. Different results could have been obtained by measuring satisfaction with a specific service Operational efficiency of the CAHMA It was generally noted that the Afyapro software was not fully utilized, not even at Bombo hospital which had better results than the other hospitals. The main reasons for poor or nonutilization of the system were: lack of commitment from hospital s leadership, poor technical know-how among staff and the leadership, lack of constant technical support, frequent power 7 Grossman, M. (1972), On the concept of health capital and demand for health, Journal of Political Economy 80(2): Weiner, J. (1993), The demand for physician services in a changing health care system: A synthesis, Medical Care Review 50(4): McGlynnE A(1997):Six challenges in measuring the quality of health care. Health Affairs, 16(3):

65 interruptions and the perceived increase (instead of a decrease) in workload due to repetition of records in the register books. Hospital leadership s commitment was the most serious factor. This was measured by the size of budget the hospital had committed to Afyapro since its introduction, and its willingness to handle CAHMA implementation problems. Bombo was the only hospital that committed its own funds in servicing the Afyapro. Other hospitals had budgets for computer maintenance, but not for those used for the CAHMA. In Lushoto hospital for example, the administration reported during the evaluation that they had not been using the Afyapro for a while because the switch linking the server to other service delivery points had broken down, and for this they were waiting for the GIZ technical team from Tanga to come and fix! If they had considered the system as their own, they would have been fully committed to its maintenance, including spending some money on it since the eventual benefits would be for the hospital. The scenario in Muheza was almost the same, where the administration was waiting for the GIZ to inform them on the next phases of the project. Administration and other staff in some of the other hospitals mentioned openly that the program was imposed on them from the CHMT. Interestingly, a respondent from the DMO s office for the same hospital which had CAHMA imposed on them said their office had absolutely no idea about the objectives of the project, and whether it was phasing out or not! In such a low level of commitment any project is likely to fail. There was also an element of inadequate technical skills required for smooth running of an ICT-based intervention. Most service providers in the project implementation areas admitted that they had little or no knowledge and skills in computer applications. Although the project conducted some training, there were complaints that the training excluded other users especially those at managerial levels. One manager at Muheza said he attempted to have himself oriented to the system through his own efforts but it didn t work easily. None of the managers in other hospitals reported such an attempt, and actually many were waiting for GIZ to move the project to the next phase (assuming there were implementation phases known only to GIZ). Some managers felt that they were left on their own too early (or rather prematurely) at a time when their technical skills and those of their service providers were not developed enough to sustain the project. Constancy of and access to technical support was a serious issue. Only one hospital (Bombo) had constant and assured access to technical support through a Computer Systems Analyst who is based at the hospital. The other hospitals reported they had to wait for very long for such support to be available. Lushoto for instance reported they had a computer technician employed in the District Council, but was not oriented to the Afyapro system so he could handle general technical issues, but not those specifically relating to the Afyapro system. Handeni hospital attributed their failure mostly to lack of technical support. Power interruptions were mentioned in all the six hospitals as a significant obstacle to constant use of the Afyapro. Whenever electric power went off, system users had to revert to the manual recording system, and always the manually recorded data was not entered into the electronic system. A power back-up system would have been useful in such a situation. The choice of a power back-up system should consider the duration of power cuts, which in some cases last during the whole day. The power interruption also interferes with the functioning of the system and the server.there are reports that some servers have 65

66 malfunctioned as a result electric faults. In such instances, the power fault affects the data as well, hence a data back-up would be required in addition to a power back-up. Data redundancy and the apparent requirement for double-work were also quite stressing. The registry sections of all the six hospitals had a manual registry alongsidethe electronic one. Most other sections had a similar situation. All this made the work appear more difficult with computers than without them More efficient alternatives in the implementation of a CAHMA Consideration of an alternative is justifiable only in the light of obvious deficiencies in the current system. For this reason, a technical assessment of the Afyapro software was conducted against the standard requirements of a data management system for a hospital. The assessment addressed the following areas: ICT infrastructure for HMIS (level of computerization, connectivity, maintenance etc Human Resource Capacity, specifically addressing the skills gap and the ICT maturity among the staff? Standards and interoperability issues for HMIS in relation to Tanga CAHMA Project Patients confidentiality and privacy Backup plan and maintenance plan Performance of the system Information flow analysis Areas assessed: The assessment covered the following HIS component System architecture Network architecture Maintenance plan Data security and Back up policy Information flow analysis 66

67 Data quality Data recording Data accuracy check Data processing and analysis HMIS system analysis System architecture Suitability Performance Usability The key components evaluated include installation and commissioning, end-user training, maintenance and support:- System Architecture The system is a desktop application three tier client server architecture that includes the user (client) interface layer, the business logic (transaction process) layer, and the database management (SQL Server) layer, This feature enables the system to support a large number of users without compromising performance. The observations have shown that in the five hospitals Afyapro-HMIS is running but there are differences in the used platforms. While in all the districts the version used is built on Visual basic,in Bombo Regional Hospital the front end used is C-Sharp although both applications are running in the same databases(mysql). The Bombo hospital therefore has the latest version of the system with more functionalities than the other hospitals. Afyapro was implemented together with stock management system in all district Hospitals (SONPAN) this system is either not working or doesn t have any technical support from the vendor,while at Bombo regional Hospital GIZ supported the implementation of WEBERP an intergrated (FMIS) but also the system is not functioning. 67

68 System Modules Medical Records Outpatient Module Pharmacy Module Laboratory Module Consulting Doctor Inventory Management Purchases Management None of the six hospitals had all these modules working. PERFOMANCE OF THE IMPLEMENTED HMIS The aim of implementing CAHMA was to eliminate data redundancy and offer faster data access within the Health facilities Due to the fact that the utilization of the system in all facilities is very minimal, the system failed to promote operational efficiency and improve satisfaction through provision of safer and better patient care. It also didn t meet the need of automating business process and integrate all departments as well as facilitate control of patient workflow. However the Afyapro-HMIS is versatile, easy to use software with a very intuitive user interface apart from the fact that the system isn t web-based, and it only runs on windows applications and it doesn t run in Linux and Mac s OS.. The system doesn t provide the benefits of streamlined operations whether reduce costs, increase quality of patient care( still double work to the most of the staff ), and gain the greatest value from information systems with sound strategic and tactical planning. The system hasn t synchronized across the entire enterprise supporting patient care, as a result although the implemented sites have automation, there is still a lot of fragmentations of 68

69 several systems within the Health facilities thus failed to improve operational efficiency, clinical workflow as well as clinician satisfaction. STANDARDS AND INTEROPERABILITY The system Afyapro was assessed according to the National HMIS standards requirements,where any (Electronic medical records system) should be able to provide the inputs to develop data sets needed for National HMIS datawarehouse (DHIS-2). The system can t meet this very important requirement as of now the MoHSW is rolling out DHIS as computerized HMIS and phasing out manual MTUHA reporting system. Database Back-up, Archiving and Maintenance plan It was only in Bombo hospital whereback-up and archiving are done, mainly due to the presence of Technical support. In all the remaining facilities there is no backup plan performed until the technical support person from Tanga goes for supervision. With the exception of Bombo hospital, there is generally no maintenance plan for the Afyapro. As a result, any technical fault or breakdown takes a long time to fix. Web Accessibility Afyapro has the challenge of not supporting web accessibility. Currently with an increase of newer innovations and technological advancement, the language used to develop Afyapro doesn t support remote access and accessibility via a PDA,tablets or smart phones. User Training There was no clear description of how the Consultant/Vendor trained the hospital members. All the district hospitals lacked a technical support personnel specifically trained for the Afyapro system. The vendor in collaboration with GIZ and TangaRHMT didn t prepare a Project implementation Plan which would show the project development lifecycle with the roles and responsibility of each stakeholder. Implementation plan of a Hospital information system involves engaging, planning, communicating and process improvement. In our assessment there is no evidence of 69

70 involvement of all stakeholders. In some hospitals e.g. Handeni, the staff attributed the system s underperformance to inadequate training, implementation and support plan. Data Flow Diagrams Information flows of all the public hospitals assessed are the same, but the Afyapro Vendor didn t explore the possibilities of simplifying the business processes. The manual information system can t be copied and be automated without business process re-engineering. Despite the automation done there is still a back-forth process of the patient while receiving services within the facilities (see the example of Bombo hospital patient flow charts attached) Environment The system s platform is limited and doesn t have capabilities to run on Linux or Mac OS; it runs only on windows. In district Hospitals Afyapro was supposed to be linked with SONPAN (Stock management system) but both systems don t have API (Application programming Interface) which allows integration of the different applications. Integration wasn t done and the Hospitals run different applications Security The system can facilitate user groups definitions and set access privileges for every user. The System isn t able to validate user s access privilege and record user signatures such as computation of IP address and time in every transaction. The user can login to a workstation using client local user account. Database access provides authentication on the database. To log into the system the user enters the login name and password; all usernames and passwords are encrypted and stored in the database. The system administrator has the option of providing selective read or writes access to the various parts of the software, which allows an almost unlimited number of security levels. Integration capabilities 70

71 The system is not able to provide an interface to facilitate interaction with third party service providers such as NHIF, Insurance Companies Systems, Corporate Client s databases; also the system can t be integrated with any third party devices like laboratory equipment etc. Human resource audit All visited hospitals had serious human resource challenges especially in relation to ICT skills. Most of the medical recorders were nursing assistants who didn t even have training in medical records technology, let alone computer literacy. In one of the sites, the Hospital was not using a filing system, instead they used exercise books (similar to those used for notetaking in primary schools) as their files. Data Management Data management component of CAHMA project is rated as not adequate at Tanga region (see quality of data assessment table). There is no written set of procedures for data management which are implemented within the region, which includes procedures for data collection and storage, data cleaning, quality control, processing and analysis. The hospitals are reporting to higher levels manually, and there is small demand from the districts and regional health management teams for computerized reports. Health data and information is of limited value until it is communicated to decision-makers in terms of issues confronting the health system, and is applied through the planning process to result in action and change. Therefore, dissemination and use of health information constituted an integral part of HMIS assessment. This component was rated as Not present and it is highly important. The assessment found that there is no continual demand for good quality and timely health information. It is usually demanded only on an ad-hoc basis, and in almost all such cases data is required on hard copy templates and forms with no emphasis on electronic reporting. 71

72 5.0. CONCLUSIONS AND RECOMMENDATIONS Conclusion 1: Although the aims and objectives of the CAHMA addressed real challenges in the health sector in Tanga, they didn t measure up to the totality of the purposes for which hospital data is collected in an integral manner. There was still a misalignment between data collection and data utilization. Recommendation 1: The aims and objectives of the CAHMA should be revised to reflect the current hospital management challenges. A revision of the objectives should be preceded by a needs assessment to identify a desirable hospital data management system that would fit the situations in which the hospitals operate while supporting the health system in its entirety that is integrating at all the levels in which the data is required. Conclusion 2: Some activities and outputs of the project were consistent with the project s objectives. These activities however did not feature in any implementation plan (which was also missing); hence the measurement of consistency lacked the component of timeliness. A standard was also lacking of the targeted outputs against the actual (observed) outputs were to be compared. This aspect of the evaluation was challenged by the lack of monitoring indicators in the project plan. Recommendation 2: The project should have a comprehensive plan with an implementation plan and a logical framework, clearly showing the types of expected outputs, their measurements and the time at which each should be measured. Baseline data at the start of project implementation would provide reference values for appraising change. Some of the indicators measured during this evaluation can be used as baseline values, but more should be developed and a detailed baseline study be conducted before scaling up this project. The variables to be used in the baseline study should be developed from the needs assessment study mentioned in recommendation 1 above Conclusion 3 The aims and objectives of CAHMA, namely those addressing improved business processes (patient management including registration and medical records, logistics and drugs management, and financial management in particular accounting) have not been achieved since there are still lots of redundancies and double-work in all these areas. There were improvements in revenue collection in cost sharing and drug revolving fund for Bombo Hospital which also happened to be the hospital with the best (though not optimal) utilization of the CAHMA), where up to 64% of its revenue was traceable in the Afyapro system. The 72

73 CAHMA may have produced a mechanism for stricter control of revenue at Bombo hospital, hence contributing towards increased revenue collection. This objective was not achieved in any of the other hospitals. Recommendation 3: Since hospital revenue is not an issue of better record keeping alone, revenue increases may not be a good indicator of the success of a data management system. Instead a system tracing revenue leakages can be used to check on contribution of the data system on revenue retention. In such a system, it would be possible to track the amount of revenue that would be lost in the absence of the data management system. Such a system can only work if all departments in the hospital with the potential to generate revenue are linked to a central revenue tracking system embedded in the CAHMA, so that the actual money collected for each product or service can be reconciled with the actual price and quantity of services provided. It is therefore recommended here that instead of computerizing many hospitals each of which partially, better results can be obtained by computerizing one hospital fully, linking all sections, then use the experience to scale up the intervention to other hospitals. Conclusion 4 The main factor behind the project s inability to measure up to expectations was the approach used, which delineated a significant proportion of the stakeholders turning them into beneficiaries rather than participants in generating the benefits. This was demonstrated by some hospitals lack of commitment to the project which apparently resulted from the lack of sense of ownership. Recommendation 4 In the re-designing of this project, it must be ensured that the intervention is owned locally, and such ownership is demonstrated by commitment of resources for its implementation by the participating hospitals. The level of commitment should be periodically monitored for its growth. Conclusion 5 The efficiency of the Afyapro system has not been realized because it has not been fully utilized. However, there are some areas in it which need improvement (as pointed out in the technical assessment) Recommendation 5 The technical issues raised in the software assessment should be addressed to make the CAHMA more efficient. This might require re-designing the Afyapro to match the current hospital data management needs, or developing a new software altogether that carries the potential to integrate, collect and collate, and promote hospital data utilization in all aspects for which it is currently utilized or intended to be utilized in future. 73

74 Conclusion 6 Although some of the aims and objectives of CAHMA were not achieved, there were some significant changes that were recorded in the implementing hospitals as a result of the CAHMA. One of such changes was an increase in the demand for ICT training and improvements in ICT literacy levels which have had positive impacts on staff morale, professionalism and proficiency in other tasks that require ICT skills. Recommendation 6 Hospitals should provide basic ICT training to their staff, so that it would cost less to train them later on applied ICT. The hospitals should maintain this positive impact by ensuring that the staff members who have received the ICT training are allocated duties which utilize those skills. 74

75 ANNEXES Annex 1: Hospital Client Flow Charts Bombo Regional Hospital. Flow charts narration for Outpatient department. Step I: The patient queues at the registration department for registration Step II: The hospital receptionist registers the patient in the system. Step III: The patient the goes to the billing department. If he/she falls into the cost sharing/cash the patient pays for services or if non cost sharing goes to Bima services. Step IV: The patient is given his/her medical file while the Bima clients are registered for Bima Step V: The patient goes for consultation and the Doctor will decide whether the patient needs investigations or medication. If the patient has been sent for investigations they will have to go back for consultation otherwise if they have been given prescription for medication they will go to step VI. Step VI: The patient leaves the Hospital. 75

76 Bombo Regional Hospital. Flow Charts for Outpatient Department Step I Patient queue for registration Step II Enter patient s details including billing Step III Yes No Cost Sharing/Billi Cost Sharing Non Cost Sharing Step IV OPD Card Queue for Bima Step V Pharmacy Consultatio n Investigation Pharmacy Consultatio n Investigation Step VI Exit Exit 76

77 Bombo Regional Hospital. Flow charts narration for inpatient department. Step I: The patient queues at the registration department for registration Step II: The hospital receptionist registers the patient in the system. Step III: The patient the goes to the billing department. If he/she falls into the cost sharing/cash the patient pays for services or if non cost sharing goes to Bima services. Step IV: The patient goes for consultation and the Doctor admits the patient. Step V: If the patient falls under cost sharing category they will then go for payments at the billing station then go to step VI. In case the patient falls under Bima they will then go straight to step VI. Step VI: Admission 77

78 Bombo Regional Hospital. Flow Charts for Inpatient Department. Step I Patient queue for registration Step II Registration Step III Yes No Billing Cost sharing/ Billing Bima Step IV Consultatio n Consultatio n Step V Billing Admission Step VI Admission 78

79 Bombo Regional Hospital. Flow charts narration for eye Department. This department operates on its own, it has a different information flow from other departments and it stores its own information. This is the observed information flow in the eye department. Step I: The patient queues at the registration department for registration Step II: The department receptionist registers the patient in the system and the patient pays for the file Step III: The patient goes for consultation and the Doctor decided whether the patient needs to go to theatre, refraction or medication. Step IV: The patient goes to billing to pay for the procedure or medication decided by the Doctor in Step III Step V: The patient then undergoes the procedure decided in Step III or goes to the pharmacy for medication Step VI: The patient leaves the hospital 79

80 Bombo Regional Hospital. Flow Charts for Eye Department. Step I Queue for registration Step II Registration and Billing Step III Consultation Theatre Refraction/ medication Step IV Billing Step V Procedure/pharmac y Step VI Exit 80

81 Bombo Regional Hospital. Flow charts narration for CTC Department. This department has its own way of keeping medical records. CTC clinics all over the country have unique Identification numbers for all their clients. All regions, districts and health facilities have unique codes. Currently NACP (National Aids Control Programme) introduced CTC-2 database and rolled out all over the country. The main functionality is to store demographic information as well as patient follow up. Meanwhile CTC-2 Database does not store electronic medical records of the patients thus there is a need to in cooperate CTC clinics in the EMR (Electronic Medical System Projects) The system uses MYSQL Database in the back end with MS Access in the front end; the system does not provide a link of information with other existing systems operating in the hospital. Step I: The client queues at the registration department for registration Step II: The department receptionist registers the client in the system. Step III: The client goes for consultation and the doctor decides whether the client should be given medication or go for investigations. If client was sent for investigation they will then have to go back to consultation. Step IV: Incase the doctor prescribed drugs, the patient will go to step V Step V: Pharmacy Step VI: The client leaves the hospital. 81

82 Bombo Regional Hospital. Flow Charts for CTC Department. Step I Queue for Registration Step II Registration Step III Consultation Investigation Step IV Pharmacy Step V Exit 82

83 Annex 2: Names of interviewed individuals Bombo Hospital Name Mr Patrick Mchami Happy Mahadia Bendera Dr Naima Dr FridaCassian Mr OmariKakuri Dr Minja Dr Fadhili Ms Hidaya Hussein Designation/section Regional Health Secretary Pharmacy Pharmacy Bima Eye department Laboratory technologist CTC OPD Registration Handeni District Hospital name UpendoMkalla RehemaZayumba Designation/section Health Secretary Laboratory HMT members (list them) 83

84 Korogwe District Hospital Dr Irira Frank Mhilu Peter Titus Victor Kabiyo ChedielMjema Acting Medical officer in Charge Hospital Secretary Clinical Officer Maintenance Officer Laboratory Manager C. K. Joseph AMO S. C. Ilembo Nursing Officer, CTC E. Kimweri Acting Matron C. Mkufya In-charge, Ward 3 M. Kusaga In-charge, Ward 4 A. Muya In-charge, ward 1 P. Mjema CTC coordinator V. Mhando In-charge, Ward 5 A. Mnguto In-charge, Ward 2 S. Mnauleke In-charge, medical records Aloyce Gregory Pharmacist Lushoto District Hospital name Jerome Malando Shakila Jumanne Designation/section Laboratory Labour Ward Jane Julius Ward 4 Andrew Zuakuu Grade 1 Levina Swai Dr Kabangube CTC in-charge Eye department K. Macha Clinical Officer 84

85 A. Baji ANO, ward 5 F. Msabaha Radiology M. Mweta RCHC E. Bendera Pharmacy Dr Beatrice Machuve Acting Medical Officer in-charge I. Maliyao ANO in-charge, Theatre Dr J. S. Shetumba Michael Mtoi Bima Medical Records Technician T. Tesha ANO S. Lupatu Hospital Secretary Muheza DDH name Dr RajabuMallahiyo Agatha Kombo Charles Titus Shaffii Dr Mbagga Designation/section Medical Officer in-charge Pharmacy Health Secretary Medical records Surgery Pangani district Hospital Name Dr R. Makoko Dr Yusuf Makange Sr Edith Chundu MwengereWaziri Dr EligiMsechu Designation/section CHMT CHMT CHMT Head, Laboratory TB coordinator 85

86 Timothy Mgaya AbdallahUssi JosephatMakombe John Ndunguru Christina Gao William S. Monyo Asia Mdoe CHMT Medical Officer in-charge Acting DMO Radiographer RCH in-charge MTUHA Acting CTC coordinator 86

87 Annex 3: Tools used in the evaluation Tool 1: Interview with Hospital Management Team Structured Questionnaires for Evaluation of CAHMA, Tanga region Part A: Introduction 1. Site 2. Users group : Hospital Management Team 1. Group composition: a. Gender: (numbers) male, female. b. Age (range): to years Part B: The interview/discussion 1. Let us discuss your experience with HMIS in this hospital a. What is your personal experience with HMIS? (probe for the experience before and after the introduction of the CAHMA) b. What do you know about the CAHMA project in in this hospital? (probe: objectives? Ownership?) 2. To what extent do you think these objectives have been met? 3. What are your desired characteristics of HIMS, i.e what do you wish HMIS could be or do? a. Desired features b. Desired system capabilities 4. To what extent does the CAHMA address the desired characteristics you have mentioned 5. What do you see as the benefits of the CAHMA for management in this hospital? (probe for its effect on staff retention and any evidence to substantiate this) 6. What are the challenges for maintaining the CAHMA in this hospital? (Probe for: financial implications, acceptability, user-friendliness, sustainability) 7. Data utilization and networking: a. How do you utilize the HIMS data collected here, ie what do you use it for? (probe for evidence of its use in management decisions, eg ask for minutes of at least one management meeting in which data from the CAHMA was used in decision making) 87

88 b. How is the HIMS data linked internally, ie with the other departments or sections of this facility c. How is the HIMS data linked externally, ie with the nationally HIMS collected data? (probe for conformity with national HIMS data collection formats) 8. How available is technical support? Do you have a computer technician in the hospital, district, regional secretariat? How available are these technicians for the service of the CAHMA? 9. Are you using different forms/ or planning systems for management? If yes, add examples. 10. How have you changed your reporting as a result of CAHMA? 11. Are there changes in the control of staff? If yes, what kind of changes? 12. How have you changed your way of managing the hospital as a result of the implementation? 13. Has professionalism of staff increased as a result of the capacity building and use of computer based hospital data management system. If so, how? 14. Do you wish HIMS in this facility was different? In what ways? THANK YOU VERY MUCH FOR YOUR TIME, WE WISH YOU ALL THE BEST. 88

89 Tool 2: Interview with Head of Sections Structured Questionnaires for Evaluation of CAHMA, Tanga region Part A: Introduction 3. Site 4. Section: 5. Respondent s title (position in the hospital): 6. Respondent s position (cadre) 7. Respondents age: Part B: The interview/discussion 15. Let us discuss your experience with HMIS in this hospital a. What is your personal experience with HMIS? (probe for the experience before and after the introduction of the CAHMA) b. What do you know about the CAHMA project in in this hospital? (probe: objectives? Ownership?) 16. To what extent do you think these objectives have been met? 17. What are your desired characteristics of HIMS, i.e what do you wish HMIS could be or do? a. Desired features b. Desired system capabilities 18. To what extent does the CAHMA address the desired characteristics you have mentioned 19. What do you see as the benefits of the CAHMA for management in this section? (probe for its effect on staff retention and any evidence to substantiate this) 20. What are the challenges for maintaining the CAHMA in this section? (Probe for: financial implications, acceptability, user-friendliness, sustainability) 21. Data utilization and networking: a. How do you utilize the HIMS data collected here, i.e. what do you use it for? (probe for evidence of its use in management decisions, e.g. ask for minutes of at least one management meeting in which data from the CAHMA was used in decision making) 89

90 b. How is the HIMS data linked internally, i.e. with the other departments or sections of this facility c. How is the HIMS data linked externally, i.e. with the nationally HIMS collected data? (probe for conformity with national HIMS data collection formats) 22. How available is technical support? Do you have a computer technician in the hospital, district and regional secretariat? How available are these technicians for the service of the CAHMA? 23. How have you changed your reporting as a result of CAHMA? 24. Are there changes in the control of staff in this section? If yes, what kind of changes? 25. How have you changed your way of managing your section as a result of the implementation? 26. Has professionalism of staff increased as a result of the capacity building and use of computer based hospital data management system? If so, how? 27. Do you wish HMIS in this facility was different? In what ways? THANK YOU VERY MUCH FOR YOUR TIME, WE WISH YOU ALL THE BEST. 90

91 Tool 3: Interview with Health Care Providers Structured Questionnaires for Evaluation of CAHMA, Tanga region Part A: Introduction 8. Site 9. Section: 10. Respondent s title (position in the hospital): 11. Respondent s position (cadre): 12. Respondents age: 13. When did you join this hospital? (year): Part B: The interview 1. Current needs related to hospital data management a. What kind of data do you often need when you provide services to your clients? b. What kind of data is required from you (which you are supposed to collect) when you provide services to your clients c. How does the CAHMA support you in a and b above? d. Please comment on the adequacy of the support that the CAHMA system provides for your work (does it attend to all your data management needs? How better would you like it to be?) 2. Actual project results and outcomes (as reported and as observed by stakeholders) What have you observed as the results of the introduction of the CAHMA system in this hospital? 3. Frequency of patient data losses before and after a. In your career as a health care provider have you witnessed instances of patient information loss at this hospital? b. How often did this occur before the introduction of the CAHMA? (skip this qn if employee was not in this hospital by then). Probe for frequency of reported losses per day, week or month c. How often does this occur in this hospital now? Probe for reported losses per day, week or month 4. Patient information retrieval time before and after 91

92 a. How would you describe the ease at which patient data was retrieved before the introduction of the CAHMA? Probe for time and convenience. (Skip if employee had not joined the hospital then) b. How would you describe the ease at which patient data is retrieved currently at this hospital? Probe for time and convenience. 5. Frequency and ease of patient information sharing before and after: a. How easy (or difficult) was it for you to share patient information between departments before the introduction of the CAHMA? b. How easy (or difficult) is it for you to share patient information between departments after the introduction of the CAHMA? 6. What have you observed or experienced as the benefits of the CAHMA project in this hospital? 7. What do you think are the reasons for the benefits you have mentioned above 8. Please give any other suggestions for the improvement of data management system in this hospital THANK YOU VERY MUCH FOR YOUR TIME, WE WISH YOU ALL THE BEST. 92

93 Tool 4: Observation Checklist for project activities 9. CHECKLIST FOR Observation of CAHMA project activities Part A: Introduction 14. Site 15. Section: Part B: The Observation 1. Actual on-going activities as observed during the evaluation (compared to those listed in the project logical framework for the particular time) CAHMA Project activities listed in project document Observed during the evaluation (Yes or No) Explanation 93

94 Tool 5: Observation checklist for service delivery time 10. CHECKLIST FOR Observation of service delivery Part A: Introduction 16. Site 17. Section: reception, consultation, pharmacy, laboratory and cash/payment points) Part B: The Observation (start from 0800am, stop at 0300pm) Procedure: Stay at the entry point to the service delivery point/section where every client starts to queue up for the service. Issue every newly arriving client with a numbered card and record the time you issued the card to him/her as TIME IN. Request the client to return the card to you when he/she has received the service at the particular station. When the client returns the card to you, record the time as TIME OUT. Use the 24hr system. Client number Time in Time out Time Spent Remarks 94

95 Tool 6: Client Exit Interviews Exit interviews with clients (English version) Part A: Introduction 18. Site 19. Respondents age: 20. Respondent s sex: Part B: The interview 9. Which section of this hospital did you visit for services today? (tick all that apply) a. Reception b. OPD c. Laboratory d. Pharmacy e. Other (please specify) Have you ever visited this hospital for service before? (Y/N) 11. When was the last time you visited this hospital for service? a. Within the past week b. Within the past month c. Within the past year d. More than a year ago 12. Did you visit this hospital for service 5 years ago? (Y/N) 13. How different was your experience with services in this hospital today compared to when you were here last time? a. Worse today b. Same as last visit c. Better today d. Never visited this hospital before 14. Please give reason(s) for your answer in qn 5 above: 95

96 15. How different was your experience in this hospital today compared to what it was 5 years ago? a. Worse today b. No difference c. Better today d. Never visited this hospital then 16. With reference to your previous experience with this hospital, have you noticed any improvement in the service delivery points (eg reception, laboratory, cashier) relating to handling of information? a. Yes b. No c. Never been here before 17. If your answer in 8 above is Yes, please explain: 18. With reference to other hospitals you have visited elsewhere in Tanzania, how would you rank this hospital s record keeping system? a. Worse than other hospitals b. Same as other hospitals c. Better than other hospitals d. I have never been to any other hospital 19. Please give reasons for the answer you gave for question 10 above: 20. Did you see a computer in use in any section of this hospital? a. Yes b. No 96

97 c. I don t know a computer 21. If your answer for qn 12 above is yes, please mention those sections: 22. From your experience with this hospital have you witnessed instances of patient information (eg file, prescription, card) loss? a. Yes b. No c. I have no previous experience with this hospital 23. If the answer for qn 14 above is yes, when did this happen to you? (Year) 24. How did the hospital deal with the information loss you reported in qn15 above? a. The same file/records were retrieved b. I was restarted on a new file 25. How satisfied are you with the way this hospital handles patient information? a. Very satisfied b. Satisfied c. Unsatisfied d. Very Unsatisfied 26. Please give any other suggestions for the improvement of patient information handling in this hospital THANK YOU VERY MUCH FOR YOUR TIME, WE WISH YOU ALL THE BEST. 97

98 Exit interviews with clients (Swahili version) Part A: Introduction 21. Site 22. Respondents age: 23. Respondent s sex: Part B: The interview 27. Naombaunitajievitengovyahospitalihiiulivyopitialeokwaajiliyahuduma (tick all that apply) a. Mapokezi b. OPD c. Maabara d. Pharmacy e. Nyingine (taja) Umewahikujatenakatikahospitalihiikwahudumakablayaleo? (NDIYO/HAPANA) 29. Ni linimarayamwishoulipokujakatikahospitalihiikwaajiliyahuduma? a. Ni kama wiki mojailiyopita b. Ni kamamwezimmojauliopita c. Ni kamamwakammojauliopita d. Ni zaidiyamwakammojauliopita 30. Uliwahikufikakatikahospitalihiikwaajiliyahudumamiaka 5 iliyopita? (Ndiyo/Hapana) 31. Umeonatofautiganikatikahudumaukilinganishahudumauliyopataleonaileuliyopataulipof ikahapamarayamwisho? a. Yaleonimbayazaidi b. Hainatofauti c. Yaleo bora zaidi d. Sijawahikupatahudumakatikahospitalihii 32. TafadhalitoamaelezoyajibulakokwaSali la 5 hapojuu: 98

99 33. Umeonatofautiganikatikahudumaukilinganishahudumauliyopataleonaileuliyopataulipof ikahapamiaka 5 iliyopita? a. Yaleonimbayazaidi b. Hainatofauti c. Yaleo bora zaidi d. Sikuwahikupatahudumakatikahospitalihiimiaka 5 iliyopita 34. Kwakuzingatiauzoefuwakowahospitalihii, leoulipofikahapaumeonanafuuyoyoteyautendajikazikatikavitengoulivyopitia (mapokezi, maabarank), hususankatikakushughulikiataarifazako? a. Ndiyo b. Hapana c. Sijawahikuhudumiwakatikahospitalihii 35. Kama jibu la swali la 8 hapojuuni ndiyo, naombaufafanue. 36. Ukilinganishanahospitalinyingineambazoumewahikutembeleahapa Tanzania, unauelezajemfumowautunzajiwataarifakatikahospitalihii? With reference to other hospitals you have visited elsewhere in Tanzania, how would you rank this hospital s record keeping system? a. Ni mbayakulikohospitalinyingine b. Ni sawanahospitalinyingine c. Ni bora kulikohospitalinyingine d. Sijawahikufikakatikahospitalinyiginezaidiyahii 37. Naombaunitajiesababuzajibuulilotoakatikaswali la 10 hapojuu: 38. Umeonakompyutaikitumikamahalipopotekatikavitengoulivyopitialeokatikahospitalihii? a. Ndiyo 99

100 b. Hapana c. Sifahamukompyuta 39. Kama jibu la swali la 12 hapojuuni ndiyo tafadhalitajavitengohivyo: 40. Kwauzoefuwakonahospitalihiiumewahikushuhudiatukiololote la upotevuwataarifazamgonjwa (mfanojalada, kadi, cheti)? a. Ndiyo b. Hapana c. Sinauzoefuwowotenahospitalihii 41. Kama jibu la swali la 14 hapojuuni ndiyo, nilinitukiohilolilitokea? (mwaka) 42. Hospitaliilishughulikiajetatizoulilotajakatikaswali la 15 hapojuu? a. Taarifa/jaladalilelilelilitafutwanakupatikana b. Nilianzishiwajaladajipya 43. Umeridhikajenanamnahospitalihiiinavyoshugulikiataarifazawagonjwa? a. Nimeridhikasana b. Nimeridhika c. Sijaridhika d. Sijaridhikahatakidogo 44. Tafadhalitoamaonimengineyoyoteyakuboreshanamnayakushughulikiataarifaza wagonjwakatikahospitalihii. ASANTE SANA KWA MUDA WAKO, UGUA POLE. 100

101 Tool 7: Checklist for technical assessment of the Afyapro software Desired Hospital information system Afyapro solution Requirements Remarks o Automate patient registration o Accomplish In-patient, Outpatient admission formalities o o o o o o Integrated view to Patients for Billing Collection, Discharge Details, Patient medical history etc Support adaptability and scalability of software making it more robust Authentication and verification of entries through Audit trail Facility Easy query handling for instant decision of Bed Allocation for patients, and request for Bed Transfers. Effective search facility to search any type of information related to patient history Graphical presentation of Data for top management analysis Availability of source code o Comprehensive performance reports o o o o o Built in workflow management for all functional areas Multiple store accounting Interface with various Laboratory Equipment for Data Capturing and Reporting Facilitate receipts and payments An integrated accounting system that automates financial activities. 101

102 Desired Hospital information system Requirements o Support company insurance registration & online billing Afyapro solution Remarks o Automate NHIF & Insurance claims o Easy to understand financial reporting including general ledger, accounts receivable, accounts payable with trace back feature o Integrated with smart card services o Facilitate online prescription, ordering and diagnosis requests o o Support multi store pharmacy Complete digitized medical records o User definable Pathology test reports o o o o o o Support ICD 10 & OPT Codes Automate order processing Keep track of stock movements; facilitate Bin card, Stock register, stock take, stock valuation, and analysis and consumption details Integrated General Ledger, Sales Ledger, Fixed Asset register, Purchase Ledger, receivables and payables. Enriched with superior graphical user interface Support bar-coding of inventory, assets, and specimens o Facilitate visiting doctors and package billing o o Night audit and auto billing Smart queries and FAQ and Search 102

103 Desired Hospital information system Requirements Engine o Fully integrated with sales, inventory, purchase, asset and financial accounting system o Automate purchase transactions including requisitions generation, approval purchase order generation, costing, supplier invoice posting, payment and purchase returns. Afyapro solution Remarks o o Facilitate invoice production, debit note, credit notes, customer statement, receipts Maintain information and transaction history of all customers and suppliers o Facilitate stock issues, delivery notes, job cards, sales invoice, stock history, sales returns and stock adjustments o Aging analysis of Outstanding details of Creditors and Debtors o o o o o Multi Currency Billing system with transaction and re-translation features Computerized treatment sheets for doctors and nurses Flexible management of all kinds of hospital activities System security at several levels. Control of data access, read, and write capabilities in the hands of the system administrator Flexibility has to be added that helps to customize the system for hospitals unique operating 103

104 Desired Hospital information system Requirements environment o System parameters should be engineered in such a way that they are understood by the users. Afyapro solution Remarks o o o o Many generic reports as well as a broadly recognized reporting tool for custom reports Elimination of data redundancy and offers faster data access User definable parameter driven system User definable hierarchy structure for document approval o Secure login and encrypted passwords with access privilege with multi levels SSL, client and database level authentication o Easy to install and user friendly 104

105 Tool 8: Checklist for data extraction from ledgers, cashbooks, annual reports, and registers CHECKLIST FOR data extraction from specific documents Part A: Introduction Site Section: Part B: The Observation 2. Cost of the CAHMA investment in the hospital Year Amount invested (TSh) Partners involved Percentage of the hospital revenue that is recorded in the CAHMA Year Hospital revenue recorded in the CAHMA (in TSh) Hospital revenue as verified in receipt books(in TSh) 4. Percentage of hospital budget set for CAHMA maintenance 105

106 Year Hospital s budget (TSh) Allocation for CAHMA (Tsh)