Document of The World Bank FOR OFFICIAL USE ONLY INDONESIA FIFTH POPULATION PROJECT (FAMILY PLANNING AND SAFE MOTHERHOOD) LOAN 3298-IND.

Transcription

1 Document of The World Bank FOR OFFICIAL USE ONLY Report No IMPLEMENTATION COMPLETION REPORT INDONESIA FIFTH POPULATION PROJECT (FAMILY PLANNING AND SAFE MOTHERHOOD) LOAN 3298-IND June 26, 1997 Population and Human Resources Division East Asia and Pacific Department East Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

2 CURRENCY EQUIVALENT Currency Unit = Rupiah At Appraisal in 1990 US$1.00 = RP 1,901 Rp 1 million = US$ 526 Annual Average ,950 2,030 2,080 2,160 2,237 2,372 FISCAL YEAR OF BORROWER April 1 - March 31 ABBREVIATIONS BDDs BKKBN DHS GOI ICR IEC LSS MIS MOH NGOs NOLs NPV TFR Village Midwives Family Planning Coordinating Board Demographic and Health Survey Government of Indonesia Implementation Completion Report Information, Education and Communication Life Saving Skills Management Information System Ministry of Health Non-Government Organizations No Objection Letters Net Present Value Total Fertility Rate Vice President Director Division Chief Task Manager Jean-Michel Severino, EAP Marianne Haug, EA3DR Samuel Lieberman, EA3PH Fadia Saadah, EA3PH

3 FOR OFFICIAL USE ONLY TABLE OF CONTENTS Preface... Evaluation Summary...ii i PART I: PROJECT EVALUATION ASSESSMENT A. Statement of Objectives.I B. Achievement of Objectives.2 C. Major Factors Affecting the Project.7 D. Project Sustainability.8 E. Bank Performance.9 F. Borrower Performance G. Assessment of Outcome.10 H. Future Operations.10 I. Key Lessons Learned.11 PART II: STATISTICAL TABLES Table 1: Summary of Assessment Table 2: Related Bank Loans Table 3: Project Timetable.16 Table 4: Loan Disbursements: Cumulative Estimated and Actual.16 Table 5: Table 6: Studies Included in Project.19 Table 7A: Project Costs.22 Table 7B: Project Financing.22 Key Indicators for Project Implementation.17 Table 8: Status of Legal Covenants.23 Table 9: Bank Resources: Staff Inputs Table 10: Bank Resources: Missions.25 PART III: BORROWER'S ASSESSMENT This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

4 I

5 IMPLEMENTATION COMPLETION REPORT INDONESIA FIFTH POPULATION PROJECT (Family Planning and Safe Motherhood) LOAN 3298-IND Preface This is the Implementation Completion Report (ICR) for the Fifth Population Project (Family Planning and Safe Motherhood), for which Loan 3298-IND in the amount of US$ 104 million equivalent was approved on March 5, 1991 and made effective on June 20, The loan was closed on schedule on September 30, 1996, with an informal extension through January 31, 1997 to allow for payment of eligible expenditures incurred on or before the closing date. The final disbursement took place on January 28, 1997, at which time a balance of US$ 3.9 million was canceled. (There was a previous cancellation of US$0.2 million in November 1995). No cofinancing for the project was provided. The ICR was prepared by Eduard Bos, Population Specialist in HDDHE and Fadia Saadah, Population and Health Specialist, EA3PH. The report was reviewed by Samuel Lieberman, Chief, Human Resources Operations Division, EA3PH, and Oscar de Bruyn Kops, Acting Project Advisor, EA3DR. Preparation of this ICR was based on a mission carried out during November 1996, on materials in the project file, and on data and reports provided by the implementing agencies. The Borrower contributed to the ICR by preparing an evaluation of the project and by commenting on the draft ICR.

6 ii IMPLEMENTATION COMPLETION REPORT INDONESIA FIFTH POPULATION PROJECT (Family Planning and Safe Motherhood) LOAN 3298-IND Evaluation Summary Introduction i. By the late 1980s, Indonesia was well advanced in its fertility transition with the total fertility rate (TFR) having fallen from 5.6 to 2.9 (Demographic and Health Survey; 1994) in the previous two decades. This fertility decline was due in no small measure to the activities of the country's family planning program which the Bank had supported through four earlier loans. Under the leadership of the Family Planning Coordinating Board (BKKBN), family planning became sanctioned and accepted normatively, widely practiced, and institutionalized, the hallmarks of an advanced program. Accordingly, a key aspect of the context when this operation was designed was the need to refocus the agenda of an agency -- BKKBN -- (which became a ministry with a wider mandate early in project implementation) and to pursue goals and initiatives in keeping with the requirements of a maturing program. ii. A second important feature of the context when the project was designed was Indonesia's more limited achievements as regards morbidity and mortality due to reproductive causes. The project was prepared when the government's initial policies as regards safe motherhood were being formulated and comprised a first step towards improving reproductive health services. These initiatives preceded by several years the 1994 International Conference on Population and Development in Cairo. However, the Cairo meeting and its aftermath encouraged Indonesian policy makers to press ahead with reproductive health interventions, a decision which became part of the background for this project. Project Objectives iii. Of the project's two broad objectives, the first was to assist BKKBN in carrying out its tasks and responsibilities in a setting defined by a maturing family planning movement. These included improving access to contraceptive services for remaining difficult-to-reach groups, improving the clinical quality of services, and responding through alternative approaches to service delivery and communications campaigns to the diversity of clients' needs and to the growing number of contraceptive delivery channels. The second objective was to help the Government of Indonesia (GOI) expand access to maternal health services by training and deploying village midwives. This was a crucial part of the government's overall approach to making motherhood safer.

7 iii iv. Corresponding to these objectives, the project had the following components: PartA (BKKBN) * Targetedfamily planning and safe motherhoodpromotion activities intended to improve service accessibility and utilization among the urban poor and populations in coastal and transmigration areas; promote family planning services in the organized sector; improve quality of family planning services; and to publicize safe motherhood interventions. * Information, education and communication (IEC) and community outreach designed to support improvements in BKKBN's IEC strategy and system capacity, promote participation of youth in family planning and develop community institutions in specific areas. + Institutional development that provided for staff development training activities, reform of BKKBN's management information system (MIS), evaluation and research capacity and strengthening of the field operations through the provision of vehicles, equipment, contraceptives and other materials. Part B (Ministry of Health) * Strengthening ofpolicyframework on the objectives and related training and planning principles that would govern the deployment of village midwives (BDDs). * Strengthening training capacity through training of trainers and improving teaching materials and equipment. * Training and improving the effectiveness of about 16,000 BDDs, including establishing accreditation standards for midwifery training schools and certifications procedures for BDDs. v. The objectives were carefully formulated, drawing on the findings of a major sector study completed in The project's family planning components responded to the needs of a maturing family planning program, while the operation's support for midwife training and policy research was in keeping with GOI's decision to improve maternal health. The Bank's involvement was consistent with its country assistance strategy and its approach to human resource development in Indonesia. Implementation Experience and Results vi. The loan was approved by the Board in May 1991, became effective on June 20, 1991, and was closed on schedule on September 30, Total project costs were US $104.0 million, of which US$ 4.1 million was canceled. The project achieved its objectives and all of the major components in Part A and Part B were implemented in a satisfactory manner.

8 iv vii. The project recorded a number of achievements. Components dealing with targeted family planning services were implemented and provided valuable insights on how to improve services in these hard-to-reach areas. The importance of quality in family planning services was documented through project-funded research. Quality improvements in contraceptive services were brought about through expanded method choice (via procurement of contraceptives), and by upgrading through training the clinical and counseling skills of providers. The challenge of removal needs for implants was addressed after project-supported research revealed the magnitude of the issue. Capacity in IEC development and delivery was strengthened through equipment purchase and audience research. Institutional development was accomplished by updating and streamlining the MIS, in-country and overseas training for BKKBN staff, and operations research on various topics, including those just cited. viii. Part B attained the policy objectives concerned with clarifying the roles of midwives and setting up an accreditation mechanism for midwife schools and a ministerial decree defining the role of midwives was issued. Training capacity was enhanced through training of trainers and procurement of teaching equipment and materials and about 16,000 village midwives were trained and deployed. ix. In terms of broad outcomes, the project's design as a de facto program loan to support BKKBN's entire effort militates against measuring project specific impacts on fertility and maternal mortality and morbidity. However, it is reassuring that key fertility and mortality indicators moved in the desired direction during implementation. The project also yielded interesting results on the institutional front. This operation was one of the first for the Bank in Indonesia to concentrate on support for software, i.e., training, studies, and so forth, and provided useful lessons in this regard (see para. xiii below). The flexibility built into the project allowed for key activities to be revised several times to reflect changing needs and priorities, e.g., expanding efforts related to improving clinical skills of health providers and enhancing the quality of midwifery training. However, this flexibility was not always used optimally. This was due partly to continuing rigidities in GOI's planning and budgeting procedures, also because of the absence of clearly defined goals and targets for some interventions, and because managers within BKKBN and MOH gave higher priority to implementing long planned activities than to developing and pursuing alternative strategies. Summary of Findings, Future Operations, and Key Lessons Learned x. The project achieved its objectives. Overall performance by both the Borrower and the Bank were rated satisfactory. The Borrower's implementation performance was rated highly satisfactory. xi. The investment appears, on balance, to be sustainable. Key components and activities relating to service and information provision and quality improvements have been absorbed into BKKBN's ongoing program. As for Part B activities (MOH), follow up activities and projects have been designed to address factors bearing on the professional and economic viability of village midwives.

9 v xii. Key lessons learned: Substantive issues * Local specificity of family planning interventions -- Addressing needs of specific population groups as well as increasing the effectiveness of existing programs requires plans that better reflect local needs and conditions. This was clearly illustrated in the component dealing with targeted family planning. * Quality of family planning and maternal health services. Although the initial project design included interventions to improve quality of family planning and maternal health services, the mid-term review of the project as well as other sector studies indicated that the unfinished agenda was significant. In addition, improving quality of services is essential for addressing the unfinished agenda in family planning and maternal health services. In response, project resources were reprogrammed to place more emphasis on improving quality of services. Additional programs for improving quality of care have been launched in follow-up initiatives. * Effectiveness and sustainability of BDDs. As efforts to train and deploy BDDs were underway, it became apparent that the completion of the BDD program (as initially designed) will not be adequate to address maternal health needs. The BDDs were not adequately trained and many had difficulties in gaining credibility as health providers and integrating into the village community. Moreover, it was not clear if these BDDs would be sustained in the villages after the completion of their three (or six) year contracts. GOI needs to address this important policy issue in the near future. * Linkages between supply- and demand- side interventions. The project financed pilots to address demand and supply side interventions related to safe motherhood. These pilots were effective in demonstrating the need for intensifying such efforts and for enhancing the coordination of demand and supply side activities -- a lesson that has been incorporated in follow-up operations. Implementation/process issues * Investment projects as tools for addressing policy reforms. Projects can be useful in addressing some policy issues if (a) they have very clearly defined objectives and linkages to the program; (b) there is strong ownership by the government; and (c) they are very closely supervised. This was illustrated in the component dealing with midwifery policy issues. * Annual Planning and flexibility in project design. The project adopted an annual planning process that allowed for flexibility in project implementation and strengthened the ability of the project to respond to sector issues that emerged during execution. However, the lack of flexibility in some government procedures (especially the DIP process) and the lack of well identified monitoring indicators limited the optimal use of this feature. Although the lack of felxibility in the planning and budgeting process is a systemic issue that affects most sectors in Indonesia, future operations and discussions with GOI should continue to address this constraint.

10 vi * Performance-based-contracts for research. The completion and/or quality of some studies did not always meet the expected standards. Future projects may wish to consider performance-based contracts that ensure the quality and timeliness of the delivery of the studies. * Ownership/leadership. The project enjoyed the support of strong leadership at BKKBN and high degree of ownership among its implementing agencies. These were key elements in its success and timely completion.

11 IMPLEMENTATION COMPLETION REPORT INDONESIA FIFTH POPULATION PROJECT (Family Planning and Safe Motherhood) LOAN 3298-IND Part I: PROJECT EVALUATION ASSESSMENT A. Statement and Evaluation of Objectives 1. The project's objectives were to support the strategies of the Government of Indonesia (GOI) in family planning and maternal health. Project activities were intended to increase access to family planning methods, to improve the quality of family planning services, and to improve maternal health through expanded community health services. The Bank's involvement was consistent with its strategy for assistance in human resource development in Indonesia, which emphasized a focus on less developed areas and on the poor. 2. Project objectives were developed in a context of successful realization of GOI's longstanding family planning goals. Thus project design focused on the need to refocus the agenda of an agency and to pursue goals and initiatives driven by the requirements of a maturing program. A second important feature of the context when this operation was designed was Indonesia's more limited achievements as regards morbidity and mortality due to reproductive causes. Moreover, the project was prepared when the government's initial policies as regards safe motherhood were being formulated and comprised the first step towards improving reproductive health services. 3. The specific objectives of the project were first to assist BKKBN in carrying out its tasks and responsibilities in a setting defined by a maturing family planning movement. These included improving access to contraceptive services for remaining difficult to reach groups, upgrading the clinical quality of services, and responding through alternative approaches to service delivery and communications campaigns to the diversity of clients' needs and to the growing number of contraceptive delivery channels. The second objective was to help GOI expand access to maternal health services and training and deploying village midwives. This was a crucial part of the government's overall approach to making motherhood safer. 4. Corresponding to these objectives, the project had the following components: Part A (BKKBN) * Targetedfamily planning and safe motherhood promotion activities intended to improve service accessibility and utilization among the urban poor and populations in coastal and transmigration areas; promote family planning services in the organized sector; improve quality of family planning services; and to publicize safe motherhood interventions.

12 Part B (MOH) 2 * Information, education and communication (IEC) and community outreach designed to support improvements in BKKBN's IEC strategy and system capacity, promote participation of youth in family planning and development of community institutions in specific areas. * Institutional development that provided for staff development, reform of BKKBN's management information system (MIS), evaluation and research capacity and strengthening of the field operations through the provision of vehicles, equipment, contraceptives and other materials. + Strengthening ofpolicy framework on the objectives and related training and planning principles that would govern the deployment of village midwives (BDDs). * Strengthening training capacity through training of trainers and improving teaching materials and equipment. * Training and improving the effectiveness of about 16,000 BDDs, including establishing accreditation standards for midwifery training schools and certifications procedures for BDDs. 5. The objectives were carefully formulated, drawing on the findings of a major sector study completed in The project's family planning components responded to the needs of a maturing family planning program, while the operation's support for midwife training and policy research was in keeping with GOI's decision to improve maternal health. The Bank's involvement was consistent with its country assistance strategy and its approach to human resource development in Indonesia. B. Achievement of Objectives 6. The project substantially achieved its major objectives for both Parts A and B. An assessment of the contribution of the project towards the objectives is provided below for each component. Part A: Family Planning Program (BKKBN) 7. Expansion of family planning services in hard-to-reach areas: This component included: targeted services to the coastal areas, transmigration areas, the organized sector, and the urban poor. The exact impact of these services cannot be measured for the specific areas. However, the results of the Demographic and Health Survey' results indicate that, overall, the achievements of the family planning program are consistent with the project objectives. For instance, the contraceptive Indonesia Demographic and Health Survey Central Bureau of Statistics; State Ministry of Population/National Fanily Planning Coordinating Board; Ministry of Health; and Demographic and Health Institute- Macro International Inc. October, 1995.

13 3 prevalence for modern methods increased from about 47 percent in 1991 to about 52 percent in 1994; unmet need for family planning services dropped from 12.7 percent in 1991 to 10.6 percent in 1994 among married women and from 13.9 percent in 1991 to 11.7 percent in 1994 among married women with no education. 8. Implementation of planned activities for the coastal and transmigration areas was completed. As these activities occurred in small areas in districts scattered across Indonesia, an accurate estimate of the increase in contraceptive use is unknown. However, these relatively small scale efforts provided important lessons for improving the effectiveness of family planning services that benefited the general programs dealing with these issues. For instance, the project demonstrated that the effectiveness of the interventions is greater if local conditions and needs were better reflected in the project annual plans -- a finding that guided the project implementation after its mid-term review. 9. The third component dealing with targeted service delivery focused on family planning services for the organized sector. The implementation targets for this component exceed the appraisal estimates. The project supported provision of family planning services and IEC to employees of more than 7,000 businesses. This model has worked better in larger firms with existing company clinics. Moreover, as the activities of this component were completed, a health insurance scheme for organized sector workers expanded its coverage to include family planning, ensuring sustainability of this approach. 10. The achievements of the pilots for delivery of family planning services to poor urban areas were more limited. Delays in implementation resulted from the difficulties encountered in defining the urban poor communities. While implementation of this components was achieved, the effectiveness of the activities could have been enhanced if (a) contractual arrangements with NGOs to deliver services for the urban poor were conducted in a more competitive manner and were completed more efficiently; and (b) more clearly defined outputs and indicators were defined at appraisal. 11. Safe motherhood promotion. Activities to publicize safe motherhood interventions were carried out by BKKBN. The original plan was that such demand-side interventions will support and supplement the activities supported under Part B of the project -- i.e., supply-side interventions. However, the demand- and supply-side interventions were not fully coordinated. 12. Improving the quality of family planning services. The project made important contributions towards improving the quality of family planning services through improving the method mix by procurement of long-term contraceptive methods, training providers in service delivery, and improving counseling skills of providers. These activities were completed and about 15,000 doctors, midwives, and family planning field workers were trained in clinical skills and counseling 13. After the start of the project, it became clear that not enough providers were available for skilled removal of implants, which had become a popular contraceptive method in Indonesia, after the end of its effective period. Furthermore, recording and tracking of acceptors of implants was

14 4 inadequate. These issues were exacerbated by the rapid expansion in implant use during the first two years of the project. Several initiatives were undertaken to address these issues. Studies financed by the project were carried out on the magnitude of the implant use, resulting in projections of the expected number of removals needed in the future. Training materials were produced and training of doctors and midwives in removal of implants was carried out. The addition of midwives trained in removal of implants may be expected to alleviate the backlog of women who did not have the removal done after five years (estimated at 10 percent of the 1990 cohort of acceptors). Future efforts need to continue efforts to improve tracking of implant acceptors and developing the implant strategy. 14. The importance of quality for a successful family planning program cannot be overstated. Research conducted as part of the project indicated that health concerns and side effects were the main reasons for discontinuation of a method or for not using one at all. Counseling and follow-up could in many cases address such concerns by directing couples to the most appropriate contraceptive method. One of the Norplant studies supported by the project found that women who had access to removal on demand were more likely to continue practicing family planning than those who did not. And while proximity to a service provider was the most important reason for choosing a source of contraceptive supplies, the competence of staff and the quality of the facility were also found to be important reasons. The documentation of the importance of quality and the project's efforts to improve quality must be seen as among the most important contributions of the project. 15. Improving LEC and youth outreach. Dissemination of information is an appropriate area for a mature program to be active in, and the subcomponent on youth outreach were particularly relevant. This component was designed to pilot and develop new approaches, strengthen IEC message design capabilities, and carry out audience research. The equipment was procured as planned and most activities were undertaken. However, the province specific audience research and market segmentation were only completed in the second half of the project. Thus, the expected changes in the content of the IEC messages did not materialize during implementation. Nevertheless, the investments made in this project are likely to be useful for future program development, especially as the country implements its decentralized strategy. 16. Institutional development. The training subcomponent was designed to strengthen skills at all levels of the family planning program to meet the needs of a mature program and to allow the program to change strategically. Staff development was seen as an important organizational tool when technological changes and external developments imposed new requirements. In-country training and refresher courses appeared to have been particularly successful for upgrading skills of BKKBN staff. The project supported about 132 Master degree and 11 Ph.D. students overseas. In addition, 3290 candidates received D3 degrees, 406 received masters degrees and 124 candidates received post-graduate degrees in country. Moreover, a large number of staff received in-service technical training in family planning related fields. Although several of these training activities were completed and the educational level of the staff was upgraded, BKKBN did not develop a clear manpower strategy to guide the staff development program. As a result, BKKBN did not train adequate numbers of staff in areas like reproductive and maternal health or adolescent reproductive

15 issues. Moreover, the mangers of the staff development program were generally more concerned with completion of the implementation as opposed to the effectiveness of the program. 17. Program monitoring and research. The research subcomponent was intended to build up research capacity in BKKBN, while simultaneously providing useful information for program implementation. A total of 44 research activities was carried out under Part A, most of which were directly linked to program issues. The 1994 DHS funded by the project has become a valuable reference for health and mortality information for many other purposes, especially service delivery related issues. Studies carried out on implants also provided useful information for program delivery, leading to additional efforts to improve program quality 2. Although research topics and terms of reference were generally good, the final quality and/or the completion of the relatively small scale studies and operational research did not meet expected standards The process for developing and seeking approval for a large number of research proposals proved to be burdensome for the Borrower, and the process of reviewing the proposals was timeconsuming for the Bank. As a result, obtaining the required approvals from the Bank for each proposal often resulted in delays in conducting the research. This was at times due to the inappropriateness or other deficiencies of the proposed studies, at other times this was due to administrative delays. Providing technical assistance at the design and/or report writing stages of research projects as well as performance-based-contracts for such projects could be useful tools that may be adopted by future projects to enhance the quality of Bank funded research. 19. The revision in the MIS to improve monitoring was successfully implemented through simplified reporting requirements and upgraded computer hardware. As a result, the number of data items in the recording and reporting system decreased from 367 to 240. Future challenges for the MIS system involve the growing role of the private sector in service delivery and identifying needed information from the private sector for program planning purposes. Part B: Midwives Development (MOH) 20. Strengthening the Policy Framework. One of the most important successes of Part B was the institutionalization of Ministerial Decree 572 that defines the role of the midwife, thereby legitimizing her role as a (private) health provider. The Decree also requires that the licenses for midwives be renewed every five years -- a measure that can be used to ensure quality control of midwifery practice. A working group has been established to operationalize the Decree and address issues related to its implementation. However, the establishment and implementation of midwifery standards are still in the early stages of development with assistance from WHO. Moreover, the establishment of a Midwifery Board, another objective of this subcomponent, did not materialize 2 This study represents an important contribution to the literature in its approach and methodology. It utilizes data that is not available in most other countries and provides important insights to the introduction and implementation of Norplant contraceptives.

16 despite several efforts and activities to create such an entity. It is likely that future projects dealing with health professionals will deal with the issue of Boards for health professionals including midwives. 21. The project also provided support for study tours related to midwifery training and deployment and a series of evaluation studies of the different midwifery training programs. These evaluative efforts provided useful information for guiding the next steps needed for improving the quality of midwifery services. Many of the recommendations put forward by these evaluation studies have already resulted in important follow-up action by MOH including a revised in-service training strategy for village midwives Strengthening the training capacity. The project was successful in developing an accreditation system for midwifery schools. By the time the project closed, this was under implementation, with 10 percent of the schools already receiving an accreditation rating. However, the implementation of the accreditation system remains a reactive process that is dependent upon individual initiative from the schools to complete the self-assessment. Accreditation needs to be more proactive for it to be effective in standardizing quality. 23. As part of the support for strengthening the training capacity, the project funded curriculum revisions. Two sets of curriculum changes were introduced in 1991 and The major difference was omitting the two-year work experience requirement before acceptance into the midwifery program. Although the 1994 curriculum is more systematic and process oriented than the 1991 program, further revisions may be needed in the future to further enhance the quality of midwifery training. 24. Another constraint affecting training quality and capacity related to the quality of trainers and teaching materials. The project responded by supporting several training programs to enhance the capacity of teachers, especially following the decree that accelerated the training and deployment of community midwives and created an immediate need to train more teachers. Forty two midwifery teachers were sent overseas for a course in education techniques and advanced midwifery, and another 20 teachers were sent for short-courses in problem-based learning in midwifery education. In-country training was also provided for midwifery teachers and clinical instructors. Although these efforts were critical to strengthening the quality of the clinical training, they could not compensate for the low exposure of the midwifery students to clinical cases -- a consequence of GOI's acceleration of the training of village midwives. 25. Training and improving effectiveness of community midwives. The planned target of training about 16,000 community midwives was achieved. In addition, the project supported efforts to develop in-service training on life savings skills (LSS) to improve the skills of village midwives. Assessment of the LSS training pointed to the need to develop an in-service training strategy for these midwives -- an effort that has already started as part of the preparation of the upcoming Safe Motherhood Project.

17 7 26. As for the deployment of community midwives, a position for a Midwife Coordinator, located at the district and sub-district level, was created to assist with management of the community midwives after deployment. However, the role and effectiveness of these Coordinators still would require future evaluation, especially since many community midwives still receive little supervision and support. 27. In addition, the project supported the establishment of a Midwife Baseline Data registration that allows for tracking midwives after their deployment. Minimal information is recorded such as: name, age, date graduated, and employment location. Computers were sent to 26 provinces but apparently the system is currently underutilized. 28. Project management. At BKKBN, a Project Secretariat was established during the first year of the project, and the Deputy for Program Planning and Analysis was the Project Director. During year 3 of the project, the Secretariat was dismantled, and project activities were institutionalized within the BKKBN organizational structure and handled by a small team. One consequence was that project financial reports were sometimes incomplete or late. In addition, and given that not all of BKKBN's Bureaus were familiar with the Bank's procedures, procurement of certain activities took longer than needed. 29. As for Part B, after initial delays in the first 18 months of implementation, project management improved significantly. This was party due to the change in the Project Secretariat. Project activities progressed smoothly thereafter due to diligent management efforts. The Secretariat efficiently executed project activities through their consistent follow-up. 30. A steering committee was established to carry out joint assessments and policy review, and to coordinate the work of the separate project secretariats that had responsibilities for implementing Part A and Part B. However, Part A and Part B were essentially implemented as two parallel projects. Factors generally outside Government control C. Major Factors Affecting the Project 31. One of the key external factors that positively affected the project was the 1994 International Conference on Population and Development. This Conference and its agenda for follow-up action represented a strategic shift in the prevailing approach as regards family planning and reproductive health issues. The Conference called for more emphasis on reproductive health concerns and stressed the importance of the quality of family planning services. Indonesia, which was an important participant and supporter of the Conference, embraced its recommendations -- a decision that supported the project and its objectives.

18 Factors generally subject to Government control 32. A number of factors affected the project's implementation and ability to achieve its objectives. These include: 8 * Expansion of BKKBN's mandate. During the third year of project implementation, BKKBN's mandate was expanded to include family welfare in addition to family planning. Although the family welfare activities are complementary to those of family planning, adjustment to the new organizational structure was time consuming and affected the implementation of various project activities at the time. However, when some of these issues were identified at the mid-term review, BKKBN's management took corrective steps and gave more attention to the project. It is not possible to assess the impact of the reorganization on service delivery and/or field activities. * Acceleration of the village midwife program. A Presidential decree issued in January 1993 accelerated the training and deployment of midwives, with the placement of a midwife in every village to be achieved by This led to the need for expedited training to meet the numerical targets, which offset efforts to improve the practical training offered to midwives. * Political support for safe motherhood. The last five years have witnessed increased political commitment and support for safe motherhood initiatives. This support enhanced the project's ability to achieve its objectives and strengthened the already strong ownership of Government to the project and its objectives. Factors generally subject to implementation agency control 33. Among the key factors that affected the project and that were under the control of the implementing agency were: * Coordination between the two parts of the project. The two agencies could have collaborated more -- this would have benefited both components. * Project management. In general, the project's managers performed well and as a result many of the project findings were institutionalized or followed-up. When the performance of Part B was unsatisfactory in the first 18 months of the project, GOI took action and appointed a new team at the project secretariat, resulting in significant improvements in implementation. In Part A, BKKBN provided strong support and leadership for the project. The Minister himself chaired most of the wrap-up meetings and followed-up on project performance and activities. However, the dismantling of the secretariat for Part A in the last 2 years of the project resulted in few delays that could have been avoided. D. Project Sustainability 34. Overall, The investment appears, on balance, to be sustainable. Key components and activities relating to service and information provision and quality improvements have been absorbed into BKKBN's ongoing program. Follow-up projects have been designed and will be used

19 to carry forward GOI's strong commitment to safe motherhood, specifically by addressing the factors bearing on the professional and economic viability of village midwives Part A. Many of the interventions in Part A of the project are already institutionalized or have been added to regular program activities. For instance, services for coastal areas are now part of the regular program activities and will continue beyond the life of the project. Efforts to improve quality of services have led to the establishment of quality assurance groups for family planning services at the district and province levels -- an activity that will also be supported in the follow-up investment to this project. On the other hand, efforts to revise the MIS are already in place. The results of the research studies have been incorporated into program activities and other investment plans. In addition, the project's investments in changing norms and behaviors will enhance the sustainability of its results and efforts to improve quality of services will increase the long term sustainability of the program. 36. Part B. Sustainability of Part B's achievements is likely, but will face important challenges in the next few years. Contracts of private community midwives expire after three years, with the first batch expected to "graduate" during Midwives who have completed their contracts may be attracted to private sector hospitals or private practices in cities. If these midwives are the linchpin of the safe motherhood strategy, then efforts to improve their effectiveness and sustainability are needed. In response to this urgent need, a number of policy options have been developed with GOI and will be implemented in the follow-up safe motherhood project. E. Bank Performance 37. Overall Bank performance was satisfactory. During project preparation and appraisal, appropriate skills mix during missions had a positive influence on the preparation of individual components for Part A. In terms of technical assessments, the project was well prepared. However, the monitoring of project impact should have received more attention. 38. Supervision occurred biannually during each of the project years, and was generally found to be useful by the Borrower. Missions were generally staffed with population/health specialists and supported with operational staff from the resident mission. The review missions drew upon other expertise as needed for project implementation. In addition, staff involved in project preparation were also involved in its supervision and there were only two changes in the task management of the project -- features that provided the project with the continuity needed for good implementation. On the other hand, the suggestion made in the SAR for the placement of a long term expert in family planning administration in the Bank's office in Jakarta did not materialize. This resulted in a higher supervision load for Bank staff than was anticipated at appraisal, especially for activities like research that require significant resources (in terms of time and skills) to review.

20 10 F. Borrower Performance 39. Overall Borrower performance was satisfactory. Government commitment to the project objectives and achievements was very high. Project components, especially Part A, were well prepared and both Parts of the project were well supported. Covenant compliance was satisfactory (Table 8). Implementation was especially satisfactory with effective project staff and leadership. There was good team work between the Government and Bank teams. G. Assessment of Outcome 40. The project's net present value (NPV) cannot be calculated accurately because many of the project outcomes cannot be measured quantitatively and it is difficult to assess some outcomes in monetary terms. Moreover, the project's design, as a program support loan, does not allow for the separation of influence of the project interventions, other program interventions, and other determinants of fertility and health status. 41. The project's interventions and its immediate outcomes have been achieved. However, the medium-term impact of project activities cannot be measured at this time. For instance, the impact of increased contraceptive use on fertility, the impact of IEC messages on adolescent behavior, or the effects of training on institutional development are all likely to have a medium- to long-term effects that are beyond the scope of this evaluation. It is, however, reassuring to note that sector wide indicators moved in the desired direction during implementation -- e.g., the decline in TFR (from 3.02 in 1991 to 2.85 in 1994) the increase in contraceptive prevalence rate (from 47.1% in 1991 to 52.2% in 1994), the decrease in unmet need (from 12.7% in 1991 to 10.6% in 1994) and so forth. 42. The assessment of the effectiveness of the midwifery training program and its policy is more challenging. The effectiveness of this new cadre of health providers has not been demonstrated yet. This issue will be tackled in the upcoming Safe Motherhood Project. Experience from other countries that implemented similar "village midwife" programs (e.g., Malaysia, Philippines) points to mixed findings. It is likely that this approach has its merits and can greatly increase access to services at the village level. However, efforts to improve technical competence of the midwives and sustainability need to be carefully examined in the near future. H. Future Operations 43. Most activities carried out under the project continue to be supported by GOI. Many of the interventions have been integrated into ongoing programs. In addition, GOI has followed-up on project findings and evaluations through a series of efforts that include the upcoming Safe Motherhood Project: A Partnership and Family Approach. This project responds to the lessons learned from the Population V Project, especially those related to sustainability and effectiveness of

21 11 village midwives; quality of maternal and reproductive health services (including family planning); and the linkages between supply and demand- side interventions. In addition, lessons learned from the Population V Project were incorporated in ongoing projects like the Third Community Health and Nutrition Project (Loan 3550-IND). I. Key Lessons Learned 44. The key lessons learned from the project have been grouped into two categories: those dealing with substantive issues and others dealing with implementation/process. Substantive issues * Local specificity of family planning interventions -- Addressing needs of specific population groups as well as increasing the effectiveness of existing programs requires plans that better reflect local needs and conditions. This was clearly illustrated in the component dealing with targeted family planning. * Quality of family planning and maternal health services. Although the initial project design included interventions to improve quality of family planning and maternal health services, the mid-term review of the project as well as other sector studies indicated that the unfinished agenda was significant. In addition, improving quality of services is essential for addressing the unfinished agenda in family planning and maternal health services. In response, project resources were reprogrammed to place more emphasis on improving quality of services. Additional programs for improving quality of care have been launched in follow-up initiatives. * Effectiveness and sustainability of BDDs. As efforts to train and deploy BDDs were underway, it became apparent that the completion of the BDD program (as initially designed) will not be adequate to address maternal health needs. The BDDs were not adequately trained and many had difficulties in gaining credibility as health providers and integrating into the village community. Moreover, it was not clear if these BDDs would be sustained in the villages after the completion of their three (or six) year contracts. GOI needs to address this important policy issue in the near future. * Linkages between supply- and demand- side interventions. The project financed pilots to address demand and supply side interventions related to safe motherhood. These pilots were effective in demonstrating the need for intensifying such efforts and for enhancing the coordination of demand and supply side activities -- a lesson that has been incorporated in follow-up operations. Implementation/process issues * Investment projects as tools for addressing policy reforms. Projects can be useful in addressing some policy issues if (a) they have very clearly defined objectives and linkages to

22 12 the program; (b) there is strong ownership by the government; and (c) they are very closely supervised. This was illustrated in the component dealing with midwifery policy issues. * Annual Planning and flexibility in project design. The project adopted an annual planning process that allowed for flexibility in project implementation and strengthened the ability of the project to respond to sector issues that emerged during execution. However, the lack of flexibility in some government procedures (especially the DIP process) and the lack of well identified monitoring indicators limited the optimal use of this feature. Although the lack of felxibility in the planning and budgeting process is a systemic issue that affects most sectors in Indonesia, future operations and discussions with GOI should continue to address this constraint. * Performance-based-contracts for research. The completion and/or quality of some studies did not always meet the expected standards. Future projects may wish to consider performance-based contracts that ensure the quality and timeliness of the delivery of the studies. * Ownership/leadership. The project enjoyed the support of strong leadership at BKKBN and high degree of ownership among its implementing agencies. These were key elements in its success and timely completion.

23 13 IMPLEMENTATION COMPLETION REPORT INDONESIA FIFTH POPULATION PROJECT (Family Planning and Safe Motherhood) LOAN 3298-IND PART II: STATISTICAL TABLES Table 1: Summary of Assessments A. Achievement of Objectives Substantial Partial Negligible Not applicable Macro Policies n mxi Sector Policies E n E g Financial Objectives al le lxi Institutional Development ix] i 2i [I Physical Objectives El El Poverty Reduction m Gender Issues lx E El 2I Other Social Objectives El E El Environmental Objectives [ El ] l Public Sector Management l lx a E Private Sector Development E E El Other (specify) U E L 3 (Continued) B. Project Sustainability Likely Unlikely Uncertain (i) (.1) (i) [El 1E El