Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) Report Number : ICRR0021183 1. Project Data Project ID Project Name P101160 BI-Health Project (FY09) Country Practice Area(Lead) Additional Financing Burundi Health, Nutrition & Population P131919,P126303,P126303 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-H4880,IDA-H8080,TF- 30-Dec-2012 39,800,000.00 12526,TF-13043 Bank Approval Date Closing Date (Actual) 09-Jun-2009 30-Jun-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 25,000,000.00 34,800,000.00 Revised Commitment 49,819,669.12 34,766,824.80 Actual 48,934,712.06 34,766,824.80 Prepared by Reviewed by ICR Review Coordinator Group Salim J. Habayeb Judyth L. Twigg Joy Behrens IEGHC (Unit 2) PHPROJECTDATATBL Project ID Project Name BI-Health Sect Dev Support-Add Fin P126742 (FY12 ( P126742 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) 14,800,000.00 Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) Bank Approval Date Closing Date (Actual) 21-Jun-2012 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 0.00 Revised Commitment 0.00 0.00 Actual 0.00 0.00 2. Project Objectives and Components a. Objectives Original Objectives. According to the Financing Agreement dated 7/7/2009, the objectives of the project were to increase the use of a defined package of health services by pregnant women and children under the age of five. The statement of objectives in the ICR and the Project Appraisal Document are identical. Revised Objectives. The project development objectives were revised on 12/10/2012, at which point 17.5% of the grant proceeds had been disbursed. The revised objectives were to increase the use of a defined package of health services by pregnant women, children under the age of five and couples of reproductive age. Several key associated outcome targets were also revised. The geographic scope of the project was national. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 10-Dec-2012 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components 1. Increased financing for a redefined free package of services (Appraisal, US$20 million; Actual, US$64.5 million). Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) Sub-component A: Expanded provision of a free package of health services, principally to pregnant women and children under five, through results-based payments. Sub-component B: Capacity building of community-based organizations, community health workers, and health committees for participation in related activities. 2. Strengthening the capacity of the Ministry of Public Health and other entities involved in Results- based Financing (RBF) (Appraisal US$5 million; Actual, US$20.1 million). • Capacity building of the Directorate General of Resources in procurement, financial management, and establishment of internal controls in relation to a free package of health services-related activities. • Capacity building of the Directorate General of Health, including the Directorate of Health Promotion, Hygiene, and Sanitation, with particular reference to the implementation of RBF at the community level. • Establishment of a technical unit within the Ministry of Public Health for the development and implementation of free package of health services-related activities. • Capacity building of the Ministry of Public Health in monitoring and evaluation. • Establishment of external controls in carrying out of third-party verification. The components were not revised. Additional financing was provided to the project in 2012 to continue its activities before running out of funds (Restructuring Project Paper, 9/24/12, p. 6), and to support provision of contraceptives to couples of reproductive age in line with the added family planning objective (to increase the use of a defined package of health services by couples of reproductive age), to support nutrition-related activities for pregnant women and children under five, and to strengthen verification. The additional financing therefore expanded both the duration and the scope of the project by also contributing to address two of the country’s challenges, high fertility and child malnutrition. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Costs. The initial project cost was estimated at US$25 million, and, with two additional financings, the total estimated project cost aggregated at US$84.8 million. The actual cost was US$84.6 million (ICR, p. 44). Borrower contribution. There was no direct financial contribution by the Borrower. Financing. The initial financing approved on 6/9/2009 consisted of an International Development Association (IDA) grant of US$25 million. The first additional financing on 6/18/2012 provided a grant of US$14.8 million from the Health Results Innovation Trust Fund, and the second additional financing on 12/10/2012 consisted of an additional IDA grant of US$25 million, and another US$20 million grant from the Health Results Innovation Trust Fund, in a simultaneous manner. The total amount of the grants aggregated at US$84.8 million. Dates. Appraisal was finalized on 5/13/2009; the project was approved on 6/9/2019 and became effective on 9/30/2009. The project was restructured on 12/10/2012 to revise the objectives and to update the Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) results framework. A mid-term review was undertaken on 12/1/2014. Three additional project restructurings were undertaken on 1/30/2015, 3/26/2015, and 2/2/2017 to reallocate funds between disbursement categories. The closing date was extended to accommodate the additional financing to 6/30/2017, when the project closed. 3. Relevance of Objectives Rationale At appraisal in 2009, the country of 8.2 million people was in a post-conflict transition, and the GDP per capita was US$190, among the lowest in the world. The maternal mortality ratio was high at 824 deaths per 100,000 live births; the under-five mortality was 98.7 per 1,000 live births; the fertility rate was 6.3 children per woman; and chronic malnutrition was rampant at 58% among children under-five (ICR, p. 6). Health services were underused, and health human resources were scarce and poorly distributed. Previously, starting in 2006, the country had instituted a selective policy of free health care (FHC) for pregnant women and children less than five years of age. The policy was declared by the President of the Republic, illustrating political ownership and commitment. At appraisal, the objectives were aligned with the Bank’s 2008 Country Assistance Strategy and with the government’s priorities set out in the 2006-2011 Poverty Reduction Strategy Paper, with human development identified as a key pillar to pull the country out of poverty. The Strategy Paper supported the FHC and Performance-based Financing (PBF) approach of the government for reducing maternal and child mortality, and for strengthening national health system performance. The Results Area 2 on improving access and quality of social services in the Country Assistance Strategy (Increasing Resilience by Consolidating Social Stability, 2013-2016) stated that the Bank will continue to provide support for results-based financing in the health sector with a view to strengthening the health system to improve health outcomes for pregnant women and children under five, and improving the delivery of reproductive health services essential to reducing the fertility rate. At project closing, the ICR (p. 14) noted that the Country Partnership Framework 2018-2022 was being prepared, that it highlighted challenges and opportunities linked to the health sector, and that it would maintain a strong focus on health, although no specific information was provided. The objectives remain relevant to the country policy on free care for pregnant women and children. However, with the poor quality of services in the country and the persisting high mortality levels despite the quantitative increase in coverage by the project, the ICR stated that the project could have also aimed at increasing the use of quality services (ICR, p. 15). The TTL explained that RBF reimbursements to health facilities under the project were made against both quantitative and qualitative indicators; however, most of the indicators were quantitative. The ICR stated that this concern about the quality of health services was being addressed under the follow-on project (KIRA: Burundi Health System Support Project, P156012, approved in 2017), and where the RBF gives more weight to quality components in its reimbursements (TTL clarifications, 2/25/2018). Rating Substantial Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Increase the use of a defined package of health services by pregnant women. (Under the original outcome targets) Rationale The ICR adequately explained the theory of change: The main project activities, consisting of transfer of financial resources to health facilities and capacity building of actors involved in RBF, would plausibly enhance the provision of related health care services, which in turn would contribute to increased use of health services by pregnant women and children. Outputs • 804 health facilities were supported through PBF for providing health care services. • The percentage of public and nonprofit health facilities offering the free package of services through RBF increased from a baseline of 50.9% in 2010 to 96.8% in 2017, slightly short of the target of 100%. • 18 managerial units were involved in verifying performance-based payments. • The project provided US$51.3 million to health facilities and US$11.7 million to implementing entities under PBF. • Training was provided to 670 community-based organizations. • Training was provided to 11,845 community health workers. • 670 health committees were set up. • 545 individuals were trained in PBF processes. • Several individuals were trained in financial management, information and communication technology, environmental and community health, communications, and monitoring and evaluation (M&E). • 30 controllers were hired by the General Directorate of Resources for its de-concentrated units. • Two national entities were hired to carry out third-party external control. • The project provided U$$0.88 million for provincial health office verification. Outcomes • Coverage of prenatal care services for pregnant women (at least 3 visits) increased from a baseline of 59.4% in 2009 to 66.5% in 2017, short of the original target of 70%. • The number of pregnant women receiving antenatal care during a visit to a health provider increased from a baseline of 415,799 in 2009 to 529,666 in 2017, exceeding the original target of 501,262. • The percentage of births assisted by skilled personnel increased from a baseline of 37% in 2005 Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) (subsequently updated to 64.4% in 2009, per TTL clarifications, 4/25/18) to 85.1% in 2017, exceeding the original target of 50%. • Female beneficiaries under project activities reached 1,394,317 in 2017, exceeding the original target of 1,368,232. Rating Substantial PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective Increase the use of a defined package of health services by pregnant women. (Under the revised outcome targets) Revised Rationale Outputs Outputs are same described above under the original outcome targets for Objective 1. Outcomes • Coverage of prenatal care services for pregnant women (at least 3 visits) increased from a baseline of 59.4% in 2009 to 66.5% in 2017, marginally exceeding the revised target 65.5%. • The number of pregnant women receiving antenatal care during a visit to a health provider increased from a baseline of 415,799 in 2009 to 529,666 in 2017, short of the revised target of 549,564. • The percentage of births assisted by skilled personnel increased from a baseline of 64.4% in 2009 to 85.1% in 2017, almost fully meeting the revised target of 86.2%. • The number of pregnant women living with HIV who received antiretrovirals to reduce the risk of Mother- to-Child Transmission increased from a baseline of 1,582 in 2009 to 5,026 in 2017, exceeding the target of 3,600. • Female beneficiaries covered by project activities reached 1,394,317 in 2017, attaining the revised target of 1,393,176. Revised Rating Substantial PHEFFICACYTBL Objective 2 Objective Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) Increase the use of a defined package of health services by children under the age of five. (Under the original outcome targets) Rationale Outputs Outputs are the same as described above under the original outcome targets for Objective 1. Outcomes • The average number of visits to a health provider by children under the age of 5 increased from a baseline of 1.68 in 2009 to 2.30 in 2017, short of the original target of 2.80. • The number of children under 12 months immunized against DTP3 increased from a baseline of 297,780 in 2009 to 351,470 (GP clarifications, 5/29/18) in 2017, exceeding the original target of 341,030. Rating Substantial PHREVDELTBL PHINNERREVISEDTBL Objective 2 Revision 1 Revised Objective Increase the use of a defined package of health services by children under the age of five. (Under the revised outcome targets) Revised Rationale Outputs Outputs are the same as described above under the original outcome targets for Objective 1. Outcomes • The average number of visits to a health provider by children under the age of 5 increased from a baseline of 1.68 in 2009 to 2.30 in 2017, attaining the revised target of 2.30 (GP clarifications, 5/29/18). • The number of children under 12 months immunized against DTP3 increased from a baseline of 297,780 in 2009 to 351,470 (GP clarifications, 5/29/18) in 2017, exceeding the revised target of 341,030. • The number of children between the age of 6 and 59 months receiving Vitamin A supplementation increased from a baseline of 1,265,275 in 2009 to 1,651,266 in 2017, almost fully meeting the revised target of 1,665,878. • The number of children under age five treated for moderate or severe acute malnutrition regressed from Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) a baseline of 87,170 in 2009 to 58,307 in 2017, short of the target of 83,081. The TTL explained (5/25/18) that malnutrition was not part of FHC, and that this may explain why the newly added nutrition activities received less attention. Also, the ICR noted that personnel at health facilities and at the community level did not have the necessary skills to deliver quality nutrition services under the newly added nutrition activities (ICR, p. 32). The ICR concluded that multisectoral interventions were needed to address malnutrition in Burundi, beyond supply-side incentives provided by PBF. Revised Rating Substantial PHEFFICACYTBL Objective 3 Objective Increase the use of a defined package of health services by couples of reproductive age. Rationale Outputs This objective reflected a greater emphasis on family planning services to address the high fertility rate in the country. It resulted in higher fees being paid for family planning services (ICR, p. 11). No further information was provided by the ICR. Outcomes The ICR recorded a contraceptive prevalence rate that increased from a baseline of 19.80% in 2009 to 37.40% in 2017 (updated to 42.50% per GP clarifications, 5/29/18), exceeding the target of 34.85%. However, the information provided by the ICR raised significant concerns about the validity and reliability of the evidence. According to the ICR, the contraceptive prevalence rate was not well captured, as data were not specific to couples, but rather more related to women of reproductive age (ICR, p. 24). The intermediate results indicator (number of couple-years of contraceptive protection using modern methods), which could be better tracked over time, increased from a baseline of 228,082 in 2009 to 555,702 in 2017, markedly below the target of 734,520, thus contributing to the lack of demonstration of a full results chain. According to the ICR, the reasons for low achievement in family planning include that the drop in contraceptive prevalence in recent years was probably due to the electoral process and political crisis (ICR, p. 17); and that the role of religious leaders contributed to the slow uptake of contraceptives (ICR, p. 7). For contextual information purposes, the ICR also noted that the 2017 Demographic Health Survey showed a low contraceptive prevalence rate at 23%. Rating Modest Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) PHREVDELTBL PHOVRLEFFRATTBL Rationale In conclusion, overall efficacy is rated Substantial, based on the following: • Achievement of the first objective (increase the use of a defined package of health services by pregnant women) is rated Substantial (consistent with almost fully achieving the objective) under both the original and revised revised outcome targets. • Achievement of the second objective (increase the use of a defined package of health services by children under the age of five) is rated Substantial (consistent with almost fully achieving the objective) under both the original and revised outcome targets. • Achievement of the third objective (increase the use of a defined package of health services by couples of reproductive age), which was added on 12/10/2012, is rated Modest (consistent with partly achieving the objective). Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency The PAD conducted a cost-benefit analysis at appraisal, using the Marginal Budgeting for Bottlenecks tool to translate the expected increase in health service utilization into lives saved. It estimated the benefit-cost ratio at 1.79, suggesting that each US$ invested would yield an economic return of US$1.79. The ICR analysis included all costs related to the FHC/PBF programs. The impact was translated into economic benefits in terms of average lost future earnings of an individual, and benefits were assumed to accrue over the working lifetime of pregnant women (28-60) and children (15-60). The analysis assumed a long-term economic growth of 2.24%, used a discount rate of 11%, and used a total investment of US$213 million over 7 years (2010-2017). A sensitivity analysis was also conducted. The estimated numbers of maternal and child under-five lives saved were 86,013 and 89,755 respectively. The reduction in maternal and child mortality yielded an estimated internal rate of return of 13% and generated economic benefits with a net present value of US$43.9 million. The analysis had a favorable benefit-cost ratio of 1.67, suggesting that each US$ invested would yield an economic return of US$1.67. According to the ICR, there were no major issues in the efficiency of implementation, except for disruptions caused by external factors. The civil unrest in 2015 temporarily hampered access to health care services, and a malaria outbreak in 2016-2017 resulted in high mortality among pregnant women due to poor access to drugs. Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 100.00 ICR Estimate  13.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome This ICR Review applied a split evaluation for determining the overall outcome, as development objectives and several key associated outcomes targets were revised. Relevance of objectives is assessed across the entire project and is rated Substantial, as the objectives were consistent with country priorities, and with government and Bank strategies. Under both the original and revised objectives, as well as revised outcome targets, efficacy is rated Substantial, consistent with almost fully achieving the project objectives. Efficiency is rated Substantial across the entire project because of favorable returns. The outcome under both original and revised objectives and outcome targets is rated Satisfactory, and therefore overall outcome is rated Satisfactory, indicating minor shortcomings in the project's preparation, implementation, and achievement. According to IEG/OPCS guidelines, when a project’s objectives are revised, the final outcome is determined by the weight of bank disbursements under each set of objectives (17.5% under the original objectives, and 82.5% under the revised objectives). However, since the outcome ratings are the same for both, a formal calculation is unnecessary, and the overall Outcome is rated Satisfactory, indicative of minor shortcomings in the project’s preparation, implementation, and achievement. a. Outcome Rating Satisfactory 7. Risk to Development Outcome There are two main risks that development outcomes may not be maintained. First, potential political instability and conflict would affect access to health care. Security issues would negatively affect staff and health worker Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) mobility, as well as the verification process for PBF. Second, financial sustainability remains at risk, as development partners reduced their funding contributions after the 2015 political crisis. Public health expenditures remain constrained and dependent on external assistance. However, the government maintained its commitment to allocate 1.4% of its budget to FHC/PBF, and according to the ICR, the government has also made a commitment to increase this allocation by 0.1% every year under the follow-on project. According to the ICR, institutional strengthening was noteworthy, as the General Directorates of Resources, Services, and Health Promotion, Hygiene and Sanitation, and provincial health offices benefited from capacity building (ICR, p. 22). The project strengthened the functioning of health facilities, as it allowed them to manage their own resources. 8. Assessment of Bank Performance a. Quality-at-Entry The design supporting the national RBF scheme was pertinent, as the program was the main mechanism to implement the national policy for the provision of free health care to pregnant women and children under five, while concurrently strengthening government capacity to implement RBF. However, the project design focused on increasing the quantity of health services, with less provision of incentives to promote quality aspects of health services under RBF reimbursements (see Section 3). The ICR noted this shortcoming in the context of poor quality of health services in the country and the continued high mortality levels despite increased coverage (ICR, p. 15). Lessons learned from a previous health operation in the country and from other post-conflict settings were considered and incorporated, including rapid provision of service delivery so that communities can quickly see the benefits of the peace dividend; focused strategy that prioritizes key interventions rather than trying to do everything all at once; and the importance of public sector capacity building (PAD, p. 11). The Bank provided US$0.6 million (out of which US$0.17 million was used) as a Project Preparation Facility Advance to facilitate project preparation, including carrying out of studies and supporting technical advisory services. Both government and Bank teams held participatory public consultations with local beneficiary communities and stakeholders throughout the country (PAD, p. 19). In 2009, the Bank team held a Quality Enhancement Review that contributed to planning implementation arrangements and safeguard policies (ICR, p. 28). Risk assessment was reportedly adequate with mitigation measures defined, including for capacity building. Adequate plans were developed to address compliance with safeguard policies that were triggered by the project (Environmental Assessment and Indigenous People). Financial management capacity of implementing agencies was assessed in accordance with the Bank’s Financial Management Practices, and the proposed financial management arrangements put in place for the project were adequate, and included capacity-building inputs (PAD, pp. 17-18). Quality-at-Entry Rating Moderately Satisfactory Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) b. Quality of supervision There was stability in implementation support, as the Task Team Leader changed only once during implementation. The Team undertook regular missions that included fiduciary specialists, and, as the project dealt mainly with performance-based financing, a PBF specialist was part of 14 supervision missions (ICR, p. 28). M&E specialists were team members in the earlier missions. Bank staff and consultants were involved when needed, such as with safeguards. The Bank posted a financial management specialist in Bujumbura from 2014 to 2017 to support the project's financial management unit. Follow-up and reporting were regular, with 16 Implementation Status Reports (ISRs) filed during implementation, and the ICR noted that each mission started with a review of the previous aide-memoire to ensure that there were no pending issues (ICR, p. 29). The Bank Team reportedly showed flexibility and pro-activeness in restructuring the project; however, the restructuring expanded project implementation to activity areas in malnutrition where capacities and personnel skills were limited (Section 4, and ICR, p. 32). Staff at both facility and community levels did not have the necessary skills to deliver quality nutrition services, and the ICR described the lack of availability of supplies, equipment, protocols and related supervision quality as major weaknesses (ICR, p. 32). The temporary disruptions in health care services during the 2015 civil unrest were beyond the control of the supervisory Team. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The objectives were well specified, and the indicators reflected the contribution of the operation’s activities and outputs toward achieving the intended outcomes. Overall M&E arrangements, data collection, and analysis were adequate, except for the contraceptive prevalence rate for couples of reproductive age, which was not well captured (as explained in Section 4, Objective 3). Baseline data were available, and M&E arrangements were aligned with existing government systems (ICR, p. 24). Also, RBF arrangements and verification mechanisms were adequate. b. M&E Implementation Implementation was adequate, and indicators were reported in 16 ISRs (ICR, p. 29). Data sources included routine monitoring and surveys. Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) c. M&E Utilization M&E findings were used for monitoring and project assessment, and performance data were made publicly accessible. RBF arrangements benefited from M&E; for example, fees linked to performance were raised for national hospitals when costs were found to be underestimated. The government and the Bank built on M&E results to initiate the follow-on project (KIRA - Burundi Health System Support Project - P156012), which was approved in 2017. M&E Quality Rating Substantial 10. Other Issues a. Safeguards There was full compliance with safeguard policies (ICR, p. 26). The project was classified as category B and triggered Bank safeguard policies on Environmental Assessment (OP/BP 4.01) and Indigenous Peoples (OP/BP 4.10). The project was not expected to generate adverse environmental impacts, as there was no infrastructure development, but there was risk of inappropriate handling of hazardous medical waste. The government of Burundi updated its National Biomedical Waste Management Plan and conducted a social assessment titled "Specific Plan for Very Vulnerable Peoples," focusing on the Batwa population. The purpose of the vulnerable people plan was to build the capacity of the Batwa people so that they could take more charge of their own health, and to participate in decision-making fora and the management of health programs. The plans were publicly disclosed on 4/1/2009 (PAD, p. 19). In addition to environmental safeguard implementation, training, and installation of disposal equipment, the project developed and implemented a community health plan. Six coaching agencies were hired to support community health and safeguard activities. As for the social safeguard, related activities that were implemented by the Ministry of Public Health, with the assistance of a non-governmental organization dealing with the promotion of the Batwas, included sensitization outputs for both health providers and beneficiaries; training Batwa health workers; and encouraging Batwa women and households to increase their attendance at health facilities and to use civil registry services. At the end of the project, a change in behavior of the Batwa beneficiaries was noted, with an increase in the number of Batwa women who receive antenatal care and who deliver and immunize their children at health facilities (ICR, p. 26). b. Fiduciary Compliance Financial Management. According to the ICR, financial management complied with Bank procedures and policies. The project developed a PBF Procedures Manual, and configured the accounting system to link Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) with the accounting system of the General Directorate of Resources. Training was provided to financial management staff. Interim financial reports were provided regularly twice a year, and then quarterly after the additional financing was provided in 2012. External auditing was undertaken regularly, and there were no reservations or qualifications (ICR, p. 27). Procurement. Under the project, there were no civil works, and the procurement of goods was limited. An international consultant was hired to assist in the procurement of technical assistance and consultant services, which were undertaken in accordance with Bank Guidelines on the selection and employment of consultants by World Bank Borrowers. There were occasional procurement delays, but with no notable impact on project implementation. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- This ICR Review rated Quality at Entry as Moderately Satisfactory because of insufficient RBF arrangements to promote quality aspects of health services; and it rated the quality of supervision as Bank Performance Satisfactory Moderately Satisfactory Moderately Satisfactory because the project restructuring added activity areas where skills and capacities were limited to undertake effective implementation. Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) 12. Lessons The ICR (pp. 30-32) offers several useful lessons, including the following lessons restated by IEG: • Linking performance-based financing with national funding priorities facilitates effective implementation. The selective free health care policy for pregnant women and children was already a national policy declared by the President of the Republic in 2006. Hence, the project built on pre-existing ownership and commitment (see Section 3). • Strengthening local capacity facilitates RBF approaches. Under the project, training was provided to hundreds of community-based organizations and thousands of community health workers (see Section 4). Non-governmental organizations, health committees, and community health agents were also sensitized to health priorities and operations (ICR, p. 22). • Publicly accessible performance data can potentially enhance oversight (see Section 9). The project’s performance-based financing data was publicly accessible, and the ICR noted that this enhanced credibility, including with development partners (ICR, p. 31). 13. Assessment Recommended? Yes Please explain It may be of interest to examine how Burundi has scaled up PBF and how PBF could potentially be made effective in broader program areas in the health sector, particularly given current interest in universal health coverage. 14. Comments on Quality of ICR The ICR provided a comprehensive overview of the project. The report clearly explained the project’s context at appraisal, its economic analysis, and compliance with safeguards. Its lessons were directly obtained from project experience. The ICR was candid and aligned to development objectives. The results chain was reasonably well elucidated in illustrating how activities contributed to the desired outcomes, but was not well demonstrated for family planning (contraception for couples of reproductive age). The quality of evidence is adequate for the first two objectives, but questionable for the third family planning objective, where the narrative and the findings did not support the ICR's conclusions in that area. Also, the report showed a discrepancy in actual project costs between Annex 3 and the Financing Section, p. 2. The report was consistent with the guidelines, except for the lack of applying a split evaluation, which was Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BI-Health Project (FY09) (P101160) indicated, as both development objectives and key associated outcome targets were revised. a. Quality of ICR Rating Substantial Page 16 of 16