Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004542 IMPLEMENTATION COMPLETION AND RESULTS REPORT < H751-BF, TF11678 and D1080-BF> ON AN IDA GRANT IN THE AMOUNT OF SDR 42.8 MILLION (US$63.9 MILLION EQUIVALENT) AND A GRANT FROM THE MULTI-DONOR HEALTH RESULTS INNOVATION TRUST FUND IN THE AMOUNT OF US$12.7 MILLION TO THE REPUBLIC OF BURKINA FASO FOR A BF-Reproductive Health Project (FY12) October 23, 2019 Health, Nutrition & Population Global Practice Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective Jun 28, 2018) Currency Unit = FCFA (XOF) FCFA 566 = US$1 US$1 = SDR 0.729836 FISCAL YEAR January 1 - December 31 Regional Vice President: Hafez M.H Ghanem Country Director: Soukeyna Kane Global Practice Director: Muhammad Ali Pate Practice Manager: Magnus Lindelow Task Team Leader(s): Paul Jacob Robyn, Zenab Konkobo Kouanda ICR Main Contributor: Maud Juquois ABBREVIATIONS AND ACRONYMS AF Additional Financing ANC Antenatal Care BCC Behavioral Change Communication CAS Country Assistance Strategy CBHI Community Based Health Insurance CBT Community-based targeting CEmONC Comprehensive Emergency Obstetric and Neonatal Care CPR Contraceptive prevalence rate CPS Country Partnership Strategy CSO Civil Society Organization CT/PLNS National Committee CYP Couple Year Protection DHS Demographic and Health Survey ENSP National Public Health School (Ecole Nationale de Santé Publique) FP Family Planning GFF Global Financing Facility IMCI Integrated Management of Childhood Illness IE Impact Evaluation IEC Information, Education and Communication INSD National Institute of Statistics and Demography (Institut national de la statistique et de la démographie) M&E Monitoring and Evaluation MCTC Mother to Child MDGs Millennium Development Goals MoH Ministry of Health MTR Mid-Term Review NGOs Non-Government Organizations PADS Health Development Support Program (Programme d’Appui du Développement Sanitaire) PDO Project Development Objective PMTCT Prevention of Mother to Child Transmission PNC Postnatal Care PRSP Poverty Reduction Strategy Paper PSR Reproductive Health Project (Projet Santé de la Reproduction) RBF Results-Based Financing RF Results Framework RGPH General Census of Population and Habitat (Recensement Général de la Population et de l’Habitat) RH Reproductive Health SCADD Strategy for Accelerated Growth and Sustainable Development (Stratégie de Croissance Accélérée et du Développement Durable) UNDP United Nations Development Programme VCT Voluntary Counseling and Testing TABLE OF CONTENTS DATA SHEET ................................................................................................................................1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES .........................................................5 A. CONTEXT AT APPRAISAL ............................................................................................................. 5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION................................................................... 9 II. OUTCOME .........................................................................................................................14 A. RELEVANCE OF PDOs ................................................................................................................ 14 B. ACHIEVEMENT OF PDOs (EFFICACY) .......................................................................................... 15 C. EFFICIENCY ................................................................................................................................ 19 D. JUSTIFICATION OF OVERALL OUTCOME RATING....................................................................... 20 E. OTHER OUTCOMES AND IMPACTS (IF ANY)............................................................................... 20 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ..................................21 A. KEY FACTORS DURING PREPARATION....................................................................................... 21 B. KEY FACTORS DURING IMPLEMENTATION ................................................................................ 21 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ...23 A. QUALITY OF MONITORING AND EVALUATION (M&E) .............................................................. 23 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ....................................................... 25 C. BANK PERFORMANCE ............................................................................................................... 26 D. RISK TO DEVELOPMENT OUTCOME .......................................................................................... 27 V. LESSONS AND RECOMMENDATIONS .................................................................................27 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ..............................................................29 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION...........................42 ANNEX 3. PROJECT COST BY COMPONENT ...............................................................................44 ANNEX 4. EFFICIENCY ANALYSIS ...............................................................................................45 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ....49 ANNEX 6. RESULTS-BASED FINANCING AND IMPACT EVALUATION .........................................54 The World Bank BF-Reproductive Health Project (FY12) (P119917) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P119917 BF-Reproductive Health Project (FY12) Country Financing Instrument Burkina Faso Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Republic of Burkina Faso National AIDS Council (SP-CNLS) Project Development Objective (PDO) Original PDO To improve the utilization and quality of reproductive health services in the Recipient's territory, with a particular focus on selected regions of the Recipient. Revised PDO To improve the utilization and quality of maternal and child health, reproductive health and HIV/AIDS services in the Recipient's territory with a particular focus on the poor and vulnerable. Page 1 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 28,900,000 28,877,559 26,310,163 IDA-H7510 12,700,000 12,679,198 12,679,198 TF-11678 35,000,000 34,755,566 35,530,121 IDA-D1080 Total 76,600,000 76,312,323 74,519,482 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 76,600,000 76,312,323 74,519,483 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 20-Dec-2011 22-Oct-2012 30-Jul-2015 31-Dec-2016 30-Jun-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 23-Feb-2016 30.83 Additional Financing Change in Implementing Agency Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Disbursements Arrangements Change in Legal Covenants Change in Institutional Arrangements Change in Financial Management Change in Implementation Schedule 29-Jan-2018 61.53 Reallocation between Disbursement Categories Page 2 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 27-Jun-2012 Satisfactory Satisfactory 0 02 26-Dec-2012 Satisfactory Moderately Satisfactory 0 03 22-Jun-2013 Satisfactory Moderately Satisfactory 3.88 04 28-Dec-2013 Satisfactory Moderately Satisfactory 3.98 05 22-Jun-2014 Satisfactory Moderately Satisfactory 7.56 06 24-Dec-2014 Satisfactory Moderately Satisfactory 15.37 07 08-Jun-2015 Satisfactory Satisfactory 23.59 08 29-Dec-2015 Satisfactory Satisfactory 30.83 09 29-Jun-2016 Satisfactory Satisfactory 33.71 10 29-Dec-2016 Moderately Satisfactory Moderately Satisfactory 45.18 11 28-Jun-2017 Moderately Satisfactory Moderately Satisfactory 52.87 12 22-Dec-2017 Moderately Satisfactory Moderately Unsatisfactory 59.13 13 20-Jun-2018 Moderately Satisfactory Moderately Satisfactory 74.52 SECTORS AND THEMES Sectors Major Sector/Sector (%) Health 71 Public Administration - Health 33 Health 38 Page 3 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Social Protection 29 Social Protection 29 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 0 Health Systems and Policies 100 Health System Strengthening 33 Reproductive and Maternal Health 67 ADM STAFF Role At Approval At ICR Regional Vice President: Obiageli Katryn Ezekwesili Hafez M. H. Ghanem Country Director: Madani M. Tall Soukeyna Kane Director: Ritva S. Reinikka Amit Dar Practice Manager: Jean J. De St Antoine Magnus Lindelow Task Team Leader(s): Haidara Ousmane Diadie Paul Jacob Robyn ICR Contributing Author: Maud Juquois Page 4 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Country context. Burkina Faso is a landlocked country south of the Sahara. It covers an area of 274,200 square kilometers and had a population estimated in mid-2010 at 15.2 million, which had been growing at 3.4 percent per annum. The vast majority of the population (77 percent) lived in rural areas and was affected by illiteracy (71 percent in 2007). Burkina Faso remained one of the poorest countries in Africa with a per capita income of US$510 in 2009 (Atlas method). In 2010’s UNDP Human Development Report ranked Burkina Faso 181th out of 187 countries. The poverty headcount ratio at national poverty line was estimated at 47percent of the total population in 2009 (Source: L'Institut national de la statistique et de la Démographie - INSD). Political instability. At appraisal stage, the first signs of the political crisis happened in Burkina-Faso and had prompted a government reshuffle already in 2011. The protest incidents included a military mutiny led by disgruntled members of the presidential guard in April 2011. During the same month, large-scale rallies were organized to protest against the rising cost of basic goods. Sectoral and institutional context. At appraisal in 2011, Burkina Faso remained off track to meet the Millennium Development Goals (MDGs) 4 (reducing child mortality) and 5 (reducing maternal mortality). A major constraint to their achievement has been the rapid population growth rate, especially with respect to fertility. Burkina Faso’s total fertility rate remained very high, with an average of six pregnancies per woman. Despite major efforts in the education sector, the country has not been able to enroll all its children in primary school, not to mention the secondary and tertiary levels of education where attainments and quality have been poor. The rapid population increased also has far-reaching consequences for the economy in the areas of dependency ratios, labor productivity, savings, growth, living standards, the demand for public services, and poverty reduction. Reproductive Health outcomes. In order to harness Burkina Faso’s economic potential to complete its demographic transition, it was important to improve reproductive health (RH) outcomes, including child survival. The average age at first pregnancy has remained 19 years over the last two decades and the adolescents 15-19 years contributed to fecundity for 11 percent (DHS 2010). The contraceptive prevalence rate (CPR) for modern methods remained very low at 15 percent (DHS 2010). Burkina- Faso was one of the Sub-Saharan countries with the highest unmet need: 33 percent of married women express a desire to avoid or space pregnancies but do not use any contraception method. While 89 percent of women knew at least one modern contraceptive method for limiting or spacing births, fewer than 10 percent of them were using a long-term family planning (FP) method. One of the reasons for the high desired fertility was the poor child survival rate that made families want to have more children in case some do not survive. Maternal health outcomes must be improved, as only 34 percent of pregnant women completed four prenatal care visits; and postpartum care was insufficient, covering only 26 percent of newborns and 72 percent of women who gave birth (DHS 2010). The proportion of deliveries by skilled attendants was 67.1 percent (DHS 2010), and the maternal mortality ratio is high at 307.3 per 100,000 (RGPH 2006). Child survival was also critical as one in five children did not reach the age of five. Health system issues. At appraisal, access to health services remained inadequate, with the population living at an average distance of 7.5 km from a health center, more than an hour walk. There were large variations in access to services and health outcomes between urban and rural areas, and between the wealthiest 20 percent and the Page 5 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) poorest 20 percent of the population. Doctors and midwives remained disproportionately concentrated in urban areas, and service quality was undermined by the inadequate motivation of public sector health workers due to low salaries, poorly developed career structures, and limited accountability for performance. Low quality of care also contributed in poor health outcomes and high maternal mortality ratio (insufficient equipment and drugs, and a lack of skills of health staff). Health financing. In 2009, the government’s health budget represented 15 percent of the total government budget translating into US$11.9 per capita, a level that should bring better health outcomes. However, 39 percent of total health costs were paid out of pocket, thus reducing access to services and efficiency of spending. The allocative efficiency of the government health budget was limited, and the quality of public expenditure management needed to be improved. The government has demonstrated its commitment to improving the efficiency and effectiveness of health care spending. In previous years, it introduced a pooled fund (panier commun) to support the delivery of a minimum package of health services into its Health Development Support Program- Programme d’Appui du Développement Sanitaire (PADS). PADS is a project coordination unit shared by several donors and development partners, set up by the government and established under the General Secretariat of the MOH. It manages the funds put in a common pool by donors and operates by contracting with health districts and Non-Government Organizations (NGOs). The Bank assisted the Government in the creation of the PADS and promoted harmonized implementation arrangements and pooled funding, including all major health partner such as UN agencies (UNICEF, UNFPA and WHO), the Dutch cooperation, the German cooperation, GAVI, the Global fund and the Bill and Melinda gate Foundation. In 2011, seven partners (including Government) used the common fund. The PADS used a series of transparent criteria to allocate funds to structures (central, hospitals, regional directorates and health districts) and functions (training, common expenditures and operations). At appraisal stage, performance of the PADS in managing IDA’s and other partners funds was adequate. Burkina-Faso national health priorities. The third Poverty Reduction Strategy Paper (PRSP) for Burkina Faso (2011- 2015) emphasized demographic issues. Additionally, one of the strategic objectives of the National Health Plan 2011- 2020 is to “Improve health services delivery” (strategic objective 2) by strengthening the supply of health services, improving quality of maternal and child health and also improving quality and motivation of human resources for health (objective 3.2). Rationale for Bank’s engagement and higher-level Objectives to which the Project contributes. The Project was strongly aligned with the national priorities described above and also with the 2010-2012 Country Assistance Strategy (CAS, completed on August 10, 2009). Strategic Theme 2 of the CAS is : “Promoting shared growth through improved social service delivery” and CAS priorities and goals were “ to better serve the needs of the poor; increase access to services; strengthen institutional capacity; upgrade quality and effectiveness of services delivery; enhance the role of the private sector in achieving important public health goals; and decentralize through enhanced participation of the local bodies and the community”. Finally, the proposed project was fully consistent with the Bank’s HNP strategy (2007), and the Bank’s Reproductive Health Action Plan (2010-2015) which commits the Bank to improving access to quality family planning and reproductive health services Thus, the Project was intended to help improve reproductive, maternal and child health, by introducing Results- Based Financing (RBF) which would strengthen accountability at the central and decentralized level in the health sector, as well as improve financial access to quality health services. It also aimed to strengthen reproductive health services by addressing gaps in critical inputs and capacity constraints. Page 6 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Theory of Change (Results Chain) Figure 1: Theory of change at appraisal (ICR author) Project Development Objectives (PDOs) “The objective of the Project is to improve the utilization and quality of reproductive health services in the Recipient's territory, with a particular focus on selected regions of the Recipient”, as stated in the Financial Agreement signed February 14th, 2012. Key Expected Outcomes and Outcome Indicators The two key expected outcomes were: (i) improved utilization of reproductive health services and (ii) improved quality of reproductive health services. The following four PDO indicators were to measure the success of the project (the four of them measuring the increased utilization while one of them also measure the improved quality of services). Page 7 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Table 1: PDO indicators and outcomes at appraisal PDO Indicators Measure of which outcome of the PDO 1. Contraceptive prevalence rate Measuring improved utilization of reproductive health services 2. Proportion of births assisted by Measuring improved utilization of reproductive health services skilled personnel and improved quality of reproductive of health services 3. Proportion of women attending Measuring improved utilization of reproductive health services post-natal consultations 4. Number of children immunized Measuring improved utilization of reproductive health services In addition, there were eleven intermediate outcome indicators: (i) Percentage of pregnant women receiving at least 2 doses of anti-tetanus immunization; (ii) Percentage of facilities that received RBF credits on time; (iii) Pregnant women receiving antenatal care during a visit to a health provider; (iv) Couple Years Protection (CYP); (v) Percentage of Cesarean-sections; (vi) Percentage of women of reproductive age that are satisfied with the quality of RH care and services provided in public sector facilities; (vii) Health personnel receiving training; (viii) Number of CEmONC public sector facilities; (ix) Pregnant women living with HIV who received antiretroviral to reduce the risk of MCTC; (x) Percentage of facilities with no contraceptive stock-outs in last three months; (xi) Percentage of women aged 15-19 who know at least one modern family planning method. Components The original Project was supported by an IDA grant of US$28.9 million and a grant from the Multi-Donor Health Results Innovation Trust Fund (HRITF) of US$12.7 million. The project consisted of two components: Component 1: Improving the delivery and quality of a Reproductive Health Service Package through Results-Based Financing (US$22.3 million- IDA US$9.6 million and HRITF US$12.7 million). The first component was financing a Results-Based Financing (RBF) scheme in some targeted districts of the country (fifteen health districts in six regions, accounting for 25 percent of the population) to improve the delivery and quality of health services. The six regions were Centre Nord, Centre Ouest, Nord, Sud-Ouest, Boucle du Mouhoun and Centre Est. The core activities supported by Component 1 included: (i) financing the RBF grants to health centers in the targeted districts (details of the RBF approach in Burkina-Faso provided below) and; (ii) supporting RBF implementation and supervision with: Page 8 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) - An international firm in charge of providing technical support (at national and at district levels) and external monitoring/verification. The international firm was also in charge of subcontracting local Civil Society Organizations (CSOs) to carry out consistency checks between facilities records and exit surveys of patients. - Financing trainings on RBF and its implementation to stakeholders at all levels (local government, health workers, district verification teams, etc.) - Support research on RBF. The Results-Based Financing Model in Burkina-Faso is presented in Annex 6 (and detailed in an RBF implementation manual). Component 2. Supporting critical inputs for reproductive and family planning services (US$19.3 million). The second component of the Project, entirely financed by the IDA Grant, was implemented at the national level and included the following core activities: (i) Training of nurses, skilled birth attendants and doctors. The Project supported pre-service training (60 skilled birth attendants and 60 nurses) at the National Public Health School (Ecole Nationale de Santé Publique – ENSP) and in- service training (for around 360 nurses, 360 skilled birth attendants and doctors) on emergency obstetric and neonatal care, family planning, integrated management of child illnesses, management of health services, etc. (ii) Provision of drugs and equipment to improve obstetric and neonatal services. Key equipment included ambulances, motorcycles, emergency delivery kits, drugs, contraceptives, equipment for waste management, etc. (iii) Strengthening demand for family planning and reproductive health services. The Project supported NGOs to carry out information, education and communication IEC) and behavioral change communication (BCC) strategies at community level, in partnership with Community-Based Organizations. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Additional Financing 2016 (US$35 million). An Additional Financing (AF) was approved on March 15th, 2016 along with restructuring of the parent Project. The Bank received a letter (ref #No 0044-Presidence du Burkina Faso) dated April 7, 2014, requesting the Association to provide additional support for the Government’s efforts in the fight against HIV/AIDS. And in July 2015, a Mid-Term Review (MTR) was conducted and the main recommendations were: (i) continue implementation of the RBF pilot given promising initial results in improving utilization and quality of health services; (ii) expand Community-Based Health Insurance (CBHI) and targeting of the poor within the operation (piloted at a small scale before) to improve financial protection of the poor; (iii) extend the end line of the RBF impact evaluation from early 2016 to early 2017 (due to initial delays on implementation of the pro-poor mechanisms) and (iv) strengthen capacities of health workers and demand for family planning and reproductive health services. The MTR also identified a financing gap to support RBF until 2017. The HIV epidemic was still a generalized one and national prevalence rate was estimated at its lowest level at 1.0 percent and 1.2 percent for women and 0.8 percent for men (DHS 2010). Direct financing through the Project of the HIV/AIDS multisectoral strategy was not considered at appraisal as another Project was already supporting it (Health Sector Support and Multisectoral AIDS Project, P093987, which closed in December 2014). Moreover, before the Additional Financing, the Project included support to supply-side activities related to HIV/AIDS with prevention of Page 9 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) mother-to-child transmission (PMTCT) and HIV/AIDS treatment being part of RBF incentivized services. Several partners contributed to fund the HIV/AIDS multisectoral strategy, including UNPD, UNAIDS and UNICEF. At closing of the Health Sector Support and Multisectoral AIDS Project, the Government requested additional financial support from the Bank to fill some financing gaps and to sustain good results achieved by the national program, especially on promoting HIV preventive activities among high risk and vulnerable groups. Thus, the AF aimed to scale up the impact and development effectiveness of the Reproductive Health Project by: (i) helping address the RBF financing gap; (ii) supporting interventions that address the needs of groups vulnerable to HIV/AIDS; (iii) strengthening critical inputs for reproductive health (training of health workers and equipment’s and drugs); and (iv) helping to finance the costs associated with the expansion of ongoing piloted strategies such as community-based targeting (CBT) of the poor and CBHI. With the expansion of the scope, the Project was targeting 4,000,000 beneficiaries (including an additional 130,000 poor beneficiaries in support of universal health coverage under the proposed AF). In consequence, numerous changes were made and are described below. Level II Restructuring (approved on January 29th, 2018). The only change was a reallocation of funds between components. A restructuring request was received on December 20th, 2017 to proceed with a reallocation of funds six months before Project closing date. The reallocation to component 1 of the Project (Results-Based Financing) was needed as insufficient resources were allocated to the RBF (full disbursement was achieved after one year of the AF implementation). The request also included an extension of the Project closing date but due to lack of performance of implementation unit in the MoH (overall project implementation was downgraded to Moderately Unsatisfactorily), the extension was not accepted (explanations provided under section III on Key Factors affecting implementation and outcome). Revised PDOs and Outcome Targets To improve the utilization and quality of maternal and child health, reproductive health and HIV/AIDS services in the Recipient's territory with a particular focus on the poor and vulnerable (wording from the Financing Agreement, dated April 4th, 2016). The PDO was revised as the AF incorporated new interventions related to HIV/AIDS and interventions to improve access to essential services for the poor and vulnerable. The two initial key expected outcomes were therefore expanded to the three following expected outcomes: (i) improved utilization of maternal and child health, reproductive health and HIV/AIDS services; (ii) improved quality of maternal and child health, reproductive health and HIV/AIDS services and (iii) increased utilization and quality of these services for the poor and vulnerable. Revised PDO Indicators As part of the AF, the Results Framework was revised to: (i) add specific indicators to track progress on new interventions added through the AF and (ii) revised some of initial indicators to convert them into cumulative numbers (instead of percentage) and align the targets to the revised closing date of June 30, 2018. Page 10 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Table 2: Revised PDO indicators through 2016 Additional Financing PDO indicators after AF Initial PDO indicators Changes made and rationale New acceptors of modern Contraceptive prevalence Conversion from coverage (percentage) contraceptive methods rate to number Proportion of birth assisted by skilled Proportion of birth assisted Target revised personnel by skilled personnel Percentage of women who received Proportion of women Conversion from coverage (percentage) postnatal consultation between the attending postnatal to number 6th days/6th week of delivery consultations Children immunized (number) Children immunized Conversion from coverage (percentage) (percentage) to number Number of poor people and New indicator to cover new sub- vulnerable persons who receive component 1.3 (community-based health services free of charge targeting, community-based health insurance and support to UHC) added through AF Number of pregnant women living New indicator to cover new sub- with HIV who received antiretroviral component 2.4 (Strengthening creation of to reduce the risk of PMTCT HIV/AIDS and STI demand for prevention and treatment among PLWHA, youth and women) added through AF Intermediates outcome indicators were also revised, five of the original ones being deleted, two added (to track progress on new interventions funded) and six revised (target revised to align with extension of closing date, change from percentage to number). From eleven in the original Project, the restructured Results Framework included eight intermediate outcome indicators (See Annex 1 – the revised Results Framework). Revised Components Changes in the components made through the 2016 Additional Financing (US$35 million) included: - Continuing support to activities under components 1 and 2 (US$19 million additional financing): o Under the RBF approach (component 1), the list of indicators to be incentivized was expanded to services to be provided free of charge for the poorest (beneficiaries of community-based targeting and CBHI) and new HIV/AIDS-related indicators; o Under component 2, additional financing was provided to strengthen interventions to support critical inputs with a particular focus on training of health workers and provide drugs and equipment for obstetrical and neonatal services and biomedical waste management. - Introducing a new sub-component to support provision of health services to the poorest through health insurance (new sub-component 1.3: Supporting Universal Health Coverage through improving financial access to health services for poor and indigent populations, US$6 million). Indeed, during the first phase of the Project, a pilot of CBHI and community targeting was conducted, and its results were promising to protect the poorest from financial costs of health services. Page 11 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) - Introducing a new sub-component to increase demand for and utilization of HIV/AIDS preventives and preventive health services, with a particular focus on high risks groups (sub-component 2.4: Strengthening creation of HIV/AIDS and STI demand for prevention and treatment among PLWHA, youth and women, US$10 million). With the Level II Restructuring approved on January 29th2018, a reallocation of funds between components was done: resources from sub-components 1.3, 2.1 and 2.2 (all non-essential activities cancelled) were reallocated to sub-components 1.1 and 1.2 (as explained above): The table below summarizes the changes from the 2016 AF/restructuring and restructuring of 2018 to reallocate funds: Table 3: Summary of Project components and costs (Original, 2016 AF-Restructuring and 2018 Restructuring) Project components (after AF restructuring 2016) Original AF/Restruc Reallocation Total Original, AF turing 2016 2018 and restructuring Component 1: Improving the delivery and quality 22.3 21 0.7 44 of a Reproductive Health Service Package through Results-Based Financing Sub-component (i) - Delivery of Packages of Basic 16.8 10 1.2 28 Health Care (PBHS) Subcomponent (ii) - Support to RBF implementation 5.5 5 1 11.5 and supervision Subcomponent (iii) – Supporting Universal Health 6 -1.5 4.5 Coverage through improving financial access to health services for poor and indigent populations (NEW AF) Component 2. Supporting critical inputs for 19.3 14 -0.7 32.6 reproductive health and HIV/AIDS services Subcomponent (i) - Training of nurses, skilled birth 2.3 2 -0.05 4.3 attendants and doctors Subcomponent (ii) - Provision of drugs and 8.5 2 -0.7 9.8 equipment to improve obstetrical and neonatal services Subcomponent (iii) - Strengthening demand for 8.5 0 0 8.5 family planning and reproductive health services Subcomponent (iv) -Strengthening creation of 10 10 10 HIV/AIDS and STI demand for prevention and treatment among PLWHA, youth and women (NEW AF) Total 41.6 35 76.6 Other Changes As part of the 2016 Additional Financing, the other following changes were made: 1- Changes in Implementing Agency, Institutional arrangements and Financial Management Page 12 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) The Permanent Secretariat of the National Committee for the Fight Against HIV/AIDS (SP/CNLS-IST) was made the implementing agency responsible for HIV/AIDS interventions under the new sub-component 2.4. Accordingly, the SP/CNLS-IST had fiduciary and coordination responsibilities for the financing received to implement these interventions. Institutional arrangements for the other interventions (component 1 RBF and component 2 except sub-component 2.4) remained the same through the PADS (Programme d’Appui au Développement Sanitaire- Health Development Support Program) 2- Change in closing date and implementation schedule The closing date of the original IDA Grant of December 31, 2016 was extended to June 30, 2018. Thus, the implementation schedule was updated to allow planned activities to be implemented during the eighteen months extension period. Rationale for Changes and Their Implication on the Original Theory of Change As noted earlier, the 2016 AF aims to scale up the impact and development effectiveness of the Reproductive Health Project and the scope of the Project was extended to include interventions addressing the needs of groups vulnerable to HIV/AIDS and finance the costs associated with the expansion of ongoing piloted strategies such as community- based targeting (CBT) of the poor and CBHI. Thus, the PDO was revised as to improve the utilization and quality of maternal and child health, reproductive health and HIV/AIDS services in the Recipient's territory with a particular focus on the poor and vulnerable. The scope was extended to HIV/AIDS services, maternal and child health and to focus on the poor and vulnerable. Figure 2: Theory of change (after 2016 AF) Page 13 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) II. OUTCOME Through the 2016 restructuring and AF, the scope of the Project was expanded (addition of HIV/AIDS services and the focus on the poor and vulnerable), outcomes were clarified, and targets were increased. Thus, as per the ICR guidelines, a split rating is not required and the outcome of the Project is assessed against the revised outcomes/targets. It is also worth-noting that the Project was implemented in a context of instability and fragility in-country, with a political and security crisis started in 2011. A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating The relevance of PDO (initial and revised) is rated as High, as it remains on line with past and current national priorities in Burkina-Faso, as well as fully aligned with Bank’s strategies. One of the three axes of the Burkina-Faso National Economic and Social Plan 2016-2020 (PNDES-Plan National de Développement Economique et Social 2016-2020) is to “improve human capital”, and objective 2.1 focuses specifically on promoting population health and accelerating the demographic transition. Additionally, the National Health Plan 2011-2020 (PNDS: Plan National de Développement Sanitaire) sets among its key sectoral priorities « improvement of health service delivery » and expected results are the following: (i) leadership and governance in the health sector are strengthened; (ii) health services are more performant and offer health services of better quality at the different levels (including community level) and (iii) specific mortalities are reduced significantly compared to 2010. Specifically, objective 2.1 of the PNDS is to increase utilization of health and nutrition services for the most vulnerable (mothers and children). Furthermore, Burkina-Faso passed in 2015 a universal health insurance law 1, and including CBHI in the Project design was a way to pilot the proposed model for the voluntary arm of the scheme and generate evidence on it prior to making national-scale decisions. Thus, the original and revised project development objectives directly supported the Government priorities. Indeed, the project intended to increase utilization and quality of maternal and child health, reproductive health and HIV/AIDS services with a particular focus on the poor and vulnerable. The proposed project was also fully in line with the World Bank Group’s twin objectives of reducing poverty and promoting shared prosperity, the Human Capital project and with the Sustainable Development Goals (SDG), in particular Goal 3: Ensure healthy lives and promote well-being for all at all ages. Goal 3 of the SDGs has several targets for which the project directly supported: reduction of maternal mortality (Target 3.1), reduction of under-5 and neonatal mortality (Target 3.2), achieving universal access to sexual and reproductive health-care services (Target 3.7), achieving Universal Health Coverage (Target 3.8). Moreover, the PDO is fully aligned with the Bank’s current Country Partnership Strategy (CPS) for Burkina-Faso (2018- 2023): it directly contributed to achieving the CPS objective 2.2: Expand access to reproductive health and nutrition. The CPS (2018-2023) makes gender a key cross-cutting area for Burkina-Faso. By reducing gender bias against women particularly in their access to quality health care and by tackling the high rate of fertility with a well-functioning family planning system, it will improve their socio-economic conditions. Thus, the PDOs are highly relevant to the long-term outcomes of improving maternal, child and reproductive health outcomes and supporting the demographic transition 1 Law number 060-2015/CNT Page 14 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) in Burkina-Faso. B. ACHIEVEMENT OF PDOs (EFFICACY) Rating: Modest The assessment of the project’s revised PDO for this ICR is unpacked into the following 3 objectives (as indicated in the revised results chain, and detailed in Annex 1): (i) To improve utilization of maternal and child health, reproductive health and HIV/AIDS services; (ii) To improve quality of maternal and child health, reproductive health and HIV/AIDS services; and (iii) To increase utilization and quality of these services for the poor and vulnerable. While two out of the six PDO surpassed their targets and two were substantially achieved (and the last two only partially achieved), and five of the eight IO were substantially achieved, two partially and one not achieved, assessment of the quality and equity parts with indicators of the results framework is limited. In addition to the indicators of the results framework, discussion and assessment of the efficacy of the Project in the following section also builds on additional results from the Results-Based Financing annual data reports, as well as from the results of the Impact Evaluation (IE). Results-Based Financing for Health Impact Evaluation in Burkina-Faso (final report June 2018): An impact evaluation was conducted prior to implementation of the RBF approach, to rigorously assess the impact of the RBF. Baseline data collection took place between October 2013 and March 2014 and endline between April and June 2017. Twelve health districts were part of the IE (out of the 15 where the RBF was implemented through the Project, as the 3 other ones were part of the RBF pre-pilot thus excluded from the Impact Evaluation). The IE provides counterfactual thus attribution for Project’s interventions. Health facilities were randomized in four interventions groups (see Annex 6 for details): standard RBF and three interventions groups where RBF was combined interventions to increase access to the poorest to health care services. Assessment of Achievement of Each Objective/Outcome 1- Improved utilization of maternal and child health, reproductive health and HIV/AIDS services: Rating-Modest One key objective of the Reproductive Health Project was to increase the utilization of maternal, child, reproductive and HIV/AIDS services, using a Results-Based Financing approach. It was indeed expected that the RBF would support improvement in health services delivery by increasing motivation and accountability of health providers, providing additional financing at the local level while also increasing autonomy of health facilities and strengthening the monitoring and evaluation system. Approximately 600 primary and secondary care health facilities were part of the RBF (in 15 health districts in 6 regions), covering a population of approximately 4.9 million (at the end of the Project), 25 percent of the population of Burkina-Faso. At the start of the project, the different key activities to ensure a proper RBF implementation took place: hiring of three contracting and verification agencies; substantial training and capacity building activities at regional, district and local levels (3,604 health staff and 194 community verification agents were trained on RBF in 2013 and early 2014); local community organizations contracted to conduct the RBF beneficiaries surveys; necessary equipment was purchased (152 bicycles, 21 field supervision vehicles and 15 ambulances); a cloud computing system was set up (www.fbrburkina.org). The first RBF payments to health facilities happened at the end of Page 15 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) 2014. In 2016 and 2017, financing gaps (before effectiveness of the AF) and delays in payments (in 2016 and 2017) impacted negatively the results of the RBF approach as it created demotivation and uncertainties from health providers (details on Section III). As per the revised RF, the four following PDO indicators were used to assess achievement of this objective: (i) new acceptors of modern contraceptive methods; (ii) number of immunized children; (iii) percentage of women who received postanal consultations between the 6th days/6th week of delivery; and (iv) number of pregnant women living with HIV who received antiretroviral to reduce the risk of PMTCT. By the closing date, one PDO indicator surpassed its target: 58.7 percent of women received postanal consultations between the 6th days/6th week of delivery (revised target was 42 percent, achievement against the target is 140 percent); two PDO indicators were partially achieved: the project contributed to provide modern contraceptive methods to 586,980 new acceptors (against a target of 770,000, i.e. achievement against the target of 76 percent); the total cumulated number of immunized children was 585,566 (against a target of 860,000, i.e. achievement against the target of 68 percent) and the last one was substantially achieved as 5,348 pregnant women living with HIV have received antiretroviral to reduce the risk of PMTCT (against a target of 6,135, i.e. achievement against the target of 87 percent). Out of the five intermediate indicators linked to this increased utilization of health services objective, three of them were substantially achieved: 1. Percentage of pregnant women receiving at least 2 doses of anti-tetanus immunization: 75 percent (target was 86 percent); 2 Percentage of pregnant women receiving at least 2 antenatal care visits during a pregnancy: 69 percent (target was 86.5 percent); 3. Number of pregnant women receiving VCT services: 874,064 (target was 889,090); one partially achieved 4. Percentage of Cesarean-sections among planned pregnancy: 2.43 percent (target was 3.1 percent) and the last one not achieved: 5. Percentage of facilities that received RBF credits on time: 0 percent (target was 80 percent, this is due to the fact that payments for last quarter exceeded the 35 days that was in the RBF Manual). Results from the IE reflect these mixed results: the RBF intervention had a positive impact on utilization of maternal health services (for example +4.4 percentage point between RBF and control groups for facility-based delivery and +6.6 percentage point on 3 postnatal care visits) and some positive impact on child and adult consultation. However, results from the IE didn’t demonstrate any impact of the RBF on the utilization of preventive child services, vaccination of children and mixed effects on growth monitoring. 2-Improved quality of maternal and child health, reproductive health and HIV/AIDS services: Rating-Modest The RBF approach as well as key interventions to provide critical inputs for maternal, child, reproductive health and HIV/AIDS services under component 2 aimed at improving the quality of care that was delivered in Burkina-Faso. A quality check-list was part of the RBF quarterly assessment of health facilities, in addition to the increased accountability and motivation of health providers and increased financing and autonomy that were expected to increase the quality of care. As part of the quality check-list, technical quality was measured for specific services such as family planning, assisted deliveries, antenatal care, etc. As part of activities under component 2, the project supported the training of 120 nurses and midwives, provision of drugs and equipment to improve obstetric and neo-natal services, teaching aids to improve quality of training distributed to the five regional nursing hospitals. Indicators to assess the quality of care from the results framework are limited, thus complemented by other data sources. One PDO indicator was used to assess achievement of this objective: proportion of births assisted by skilled personnel. Its target was substantially achieved: 79 percent of births were assisted by skilled personnel (revised target was 88 Page 16 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) percent, achievement against the target is 90 percent). However, out of the two intermediate indicators linked to this improved quality of health services objective, only one was substantially achieved and the other one partially achieved:1. Health personnel receiving training: 3,700 (target was 4,000, baseline 0, achievement against the target 93 percent) and 2. Percentage of facilities with no contraceptive stock-outs in last three months: 90 percent (target was 100 percent, baseline 71.1 percent, achievement against target 65 percent). As indicated earlier, a strong focus was put on quality improvement within the RBF approach. A quality checklist was administered every quarter to health facilities and impacted the amount of RBF bonuses that was received. The graph in Annex 1 highlights the positive trend on specific services quality from the start of the RBF (quarter 1 of 2014) to end of 2017. For maternal and reproductive health services (antenatal care, assisted delivery and family planning), in four years the quality score improved from between 31-36 percent to more than 75 percent. On ICMI quality, it seems that the level of quality was quite high at the launch of the RBF (78 percent) and was at same level by end of 2017 (with some variation during project implementation) (graphs in Annex 1). For quality of HIV/AIDS services, there was no specific indicator to measure this in the RF, but this was measured as part of the RBF’s technical quality score related to monitoring of people living with HIV under ARV. The quality of the care did not improve much as it went from an annual average score of 39 percent in 2015 to 39.75 percent in 2017. Additionally, results of the IE demonstrated that the RBF had a positive impact on availability of key infrastructure (such as a better availability of power and safe water at 55 percent, same as in baseline but decreased availability in control facilities) and ANC routine services (in comparison with status quo), but no effect on drug availability and negative impacts on quality of child care services and perceived quality of care. Based on achieved of PDO indicators and IO indicators, as well as results from the IE, the rating for this objective is Modest. 3- Increased utilization and quality of maternal and child health, reproductive health and HIV/AIDS services for the poor and vulnerable: Rating-Modest This equity related objective of the Project was specifically added through the 2016 restructuring and AF of the Project to reflect interventions to improve utilization and quality of health services for the poor through RBF and demand-side interventions. User fees for some key health services were indeed preventing poor and vulnerable people to access services and thus a focus was put on limiting inequalities. That is the reason why, some demand-side interventions were combined to the standard supply-side RBF interventions, namely: targeting of the poorest (‘indigents’), subsidization of health services for the poorest, additional providers’ incentives to offer services to the poorest for free and developing community-based health insurance (CBHI). Some of these interventions were already implemented prior to the restructuring. Details of the interventions and design of the RBF Impact Evaluation are described in Annex 6. The Project has successfully supported the extension of the Community-Based health insurance, which is now covering 40 percent of communes across Burkina-Faso, and complementarily supported the focus of RBF on providing health services for the poorest free of charge. 17 health insurance schemes were put in place in the Boucle du Mouhoun region and enrollment started in May 2015. ID cards for Community Based Health insurance enrollees have been finalized and distributed from November 2015. Then, from March 2017 to March 2018, the set-up of 71 new health mutuals was financed as well as nine regional unions, thanks to a contract with the NGO ASMADE which relied on local organizations Page 17 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) to support the health mutual roll-out. The support from the Project was building the foundation for the expected roll- out of the national health insurance scheme. The following PDO indicator was used to assess achievement of this objective on utilization: number of poor people and vulnerable persons who receive health services free of charge. At completion, 130,550 poor people have benefited of health services free of charge (target was 100,000). And 14 percent of the target population was enrolled in CBHIs scheme, while the target was 15 percent (intermediate indicator). However, from the results framework and available other data sources, it is not possible to assess the improvement of quality of care for the poor and vulnerable. Despite these successes, the IE provides a more mixed picture. Overall, the IE showed that the RBF had an impact on the use of modern family planning among the poorest 20 percent (+7.6 percentage point in comparison with control groups). These impacts were even higher when RBF was combined with demand-side interventions (CBHI or subsidization of services provided to the poorest). On health status, no impact was found, except for reduction of severe acute child malnutrition among the poorest 20 percent. However, results from the IE didn’t demonstrate clear impact of the PBF (even combined with demand-side interventions) on the utilization of preventive child services for the poorest or other maternal or reproductive health services. Overall, sub-component 1.3, which was directly addressing this equity issue, represented less than 6 percent of overall Project financing at closing. While the Project supported setting up the CBHI approach, its impact was limited on improving utilization of health services for the poorest. Table 4: Summary of indicators achievement PDO Indicators Intermediate Total Outcome Indicators Surpassed (100%+) 2 2 Achieved/Substantially (85%+) 2 5 7 Partially Achieved (65%-84%) 2 2 4 Not Achieved (less than 64%) 1 1 Total 6 8 14 % surpassed and achieved 67% 63% 64% Page 18 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Justification of Overall Efficacy Rating Efficacy Rating based on achievement of indicators (and additional data) Outcome 1: To improve utilization of maternal and child health, Modest reproductive health and HIV/AIDS services; Outcome 2: To improve quality of maternal and child health, Modest reproductive health and HIV/AIDS services; and Outcome 3: To increase utilization of these services for the poor and Modest vulnerable. Overall rating Modest C. EFFICIENCY Assessment of Efficiency and Rating- Substantial A cost-benefit analysis was carried out in 2011 for the original Project (but not updated in 2016 for the AF due to the limited available data to develop a formal cost-benefit analysis). The initial cost benefit analysis estimated that the Project would yield to significant economic returns to Burkina-Faso by improving utilization and quality of reproductive, maternal, child health and HIV/AIDS services. It was estimated that the Project would generate a return of around 35 percent. Details of the Efficiency assessment are provided in Annex 4. Implementation efficiency is considered Substantial. Implementation arrangements of the Project (using existing and performing implementing agencies with the PADS and SP/CNLS) were defined to ensure a smooth implementation of the Project activities. During the life of the Project, a comprehensive Procedure Manual and a detailed operational manual for the Results-Based Financing component were used by implementing entities. Despite delays to declare effectiveness (9 months delay) and some challenges during implementation (financial constraint for the RBF component, and issues with management of the Project from the PADS after 2016), these issues were addressed in a timely manner. The Project was implemented efficiently in terms of procurement, financial management and RBF activities. Moreover, the Project was implemented in a difficult context, considering the political instability and security issues in-country, but still managed to disburse more than 98 percent of the allocated financing was disbursed (original IDA Grant, HRITF Trust Funds and Additional Financing) and all planned activities were implemented. Allocative efficiency is considered Substantial. Global evidence shows that high impact reproductive, maternal, child and HIV/AIDS interventions, as the ones supported by the Project, are very much cost-effective. For instance, training of community health workers and midwives show a cost of 150 to US$1,000 per DALY 2 averted depending on the 2DALYs are a measure of overall burden of disease, expressed as the number of years lost due to ill-health, disability or early death. Page 19 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) national context, as well as safe motherhood initiatives based on a package combining ante and post-natal care by trained health attendants (all figures, as reported in DCP3, volume 2, chapter 17). The Project also contributed to improve allocative efficiency in the health sector with the Results-Based Financing approach. The efficiency of the project was assessed considering its costs and its direct and indirect benefits (details in Annex 4). Impacts of the Project will be conserved by the benefits for the next generation. Indeed, investments in maternal, child and reproductive health have intergenerational payoffs. Some of the benefits of the Project investment are not short-term effects; the returns on investment will take longer than the project implementation period to emerge. These investments have strong impact on the development of human capital of the children (better health, nutrition and education outcomes for children from healthier, better educated and more autonomous mothers). Therefore, investing in child, maternal and reproductive health contributes to a virtuous cycle of better-health and socioeconomic status, and also impacts the development and economic growth of a country. D. JUSTIFICATION OF OVERALL OUTCOME RATING Project Relevance High Efficacy Modest Efficiency Substantial Outcome Rating Moderately Unsatisfactory E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender The Project directly targeted women and adolescents, as the focus was on improving maternal and reproductive health utilization and quality of care and access to HIV/AIDS services and prevention. By reducing gender bias against women particularly in their access to quality health care and by tackling the high rate of fertility with a well- functioning family planning system, the Project intended to improve their socio-economic conditions. Results presented in the Results Framework in Annex 1 shows that the Project contributed to improvements in their utilization in key maternal, reproductive and HIV/AIDS services. Institutional Strengthening The Project, through the RBF approach and expanding the Community-Based Health Insurance, contributed strongly to strengthen Burkina-Faso health institutions. Indeed, as part of the Results-Based Financing component, health personnel were trained to the approach, technical assistance (national and international) was provided and mechanisms to transfer competencies were put in place. Additionally, a community-based approach to target the poor was set up, in collaboration with Social Protection, and CBHIs was expanded significantly with the Project financing, building the basis for a national health insurance in Burkina-Faso. Page 20 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Poverty Reduction and Shared Prosperity The Project was directly pro-poor as it targeted the most vulnerable households in the target regions of Burkina-Faso, and specific mechanisms to ensure free access to health services to the identified poorest was implemented, combined with the RBF approach. Additionally, the RBF and CBHI programs supported indirectly employment and financial resources into villages and vulnerable communities. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION Overall, the following key factors during preparation impacted positively implementation and outcome of the Project: - The Government of Burkina-Faso was taking a more active approach to tackle issues related to population growth. At the time of project preparation, initiatives from other donors to improve modern contraceptive prevalence, were not met with opposition, rather some traditional and religious leaders were championing progress on family planning. - Using the PADS, experienced unit and pooled fund for other partners. Building on lessons learnt from the previous HIV/AIDS Bank project in Burkina-Faso, the PADS was used to implement the Reproductive Health Project. The PADS had significant and demonstrated experience to manage contracts between central and local levels, and strong multidisciplinary team. Additionally, other donors were also engaged in the PADS (GAVI, Global Funds, WHO, UNFPA, UNICEF, Sweden and the Dutch Cooperation), which facilitated donors’ coordination. - RBF pre-pilot experience in country and consensus from Government on the policy. Results of an independent evaluation of the RBF pre-pilot (implemented between 2011-2013 in three districts Titao, Leo and Boulsa) showed positive effect in utilization and quality of services, but also demonstrating the need to add equity measures to ensure access to health care services for the poorest. There was a clear buy-in from the Government for the Results-Based Financing approach and the creation within the MoH in October 2009 of a highly competent RBF Technical Unit prior to project effectiveness (who designed and led implementation of the pre-pilot, produced RBF tools and manuals). B. KEY FACTORS DURING IMPLEMENTATION Factors that positively affected project implementation: - Despite long delays between Board Approval (December 2011), Financing Agreement signing (February 14th 2012) and effectiveness (October 2012, institutional delays), disbursements then rose quickly due to well-planned roll-out of the RBF component and preparedness of other interventions under component 2 of the project (baseline Impact Evaluation data collection end of 2013; training of health providers, managers, community verifiers; launch of majors procurement (equipment, commodities, contract and verification agencies, etc..). Page 21 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Factors subject to implementing entities control: - Institutional changes at the Ministry of Public Health and Hygiene (five different Ministers and five general secretaries during implementation of the Project). These changes did not facilitate the understanding and sustainability of the innovative RBF approach. This also contributed, as noticed during the MTR, to the weak-buy in by other government actors. In addition, the coordinator of the PADS was unexpectedly replaced in 2016; the new one was nominated without any competitive hiring process. - The RBF technical unit (CCTN FBR) institutional changes from one direction to another during Project implementation also affected negatively Project implementation. Initially, the RBF technical unit was in a position to have significant leverage to influence and contribute to health sector policies, as it was under the General Secretariat of the Ministry of Health, but after some changes it ended up under a technical directorate in early 2015. The delays in RBF payments (more than 6 months in 2016) were due in part to lack of funds but also lack of coordination between MoH’s different departments, especially between the PADS and the RBF technical unit establish under a technical directorate without any functional reporting relationship. - Introduction of a nation-wide user fee exemption policy in June 2016. The free health care policy targeted pregnant women as well as under five children, thus covering many of the services incentivized by PBF, started in April 2016 in two cities (Ouagadougou and Bobo-Dioulasso) and was scaled-up nationwide two months later. It may have diverted the focus of health care providers from preventive services already free, in favor of delivering services both incentivize by RBF and reimbursed by the free health care policy. - Decreased performance of the PADS and lack of proactivity from the Ministry of Health to address some key issues that were negatively affecting project implementation and outcomes. Financial gap of the RBF program (2016 before effectiveness of the Additional Financing and 2017 before reallocation of funds between categories) constrained the RBF program and outcomes and created demotivation for health providers and uncertainties on the program. While the PADS was ‘well integrated’ within the MoH at the project design state and first years of implementation, over time during implementation the relationship between PADS and MoH changed. It turned into some sort of “competition” between the two entities. That is also the reason why the management of the new health project was moved from PADS to MoH. It took few months for the issue of the decreased performance of the PADS from 2016 (due to changes in PADS staff) to be fixed by the Ministry of Health. Additionally, mid-2016, the National Purchasing Center for the generic and essentials drugs (CAMEG) experienced an unprecedented institutional crisis that led to a national medicines stock-outs including contraceptive and AAV vaccines in 2017. Factors outside of Government control: - Political crisis and instability. A political and security crisis started in 2011 (April 2011: presidential guards mutiny and thousands of people protest over food prices) and culminated in widespread population protests: July 2013, thousands of demonstrators protested over plans to create a Senate and in January 2014, demonstrators across the country opposed plans by President Compaore to prolong his term. That Page 22 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) led to the change of Government in October 2014, marking a historic turning point for the country. The political turmoil was concluded in democratic elections in November 2015. Despite the prevalent security context in the Sahel region, and deterioration of the situation in Burkina-Faso during the time of implementation of the Project (for example, Islamist attack on hotel and café in the capital in January 2016, another attack in Ouagadougou in August 2017, attack on French Embassy in March 2018),the Project succeeded to implement all planned activities and fully disbursed. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design The theory change behind the design of the M&E for the Project was clear, with a straightforward PDO and indicators defined to measure progress on the different interventions financed by the Project. However, and as discussed in the Efficacy section, a limited number of indicators were related to the quality of care which was one of the expected outcomes of the Project. The design of the Project included a robust monitoring and evaluation system: M&E arrangements were well-embedded institutionally as data collection was aligned to the national health information and management system (HMIS), i.e. same indicators were collected to avoid additional data collection and creating a parallel system, and quality of data was strengthened with the RBF approach. Indeed, it was planned that quantitative indicators bought through RBF contracts (most of them part of the RF) would be verified monthly by third parties (ACV-Contract and Verification Agencies) and some counter-verification would also take place every six months (by independent agents) on quantitative and qualitative indicators. PDO level indicators were thus measured by the HMIS (and reported in the Annual Health Statistic Reports) and quality of data was ensured by the RBF mechanism. Additionally, the monitoring system included training and capacity building at national, regional and local levels and annual program evaluations and strategic planning exercise for each component. With 2016 restructuring and AF of the Project, the design of the RF was improved (indicators that were monitored using coverage were converted into indicators in cumulative numbers to be consistent with core indicators requirements and the nature of data collected with the RBF). Targets were also aligned at that time with the extension of the closing date. However, the RF showed some significant limitations to allow the assessment of the different outcomes of the PDO. While the PDO was expanded to an equity outcome (with the target of the poor and vulnerable), the indicators to measure this outcome were limited. Moreover, indicators to measure quality of services were not added nor improved, which required the use of proxies to assess quality of care outcome of the revised PDO. A rigorous RBF Impact Evaluation was also part of the M&E design of the Project (the IE was planned to be prospective, controlled and randomized). Indicators of the IE were the same as the ones in the RF, and the National Institute of Public Health (at the end, it was Centre Muraz which was selected as the most qualified of national research institutes) would be involved with an international firm (Heidelberg University, who was well respected and trusted in the country, having close to five decades of experience working with and investing in the capacity of Burkinabe research institutes), thus strengthening its capacity. Results of the IE would inform the impact of the RBF program. Page 23 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) M&E Implementation Regular data collection was done as part of the national health management information system, providing data to inform the results framework as required to closely monitor progress of the Project. Additionally, the rigorous data collection set-up as part of the RBF approach were functional. However, some aspects of the RBF verification mechanism were dropped by the MoH as perceived too costly (such as the community verification which or the Contracting and Verification Agencies replaced by local health staff in charge of verification). The Bank raised concerns that the monitoring system would become weaker because of these changes from the original design. The teams from the Ministry of Health (especially the RBF technical unit) and the CT/CNLS had strong M&E experts who produced regular reports on progress towards the Project indicators. A cloud computing system was set up (www.fbrburkina.org) to enter RBF data and allowing monitoring of the results. However, this RBF web-based portal was not anymore functional the last few months of the Project, creating some issues for the RBF technical unit to produce proper RBF invoices. The Impact Evaluation of the RBF was completed (Baseline data collection took place between October 2013 and March 2014 and endline between April and June 2017) and results were presented and discussed in-country. IE baseline data collection took place and endline data collection happened (12 intervention districts out of the 15 where the RBF was implemented through the Project, as the 3 other ones were part of the RBF pre-pilot thus excluded from the Impact Evaluation). The IE results were presented, discussed and validated through several forums, first through a retreat in Heidelberg with a select number of stakeholders and implementers to explore initial results and try to understand them, followed by several in-country workshops that included national, regional and district level implementers. An additional qualitative study was conducted to convert the IE to a mixed methods evaluation in an attempt to better understand the factors that contributed to the results found through the IE. M&E Utilization Data collected during the Project were used to inform on the overall project achievements and recommendations were made for timely actions. Annual and detailed RBF reports were produced from 2014 to 2017 by the RBF technical unit, detailing the results achieved by health facilities, trends, but also comparing RBF intervention districts with control health districts and the national average. Results from the RBF bi-annual counter-verification were shared at regional level during workshops, thus ensuring corrective measures to be taken by health care providers. Additionally, revisions on the RBF operational manual were based on the progress on the quantitative indicators, adjusting incentives to results monitored and verified. Justification of Overall Rating of Quality of M&E The M&E design of the Project is rated as Substantial: Overall rating M&E Design M&E Implementation M&E Utilization Substantial Modest Substantial Substantial Page 24 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social Safeguards According to the OP 4.01 on Environmental Assessment, the Project was classified as Environmental Category “B” and the environmental classification was due to the expected increase of medical equipment and drugs through the Project interventions, thus increased medical waste. A Medical Waste Management Plan was prepared and disclosed in country in March 2011. No construction works were planned under the Project (only minor refurbishing). The MWMP has been regularly monitored during project implementation and judged satisfactorily (ISR, December 2017). Aspects related to waste management were also monitored as part of the RBF quality check list. Additionally, some health facilities used the RBF bonuses to improve hazardous medical waste management and bought incinerators. The Project did not pose any social Safeguards risk and was expected to have a positive social impact by strengthening community voice through monitoring of the quality of health services (part of the RBF approach) and also improving health care services access, especially for the poorest. Annual community client satisfaction surveys were conducted to track patient satisfaction, grievances and provided feedback to health facilities on ways to improve service delivery. In addition, mechanisms were in place for grievances related to fiduciary aspects such as procurement processes implemented by the PIU. Fiduciary Fiduciary management system was built on strong and experienced teams used to Bank procedures, which explains strong fiduciary performance until 2016.The PADS-Programme d’Appui au Développement Sanitaire (entity in charge of managing the fiduciary aspects of the Project, except Component 2.4 that was managed by the SP-CNLS-IST as part of 2016 restructuring) was a strong team, used to Bank procedures and with experienced and multidisciplinary staff, managing also other WB Project such as the SWEDD (Sahel Women’s Empowerment and Demographic Dividend Regional Project). The SP/CNLS-IST had the fiduciary responsibility (as well as technical and implementation) for the component 2.4 (demand creation strengthening HIV prevention to support PLWHA and reproductive health services for youth, women and high-risk groups) introduced through the 2016 restructuring. The PADS and SP/CNLS managed the previous WB Project in Burkina-Faso (Health Sector Support and AIDS Project-P093987) that closed in December 2014. But changes in the PADS team in 2016 impacted negatively the performance of the Project (until then, FM and procurement were rated Satisfactorily). Specifically, the Project Coordinator, who the Bank found as a high performing coordinator, was removed. Both ratings for procurement and financial management were thus downgraded to moderately satisfactory end of 2016. These delays in payment from the PADS, especially on RBF payments, have resulted in stagnating results of the program. In addition, procurement for activities related to Component 2, in particular the reproductive health subcomponents, were significantly delayed (while those related to HIV/AIDS and UHC executed on time). Despite these issues, Project fully disbursed at closing and implemented all planned activities. As detailed in Section III about Key factors, it affected implementation and outcomes of the Project. Page 25 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) C. BANK PERFORMANCE Quality at Entry The ICR rates the Bank performance during project preparation as Satisfactory. Design of the Project. The design of the Project with two components (increasing performance of health service delivery with RBF and supporting critical inputs, including strengthening demand for reproductive health) was addressing major issues identified to improve reproductive health. A thorough analysis of the sector was done to propose the design, notably using latest results of the Demographic and Health Survey 2010 but also the evaluation of needs in Emergency Obstetric and Neo-natal care (conducted in 2011 by the Bank, UNFPA, UNICEF and WHO). Additionally, the technical design of the proposed Project was based on promising results of a pre-pilot experience of Results-Based Financing in three health districts in Burkina-Faso, as well as international RBF lessons (such as ensuring autonomy of health facilities, involvement of external entities for verification, decentralization of decision- making, training of health workers and supervisors, …). The package of health services to be incentivized with the RBF approach (especially on maternal, reproductive and child health) was evidenced based (high impact interventions). To mitigate the risks associated with the Project (rated Substantial at appraisal, mainly due to the introduction of the RBF), strong technical assistance, training and gradual scale-up were part of the design of the Project. Government commitment and institutional arrangements. Before the start of the Project, that was a political commitment to introduce Results-Based Financing in the health sector to improve health service delivery. To ensure a smooth implementation of the Project, institutional arrangements proposed by the Bank team used existing and performing financial and disbursement arrangements already in place, using the PADS and the SP/CNLS (both already managed the previous health Bank Project). To mitigate Capacities strengthening of the PADS was adequately planned. Strong monitoring and evaluation system and learning agenda/capacity strengthening. A strong monitoring and evaluation system, combined with capacity strengthening and a clear learning agenda with the Impact Evaluation, was embedded in the Project from the start. Quality of Supervision Despite delays (for effectiveness of the Project and processing of the last restructuring), that were beyond the Bank’s control, the ICR rates the Bank performance during project supervision as Satisfactory. The task leadership was changed only one-time during Project implementation (in 2017 and the incoming TTL was part of the task team and co-TTL before of the Project), thus ensuring a smooth transition. The first TTL (who was also the TTL during preparation) was based in-country, which allowed a close supervision over implementation of the Project. Thirteen ISRs have been produced (adequate for duration of the project) which were well detailed on the implementation status, progress towards achievement of the PDO and keys issues. The Mid-Term review mission in July 2015 was used to assess successes of the RBF approach but also have highlighted some bottlenecks and proposed corrective measures. Throughout Project implementation, the Bank team provided strong technical support and capacity strengthening Page 26 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) on Results-Based Financing: South-South exchanges were facilitated and an international expert on RBF (consultant) was part of missions conducted by the team. The Bank team led a strong policy dialogue on the RBF approach with the Government and with other partners involved in the sector (such as UNFPA, UNICEF, WHO…etc), to ensure sustainability of the mechanism. The Bank team demonstrated proactivity to supervise the Project, even though there were delays from the Government side to fix them. To address the financing gaps identified on the RBF component, the 2016 Additional Financing and Restructuring was processed, and the January 2018 was also processed by the Bank to reallocate financing between disbursement categories. When faced with the decline of the PADS performance in 2016: specific missions were organized to tackle the issue and management was involved in meetings with the Minister of Health as well as the Minister of Economy, Finance and Development. Justification of Overall Rating of Bank Performance Overall, the ICR rates Bank performance over the entire life-span as Satisfactorily, as both quality at entry and quality of supervision are rated Satisfactorily. D. RISK TO DEVELOPMENT OUTCOME The risk to development outcome is considered to be Moderate given current national priorities in Burkina-Faso to improve human capital (PNDES-Plan National de Développement Economique et Social 2016-2020), and specifically on promoting population health and accelerating the demographic transition. Commitment of the Government of Burkina-Faso on progressing towards Universal Health Coverage is high, and improving maternal, reproductive and child health is key to do so. Government commitment remains high (relative to other countries), although the security situation makes it harder to follow-through on commitments. Furthermore, sustainability of project investments can also be guaranteed by the new World Bank-financed Project that was approved by the Board in July 2018 (Health Services Reinforcement Project, P164696) and RBF will be scaled up substantially as part of the national strategic purchasing strategy. Indeed, in 2018, the Ministry of Health has been working on developing a Strategic Purchasing document which is building on the lessons learnt from the RBF approach. Additionally, Burkina-Faso is now part of the Global Financing Facility (GFF) for Every Woman Every Child, and the GFF Investment Case would support priority high-impact interventions sustaining interventions and results achieved as part of component 2 of the Reproductive Health Project. V. LESSONS AND RECOMMENDATIONS The Reproductive Health Project provides useful lessons that could inform other health projects, especially those including Results-Based Financing approach: - The RBF needs to be integrated as a reform process. To allow application of best practices and then full potential of the approach, additional reforms needs to happen. For example, the RBF mechanism is a way to better allocate domestic resources to health facilities and ensure its sustainability. The RBF technical unit should also be put at a level where it can influence policy decision to integrate RBF reforms. Page 27 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) - A strong national leadership, institutional anchoring of the RBF technical unit and a stable and predictable environment is key to ensure a steady and successful implementation of a reform such as the Results-Based Financing strategy. It was observed in Burkina-Faso some institutional instability at the Ministerial level, specifically turnover of staff and decisions-makers, and multiple changes in the anchoring of the RBF technical unit (which ended up under a technical direction). Both impacted the buy- in of Results-Based Financing as an overall approach to strengthen the performance of the health system by the different departments of the Ministry of Health. Moreover, health financing became very political during political instabilities in-country with different groups with their own priorities (pro-PBF, pro-free health care, pro-national health insurance) resulting in a vicious cycle of building up and tearing down the RBF program. - Delays in payment of RBF bonuses could affect negatively the impact of the approach and its credibility, as seen in other RBF countries. During implementation of the Reproductive Health Project, financial constraints and delays in payments at different times created uncertainties and demotivation of health services providers. It also prevented them to do the planned investments on-time to improve their health facilities and service delivery, impacting health results. - Embedding a strong research agenda and technical assistance in the design of the Project ensure a smoother implementation and transfer of capacities. The RBF Impact Evaluation was conducted by the University of Heidelberg and the Centre Muraz of Burkina Faso and provided insightful results on the impact of the interventions financed and informed the development of new health project. - Implementing prevention and sensibilization activities related to reproductive health and HIV/AIDS services through local and experienced NGOs is a successful approach. The SP/CNLS-IST relied on local NGOs to target youth, women and high-risk groups for prevention and sensibilization, and targets were substantially achieved, and messages and activities developed were appropriate to target groups and local context. - Different mechanisms of health financing (free health care, RBF and CBHIs) needs to be better integrated. While the design of the Reproductive Health Project had proposed a synergy between RBF and Community-Based Health Insurance to strengthen health results, the introduction of the nationwide free health care policy in 2016 without considering a streamlined and integrated RBF and free care services approach created potentially contrary incentives for health providers. The design of the new health Project is aiming at reducing inconsistencies on health financing with a clear focus on a strategic purchasing vision and health insurance and helping coordination and institutional stability by sharing within and outside the ministry. The current health project in Burkina-Faso also built on the lessons learnt from this Project: management of the Project is within the MoH (not anymore with the PADS) and the Project is supporting harmonization and . alignment of health financing reforms, and expanding strategic purchasing mechanisms (RBF). Page 28 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To improve utilization of maternal and child health, reproductive health and HIV/AIDS services in the Recipient's territory Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion New acceptors of modern Number 0.00 0.00 770000.00 586980.00 contraceptive methods (number) 19-Dec-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 Comments (achievements against targets): Achievement against target: 76% Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of women who Percentage 0.00 0.00 42.00 58.70 received postnatal consultation between the 6th 01-Jan-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 days/6th week of delivery Page 29 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Comments (achievements against targets): Achievement against target: 140% Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Children immunized Number 0.00 0.00 860000.00 583566.00 (number) 18-Apr-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 Comments (achievements against targets): Achievement against target: 68% Cumulative number for fully immunized children (under 1 year old). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of pregnant women Number 0.00 0.00 6135.00 5348.00 living with HIV who received antiretroviral to reduce the 01-Jan-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 risk of MTCT (number) Comments (achievements against targets): Achievement against target: 87% Page 30 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Objective/Outcome: To improve quality of maternal and child health, reproductive health and HIV/AIDS services in the Recipient's territory Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of births assisted Percentage 0.00 80.00 88.00 79.09 by skilled personnel 01-Jan-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 Comments (achievements against targets): Achievement against target: 90% Objective/Outcome: To increase utilization and quality for the poor and vulnerable of maternal and child health, reproductive health and HIV/AIDS services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of poor and Number 0.00 0.00 100000.00 130550.00 vulnerable persons who receive health services free 01-Jan-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 of charge Comments (achievements against targets): Achievement against target: 131% Page 31 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) A.2 Intermediate Results Indicators Component: Component 1: improving the delivery and quality of a Reproductive Health Service Package through Results-Based Financing Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of pregnant Percentage 0.00 0.00 86.00 75.09 women receiving at least 2 doses of anti-tetanus 01-Jan-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 immunization Comments (achievements against targets): Achievement against target: 87% Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of facilities that Percentage 0.00 0.00 80.00 0.00 received RBF credits on time 31-May-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 Comments (achievements against targets): Achievement against target: 0% (important delays and RBF portal non operational. Payments for last quarters were not made under 35 days). Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 32 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Target Completion Percentage of pregnant Percentage 0.00 0.00 76.50 69.00 women receiving at least 2 antenatal care visits during a 31-May-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 pregnancy Comments (achievements against targets): Achievement against target: 90% Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of Cesarean- Percentage 0.00 0.00 3.10 2.43 sections among planned pregnancy 01-Jan-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 Comments (achievements against targets): Achievement against target: 78% Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of target Percentage 0.00 0.00 15.00 14.00 population enrolled in community-based health 01-Jan-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 Page 33 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) insurance schemes Comments (achievements against targets): Achievement against target: 93% Component: Component 2: Support critical inputs for reproductive health and HIV/AIDS services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Health personnel receiving Number 0.00 0.00 4000.00 3700.00 training [cumulative] 31-May-2012 31-Dec-2016 30-Jun-2018 30-Jun-2018 Comments (achievements against targets): Achievement against target: 93% Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of pregnant women Number 0.00 0.00 889090.00 874064.00 receiving VCT services 02-Feb-2016 31-Dec-2016 30-Jun-2018 30-Jun-2018 Comments (achievements against targets): Page 34 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Achievement against target: 98% Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of facilities with Percentage 71.10 0.00 100.00 90.00 no contraceptive stock-outs in last three months 31-Dec-2010 31-Dec-2016 30-Jun-2018 30-Jun-2018 Comments (achievements against targets): Achievement against target: 65% Page 35 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) B. KEY OUTPUTS BY COMPONENT Revised PDO: To improve the utilization and quality of maternal and child health, reproductive health and HIV/AIDS services in the Recipient's territory with a particular focus on the poor and vulnerable. Objective/Outcome 1: To improve utilization of maternal and child health, reproductive health and HIV/AIDS services 1.New acceptors of modern contraceptive methods 2. Children immunized (number) 3. Percentage of women who received postnatal consultation Outcome Indicators between the 6th days/6th week of delivery 4. Number of pregnant women living with HIV who received antiretroviral to reduce the risk of PMTCT 1. Percentage of pregnant women receiving at least 2 doses of anti- tetanus immunization 2. Percentage of facilities that received RBF credits on time Intermediate Results Indicators 3. Percentage of pregnant women receiving at least 2 antenatal care visits during a pregnancy 4. Percentage of Cesarean-sections among planned pregnancy 5. Number of pregnant women receiving VCT services 1.Roll-out of the RBF approach in 15 health districts (25% of the Key Outputs by Component country) (linked to the achievement of the Objective/Outcome 1) 2. NGOs recruited to promote demand for HIV/AIDS services, FP and reproductive health services Objective/Outcome 2: To improve quality of maternal and child health, reproductive health and HIV/AIDS services 1. Proportion of birth assisted by skilled personnel Outcome Indicators Intermediate Results Indicators 1. Health personnel receiving training Page 36 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) 2. Percentage of facilities with no contraceptive stock-outs in last three months 1. Pre-service training of 120 midwives and nurses, and in-service Key Outputs by Component training (linked to the achievement of the Objective/Outcome 2) 2. Procurement of FP commodities and equipment Objective/Outcome 3: To increase utilization for the poor and vulnerable of maternal and child health, reproductive health and HIV/AIDS services 1. Number of poor and vulnerable persons who receive health Outcome Indicators services free of charge 1. Percentage of target population enrolled in community-based Intermediate Results Indicators health insurance scheme 1. Set-up of 71 new health mutuals Key Outputs by Component 2. Identification/targeting of the poorest with community-based (linked to the achievement of the Objective/Outcome 2) approach Additional data on improvement of quality of care: Page 37 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Figure 3: Evolution of RBF quality score for clinical services 90% 80% 79% 78% 70% 75% 60% 50% 40% 32% 36% 30% 31% 20% 10% 0% T1 T2 T3 T4 T1 T2 T3 T4 T12016 T2 T3 T4 T1 T2 T3 T4 2014 2014 2014 2014 2015 2015 2015 2015 2016 2016 2016 2017 2017 2017 2017 PCIME CPN Accouchement PF Source: RBF annual report 2017 (PCIME= IMCI; CPN=ANC; Accouchement= Assisted delivery; PF= Family Planning) The following graph demonstrates that the quality improvements, measured by the quality check-list, happened in health facilities at the different levels (primary to regional hospitals): Page 38 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Figure 4: Evolution of quality score at different levels of health care between 2014 and 2017 1er 90% Échelo 76% 79% 77% 73% 77% n 80% 71% 73% 74% 72% 75% 75% 74% 66% CMA 70% 61% 57% 76% 67% 72% 60% 71% 73% 74% 71% 72% 69% Score moyen 68% 45% 66% 63% 63% 64% 62% 67% 64% 64% 64% 63% 64% 61% 64% 67% 63% CHR 50% 58% 60% 59% 40% 52% 47% 49% 30% 34% 20% 10% 0% T1 2014 T2 2014 T3 2014 T4 2014 T1 2015 T2 2015 T3 2015 T4 2015 T12016 T2 2016 T3 2016 T4 2016 T1 2017 T2 2017 T3 2017 T4 2017 Source: RBF annual report 2017 Rating: Objective 1 PDO indicators Intermediate Outcome Total indicators To improve utilization of maternal and child health, reproductive health and HIV/AIDS services Surpassed (100%+) 1 1 Achieved/Substantially (85%+) 1 3 4 Partially achieved (65-85%) 2 1 3 Not achieved (less than 64%) 1 1 Total 4 5 9 % surpassed and achieved 50% 60% 56% Page 39 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Rating Objective 2 PDO indicators Intermediate Outcome Total indicators To improve quality of maternal and child health, reproductive health and HIV/AIDS services Surpassed (100%+) Achieved/Substantially (85%+) 1 1 2 Partially achieved (65-85%) 1 1 Not achieved (less than 64%) Total 1 2 3 % surpassed and achieved 100% 50% 75% Rating Objective 3 PDO indicators Intermediate Outcome Total indicators To increase utilization and quality for the poor and vulnerable of maternal and child health, reproductive health and HIV/AIDS services Surpassed (100%+) 1 1 Achieved/Substantially (85%+) 1 1 Partially achieved (65-85%) Not achieved (less than 64%) Total 1 1 2 % surpassed and achieved 100% 100% 100% Page 40 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Overall Rating PDO indicators Intermediate Outcome Total indicators Surpassed (100%+) 2 0 2 Achieved/Substantially (85%+) 2 5 7 Partially achieved (65-85%) 2 2 4 Not achieved (less than 64%) 1 1 Total 6 8 14 % surpassed and achieved 67% 62.5% 64% Page 41 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Paul Jacob Robyn, Zenab Konkobo Kouanda Task Team Leader(s) Mohamed El Hafedh Hendah, Haoussia Tchaoussala Procurement Specialist(s) Sandrine Egoue Ngasseu Financial Management Specialist Alexandra C. Sperling Counsel Sariette Jene M. C. Jippe Team Member Supriya Madhavan Team Member Haidara Ousmane Diadie Team Member Fatoumata Diallo Social Safeguards Specialist Leandre Yameogo Environmental Safeguards Specialist Joel Arthur Kiendrebeogo Team Member Roxane Sylvie Maria Medah Bapuuroh Team Member Benoit Mathivet Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY10 .122 3,842.30 FY11 32.313 338,884.69 FY12 27.013 127,938.12 Total 59.45 470,665.11 Page 42 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Supervision/ICR FY12 7.700 54,054.07 FY13 30.340 231,768.75 FY14 22.922 193,663.61 FY15 25.380 165,637.62 FY16 35.189 280,878.30 FY17 30.442 108,245.65 FY18 45.282 107,009.12 FY19 1.297 3,109.16 FY20 .525 1,707.65 Total 199.08 1,146,073.93 Page 43 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) ANNEX 3. PROJECT COST BY COMPONENT Amount after Actual at Amount at Approval AF and Percentage of Components Project Closing (US$M) Restructuring Approval (US$M) (US$M) (US$M) Component 1- Improving the delivery and quality of a Reproductive Health 22.30 44.00 43.30 Service Package through Result-Based Financing Component 2- Supporting critical inputs for reproductive 19.30 32.6 33.30 health and HIV/AIDS services Total 41.60 76.60 75.40 98.4% Page 44 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) ANNEX 4. EFFICIENCY ANALYSIS Global evidence demonstrates that improving utilization and quality of maternal, child and reproductive health, and HIV/AIDS services leads to decrease in morbidity and mortality, thus promoting social and economic gains. Additionally, the rationale for public intervention in health is well established: health interventions are public goods and create externalities: the societal benefit of delivering these interventions exceeds their costs. Likewise, the negative externalities of not providing these interventions is a risk for all members of the society. Methodology. A cost-benefit analysis was carried out in 2011 for the original Project (but not updated in 2016 for the Additional Financing due to the limited available data to develop a formal cost-benefit analysis). The initial cost benefit analysis estimated that the Project would yield to significant economic returns to Burkina-Faso by improving utilization and quality of reproductive, maternal, child health and HIV/AIDS services. It was estimated that the Project would generate a return of around 35 percent. To assess the efficiency of the Project at closing, the following dimensions are analyzed: implementation and allocative efficiencies. Data from the results framework of the Project and the RBF Impact Evaluation are used for the efficiency analysis. Efficiency Definition Allocative efficiency The use of resources to meet strategic development priorities and improve welfare of population. Implementation efficiency The effectiveness of implementation arrangements to achieve project’s activities. 1- Implementation efficiency- Substantial Implementation arrangements of the Project (using existing and performing implementing agencies with the PADS and ST/PLNS) were defined to ensure a smooth implementation of the Project activities. During the life of the Project, a comprehensive Procedure Manual and a detailed operational manual for the Results-Based Financing component were used by implementing entities. Despite delays to declare effectiveness (9 months delay) and challenges during implementation (financial constraint for the RBF component, and issues with management of the Project from the PADS after 2016), these issues were addressed in a timely manner. The Project was implemented efficiently in terms of procurement, financial management and RBF activities. More than 98 percent of the allocated financing was disbursed (original IDA Grant, HRITF Trust Funds and Additional Financing) and all planned activities were implemented. Page 45 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) 2- Allocative efficiency-Substantial Allocative efficiency. Global evidence shows that high impact reproductive, maternal, child and HIV/AIDS interventions, as the ones supported by the Project, are very much cost-effective. For instance, training of community health workers and midwives show a cost of 150 to US$1,000 per DALY 3 averted depending on the national context, as well as safe motherhood initiatives based on a package combining ante and post-natal care by trained health attendants (all figures, as reported in DCP3, volume 2, chapter 17). The Project also contributed to improve allocative efficiency in the health sector with the Results-Based Financing approach. In the following paragraphs, the efficiency of the project is assessed considering its costs and its direct and indirect benefits. Project Costs. The cost for this Project was minimal: for component 1 (RBF), it is estimated at US$1.76 per person per year (considering actual cost of component 1 and population covered around 5 million for a 5- year project). For component 2 which was implemented nationwide, it is estimated at US$0.33 per person per year (considering actual cost of component 2 and population of Burkina-Faso for a 5-year project). Costs of the Project can also be compared to total and domestic health expenditures in Burkina-Faso: Health expenditure Share of RBF Share of Share of the per capita (current cost (cost per component 2 Project US$), 2016 capita, US$1.76) (cost per capita, (overall cost US$0.33 per capita, US$ 2.09) Total health 40.9 4.3% 0.8% 5.1% expenditures Domestic general 16.4 10.7% 2.0% 12.07% government health expenditure Source: WDI and project costs, ICR author calculations. Benefits. The project benefits comprised several measures of direct impact, which included increased utilization of maternal, child and reproductive health services and HIV/AIDS services, as well as improved quality of these services. From 2012 to its closure in 2018, the project contributed to provide enhanced health services to the population of targeted areas of the Project (detailed in Annex 1): 3DALYs are a measure of overall burden of disease, expressed as the number of years lost due to ill-health, disability or early death. Page 46 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) PDO level indicators 2011 2018 Comment New acceptors of modern contraceptive C methods 0 586,980 Proportion of birth assisted by skilled personnel C 0 79.09% Percentage of women who received postnatal C consultation between the 6th days/6th week of delivery 0 58.70% Children immunized (number) 0 583,566 C Number of poor people and vulnerable persons D who receive health services free of charge 0 130,550 Number of pregnant women living with HIV who C received antiretroviral to reduce the risk of PMTCT 0 5,348 Note : C : Contribution of the Project; D: Directly attributed to the Project With results from RBF IE, some key and significant benefits of the RBF approach can be directly attributed to the Project investments. The table below summarizes these results (only between control and RBF groups): Indicator PBF vs control Facility-based delivery 0.044* Three PNC visits 0.066* Modern family planning 0.076* Severe acute malnutrition (under five, poorest 20%) -0.066* Note: * p<0.1. All effect estimates pertain to absolute change. Effect estimates can be converted to percentages and reflect percentage point changes. Indirect and long-term benefits. Some interventions of the Project will have longer term benefits and will last after closing of the Project, such as for example the training (pre-service and in-service) of health workers. Additionally, one of the impacts from Project investments is an increase in use of modern family planning methods. Family planning has health benefits, such as reduced maternal deaths and disability, lower newborn, infant and child mortality, better health and nutrition for mothers, but also non-health related benefits: for example, higher private health, nutrition and education expenditures per child, women’s improved ability to participate in the labor force and at macro-level reduced public expenditures in education, healthcare and other social services. Therefore, sustainability of the Project is also ensured by the benefits for the next generation. Indeed, Page 47 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) investments in maternal, child and reproductive health have intergenerational payoffs. Some of the benefits of the Project investment are not short-term effects; the returns on investment will take longer than the project implementation period to emerge. These investments have strong impact on the development of human capital of the children (better health, nutrition and education outcomes for children from healthier, better educated and more autonomous mothers). Therefore, investing in child, maternal and reproductive health contributes to a virtuous cycle of better-health and socioeconomic status, and also impacts the development and economic growth of a country. Page 48 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Summary of the Completion Report for the Reproductive Health Project (based on the executive summary) BURKINA FASO ---------- Unité – Progrès – Justice ------------------- MINISTERE DE LA SANTE PRESIDENCE DU FASO ----------- ------------ SECRETARIAT GENERAL SECRETARIAT PERMANENT DU ------------ CONSEIL NATIONAL DE LUTTE PROGRAMME D’APPUI AU CONTRE LE SIDA ET LES IST DEVELOPPEMENT SANITAIRE -------------- -------------- UNITE DE GESTION FINANCIERE Burkina Faso benefits from World Bank financing to implement the Reproductive Health Project (Projet Santé de la Reproduction- PSR). This project aims to improve the delivery and quality of the reproductive health service package through the Results-Based Financing approach. The initial project was signed on February 14th, 2012 became effective on October 22nd, 2012 and was scheduled to close on December 31st, 2016. Burkina Faso has also benefited from an Additional Financing for this project. This Additional Financing, signed on April 04th, 2016, became effective on October 17th, 2016 and was scheduled to close on June 30, 2018. According to the provisions of the Financing Agreement between the Burkinabe Government and the World Bank, a final evaluation of the Project is planned at closing which happened on June 30, 2018. Thus, and to have data to assess objectively the Project in terms of inputs and results from the indicators, and also to draw necessary lessons for future interventions, the evaluation was completed by an individual consultant. Analysis of mechanisms for developing and adopting action plans. A committee for the elaboration of stakeholders' action plans takes into account the different frameworks (RBF and HIV/AIDS). The objectives to be achieved are determined as well as the evaluation indicators. Action plans are submitted to the monitoring committee which validates and adopts for the implementation of the activities. Indicators and measurement methods have been designed to rely as much as possible on pre-existing information systems and datasets. As part of the Project, Burkina Faso prepares and submits semi-annual project reports to the Bank no later than one month after the end of the semester. Page 49 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Analysis of funding mechanisms and transfer of funds. The beneficiaries, PADS and SP / CNLS-IST, have put in place a reliable and operational financial management system in line with the provisions of the H751-BF, TF11678 and D1080-BF financing agreements, characterized by: - the establishment of an adequate accounting system. Each beneficiary has a manual of administrative, financial and accounting management procedures and accounting software (TOM2PRO), an accounting plan that complies with international accounting standards. All accounting documentation and summary financial statements are available and archived by each beneficiary; - annually, the financial statements of the Reproductive Health Program (PSR) are established by the beneficiaries and submitted for financial audit by an independent external auditor. Annual audit reports and Interim Financial Reports (RSFs) are provided to IDA in the agreed periods; - in accordance with the World Bank's disbursement letters, the project opened five (05) designated accounts at the Central Bank of Burkina Faso (BCEAO): four (04) for the PADS and one (01) for the SP / CNLS- STI. The PADS has opened an account in a commercial bank and linked to the designated account to finance component 1. The SP / CNLS-IST opened one (01) account in a commercial bank linked to the designated account and two (02) accounts in the Public Treasury of which one (01) to receive the counterpart funds and the other one for the payments of the public structures of implementation of the Project. - PADS and SP / CNLS-IST each have a well-developed financial management team, including an Administrative and Financial Officer (RAF) and accountants; respectively an internal audit service and a functional internal control service. Analysis of the management procedures of resources allocated to the sectors and sub-sectors involved in the implementation of the project. The financial implementation of the Reproductive Health Project is satisfactory with regard to the level of disbursement (around 100%) and the consumption of financial resources, ie 92.74% with a completion rate of 90.56%. PSR funds are audited annually and audit reports are provided to the World Bank by June 30 of the following year. The SP / CNLS-IST conducted the financial audit for the 2017 fiscal year. The final audit report is available at the SP / CNLS-IST and transmitted to the World Bank on 26/06/2018. The auditor who completed the 2017 audit of the D1080 financing agreement is selected for the closing audit of the Project. At the PADS level, the financial audit for the 2017 financial year is not carried out because the recruitment of the auditor is still ongoing at the time of our visit. The same applies to the recruitment of the external auditor for the closing audit of the financing agreements project (H7510 and TF11678). The procurement plan has been developed in accordance with the Bank's guidelines for procurement methods. These are the National Invitation for Tender (NCB), the Request for Proposals (RFP), the Cost-Quality Selection (QCBS) and the Collaborative Agreements under the PSR. The analysis of the scheme and the process of implementation of the RBF. The management and implementation of the reproductive health project is pyramidal and decentralized. The General Directorate of Studies and Sectoral Statistics within the MOH is responsible for the implementation Page 50 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) of the project with the support of the Health Development Support Program (PADS). It authorizes reimbursements under the PSSB for sub-grants by the payer and reviews the performance of civil society organizations (CSOs) contracted to carry out consistency checks between facilities and patient exit survey records. A National Technical RBF Unit coordinates policy development and capacity building, particularly in the areas of indicator development and RBF targets, procurement and disbursements for RBF grant payments. The Health Development Support Program (PADS) pays health facilities. It was established under the Bank- financed Health and AIDS Sector Support Project (P093987), for procurement and financial management, and the hiring of an international firm for capacity building and development. independent evaluation of the RBF. A Steering Committee was created to oversee the project. It is chaired by the Secretary General of the Ministry of Health, and includes the directors of all major departments, donors and technical partners. The analysis of the effectiveness of awareness interventions. There is a steady increase in the number of pregnant women living with HIV who have received anti-retroviral drugs to reduce the risk of mother-to-child transmission, and the percentage of women aged 15 to 19 who know at least one method of modern family planning. The analysis of the effectiveness of the health care and socio-economic management of the different targets groups. The number of pregnant women living with HIV who received ART as part of PMTCT remained above the baseline every year. The number of pregnant women who received voluntary testing remained above the target each year. Improving utilization and quality of reproductive health services, with a particular focus on selected regions of Burkina Faso. Three indicators remained above the baseline target over the years. These are: i) percentage of women receiving at least 2 doses of tetanus toxoid vaccine, ii) number of health workers trained, iii) percentage of women receiving antenatal care during a visit to a health worker. Nevertheless, the percentage of health facilities that have not experienced a stock-out in contraceptive in the last three months has often remained below the target. The assessment of the results of preventive actions. There is a steady increase in the level of all four indicators. Three of the four indicators remained above the target. This is the rate of: i) Contraceptive prevalence, ii) proportion of births attended by skilled staff, and iii) proportion of fully immunized children. The proportion of women going for post-natal consultations has certainly improved, but this indicator has often remained below the target. Appreciation of the project's contribution to improving access of the general population and the poor to health services and care in the project area. It can be seen that the number of poor and vulnerable people who received free services is not systematically compiled centrally. Analysis of the project's contribution to the strengthening of social community-based health insurance. The percentage of population enrolled in a community-based insurance system remained higher than the basic target each year. The analysis of financial management efficiency. The total cost of contracts signed under the PSR managed by the PADS is 6,519,041,461 FCFA. With regard to the total cost of the contracts for the HIV component, the initial estimated cost in January 2017 was four million nine Page 51 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) hundred and fifty-four thousand five hundred and forty-five US dollars (US $ 4,954,545), the equivalent of two billion nine hundred and seventy-two million seven hundred and twenty-seven thousand (2,972,727,000) CFA francs (US $ 1 = 600 FCFA); 2 220 000 000 in supply contracts and 752 727 000 CFA in consulting services. With the launch of STEP, the initial plan was updated and approved in the amount of CFAF 2,152,200,000 for supplies and CFAF 765,327,000 for consultancy services, for a total of CFAF 2,917,527,000. In terms of financial management, we have identified the following strengths and achievements: - the harmonization of management procedures through the Common AIDS Basket Funds (PCS); - the stability of the financial management team at SP / CNLS-IST; - the FRB is accepted and implemented by health facilities; - autonomy in the financial management of health facilities that integrate a new management method; - bank account opening by health facilities (which did not have one) to receive project funds directly. In the implementation of the project, the weaknesses related to financial management identified are: - the delay in the transfer of funds from the health districts to the CSPS which has been resorbed by the opening of bank accounts by the CSPS; - the delay in the allocation of funds by the PADS, which led to a strike in the health facilities during the year 2016; - the perception of the RBF is not sufficiently understood by the managers of health facilities at the central level (tools not informed, strike in 2016 for delay of transfer of funds by the PADS); - despite the capacity building carried out, the recurrent mobility of civil servants does not facilitate the monitoring of project funds management; - insufficient financial resources to motivate staff of NGOs and associations that are subject to high mobility of the staff responsible for the financial management of these entities; - the non-configuration of the accounting software corresponding to the components and categories of activities of the project. Analysis of the effectiveness of procurement. In terms of achievements, most of the awarded and approved contracts have been executed. The lessening of the procurement plans by the donor has reduced the procurement procedures. In terms of weakness, it is the difficulties of conciliation of procedures between guidelines of the donor and the ones from the State. The analysis of the system of coordination and monitoring and evaluation between the coordination structures at the central and decentralized levels. For the actors involved, the PSR represents an innovative experience. Indicators of coverage and utilization of health services have improved in the targeted areas. However, for this to succeed, it took a continuous support by the donor, the World Bank. The delays in financing have led to demotivation of some actors in the field. The assessment of the contribution of the Project in achieving the objectives of the PNDS and the Strategic Framework for Combating HIV, AIDS and STIs 2016-2020. Regarding ensuring access for all to basic social services, the indicators have evolved satisfactorily. These are: i) ODP # 1: Contraceptive prevalence rate, ii) ODP # 2: Proportion of births attended by qualified staff, iii) ODP # 3: Proportion of women going for postnatal consultations, iv) ODP # 4: Fully vaccinated children, v) IRI # 1: Page 52 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Percentage of women receiving at least 2 doses of tetanus toxoid vaccine, vi) IRI # 3: Percentage of women receiving antenatal care during a visit to a health worker and vii) IRI # 5: Achievement rate of cesarean section For the extension of social coverage to all socio-professional risks and the development of mechanisms to prevent shocks, one indicator has evolved satisfactorily. This is the number of poor and vulnerable people who have benefited from free health care. The percentage of population enrolled in a community-based insurance system remained higher than the basic target each year. With regard to the development of innovative practices common to targeting and transfer programs, the relevant indicators have evolved satisfactorily in the targeted areas. These are: i) IRI # 11: Percentage of women aged 15- 19 who know at least one modern family planning method and ii) the number of pregnant women who have received voluntary counseling. With regard to enhancing consistency, coordination and good governance, the relevant indicators have evolved satisfactorily. These are: i) IRI # 2: Percentage of health facilities that received RBF credits on time and ii) IRI # 10: Percentage of health facilities that have not had a break in contraceptives in the last three months Given the level of indicators after 4 to 5 years of implementation, the Reproductive Health Project has contributed in a significant way to Burkina Faso's national social protection policy 2013-2022. Conclusion. The reproductive health project has shown convincing results both in terms of public health and the management of the resources invested. To sustain the achievements of the PSR project, the various stakeholders will have to continue efforts to strengthen collaboration. Page 53 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) ANNEX 6. RESULTS-BASED FINANCING AND IMPACT EVALUATION DESIGN OF THE RBF MODEL IN BURKINA-FASO Overall objective: change behavior of health providers at facility level for them to improve service delivery and increase quality of services. Key principles of the RBF design (following best practices): - RBF contracts signed between MoH and health centers defining basic package of health services to be provided, indicators and targets; - Monthly and quarterly assessments of results achieved by external reviewers; - Payments made to health centers based on results achieved and verified; - Health services included in the RBF contract in accordance with their package of activities: reproductive, maternal and child health services (such as assisted deliveries, family planning visit, children under 5 curative consultation, immunized children) but also other services (adult curative care, ARV treatment, uncomplicated malaria, tuberculosis treatment, etc). - RBF payments adjusted for the quality of care - Utilization of RBF funds by health centers following some rules and can be used to financial incentives for health workers, equipment and drugs, training, outreach, etc. - Satisfaction surveys of health services consumers. DESIGN OF THE RBF IMPACT EVALUATION The overall objective of the impact evaluation was to assess the impact of the PBF program on health service utilization and quality of service delivery across a wide range of targeted services. In line with what described above, the specific focus of this impact evaluation was on estimating the added benefit of combining PBF with equity measures. The main research questions fitting the abovementioned objectives were: 1. What is the effect of the PBF program (irrespective of intervention package) on selected human resources, service quality, service utilization, and health status indicators, compared to status quo? 2. What is the effect of the different PBF design options on selected human resources, service quality, service utilization, and health status indicators, compared to status quo? 3. What is the added benefit of implementing T2, T3, and T4 compared to the standard T1 on selected human resources, service quality, service utilization, and health status indicators? As well as across research questions 1-3: 4. What are the effects when considering only the most vulnerable segments of society, i.e. the ultra- poor? Page 54 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Type of interventions Regions Geographical implementation (number of health facilities in brackets) T1: Traditional PBF with Standard PBF was implemented Centre Ouest (all health facilities): no specific intervention Koudougou (54) and Sapouy (18) to facilitate access to the Centre Nord: Kougossi (12) and Kaya poorest to health services (20). Nord: Gourcy (10) and Ouahigouya (28) Sud-Ouest : Batié (5), Diébougou (5) Centre Est : Tenkodogo and Ourgaye Boucle du Mouhoun: Nouna (13) and Solenzo (16) T2: Traditional PBF Systematic targeting of the poorest Centre Nord: Kougossi (11) and Kaya +Systematic targeting of using community-based approach (up to (16). the poorest and 20% of households in the catchment Nord : Gourcy (12) and Ouahigouya (23) subsidization of health area of the health facility) and unit Sud-Ouest : Batié (3), Diébougou (7) services prices for services delivered to the Centre Est : Tenkodogo and Ourgaye targeted indigents were adjusted to compensate for the loss of revenues that health facilities experience if not charging user fees. T3: Traditional PBF Unit prices applied for services provided Centre Nord: Kougossi (11) and Kaya +Systematic targeting of to the targeted poor people were higher (24). the poorest and than in T2, offering thus an additional Nord : Gourcy (7) and Ouahigouya (17) subsidization of health financial incentive to offer services for Sud-Ouest : Batié (3), Diébougou (7) services+ provider the poorest. Centre Est : Tenkodogo and Ourgaye motivation to offer services to the poorest for free T 4: Traditional PBF PBF (T1) complementarily to CBHI. Boucle du Mouhoun: Nouna (31) and +Community Based Insurance (roll out supported by the Solenzo (17) Health Insurance NGO Asmade) was offered to all (including systematic population, and premium fully targeting of the poorest) subsidized for the identified poorest. Insurance benefit package included a wide range of primary and secondary health services, then providers received for services for the poorest both payments from the RBF program and the insurance (to reimburse user fees). Page 55 of 56 The World Bank BF-Reproductive Health Project (FY12) (P119917) Page 56 of 56