Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004423 IMPLEMENTATION COMPLETION AND RESULTS REPORT ON THE IDA CREDITS 4774-HN, 5294-HN AND 5603-HN IN THE AMOUNT OF SDR59.7 MILLION (US$77 MILLION EQUIVALENT) TO THE REPUBLIC OF HONDURAS FOR THE SOCIAL PROTECTION PROJECT (P115592) July 10, 2019 Social Protection & Jobs Global Practice Latin America And Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective July 10, 2019) Currency Unit = Honduran Lempira (HNL) US$ 1 = HNL 24.46 FISCAL YEAR July 1 - June 30 Regional Vice President: Axel van Trotsenburg Country Director: Yaye Seynabou Sakho Senior Global Practice Director: Michal J. Rutkowski Practice Manager: Pablo Gottret Task Team Leader(s): Junko Onishi, Nancy Rocio Banegas Raudales ICR Main Contributor: Nancy Banegas/Junko Onishi ABBREVIATIONS AND ACRONYMS AF Additional Financing CCT Conditional Cash Transfer CENISS Center for Information on Social Sectors (Centro Nacional de Información del Sector Social) CRW Crisis Response Window EPHPM Annual household survey (Encuesta Permanente de Hogares de Propósitos Múltiples) ENCOVI National Living Standard of Measurement Survey (Encuesta de Medición de Condiciones de Vida) FSU Unique Socio-economic Form (Ficha Socioeconómica Única) GoH Government of Honduras GRM Grievance Redress Mechanism IADB Inter-American Development Bank IAPP Indigenous and Afro Honduras People’s Plan IDA International Development Association IE Impact Evaluation IMF International Monetary Fund IP Indigenous peoples IRI Intermediate Results Indicator IRM Immediate Response Mechanism ISR Implementation Status and Results Report LCR Latin America and the Caribbean M&E Monitoring and Evaluation MCH Maternal and Child Health MIS Management Information System MoE Ministry of Education MoF Ministry of Finance MoH Ministry of Health MoP Ministry of the Presidency PCU Project Coordination Unit PI Productive inclusion PMT Proxy Means Test PRAF Family Allowance Program (Programa de Asignación Familiar) PRAF/PIPS Family Allowance Program /Integrated Social Protection Program (Programa de Asignación Familiar/Programa Integrado de Protección Social) RENPI Early Childhood Registry (Registro Nacional de la Primera Infancia) RNP National Registry (Registro Nacional de las Personas) ROI Registry of Institutional Supply (Registro de Oferta Institucional) RUP Unique Registry of Participants (Registro Unico de Participantes) SACE Administrative System for Education Centers (Sistema de Administración de Centros Educativos) SDR Special Drawing Right SEDIS Ministry of Development and Social Inclusion (Secretaria de Desarrollo e Inclusión Social) SIRBHO Beneficiary Registration System (Sistema de Información y Registro de Beneficiarios y Hogares) SP Social Protection SSIS Vice Ministry of Social Integration (Sub-Secretaria de Integración Social) TA Technical Assistance TTL Task Team Leader TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) .......................................9 II. OUTCOME .................................................................................................................... 12 A. RELEVANCE OF PDOs ............................................................................................................ 12 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 12 C. EFFICIENCY ........................................................................................................................... 15 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 18 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 18 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 19 A. KEY FACTORS DURING PREPARATION ................................................................................... 19 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 19 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 21 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 21 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 23 C. BANK PERFORMANCE ........................................................................................................... 24 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 26 V. LESSONS AND RECOMMENDATIONS ............................................................................. 26 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 28 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 43 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 45 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 46 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 53 ANNEX 6. CHANGES TO PROJECT DEVELOPMENT INDICATORS ............................................. 55 The World Bank Social Protection (P115592) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P115592 Social Protection Country Financing Instrument Honduras Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency The Republic of Honduras Secretary of Development and Social Inclusion Project Development Objective (PDO) Original PDO The Project'sdevelopment objectives are to (a) strengthen the institutional capacity of the MoP and PRAF to administer and managethe Bono 10.000 Program (the "Program"), through the development of transparent mechanisms and instruments for targeting Programbeneficiaries, monitoring compliance with Program co- responsibilities, and making payments to Program beneficiaries; and (b)increase: (i) school attendance among students in grades 1 to 6; and (ii) the use of preventive health services among familiesparticipating inthe Program. Revised PDO The PDO is to: (a) improve the institutional capacity of Recipient’s institutions to manage the CCT Program, bystrengthening transparent mechanisms and instruments for targeting CCT Program beneficiaries, monitoring compliance with the CCTProgram co-responsibilities, and making payments to the CCT Program beneficiaries; (b) provide income support to eligiblebeneficiaries;(c) increase the useof preventive health services and school attendance in grades 1 to 9 among CCT Programbeneficiaries in rural areas; and (d) improvethe Recipient’s capacity to respond promptly and effectively to an eligible emergency Page 1 of 55 The World Bank Social Protection (P115592) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 40,000,000 39,877,015 40,567,451 IDA-47740 12,300,000 12,236,787 11,643,072 IDA-52940 25,000,000 25,000,000 24,906,395 IDA-56030 Total 77,300,000 77,113,802 77,116,918 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 77,300,000 77,113,802 77,116,918 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 29-Jun-2010 29-Oct-2010 08-Oct-2012 31-Dec-2014 31-Dec-2018 Page 2 of 55 The World Bank Social Protection (P115592) RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 22-Apr-2013 23.01 Change in Results Framework Change in Components and Cost Reallocation between Disbursement Categories Change in Disbursements Arrangements Change in Institutional Arrangements Change in Financial Management 08-Jul-2013 30.26 Additional Financing Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Legal Covenants Change in Procurement 20-Nov-2014 42.80 Change in Implementing Agency Change in Results Framework Change in Components and Cost Reallocation between Disbursement Categories Change in Legal Covenants Change in Institutional Arrangements Change in Financial Management Change in Procurement 26-Feb-2015 43.11 Additional Financing Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Procurement 04-Jan-2017 55.65 Change in Results Framework 10-Jul-2017 66.30 Change in Loan Closing Date(s) 29-Dec-2017 73.87 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Substantial Page 3 of 55 The World Bank Social Protection (P115592) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 08-Jan-2011 Satisfactory Satisfactory 0 02 12-Jul-2011 Satisfactory Moderately Satisfactory 7.00 03 12-Feb-2012 Moderately Satisfactory Moderately Satisfactory 9.56 04 25-Aug-2012 Moderately Satisfactory Moderately Satisfactory 14.74 05 07-Apr-2013 Moderately Satisfactory Moderately Satisfactory 22.34 06 13-Nov-2013 Moderately Satisfactory Moderately Satisfactory 30.71 07 15-Jun-2014 Moderately Satisfactory Moderately Satisfactory 36.39 08 29-Nov-2014 Moderately Satisfactory Moderately Satisfactory 42.80 09 04-Jun-2015 Moderately Satisfactory Moderately Satisfactory 43.16 10 11-Dec-2015 Moderately Satisfactory Moderately Satisfactory 46.75 Moderately 11 21-Jun-2016 Moderately Unsatisfactory 48.19 Unsatisfactory 12 16-Dec-2016 Moderately Satisfactory Moderately Satisfactory 55.49 13 26-Jun-2017 Moderately Satisfactory Moderately Satisfactory 66.31 14 15-Dec-2017 Moderately Satisfactory Moderately Satisfactory 73.87 15 19-Jun-2018 Moderately Satisfactory Moderately Satisfactory 74.84 16 24-Dec-2018 Moderately Satisfactory Moderately Satisfactory 76.15 SECTORS AND THEMES Sectors Major Sector/Sector (%) Social Protection 100 Social Protection 78 Public Administration - Social Protection 22 Page 4 of 55 The World Bank Social Protection (P115592) Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Social Development and Protection 0 Social Protection 100 Social Safety Nets 80 Social protection delivery systems 20 Private Sector Development 100 Jobs 100 ADM STAFF Role At Approval At ICR Regional Vice President: Pamela Cox Axel van Trotsenburg Country Director: Laura Frigenti Yaye Seynabou Sakho Director: Luis Benveniste Practice Manager: Helena G. Ribe Pablo Gottret Junko Onishi, Nancy Rocio Task Team Leader(s): Cornelia M. Tesliuc Banegas Raudales ICR Contributing Author: Alexander Matthew Spevack Page 5 of 55 The World Bank Social Protection (P115592) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. At appraisal in 2010, Honduras was deemed one of the poorest and most unequal countries in the Latin America and Caribbean region (LCR), threatened by the global financial and internal political crises which led to exacerbating poverty levels that further affected poor households’ ability to smooth consumption and investments in human capital. Despite reduced poverty and extreme poverty rates between 2005 and 2009 from 65 to 59 percent and from 45 to 36 percent, respectively, Honduras continued to maintain high levels of poverty. Households in extreme poverty faced difficulties in smoothing consumption which led them to use negative coping strategies, such as taking children out of school and encouraging them to work or reducing health care visits which affected their human capital development. Lack of investments in children’s health and education led to stark inequalities: malnutrition in children under-five was 43 percent relative to 5 percent;1 and, school enrollment among 12 to 14-year-olds was 65 percent compared to 92 percent, among the poorest and richest quintiles, respectively. 2. The country lacked a comprehensive and coordinated social protection (SP) strategy, relying on the social assistance program executed by the Family Allowance Program (Programa de Asignación Familiar, PRAF). Without an overarching SP strategy, social assistance programs were fragmented, some targeting the poor (Bono Escolar, school feeding), though with low coverage; while others did not explicitly target the poor (energy subsidies and old age pensions). Honduras had over 80 social assistance programs, the largest and fairly well-targeted among them being the PRAF, a two-tiered cash transfer program under the Ministry of the Presidency (MoP). The first tier, Bono Solidario (unconditional cash transfer) launched in 1990, aimed at mitigating the effects of crises and reducing structural poverty, covered about 132,000 households in the poorest villages and provided cash transfers of about US$10 per household per month. The second tier, the Integrated Social Protection Program (PRAF/PIPS),2 (conditional cash transfer (CCT)) provided an additional cash transfer against compliance with health and education co-responsibilities (up to US$25 per household per month) to about 44,000 households. PRAF also provided other cash assistance to youth, women, and the elderly. 3. Facing rising levels of poverty and in political disarray, the newly elected Government of Honduras (GoH) was committed to achieving the goals set out in its National Plan 2010-2022 (Plan de Nación 2010-2022), aimed at breaking the intergenerational cycle of poverty and developing human capital for extremely poor households. The GoH’s strategy was to reach these goals through the launch of a national CCT Program Bono 10,000 (the Program)3 in 2010, with a coverage of 7,000 households. Integrating the PRAF cash transfers program and building on the SIRHBO to identify the poorest households. CCT Bono 10,0004 Program benefits were made conditional on school enrollment and 1 ENDESA 2005/2006. Ministry of Health. Honduras. 2 Supported by the Inter-American Development Bank (IADB), Programa Integrado de Protección Social, Credit Line #1568. 3 Bono 10,000 was renamed Bono Vida Mejor and is identified as the key poverty reduction program under the Social Assistance Platform (Plataforma Vida Mejor). Besides the Bono Vida Mejor CCT Program, the Platform includes numerous in-kind as well as cash transfer programs (often called the Vida Mejor Program) of varying sizes and objectives operated and managed separately from the CCT Program. The World Bank finances and provides technical assistance to the Bono Vida Mejor CCT Program but not to the other programs under the Platform. 4 The benefit with which the Program began—10,000 Lempiras per year or US$44 per month—was estimated to be about 25 Page 6 of 55 The World Bank Social Protection (P115592) attendance (grades 1 to 6) and on the use of preventive health services for children under-five and pregnant women. In addition, the GoH aimed to increase the coverage from 132,000 to 600,000 poorest households within the first four years of the program. 4. However, given the limited institutional capacity to deliver the Bono 10,000 Program, the GoH requested the assistance of the World Bank and the IADB to co-finance and support the implementation of the Bono 10,000 Program. To scale-up and deliver the Program, the GoH needed to enhance the: (a) targeting mechanism since programs applied different targeting methods and used data from an outdated registry of beneficiaries SIRBHO (Sistema de Información y Registro de Beneficiarios y Hogares),5 (b) process for the timely verification of compliance with co-responsibilities, (c) stability in the frequency of payments, and (d) development of relevant information for decision-making and accountability purposes. To achieve these improvements, the World Bank and IADB drew on significant international SP sector expertise. The World Bank also sought to create synergies with ongoing IDA- financed operations6 aimed at improving services for the poor through supply-side interventions on health, education, and nutrition. Theory of Change (Results Chain) 5. The Project operated on the core assumption that poverty in Honduras was perpetuated by a lack of financial and technical support needed to overcome social services’ demand-side barriers at the institutional and household levels. It assumed that insufficient investments by poor households in schooling, health, and nutrition were driven by liquidity constraints, thus resulting in poor human development outcomes leading to low earnings capacity. It was also assumed that increased institutional capacity in targeting, co-responsibilities compliance verification, and stable payment cycles would improve the delivery of the program and facilitate Program scale-up. 6. The operation was designed along a results chain to achieve two results. a) Result 1: Strengthened institutional capacity for the administration and management of the CCT Program. b) Result 2: Benefit payments made to eligible households would: (i) smooth household consumption; (ii) increase school attendance among students in grades 1 to 6; and (iii) increase the use of preventive health services among the beneficiaries. percent of extremely poor households’ median income, a level comparable to other programs in the region. 5 Database built using the 2001 Population Census and the 2004 National Living Standard of Measurement Survey ( Encuesta de Medición de Condiciónes de Vida- ENCOVI) 6 Nutrition and Social Protection Project (P082242) and Education Quality, Governance, and Institutional Strengthening Project (P101218) Page 7 of 55 The World Bank Social Protection (P115592) Figure 1: Theory of Change Activities Outputs PDOs / short- and medium-term outcomes Long-term outcomes Provision of Capacity building TA Notmeasured in Project RF and Training Capacity building and TA provided to Review of targeting mechanism Strengthened Institutional Capacity of coordinate, monitor and evaluate Program Strengthened institutional capacity of Improved targeting mechanism MoP and PRAF to administerand manage Developmentand implementation of the Bono 10.000 Program (Comp 1) a monitoring and evaluation system A Improved Social Protection System Development of a Management Information System (MIS) (beneficiary payroll, co- MIS developed Human capital accumulation (i.e., Beneficiaries' Registry updated health, education/skills) Update of the Beneficiaries' Registry B Increased school attendance among Formulation of a social protection students in grades 1 to 6 among families Povertyreduction Social protection strategy approved strategy participating in the Program (Comp 2) Provision of cash transfers to families complying with co- Increased use of preventive health services Conditional cash transfers delivered to among families eligible beneficiaries in the poorest quintile participating in the Program (Commp2) Critical Assumptions: A. i) PRAF agency has existing institutional capacity associated with implementation of previous PRAF cash transfer programs; ii) improved targeting, adecuacy of cash transfer amount, enforcement of co- responsibilities, and payment cycle stability will increase the effectiveness of the Program; iii) health and education sector provide data on beneficiaries' compliance with co-responsibilities, iv) National Registry of Persons issues identification for individuals lacking identification; v) Secretary of Finance makes funds available for Project activities B. i) health and education sectors make services available to Program beneficiaries; ii) the Program transfer size is sufficient to allow participating households to smooth consumption and invest in health and education, iii) Secretary of Finance makes funds available for Project activities Project Development Objectives (PDOs) 7. The Project's development objectives were to: (a) strengthen the institutional capacity of the MoP and PRAF to administer and manage the Bono 10,000 Program (the "Program"), through the development of transparent mechanisms and instruments for targeting Program beneficiaries, monitoring compliance with Program co-responsibilities, and making payments to Program beneficiaries; and (b) increase: (i) school attendance among students in grades 1 to 6; and (ii) the use of preventive health services among families participating in the Program. Key Expected Outcomes and Outcome Indicators 8. The key performance indicators originally established were: • Percentage of conditional cash transfers received by households in the poorest quintile (goal: 65 percent of households) Page 8 of 55 The World Bank Social Protection (P115592) • Percentage of registered households of the Program whose co-responsibilities are verified and reported (goal: 80 percent of households) • Percentage of students in grades 1 to 6 participating in the Program who comply with the co- responsibility of 80 percent school attendance (goal: 80 percent of students) • Percent of children aged 0-5 years participating in the Program with complete vaccination scheme (goal: 98 percent of children) Components 9. Component 1. Institutional strengthening of the Bono 10,000 Program (US$8.7 million equivalent). This component aimed at strengthening the efficiency, effectiveness, and management of the CCT Program and contributing to the development of a SP strategy by building the capacity in the MoP and PRAF through: (a) enhancements in the organizational and technical capacity of the MoP to coordinate, monitor, and evaluate the CCT program including reviewing the targeting mechanism for Program beneficiaries, developing a Monitoring and Evaluation (M&E) system, conducting Program evaluations, piloting of alternative payment mechanisms and implementing a communications campaign, (b) improvements in the operational and administrative capacity of the PRAF agency including updating the beneficiaries registry, developing a Management Information System (MIS) to support beneficiaries payroll management and verification of compliance of co-responsibilities and creating a unit to attend grievances and complaints; and (c) the development of an integrated SP strategy to design, implement, and monitor coherent SP programs and policies and the development of Unique Registry of Beneficiaries. The responsibilities of implementation and financial management were handed to the Technical Coordination Unit, under the MoP for activities (a) and (c), and to PRAF for activity (b). 10. Component 2. Co-financing Conditional Cash Transfers (grants) (US$31.3 million equivalent). This component would finance cash transfers to eligible beneficiaries who complied with co- responsibilities, which included: (a) children between 0 and 59 months attending preventive health controls; (b) pregnant and postpartum mothers completing their pre- and post-natal check-ups; and (c) children from 1st to 6th grades (primary level) being enrolled in school and attending classes. PRAF was responsible for the implementation and financial management of this Component. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 11. The Project received two additional financings (AFs) and underwent five level-two restructurings (see Table 1) during implementation. 12. The PDO was revised twice, mainly to scale up activities, clarify the focus areas where the cash transfers under the Project would be disbursed, and reflect the changes in the institutional arrangements of the Program as it evolved. Under the first AF in July 2013, the PDO was revised on three fronts: (i) it was made explicit that the Project would continue to only co-finance cash transfers and focus in the rural areas; (ii) wording substitutions were made, including revising “the Recipient’s Institutions” for “MoP and PRAF” and the “Conditional Cash Transfer (CCT) Program” for Program “Bono 10,000” given the likelihood that the Program’s name could be changed7; and, (iii) a third objective was added to: improve the Recipient’s capacity to respond promptly and effectively to an emergency, consistent with other projects in Honduras 7 The Program’s name later changed to Bono Vida Mejor. Page 9 of 55 The World Bank Social Protection (P115592) where Contingency Emergency Response components were included. The PDO was revised a second time during the second AF in February 2015, to expand the eligibility of the Project’s target beneficiaries to also cover students in lower secondary education grades (grades 7-9). 13. The PDO at Project closing reads as follows: “The PDO is to: (a) improve the institutional capacity of Recipient’s institutions to manage the CCT Program, by strengthening transparent mechanisms and instruments for targeting CCT Program beneficiaries, monitoring compliance with the CCT Program co- responsibilities, and making payments to the CCT Program beneficiaries; (b) provide income support to eligible beneficiaries; (c) increase the use of preventive health services and school attendance in grades 1 to 9 among CCT Program beneficiaries in rural areas; and (d) improve the Recipient’s capacity to respond promptly and effectively to an eligible emergency.” 14. As the changes were made to expand the scope, and clarifications did not affect the substance of the PDO as originally stated, a split rating was not applied.8 The project outcomes in this ICR are assessed against the expanded scope. Table 1. Summary of Significant Changes to the Project Change and date Summary of Changes Restructuring Streamlined Bono 10,000 operating processes by transferring all Program executing and April 2013 fiduciary responsibilities to PRAF (previously with MoP) which became the main executor for the Program. Changes in institutional strengthening allowed for some funds to be re- directed to transfers. AF 5294-HN Revised the PDO and PDO indicators. Added an Immediate Response Mechanism (IRM) (US$12.3 million Contingent Component in the PDO for eligible emergency, IRM Component, and a equivalent) corresponding outcome indicator. Scaled-up activities under components and new July 8, 2013 institutional strengthening activities were added. Extension of original credit’s closing date from December 31, 2014 to December 31, 2015 to align with this first AF (5294-HN). Restructuring Following presidential elections, the Ministry of Development and Social Integration’s November 2014 (SEDIS) was created. The Vice Ministry of Social Inclusion (SSIS) absorbed PRAF and took over implementation and coordination of the Program and all project activities. The project was restructured so that implementation arrangements were consistent with the new institutional reorganization. AF 5603-HN Revised the PDO and PDO indicators. Expanded objective to also include education grants to (US$25 million) lower secondary school students (7th to 9th grade). Expanded scope of activities and February 26, 2015 coverage of beneficiaries in areas not previously covered. Extended the closing date of 4774- HN and 5294-HN to December 31, 2017 to align with this second AF (5603-HN). Restructuring Revised project’s results framework to align with the Program’s coverage targets and GoH’s January 2017 policies. Minor changes to some of the intermediate indicators. Restructuring Following the 2016 elections, the closing date of 5603-HN was extended from December 31, July 2017 2017 to December 31, 2018 to allow full disbursement of funds. The closing dates of 4774- HN and 5294-HN remained the same since they were 99 percent disbursed. Restructuring The closing dates for 4774-HN and 5294-HN were extended from December 31, 2017 to June December 2017 30, 2018 to allow full disbursement of the second CCT payment following the social unrest that emerged after the elections. 8 Implementation and Completion Report Guidelines, July 2017 Page 10 of 55 The World Bank Social Protection (P115592) Revised PDO Indicators 15. Revisions to expand the scope of the project through AFs, changes in targeting and coverage of the Program, and the Ministry of Health’s (MoH) protocols led to changes in the PDO indicators (PDIs) (see Annex 6, Changes to PDO indicators). Two PDIs were dropped. One due to redundancy with the education co-responsibility indicator, which initially was moved as an Intermediate Result Indicator (IRI) and later dropped. The PDI on frequency of prenatal controls was also dropped given the unavailability of administrative data to measure this indicator. While several PDI targets were revised upwards to better reflect education outcomes, targets on health outcomes related to the original PDI “Percent of children, aged 0-5 years participating in the Program with complete vaccination scheme” were reduced as the age range was narrowed to focus on early years. The children’s age range was revised twice to align it with the MOH’s revised health protocols using administrative data sources, and not the Household Surveys. The reduction of the targets for the vaccination indicator was reduced and deemed necessary given: the Program’s focus on extreme poor households who had lower vaccine coverage at baseline. The indicator was partially reported on December 2015 for the first time. Lastly, PDIs were added through AFs to: (i) reflect the expansion of the project’s scope to cover lower secondary education students (grades 7-9); (ii) document the project’s impacts on females; and (iii) improve the Recipient’s capacity to respond promptly and effectively to an Eligible Emergency. Revised Components 16. Component 1 was expanded in scope to reflect changes in the GoH’s structure and the CCT Program. The Institutional strengthening component was notably revised during the first two restructurings (April 2013, November 2014) and under both AFs (July 2013, February 2015) in response to GoH’s reorganization, particularly with the addition of the Ministry of Development and Social Inclusion (Secretaria de Desarrollo e Inclusión Social, SEDIS). These changes allowed the project to support the management of the CCT Program both at the Program9 and the sector levels10 including piloting a productive inclusion strategy. Later changes to Component 1 focused on improving the inter-sectoral coordination among SEDIS, MoH and the Ministry of Education (MoE); and, supporting regional offices through the deconcentration of the Program. 17. Activities under Component 2 also widened the breadth of the Program’s coverage of vulnerable households, by adjusting its benefit structure and increasing the purview of the program’s co- responsibilities for education. The Program refined its target population to the most vulnerable, including households: (a) under extreme poverty, as identified by the 2013 Census and 2014 Multipurpose Household Survey (Encuesta Permanente de Hogares de Propósitos Múltiples, EPHPM); (b) in communities with large indigenous peoples (IPs); and (c) in vulnerable areas, such as the Dry Region of Honduras (Corredor Seco). The benefit structure was revised in 2015 from an annual lump sum of HNL 10,000 per household (approximately US$ 44 per month) to a smaller lump sum (an average of HNL 5,967, roughly US$ 20 per month) and incremental benefit per child complying with co-responsibilities. Finally, the education co-responsibilities were expanded to include school attendance for grades 7-9 to incentivize 9 These included: refining the targeting mechanism, linking beneficiaries to existing productive inclusion (PI) interventions, and strengthening Program delivery systems including the MIS, payment mechanisms and monitoring of co-responsibilities. 10 These included investing in tools like the RUP that could help consolidate social assistance and subsidy programs. Page 11 of 55 The World Bank Social Protection (P115592) households to keep their children in school longer and the health co-responsibilities were reviewed to specify the number of health check-ups to be accounted as co-responsibility compliance. Rationale for Changes and Their Implication on the Original Theory of Change 18. Major changes to the project were largely in response to Honduras’ highly variable political, social and fiscal context and did not substantially affect the Theory of Change. The first change was the addition of a third result: improved capacity of the GoH to respond to an emergency. A critical assumption was that the GoH had the experience to execute such a fund in the context of an eligible emergency. Another modest change to the results chain, on the outcome on increased school attendance was the change from grades 1-6 to 1-9, thus expanding the Program’s coverage and reach of influence. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 19. The relevance of the PDOs is assessed as high, with no shortcomings in the relevance to the World Bank’s Country Partnership Framework (CPF) for FY 16-20 (Report No. 98367-HN). At closing, the PDOs continued to be highly relevant to achieving the CPF’s development objective “Expand coverage of social programs” and aligned to the foster social inclusion pillar by improving conditions for growth and reducing vulnerabilities to enhance resilience. The CPF is anchored in Honduras’ short- and long-term development plans - “Plan for a Better Life” (2014-18) and the Country Vision for 2038 (approved in 2010) and is organized around three strategic pillars: (i) fostering inclusion; (ii) bolstering conditions for growth; and, (iii) reducing vulnerabilities. 20. Similarly, at closing the PDOs continue to be highly pertinent to the GoH’s priorities in the SP sector. Reduction of extreme poverty remains a priority and the PDOs continue to resonate with the GoH’s continued commitment to building institutional capacity to strengthen the SP system. Finally, the outcomes supported under the Project are aligned with the twenty targets identified by the GoH to be achieved by 2038 including eradicating extreme poverty,11 reducing the incidence of poverty to below 15 percent, and, increasing the average number of schooling years to 9 years. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 21. Overall, the project mostly achieved its intended outcomes, making its efficacy substantial. The project supported the Program since its inception in 2010, contributing to reducing extreme poverty and increasing human capital accumulation through regular income support and the enforcement of health and education co-responsibilities, while enhancing the GoH’s capacity to manage a robust SP system. The Project achieved seven of its eight PDIs, the ninth one not being relevant as an emergency was not triggered during implementation. The eighth indicator related to complete vaccination schemes was 11 Extreme poor are those who live in households with income below the basic food basket, according to the GoH’s definition. Page 12 of 55 The World Bank Social Protection (P115592) partially achieved, and the ninth PDI associated with the Immediate Response Mechanism (IRM) was not applicable as the component was never activated. These outcomes are described in detail further below by result area. 22. Result 1: “Strengthened institutional capacity for the administration and management of the CCT Program” was deemed substantial. The interventions in this area aimed to improve the GOH’s institutional capacity to manage the CCT Program by strengthening transparent mechanisms and instruments for targeting CCT Program beneficiaries, monitoring compliance with the CCT Program co- responsibilities, and making payments to the CCT Program beneficiaries. The associated PDO indicator, “percentage of households in the Program receiving CCTs in rural areas that are extreme poor” was achieved and exceeded (99.89 percent of households compared to the target of 80 percent), highlighting the Program’s transparent and effective targeting mechanism. This project data is supported by an additional analysis12 using the 2016 EPHPM which indicated that 94 and 93 percent of rural Program beneficiary households were among the extreme poor in 2014 and 2015, respectively, in terms of household income.13 The publishing of the targeting methodology in the Oficial Gazzette and the establishment of a Grievance Redress Mechanism in 17 departamental offices contributed to increase the transparency and accountability of targeting. Targeting is further discussed in the efficiency section. 23. The Project strengthened the GoH’s capacity to monitor compliance of co-responsibilities, through investments in the Program’s MIS and its interoperability with health and education information systems. At the start of the Project, PRAF operated a fragmented manual process to monitor compliance with co-responsibilities. The Project financed the automated processes for verifying co- responsibilities. For education, it took advantage of the MoE’s Administrative System for Education Centers (Sistema de Administración de Centros Educativos-SACE), supported by an IDA-financed education project.14 Verification of health co-responsibilities was more challenging in the absence of a health sector MIS. The Project, therefore financed the establishment of an Early Childhood Registry (Registro Nacional de la Primera Infancia, RENPI), managed by CENISS, which collected data on vaccinations and established the first individual database for health service use for children in the country. 24. Four alternative payment mechanisms were piloted to support the achievement of the first result related with making payments; however, the project closed without the establishment of an alternative payment method. The main payment method of the Program throughout remained to be cash payments by the state bank, Banco Nacional Agricola para el Desarrollo (BANADESA). The first three pilots consisted of testing three alternative delivery mechanisms involving commercial banks, a mobile money operator, and selected cooperatives covering 13 percent of the Program’s payments.15 Each pilot contributed to an increase in the transparency, reduced Program costs, and wait times for beneficiaries. However, pilots were not scaled up because: (i) providers did not find the business profitable; (ii) there was a lack of a supervisory legal framework; and, (iii) the GoH opted for the use of basic saving accounts for payments, rolled out as part of its financial inclusion strategy but without buy-in of the banking sector. 12 Analysis done by the Socio-Economic Database for Latin America and the Caribbean (SEDLAC) (CEDLAS and World Bank). 13 The extreme poor and the moderate poor are those among the lower six and seven household income deciles, respectively. 14 “Honduras Education Quality, Governance and Institutional Strengthening” (P101218). Approved January 24, 2008 and closed June 30, 2013. Reporting to SACE is done online by each school teacher; the Program’s MIS is linked directly to the SACE, enabling it to obtain regular and updated information on compliance with education co-responsibilities. 15 Payment providers included TIGO, CEPROBAN, Banco de Occidente, and FACACH (association of cooperatives). Page 13 of 55 The World Bank Social Protection (P115592) In 2016, the project supported a fourth pilot with BanRural, a commercial bank that made payments in 2016 and part of 2017 to 8,860 beneficiary households in three departments through basic bank accounts, with accompanying financial education capacity-building efforts. Just after the project’s closure, though, the contract with BanRural expired and an extension of the contract was declined because the per- transactions costs were underestimated in the original contract, thus rendering a renewal under the same terms was not financially viable for BanRural. In the end, just four percent of Program beneficiaries received payments through basic accounts and BANDESA’s cash payments remained to be the main payment mechanism through a combination of payments through bank branches and mobile transport units. While in terms of payment delivery there were no major improvements, substantial progress was made in the processing of payments which included the automated creation of the beneficiary payment payroll, high execution of payments with respect to planed payments (not less than 90 percent at project closing) and automated verification of education co-responsibilities. All these combined, improved processes allowed for the reliable issuance of benefit payments. 25. Result 2: “Payments made to eligible beneficiaries: (i) smoothed household consumption; (ii) increased school attendance among students in grades 1 to 6; and (iii) increased the use of preventive health services among participating households,” was also deemed substantial, achieving six of the seven PDIs. By making regular payments to 218,713 extreme poor households, the Program helped smooth consumption and incentivized the use education and health services to improve human capital development of their children. The CCT transfer represented about 25 percent of the household income of the poorest quintile. 26. The project contributed to an increase in children’s school attendance and completion for grades 1 to 9, exceeding all the associated PDO indicators. With respect to the PDIs, the “percentage of students (grades 1-6; and, grades 7-9) participating in the Program who comply with the co-responsibility of 80 percent”16 the end targets were achieved with 85.5 percent of students in grades 1-6 and 92.7 percent of students in grades 7-9 attending school at least 80 percent of the time, surpassing the respective targets of 85 and 70 pecent, respectively. In terms of completion, 98.7 percent of Program beneficiaries aged 13- 15 years completed primary education (sixth grade), surpassing the 78 percent target; and, 98.6 percent of female Program beneficiaries aged 13-15 years completed primary education (sixth grade), above the 78 percent target. For lower secondary completion (ninth grade), the Project also exceeded its end targets of 60 percent: 94.4 percent of Program beneficiaries aged 16-18 completed lower secondary education (ninth grade); and, 95.7 percent of female Program beneficiaries aged 16-18 years completed lower secondary education. The Impact Evaluation (2017) confirms these results with evidence that school enrollment and attendance of Program beneficiaries who were 17 years old at the time of the study had increased by 5.1 percent and 5.2 percent, respectively.17 27. The Project also contributed to improving the use of preventive health services; however, the PDO indicator, “percent of Program beneficary children aged 0-23 months with a complete vaccination scheme,” was partially achieved. The 2013 Program Impact Evaluation indicated that infant height and 16 For the IRI on school attendance (grades 1-6), compliance with co-responsibility was increased from 80 to 85 percent in the second AF to reflect the high percentage of children attending primary school; however, the RF was never revised to reflect this change. 17 Evaluación de Impacto del Programa de Transferencias Monetarias Bono Vida Mejor, Producto 15. Informe Final de la Evaluación de Resultados de la Evaluación de Impacto, diciembre 12 2017, Econometría Consultores Page 14 of 55 The World Bank Social Protection (P115592) weight monitoring increased by 11.3 percent and 15.9 percent, respectively, as well as increased check- ups for children aged 0-3 years. Nonetheless, the Project closed with 48 percent of Program beneficiaries aged 0-23 months with a complete vaccination scheme, compared to a target of 70 percent. Weaknesses in the MoH’s vaccination information systems18 and inter-sectoral challenges (discussed below) in verifying health co-responsibilities were the main reasons for not achieving this PDO indicator. 28. Finally, result 3, improved capacity of the GOH to respond to an emergency was not applicable, as no eligible emergencies were declared during the Project. Justification of Overall Efficacy Rating 29. The overall efficacy rating of the project is considered substantial, based on the achievement of its outcomes. The overall rating took into consideration the substantial achievement of objectives 1 and 2, with the project having met and exceeded targets for seven of the eight outcome indicators, the ninth indicator not being applicable given that an emergency was not triggered during project implementation. The project made immense contributions to the SP sector, both through the establishment of the CCT Program and the registries of social programs and their beneficiaries. The CCT Program evolved with the support of the project to better identify and reach the most vulnerable and poorest households. The project also successfully supported the establishment of key mechanisms and strategies to ensure the Program was transparent and, above all, accountable. The project’s technical assistance (TA) toward the establishment of the Unique Registry of Participants (Registro Unico de Participantes, RUP) and the Institutional Offer Registry (Registro de la Oferta Institutional, ROI) to integrate the Social Registry and the RUP currently serves as a successful international reference as it received a special nomination form the United Nations for innovation. With nearly three-quarters of the country’s population registered, the GoH is now better positioned to make informed SP policy decisions. Finally, through providing cash transfers to the extreme poor and Indigenous Peoples (IPs) communities, the project contributed to an increase in children’s school attendance and completion for grades 1-9, augmenting human capital accumulation of the future generations. While the outcome target for health was not met, IE results pointed to an increased use in preventive health services, which, with time, will lead to a healthier population. C. EFFICIENCY Assessment of Efficiency and Rating 30. The overall efficiency is deemed modest, as described below. 31. In terms of economic efficiency, a standard economic analysis was not conducted for this report as programs’ administrative costs could not be obtained; however, two rigorous IEs point to the positive net gains the project contributed to. An economic analysis carried out at appraisal analyzed the potential impact of the CCT Program on both poverty and the returns to investment in human capital accumulation through education (health investments were not calculated). Assuming perfect targeting, annual cash transfers of HNL 10,000 (approximately US$ 44 per month) to the poorest, covering the originally targeted 600,000 households, it was estimated that the Program would have reduced the poverty gap by 22 percent in four years; and, an average two-year increase in schooling would have raised the beneficiaries’ average labor income by 18.8 percent by 2025. However, these assumptions changed during implementation, 18 The systems used for tracking vaccinations are not strong enough to document vaccination levels in Program intervention areas in real-time even though IE results indicate that beneficiaries did in fact comply with health co-responsibilities. Page 15 of 55 The World Bank Social Protection (P115592) particularly the targeted population, coverage, and the size of the cash transfer. Given these changes, estimation on Program impact at project closing are difficult to compare with those estimated at appraisal; although they are worth noting. In 2017, the Program had contributed to smoothing household consumption of hundreds of thousands of households, increasing the household income of Program beneficiaries by 18 percent and reducing the poverty headcount among beneficiaries by 8.2 percent (Econometría, 2017). Moreover, the Program led to an increase in school attendance among 6 to 17-year- olds by 5.2 percent, and enrollment by 5.1 percent. Undoubtedly, enhancing school attendance and smoothing household consumption, contributed to investments that are likely to increase future incomes and reduce inter-generational poverty. 32. The two IEs conducted at two different time periods also measured the program impacts with the different benefit amounts. The 2013 IE measured the Program impact when the benefit was a lumpsum amount of HNL 10,000 per household, while the 2017 IE measured the Program impact of differentiated benefit calculations based on household structure. Since the Program impacts of the different benefit sizes were not measured in one IE, the two IE results are not comparable; however, the comparison in terms of outcome areas/indicators in which the two IEs detected program impacts are a measure of efficiency gains in the use of Conditional Cash Transfers under Component 2. 33. Adequacy of benefits. At the time the project was approved, the maximum transfer amount of HNL 10,000 per year, represented 25 percent of median household income for the extreme poor (US$ 176), and roughly 46.7 percent of the per capita income of those in the poorest quintile. That transfer amount in 2017 represented almost 28 percent of median household income of the extreme poor (HNL 36,000 or US$ 132 per month). Using the 2017 data, a HNL 10,000 transfer would represent approximately 12 percent of the food staples basket in rural areas. The actual average transfer size in 2016 was HNL 5,967 (roughly US$ 20 per month), representing 15 percent of median household income of the extreme poor, before the transfer, and 25 percent of the average per capita income of those among the poorest quintile. The value of the transfer as a share of post-transfer welfare would be 13.3 percent, just shy of a worldwide average of 15.6 percent (World Bank, 2018). In 2015, at the time of the AF, the team had estimated an annual transfer size of HNL 5,500, representing 24.4 percent of the income of the poorest quintile, suggesting that the team’s ex-ante estimation of the benefit size was accurate. 34. Program impacts with different benefit sizes. The two IEs conducted both reported reduction in poverty headcount, poverty gap, as well as increase in school enrollment and school attendance among 6 to 17-year-olds. Although both IEs reported increased attendance in weight and height check-ups among infants (under-one), the program impacts on health outcomes such as acute diarrheal diseases, acute respiratory infection, childhood malnutrition, as well as reduction in childhood labor were only found in the 2017 IE. This may suggest that the improvements in the Program operation allowed for the Program to show impacts in a more comprehensive set of indicators in 2017, while the Program continued to show positive impacts in poverty-related indicators and education-related indicators. This is despite the reduced benefit sizes in 2015 of roughly 40 percent in nominal terms, and may suggest an improved Program efficiency over time. 35. Targeting efficiency: Using the government’s poverty lines, roughly 90 percent of Program beneficiaries were extremely poor and 94 percent of beneficiaries were relatively poor in both 2014 and 2015, suggesting targeting accuracy of the extreme poor households, therefore improving efficiency. The Page 16 of 55 The World Bank Social Protection (P115592) project provided TA to fine-tune the Proxy Means Test (PMT)19 to more accurately identify the extreme poor. Following the GoH’s decision to only target the extreme poor (rather than both extreme and moderate poor20), and the subsequent 2014 project restructuring, the project supported the refining of the PMT and updated the Unique Socioeconomic Form (Ficha Socioeconomica Unica, FSU) data collection tool. From thereafter, the Program collected data on approximately 120,000 new households (including IPs in agreement with the Indigenous and Afro-Honduran Peoples’ Plan, IAPP).21 It also updated the RUP using the new FSU to validate existing beneficiary profiles and re-applied the PMT, contributing to a fully updated eligibility information on Program beneficiary households, far surpassing the target of 60 percent.22 As a result, by project closing, the Program had, arguably, become one of the best targeted CCT programs in LCR.23 36. With regards to operational efficiency the project experienced delays in the implementation of key activities because of a combination of factors. First, international funds (including this project) were frozen as part of tightened measures to meet the fiscal targets set out as part of an agreement with the International Monetary Fund. This delayed all activities under institutional strengthening nearly by one year. Second, turnover of project personnel particularly involved in the Productive Inclusion (PI) strategy was high, delaying the pilot implementation until the final year of the project, however at closing the Project overcame this inefficiency by almost fully achieving its end target (See Annex 1 Results Framework. Finally, difficulties in the procurement of key technical advisory services due to inadequate pool of consultants for the very technical work, and poorly written terms of reference, caused delays in the completion or piloting of key activities including upgrade of the Program MIS, establishment of alternative payment mechanism, deconcentration of program operation and certification of the RUP. 37. Changes in the GoH’s institutional structure and the Program initially delayed Project implementation; in the end, though, these changes resulted in an efficiency gain. A source of inefficiency came in the aftermath of the institutional reform (end 2014 to beginning 2015), in which PRAF was absorbed into the SSIS under the newly formed SEDIS. Parallel to this were adjustments to the Program; the targeting of only extremely poor households and change of benefit structure. These simultaneous changes made it challenging for systems and personnel to adjust. As a result, it took close to an entire year for the new targeting parameters to go into effect. By the project’s closure, though, the new institutional arrangements drove the GoH to take a more coordinated approach to SP, strengthen its institutional framework for social policy, and streamline procedures for improved Program implementation. 38. Finally, the Program’s inability to adhere to a defined payment schedule in line with the Program’s Operation’s Regulations, arguably impacted the frequency of payments. During the first three 19 The PMT is a statistical formula to predict a household’s welfare level based on a set of characteristics, such as location, ownership of durable goods, and demographics. The PMT aims to proxy income or consumption to identify the chronic poor. 20 The moderate poor those who live with household income more than the Basic Food Basket but less than the Basic Consumption Basket (housing, education, health, transportation), according to GoH’s definition. 21 Data collection for 19,000 households was financed by the Project, including for hard-to-reach indigenous peoples in the Gracias a Dios department. The Government focused on the 141 most vulnerable municipalities identified by the government’s poverty maps as having a high rate of poverty, as well as at high risk of draught, migration, and violence. 22 Prior to closing, CENISS had begun an operative to update the FSUs of the 90% of beneficiaries for whom it was necessary to update eligibility conditions, given that such an updating is required every five years. 23 Inter-american Development Bank (2019). Presentation entitled “Programa Presidencial de Transferencias Monetarias Condicionadas “Bono Vida Mejor”. Page 17 of 55 The World Bank Social Protection (P115592) years of the project, one payment would take as much as three months. By project’s end, with the automated generation of beneficiaries’ payroll and the establishment of a payment schedule in the Program’s operations manual, this process was reduced to around 45 days; however, payments were still highly unpredictable. In 2015 and 2018, only two payments were made, short of the three payments set out in the operation’s manual. In 2015, only one payment was carried out following changes in institutional arrangements and Program reforms. In 2018, the GoH reallocated national resources for the cash transfers to provide relief to those affected by floods. 39. Based on the evidence suggesting efficiency gains achieved through the reduced benefit size while maintaining Program impacts as well as improved targeting accuracy to reach the extreme poor, combined with the inefficiencies caused by implementation bottlenecks, the overall efficiency is rated modest. D. JUSTIFICATION OF OVERALL OUTCOME RATING 40. The overall outcome of the operation is rated moderately satisfactory. This weighs the three sub-ratings: relevance of the PDOs (high), efficacy (substantial), and efficiency (modest). E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 41. Increased female empowerment. Although the Program was not designed with the explicit objective of empowering women, female beneficiaries, relative to female non-beneficiaries, were more likely to make decisions with respect to the health, education, and discipline of their children, as well as regarding daily purchases, the purchase of assets for their homes, and home repairs. To a lesser extent, they also exercised greater control over the decision of whether to work outside of the household and the number of children they and their husbands wished to have (Alemann et al., 2016). Poverty Reduction and Shared Prosperity 42. Reduced child labor. The Program contributed to reducing child labor by 2.6 percent, with a larger reduction found among children between 12-14 years old (reduced by 4.5 percent).24 Impacts are long- lasting given that reductions in child labor have implications for school attendance and remove children from potential abusive situations. Analysis presented in the Program’s second IE indicated that transfers may offset the need for children to work, allowing them to attend school rather than work (Econometría, 2017). 43. Increased home improvements. Results from the second IE also indicate that beneficiary households made improvements in their homes, with housing poverty falling and housing expenditures increasing vis-à-vis control group households. Households cited making home improvements with respect to ceilings, batteries, stoves, gutters, and latrines (Econometría, 2017). The dimension of the Multidimensional Poverty Index (MPI)25 in which beneficiary household poverty fell the most was in housing,26 with a reduction of 18.8 percent relative to the control group. In addition to spending more on 24 Child labor regulations stipulate that children under 12 should not be working any time. Between 12 and 14, children can work up to 14 hours per week (around two hours a day), so that work does not affect their school attendance. 25 The MPI is an index that takes into account the plurality of basic needs necessary for guaranteeing a minimum of decent conditions for a household. The index is composed of health, education, work and housing dimensions. 26 The MPI’s housing dimensions takes into consideration the materials used in housing construction, as well as the number of Page 18 of 55 The World Bank Social Protection (P115592) food, health, and education, beneficiary households spent 27 percent more than the control group on housing. Given Honduras’ vulnerability to flooding and other climate-related hazards, improvements in housing strengthen household resilience to natural disaster. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 44. The World Bank was well-positioned to support the GoH’s efforts to establish a rigor ous CCT program within a larger SP framework given its prior experience in engaging on health and education, knowledge of country-specific challenges, and expertise in working with CCT programs in the region. It closely collaborated with the IADB during preparation to ensure that both institutions supported common systems and processes during implementation. Since 2002, the Bank has supported the GoH in addressing health and education supply gaps, critical to ensuring the availability of social services in the context of the project. Building on this country-specific knowledge and the policy experiences around the region, the Bank worked with the GoH to design a program that best addressed some of the most pressing social challenges in Honduras. 45. The project’s design leveraged the existing capacity, experience, and systems, building on PRAF’s 20 years of experience implementing cash transfers and lessons from the pilot of the Bono 10,000 Program; however, it may have been overly ambitious given the lack of inter-sectoral coordination and the political, social and fiscal context of the country. Initial use of PRAF’s beneficiary database, SIRBHO, which contained household data in over 1,500 poorest villages, enabled the Program to effectively target the poor from the start, enrolling nearly 160,000 households in the first few months. Thereafter, the Program envisioned building on SIRBHO, systematically identifying and registering beneficiaries, and applying a phased approach for expanding coverage while maintaining strong targeting. However, the project’s design, which was based on the Program, was highly ambitious. Early on, PRAF had limited interaction with the MoE and MoH affecting the verification on compliance with co-responsibilities from the start, affecting the Program’s ability to make regular payments to beneficiaries. Furthermore, SIRBHO information became outdated and did not have and updating process established. Moreover, the Program’s coverage goals were excessive given the unpredictable political situation immediately after the change in government in 2010 following the political crisis, the deteriorating fiscal situation, and the precarious social make-up of Honduras at the time. Subsequently, without robust systems in place, many of the Project’s key results were stalled in the initial years of the Project. B. KEY FACTORS DURING IMPLEMENTATION a) Factors subject to the control of the Government and/or implementing entities 46. Throughout the Project, the GoH exhibited unwavering support for the Program and the country’s wider approach to SP and instated important legislations. The Program and the comprehensive approach to SP spanned across three political administrations, a testament to the Program’s results and the GoH’s commitment to reducing poverty and promoting shared prosperity. This was further exhibited in 2015 when the GoH approved the first SP Law in the country (Decree No. 56-2015) and the Presidential Decree in 2014 (No.71-2014) mandating the use of RUP as the targeting tool for all social programs. The SP Law in particular, signals the GoH’s commitment towards the institutionalization of the Program. people living in the home according to the number of rooms available. Page 19 of 55 The World Bank Social Protection (P115592) Furthermore, in 2018 to ensure fiscal sustainability, GoH issued an executive decree (No. 021-2018) committing to annually allocate ten percent of the GoH’s Solidario Trust Fund to the Program. 47. The GoH took concrete actions to safeguard the Program from political influence during the Presidential elections. In anticipation of the November 2017 elections, the GoH halted CCT payments 60 days prior to elections, beyond the 30 days stipulated by GoH law, and carried out workshops with and sent written notifications to beneficiaries, Program staff, and authorities at the central and local levels regarding Program rules and beneficiary rights. The Program also halted the CCT payments due to social unrest following the November 2017 elections, as it was not considered safe to carry out this process. 48. Other factors subject to the GoH’s control during implementation include the following: i. GoH’s commitment to automate key Program processes, including the RUP, the Program’s MIS, the verification of co-responsibilities, and the Program GRM. As the MIS was fully upgraded, Project management improved substantially. ii. Inter-agency coordination and engagement improved throughout the project, with the support of two key instruments: i) the Program’s Operation’s Regulations; and ii) the Program’s multi- sectorial Technical Committee,27 allowing for better effective management of the Program. Although the Program’s Technical Committee played an important role in coordinating the Program operation, it fell short in making sectoral agencies accountable in investing to meet its commitment for the Program, for example in establishing a health MIS. The newly given mandate of the Secretary of General Coordination of the Government (Secretaría de Coordinación General del Gobierno, SCGG) and its Social Cabinet is expected to play this key oversight role in future efforts requiring multisectoral programs. iii. Lack of sector specific MIS for health in particular limited verifying co-responsibilities during the first half of the project implementation. Sectoral agencies did not have the resources to verify Program co-responsibilities which caused the project to rely on executing entities PRAF and MoP to carry out this task. While the Program supported the RENPI, the MoH, by the project’s end, had decided to develop its own health MIS, -SINOVA- which was used by the Program (at project’s closing) given its advanced development and operational stage, and incorporated information from RENPI, among other sources. At closing, SINOVA was at pilot stage but not yet fully operational to replace the RENPI for compliance verification. iv. Establishment of the SEDIS and the SSIS in 2014 led to implementation delays of close to a year. The executive decrees issued to define their functions resulted in a sustainable institutional arrangement for SP by strengthening its institutional framework for social policy and streamlining procedures for improved Program implementation. However, with SSIS absorbing the PRAF, key functions such as budget management was channeled through the SEDIS, at times delaying flow of funds. Nonetheless, the PCU remained intact, preserving institutional memory of the Program, and, v. The RUP and the ROI are very effective registries that were integrated into a Social Registry that could be further applied in Program design, monitoring, and assessments of the social sector. However, the lack of protocols for information-sharing that were compatible and in accordance with the Law of Access to Information, resulted in underuse of this rich source of information for policy making. 27Which, initially, included the MoP, SDS, MoE, MoE, MoF, Technical Planning and External Cooperation, PRAF and National Institute of the Woman (INAM). Following government’s change in 2014, CENISS was added and MoP was removed. Page 20 of 55 The World Bank Social Protection (P115592) (b) Factors subject to the control of the World Bank 49. Leadership and intensive and regular supervision were key to ensuring the project’s success. Throughout implementation, the World Bank in close coordination with the IADB, supported regular meetings, virtual or in person, to troubleshoot, to asses processes, and to discuss higher-level policy decisions. Agreements on SP policies and on the Program were reached by way of high-level meetings involving policy makers, development partners, and international experts. These high-level meetings conveyed a set of unified recommendations from the World Bank and IADB, ensuring that the projects financed by both institutions achieved their objectives and addressed factors that could diminish their impacts. However, working with multiple development partnerships also come at an administrative cost and reporting burden to the PCU. At times this led to confusion and questions on the part of the PCU as to how to prioritize the needs of each partner. Nevertheless, the benefit of working closely with IADB was a clear win-win all institutions. This approach allowed for a consensus with the GoH to implement important Program reforms, including maintenance of the Program’s expansion considering the fiscal situation and financial sustainability of the Program. 50. The Bank’s reporting on Project’s progress through the implementation status and results report (ISRs) was candid, but often left one wondering whether major issues mentioned in prior ISRs were resolved. The Bank’s ISRs identified key issues and bottlenecks, with agreed actions and next steps to improve project performance and improve ratings when necessary. However, although ISRs followed up on most issues that were resolved, in some cases follow-through with next steps was not documented. (c) Factors outside the control of the government and/or implementing entities 51. A lack of private sector interest in serving Program beneficiaries on the part of commercial banks through basic bank accounts, as well as a reluctance to support the GoH’s social initiatives, prevented the project from reaching scale and achieving a sustained alternative payment mechanism. Despite success with piloting alternative payment mechanism pilots during the project, it closed without a viable alternative payment mechanism. 52. Difficult social context resulting from the contested November 2017 presidential election prevented the Program from delivering nearly 20 percent of payments from the second scheduled payment of that year to beneficiaries. While it was initially scheduled to be completed prior to the election, it was further delayed due to the political and social context following the electoral period. As a result, part of the second payment was delayed for not wanting to risk the safety of beneficiaries, Program staff, payment agencies, and the audit firm, until the first half of 2018. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 53. At the start of the Project, many of the PDO indicators were consistent with those used in other CCT programs; however, they were too ambitious given the absence of a M&E system. The original project design had four PDO indicators and 13 IRIs, later revised to nine PDO indicators (the ninth indicator being in case an emergency response would be triggered) and 14 IRIs. The original PDO indicators and IRIs were designed to capture information on the CCT Program, data that PRAF was Page 21 of 55 The World Bank Social Protection (P115592) supposed to have already been collecting in its MIS on targeting, payments and verification of co- responsibilities but this was not the case, thus no baselines were available during the design phase. Apart from data from the Program’s MIS, the M&E was designed to draw on targeting and coverage assessments based on EPHPM, IEs, process evaluations, beneficiary satisfaction surveys, social audits, and quarterly Project Management Reports. 54. Health and education indicators were not well defined nor regularly tracked at the design stage, leading to the lack of baselines and misalignment of those indicators with the GoH’s sector standards. During the design of the project, the education sector’s standard for good attendance was 75 percent; nonetheless, the World Bank considered important to adjust it to 80 percent given international standards. MCH standards in Honduras were also different, with the project’s indicators being more ambitious and in line with international standards.28 Furthermore, determination of a baseline and a set of realistic target goals for these indicators was difficult due to the lack of health and education sector MIS’. M&E Implementation 55. M&E faced moderate shortcomings early on, with a lack of available information preventing the Project from reporting on many indicators until the fourth ISR in 2012 but used other data sources to report on PDO indicators. From early on, the project encountered problems in reporting progress on the project indicators. In some cases, the project was simply unable to report on indicators, in others it could only report on Program-level indicators. The project, however, was able to report on the PDO indicators based on information from other sources such as the first Program IE. Problems reporting on indicators resulted, in part, from the fact that the Program’s MIS needed time to be developed, which only became fully functional in 2015. While reporting on indicators in the education sector was facilitated by the existence of SACE, the MIS used to verify health co-responsibilities—RENPI—took longer to develop. 56. The project team was proactive in addressing weaknesses in information systems and the project’s RF by ramping up efforts to collect data on education co-responsibilities with SACE and on health co-responsibilities via RENPI, as well as through Restructurings and AFs. Adjustments made through Restructurings involved adding/dropping indicators to better reflect the context of changing GoH priorities and address redundancies between PDO and IRIs. 57. The project compensated for information system weaknesses with rigorously evaluations. Evaluations included: (i) two IADB-financed IEs executed; (ii) one beneficiary satisfaction survey; (iii) one IADB-financed satisfaction survey on pilot payment mechanisms (Tigo Money); (iv) one social audit with the participation of civil society; and, (v) one evaluation of operational processes. Evaluations were participatory, incorporating all relevant entities, including the MoH, MoE, CENISS, PRAF, and MoP and supported continuous improvement for enhancing Program’s impact. M&E Utilization 58. The use of M&E was highly relevant. Most of the evaluation results were used as the basis for Program adjustments. Several Program adjustments were made in response to evaluation results: (i) the 28An example of this is with vaccinations. At the time, Honduras only monitored vaccinations for children aged 0-3 years; however, World Health Organization standards were for children aged 0-5 years. Page 22 of 55 The World Bank Social Protection (P115592) benefits structure was revised to increase health and education impacts for all HH members29 and to increase fiscal sustainability;30 (ii) given the relatively modest poverty impacts, the Program changed its focus to the extreme poor; (iii) the Program extended the education grant to cover children up to grade 9, as the IE found higher enrollment and attendance among the 1-6 graders; and (iv) greater emphasis was placed on alternative payment mechanism pilots as unpredictability in payment receipts and long delays at payment centers were among the major sources of dissatisfaction among Program beneficiaries. Justification of Overall Rating of Quality of M&E 59. Overall, M&E is rated as substantial given the effective reporting of Program indicators using the M&E platform and informing Program reforms for continuous improvement through IEs and other assessments. While there were initial shortcomings in the M&E design and implementation, making it difficult to assess achievement of PDOs during implementation and test the links in the results chain for much of the project, the project did, however, make effective use of M&E findings to inform Program design and implementation. The project compensated for weaknesses in M&E arrangements by adjusting the RF through Restructurings and AFs, thus enabling improved tracking of PDOs, as well as by taking advantage of results from the first IE in the absence of Project data on PDO indicators. At closing, the Program’s MIS became a reliable mechanism for monitoring performance. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 60. Environmental Safeguard policies were not triggered. 61. Indigenous. Given the presence of IPs in the project area, OP/BP4.10 was triggered. The project produced an Indigenous and Afro-Honduran Peoples Plan (IAPP, initially referred to as IPP) published in 2010 and updated in 2014 to reflect institutional and Program changes. The 2014 version of the IAPP included the expansion of coverage into remote areas with a high presence of IPs (namely, Gracias a Dios). The ISR ratings for social safeguards wavered between Satisfactory and Moderately Satisfactory for most of the project’s implementation period, finishing with the latter. The final rating was based on the difficulties in the provision of ID cards to IP households in collaboration with the RNP and delays in incorporation of mitigation activities set out under the 2014 IAPP.31 Prior to closing, a unit was established within SSIS for the attention IPs and to apply the IAPP to all SSIS Directorates’ activities. 62. Fiduciary. ISR ratings for both financial management (FM) and procurement were Moderately Satisfactory during project implementation, due to moderate shortcomings during implementation. For example, the risks associated with the deficiencies in the development of automated payment mechanisms, and the MIS’s processing payrolls for the cash transfers. Nonetheless, although the GoH was overall compliant with the interim financial reporting and the external audit requirements, during the life 29 The first iteration of the Program was not designed for maximal impact with weak conditionality structures and incentives, since only one child was required to comply with the conditionality for the family. This is confirmed by the fact that school enrollment effects were larger among families with only one child, while they were smaller and insignificant on children in larger households. Similarly, visits to health centers increased the most for children and mothers in the absence of school-aged children. 30 Benefit sizes were to be calculated according to the number of children complying with co-responsibilities (as opposed to a flat per family amount conditional on one household member complying with co-responsibilities), with a 10,000 Lempiras per household per year being the maximum amount a household could receive. 31 Revision and mainstreaming of indigenous peoples in Operational Manual and Guidelines; Inclusion of ethnicity variables in the MIS and FSU; approval of a protocol for the participation of indigenous communities; formulization of inter-institutional coordination with indigenous federations; and adaptation of the GRM’s monitoring of complaints for indigenous groups. Page 23 of 55 The World Bank Social Protection (P115592) of the project, the auditors qualified the opinion on the Financial Statements for two Credits (5294-HN & 4774-HN) due to beneficiaries not meeting the eligibility criteria. The final audit for credit 5603-HN, including an additional analysis to verify the meeting of the eligibility criteria among beneficiaries identified as not meeting them by the previous concurrent audits has been concluded. The six-month external concurrent audits played an important oversight and control role—especially as the project strengthened the Program’s MIS—by reviewing eligibility, payments and co-responsibilities for beneficiary households. Shortcomings in financial management included the uncovering of some ineligible expenditures due to beneficiaries’ non-compliance with co-responsibilities. The GoH confirmed to substitute these expenditures for eligible expenditures. There were no major issues with procurement, as most contracting were carried out in accordance with Bank guidelines; nevertheless, some activities were delayed due to delays in the approval processes of GoH’s budgetary modifications and government- imposed budget ceilings as well as issues with the Terms of Reference and Technical Specifications, lack of bidders, and changes in key personnel in the PCU during 2018. Even so, the project closed with a strong procurement unit; resulting in the completion of several large procurement processes prior to closing. C. BANK PERFORMANCE Quality at Entry 63. There were shortcomings in the quality at entry. On the heels of a political transition and the aftermath of a global financial crisis, the project was in four months, in part to qualify for the IDA’s Crisis Response Window (CRW). This meant that certain due diligence needed to determine the GoH’s readiness to implement such an ambitious project was not carried out. Immediately after the new government took office in late January 2010, the GoH, requested technical and financial support from the World Bank (and the IADB) for the creation and financing of a new CCT Program, Bono 10,000. In response to the request, the Bank carried out an intensive project preparation to help the GoH define the parameters of the new Program, which included the implementation, management and monitoring arrangements. For the GoH, many of the details and processes of the Program had not been fully thought out and there was not a strong understanding of the capacity of existing systems. Institutional arrangements and reporting lines established at the time were still being defined, resulting in duplications in efforts in the early implementation of the project. 64. At entry, the World Bank acknowledged the long-term fiscal sustainability challenges of financing a CCT Program for 600,000 poorest households in Honduras. During project preparation, the GoH was highly committed to paying households HNL 10,000 per year/US$ 407 and reaching 600,000 households in four years. At the time, the transfer amounts were one of the most generous cash transfers relative to other CCTs in the region. The project’s economic analysis assessed that the Program would have a transformational impact on poverty in the country, it also raised that the risk to fiscal sustainability of the Program would be critical. The government presented a financing plan for the Program comprised of external and national funds. However domestic funds would depend on the availability of resources, which would be a limiting factor in expanding the Program. Possible sources would come from streamlining multiple social programs and subsidies to reduce public spending but did not materialized in the short-term. This would prove to be consequential throughout the life of the project when the Program would have to change the payment structure (monthly payment of HNL 833 /US$34 per month to be the maximum amount the household can receive) and reduce the coverage target. At closing, the Program’s financial sustainability remained one of the greatest risks to the development objectives (see below). Page 24 of 55 The World Bank Social Protection (P115592) 65. That being said, with close collaboration with IADB, the World Bank leveraged on its knowledge, research and experience with other CCT Programs to help inform the design of the Program. The Project took into consideration findings from Bank studies,32 enabling a coordinated approach to sectoral policy dialogue with the GoH, structured support to operations, and a common approach to working with other development partners. 66. Finally, the World Bank accurately assessed most risks. At appraisal, the Project team captured most risks to achieving PDOs as high. In addition to the fiscal sustainability risk, risks identified were related to pressures for Ministry of Presidency (MoP) to: a) expand Program coverage without the necessary prerequisites; b) lead institutional arrangements for which reporting lines were still being defined implying a risk of duplicating institutional roles in coordination, monitoring and execution of the Program. The only risk that was not adequately identified was the one related to a lack of resources on the part of the health and education sectors for the verification of co-responsibilities. Quality of Supervision 67. Despite several changes to the Task Team, transitions were smooth with the majority of Task Team Leaders (TTLs) having previously participated in the Project as team members and several members participating from start to finish. The first TTL led the team from preparation through April 2011, with the second TTL taking over and staying on as TTL through May 2015, overseeing both additional financings and project’s restructurings. From there, the project’s leadership changed to co- TTLs, one of whom had been a team member since April 2012. Finally, both the safeguards specialist and the senior SP specialist were members of the team during the entire life of the project. 68. Supervision was uniquely effective in that missions were carried out jointly with the IADB and important policy issues were elevated through high-level encerronas. From early on, supervision missions were jointly conducted with the IADB, which allowed for supporting unified recommendations and strategies. In addition, the World Bank, in coordination with the IADB, was instrumental in leading encerrona meetings to advance on key policy discussions. Missions and follow-up meetings were important for identifying and resolving threats to the achievement of PDOs, by identifying necessary modifications to be addressed by restructurings and new activities to be included in AFs. Justification of Overall Rating of Bank Performance 69. The Bank’s performance is assessed as moderately satisfactory, with some shortcomings in the Bank’s quality at entry combined with its satisfactory quality of supervision. While the Bank leveraged significant regional and sectoral expertise to design a project that would support the institutionalization of a CCT program, it fell short in carrying out the necessary due diligence to fully understand the GoH’s capacity to implement such a Program. At closing, the Bank satisfactory supervision, though, had contributed to the consolidation of a CCT program with the necessary systems for compliance verification of co-responsibilities and monitoring Program performance. 32Institutional Governance Review (IGR), Public Expenditure Tracking Survey (PETS), Public Expenditure Review (PER), and the IADB review of options for the organizational reform of PRAF. The Project also drew on results from local analytical work analyzing the performance of existing SP programs and projects in Honduras to understand their strengths and weaknesses and to review the performance of existing targeting mechanism with the aim of informing new policies for improved targeting. Page 25 of 55 The World Bank Social Protection (P115592) D. RISK TO DEVELOPMENT OUTCOME 70. The risk to the project’s development outcomes is substantial. At closing, the GoH remained highly committed to reducing poverty and promoting the accumulation of human capital among the extreme poor. This is evidenced by the continued rigorous implementation of RUP and the increased GoH financing of the Program which contributes to the sustainability of the Program and its delivery system. Furthermore, the Government requested new operations to the World Bank (approved in April 2019) and the IADB (currently under preparation) to build on the results and consolidate its gains. Nonetheless, there is a high risk that the outcomes achieved at project closing could be jeopardized due to the following: i. CENISS, manager of RUP and ROI, has yet to be fully institutionalized. An Executive Decree reforming the GoH’s structure issued in March 2018 established that the SCGG coordinates CENISS’ activities, a critical step for enhancing CENISS’ credibility. Currently CENISS is mapped under the President’s Office, which may undermine CENISS’ potential role. ii. The Program’s budget has yet to be included in the national budget. The Program relies on the Solidarity Fund without specific earmarks for the Program under the national budget. iii. Key program procedures have yet to be further developed. Regular payment of beneficiaries through an alternative payment mechanism, and the verification of health co-responsibilities are critical to sustain the achievement of the Program objectives. V. LESSONS AND RECOMMENDATIONS 71. Ensuring a win-win for all was made possible by setting the stage for cooperation early at project design stage for development partner collaboration. The project did well in part because of the joint efforts with the IADB during design and implementation guided by common objectives, complementarity of investments at the financial, technical and geographical level, shared decision-making and promoting common reform agendas, greatly enhanced the effectiveness of both Banks working in the SP sector. 72. Implementation problems were faced in part due to poor institutional capacity analysis at entry. The short timeline for project preparation had its trade-offs. On one hand, swift project preparation resulted in the project being prepared at an ideal time for the government, ensuring government ownership of the project. On the other hand, this came at a cost of insufficient analysis of the capacity of sectorial systems which became a bottleneck for the project implementation. Soon after the launching of the project, it became clear that the GoH’s four-year goal of progressively reaching 600,000 beneficiaries was not in line with the existing capacities for increasing coverage and payment frequency. 73. A project supporting a multisectoral Program, requires a strong inter-sectoral coordination mechanism, clearly defined roles and responsibilities, and an accountability mechanism to ensure active participation of the sectors are key to effective implementation. Without a strong inter-sectoral planning and coordination mechanism, the project faced problems implementing the verification of co- responsibilities. The project could have been more efficiently implemented with clearer roles and responsibilities among the institutions established prior to project implementation. The Program’s Technical Committee fell short of making sectoral agencies accountable in investing to meet its commitment for the Program, as it was not mandated to do so. Page 26 of 55 The World Bank Social Protection (P115592) 74. Demonstrating impact through IE is key to ensuring the Program’s survival through government transitions and supported continuous improvements during Program’s implementation. Two IEs evidencing important poverty, health, education, and other impacts, including female empowerment. The existence of rigorous, objective, and irrefutable evidence of Program impact was key to protecting the Program during three government transitions and for Program’s continuous improvement. 75. Successful pilots do not necessarily mean it can be scaled up. Although the project had positive experiences with piloting alternative payment mechanisms, and many operational lessons were learned through these pilots, without an appropriate legal framework, political commitment, willing partners in the private sector, successful small-scale pilots did not translate into a national scale up. In the context of Honduras, the establishment of the alternative payment mechanism requires a long-term strategic approach. . Page 27 of 55 The World Bank Social Protection (P115592) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Institutional strengthening: (a) improve the institutional capacity of Recipient’s institutions to manage the CCT Program Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of households in Percentage 0.00 80.00 80.00 99.89 the Program receiving CCTs in rural areas that are extreme 19-May-2010 31-Dec-2015 31-Dec-2018 31-Dec-2018 poor Comments (achievements against targets): Exceeded. Objective/Outcome: Smoothing consumption to eligible beneficiaries and increasing the use of (i) preventive health services; and, (ii) school attendance in grades 1 to 9 Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of students in Percentage 0.00 85.00 85.00 85.52 Page 28 of 55 The World Bank Social Protection (P115592) grades 1 to 6 participating in 19-May-2010 31-Dec-2015 31-Dec-2018 31-Dec-2018 the Program who comply with the co-responsibility of 80% school attendance Comments (achievements against targets): Exceeded. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of students in Percentage 0.00 70.00 70.00 92.65 grades 7 to 9 participating in the Program who comply 31-Dec-2014 31-Dec-2017 31-Dec-2018 31-Dec-2018 with the co-responsibility of 80% school attendance Comments (achievements against targets): Exceeded Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of Program Percentage 0.00 78.00 78.00 95.03 beneficiary children aged 13- 15 years that completed 19-May-2010 31-Dec-2017 31-Dec-2018 31-Dec-2018 Page 29 of 55 The World Bank Social Protection (P115592) primary education (sixth grade) Percentage of female Percentage 0.00 78.00 78.00 95.97 Program beneficiary children aged 13-15 years 29-May-2010 31-Dec-2017 31-Dec-2018 10-Dec-2018 that completed primary education (sixth grade) Comments (achievements against targets): Exceeded. According to the Government's completion report, by Project closing, this indicator reached 95.03 percent. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of Program Percentage 0.00 60.00 60.00 93.51 beneficiary children aged 16- 18 years that completed 31-Dec-2014 31-Dec-2017 31-Dec-2018 31-Dec-2018 lower secondary education (ninth grade) Percentage of female Percentage 0.00 60.00 60.00 90.33 Program beneficiary children aged 16-18 years 31-Dec-2014 31-Dec-2017 31-Dec-2018 31-Dec-2018 that completed lower secondary education (ninth grade) Comments (achievements against targets): Page 30 of 55 The World Bank Social Protection (P115592) Exceeded. According to the Government's completion report, by Project closing, this indicator reached 93.51 percent. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent of Program Percentage 51.20 80.00 70.00 47.97 beneficiary children aged 0- 23 months with complete 01-Jan-2013 31-Dec-2015 31-Dec-2018 31-Dec-2018 vaccination scheme Comments (achievements against targets): Partially achieved. 69 percent of end target. Objective/Outcome: Provide funding for eligible emergencies: improve the Recipient’s capacity to respond promptly and effectively to an eligible emergency. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Time taken to disburse funds Weeks 0.00 4.00 4.00 0.00 requested by Government for an eligible emergency 19-May-2010 31-Dec-2015 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): This indicator was not activated during project implementation. Page 31 of 55 The World Bank Social Protection (P115592) A.2 Intermediate Results Indicators Component: Institutional strengthening: Institutional Strengthening of the CCT Program Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Communication Strategy Text None Implemented Implemented Implemented Implemented; Information on Program disseminated 19-May-2010 31-Dec-2015 31-Dec-2018 31-Dec-2018 through local media & adapted to the target population Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of households Percentage 3.00 50.00 60.00 100.00 registered in the Program with updated eligibility 19-May-2010 31-Dec-2015 31-Dec-2018 31-Dec-2018 information Comments (achievements against targets): Page 32 of 55 The World Bank Social Protection (P115592) Exceeded. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Management Information Text None Implemented Implemented Implemented System for the Program operating and generating 19-May-2010 31-Dec-2015 31-Dec-2018 31-Dec-2018 reports Comments (achievements against targets): Achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion System to attend complaints Text None Implemented Implemented Implemented and grievances designed, developed and implemented 30-Jun-2014 31-Dec-2015 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): Achieved. The grievance redress mechanism has been implemented in 17 field offices. Grievances and complaints are being recorded, the system allows tracking follow up to grievances and complaints and the field staff have been trained on the use of the grievance redress mechanism. Page 33 of 55 The World Bank Social Protection (P115592) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of households in the Number 0.00 2300.00 2300.00 2296.00 Program that are registered with a local productive 31-Dec-2014 31-Dec-2017 31-Dec-2018 31-Dec-2018 inclusion programs Comments (achievements against targets): At the time of the filing of the last ISR the Project reported 1,592 households and the indicator was considered partially achieved, reaching 69% of end target. However, the Government reported in its completion report that by Project closing, this indicator had reached 2,296, reaching 99.8 percent of the end target. This final figure is used as the final result in the ICR. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Strategy for streamlining Text None Strategy approved Strategy Strategy Approved social assistance programs Approved and subsidies approved and implemented 19-May-2010 31-Dec-2013 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): Achieved. Page 34 of 55 The World Bank Social Protection (P115592) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of programs using Number 0.00 9.00 18.00 90.00 the Unique Registry of Beneficiaries 19-May-2010 31-Dec-2015 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): Exceeded. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of individuals Number 3350000.00 4300000.00 4300000.00 4600399.00 incorporated in the Unique Registry of Beneficiaries 31-Dec-2014 31-Dec-2017 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): Exceeded. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of beneficiary Number 0.00 30.00 30.00 89.52 households in rural areas who receive their payment 19-May-2010 31-Dec-2017 31-Dec-2018 31-Dec-2018 Page 35 of 55 The World Bank Social Protection (P115592) through basic accounts Comments (achievements against targets): Exceeded. The contract with BANRURAL expired on December 2017 and households are being paid by the National Bank of Agriculture Development (BANADESA). The figure presented represents the coverage of basic accounts in the municipalities where offer and market interest existed to provide payments through basic accounts. However, overall program coverage achieved with basic account was minimal, and reached nearly 4 percent of program beneficiaries. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of beneficiaries Percentage 0.00 50.00 50.00 89.56 that received capacity building on the use of basic 19-May-2010 31-Dec-2017 31-Dec-2018 31-Dec-2018 bank accounts Comments (achievements against targets): Exceeded. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of children Number 380000.00 570000.00 570000.00 755425.00 registered in RENPI 01-Jan-2016 31-Dec-2017 31-Dec-2018 31-Dec-2018 Page 36 of 55 The World Bank Social Protection (P115592) Comments (achievements against targets): Exceeded. Component: Component 2. Conditional Cash Transfers Program Grants Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of households Number 0.00 300000.00 240000.00 213886.00 receiving payments in the year in the Program in rural 19-May-2010 31-Dec-2017 31-Dec-2018 31-Dec-2018 areas Comments (achievements against targets): Substantially achieved. 89 percent of end target. According to the Government's completion report, by Project closing, this indicator reached 213,886 Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of households Number 0.00 40000.00 40000.00 18807.00 receiving payments in the year financed through the 19-May-2010 31-Dec-2017 31-Dec-2018 31-Dec-2018 Project in rural areas Comments (achievements against targets): Page 37 of 55 The World Bank Social Protection (P115592) Achieved. This target is annual and not cumulative. For the years 2011-2014 the annual target was 22,000 households and it was surpassed every year. In 2015 with Program changes in targeting, this target was increased to 40,000 households for the period 2015-2018. Under this period the Project exceeded its annual target on 2016 and 2017 with 62,974 and 78,517 households respectively. However in 2015 and 2018 could only reach 6,076 and 18,807 households respectively. While there was underachievement in 2015 due to delays in implementing the new program targeting, the shortage of target in 2018 owed to a positive achievement consisting of government funding most of the households with its own funds in part because the Bank had nearly disbursed all of its credit proceeds by then. According to the Government's completion report, by Project closing, this indicator reached 18,807 households with credit proceeds. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of indigenous and Percentage 2715.00 10000.00 10000.00 8503.00 afro-Hondurans households registered in the Program 31-Dec-2014 31-Dec-2017 31-Dec-2018 31-Dec-2018 and financed through the Project Comments (achievements against targets): Substantially achieved. 85 percent of end target. Page 38 of 55 The World Bank Social Protection (P115592) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 Improve the institutional capacity of Recipient’s institutions to manage the CCT Program, by strengthening transparent mechanisms and instruments for targeting CCT Program beneficiaries, monitoring compliance with the CCTProgram co-responsibilities, and making payments to the CCT Program beneficiaries 1. Percentage of households in the Program receiving CCTs in rural areas that are extreme poor Outcome Indicators 1. Communication Strategy Implemented; Information on Program disseminated through local media & adapted to the target population 2. Percentage of households registered in the Program with updated eligibility information 3. System to attend complaints and grievances designed, developed and implemented 4. Management Information System for the Program operating Intermediate Results Indicators and generating reports 5. Number of households in the Program that are registered with a local productive inclusion programs 6. Strategy for streamlining social assistance programs and subsidies approved and implemented 7. Number of programs using the Unique Registry of Beneficiaries 8. Number of individuals incorporated in the Unique Registry of Beneficiaries Page 39 of 55 The World Bank Social Protection (P115592) 9. Percentage of beneficiaries that received capacity building on the use of basic bank accounts 10. Percentage of beneficiary households in rural areas who receive their payment through basic accounts 11. Number of children registered in RENPI 1. Communications strategy implemented 2. Updated registry of Program beneficiaries 3. GRM operating and generating reports 4. MIS operating and generating reports , and interconnected with SACE and RUP. 5. 17 deconcentrated offices for customers service 6. Program beneficiaries registered with a local productive inclusion program 7. Social Protection Law approved Key Outputs by Component 8. CENISS established with a ROI and a RUP holding a targeting (linked to the achievement of the Objective/Outcome 1) system, populated with household information and social programs integrated into RUP 9. Proxy means test for targeting updated. 10. Beneficiaries have open saving basic accounts and were trained in its use 11. RENPI populated with children vaccination information. 12. Three pilots for altermative payment mechanisms tested. Page 40 of 55 The World Bank Social Protection (P115592) Objective/Outcome 2 Smoothing consumption to eligible beneficiaries and increasing the use of (i) preventive health services; and, (ii) school attendance in grades 1 to 9 among Program beneficiaries 1. Percentage of students in grades 1 to 6 participating in the Program who comply with the co-responsibility of 80% school attendance 2. Percentage of students in grades 7 to 9 participating in the Program who comply with the co-responsibility of 80% school attendance 3. Percentage of Program beneficiary children aged 13-15 years that completed primary education (sixth grade) 4. Percentage of female Program beneficiary children aged 13- Outcome Indicators 15 years that completed primary education (sixth grade) 5. Percentage of Program beneficiary children aged 16-18 years that completed lower secondary education (ninth grade) 6. Percentage of female Program beneficiary children aged 16- 18 years that completed lower secondary education (ninth grade) 7. Percent of Program beneficiary children aged 0-23 months with complete vaccination scheme 1. Number of households receiving payments in the year in the Program in rural areas 2. Number of households receiving payments in the year Intermediate Results Indicators financed through the Project in rural areas 3. Number of indigenous and afro-Hondurans households registered in the Program and financed through the Project Key Outputs by Component 1. Beneficiary students in grades 1-6 attending school and (linked to the achievement of the Objective/Outcome 2) completing primary education Page 41 of 55 The World Bank Social Protection (P115592) 2. Beneficiary students in grades 7-9 attending school and completing lower secondary education 3. Beneficiary households receiving payments in the year in the Program (including IP population) 4. Beneficiary households receiving payments in the year in the Project (including IP population) 5. Program beneficiary children aged 0-23 months with complete vaccination scheme Outcome 3. Provide funding for eligible emergencies: improve the Recipient’s capacity to respond promptly and effectively to an eligible emergency. (elegible emergency was not activated, thus not applicable) Page 42 of 55 The World Bank Social Protection (P115592) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Junko Onishi, Nancy Rocio Banegas Raudales Task Team Leader(s) Leonel Jose Estrada Martinez Procurement Specialist(s) Luz A. Zeron Financial Management Specialist Mirta G. Sanchez Team Member Ilka Funke Team Member Maria Concepcion Steta Gandara Team Member Robert H. Montgomery Environmental Specialist Angela Maria Rubio Martinez Team Member Rodolfo Tello Abanto Social Specialist A. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY09 18.657 109,679.37 FY10 45.768 246,284.88 FY11 0 -1,324.05 FY12 0 0.00 Total 64.43 354,640.20 Supervision/ICR FY11 18.155 128,775.52 Page 43 of 55 The World Bank Social Protection (P115592) FY12 47.950 209,656.40 FY13 41.110 158,276.78 FY14 49.078 146,262.44 FY15 29.471 117,881.41 FY16 34.210 151,438.88 FY17 45.525 212,481.46 FY18 41.174 195,416.16 FY19 20.807 147,024.13 Total 327.48 1,467,213.18 Page 44 of 55 The World Bank Social Protection (P115592) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Institutional strengthening of 12.8 10.4 81 the CCT Program Conditional Cash Transfers 61.3 61.6 101 Program Grants Immediate Response 0 0 0 Mechanism Total 74.1 72.0 97 Given that the original credit and first AF were subscribed to in the Bank’s official currency (XDR) and the USD-XDR exchange rate fluctuated during the operation, the Project accumulated more resources in USD terms. Page 45 of 55 The World Bank Social Protection (P115592) ANNEX 4. EFFICIENCY ANALYSIS 1. The Project Appraisal Document of the parent project focused on the Economic and Financial Analysis, rather than an efficiency analysis. An economic analysis carried out at appraisal analyzed the potential impact of the CCT Program on both poverty and the returns to investment in human capital accumulation through education (health investments were not calculated). Assuming perfect targeting, annual cash transfers of L$10,000 to the poorest, covering the originally targeted 600,000 households, it was estimated that the Program would have reduced the poverty gap by 22 percent in four years-time; and, an average two-year increase in schooling would have raised the beneficiaries’ average labor income by 18.8 percent by 2025. However, these assumptions changed during Project implementation, particularly the targeted population, coverage, and the size of the cash transfer. Given these changes, estimations on Program impact at Project closing are difficult to compare with those estimated at appraisal. The fiscal analysis was based on estimations in the percent of GDP the Program would cost by 2013. It was estimated that by 2013, the Program would take up 2.0 percent of GDP if it continued to provide HNL 10,000 per family, but would be reduced to 1.29 percent of GDP if the program benefits were revised to half of that (up to HNL 5,000 per year per family) for families complying with only one (health or education) co-responsibility and a full HNL 10,000 for those families meeting both (health and education) co-responsibilities. 2. The Bank’s standard measure of economic analysis could not be conducted for this project as the estimate of the administrative cost of the Program could not be obtained. As such, this Annex describes the evidence with respect to the Program’s efficiency by reviewing the effectiveness of targeting the poor and by reviewing the adequacy of benefit size and comparing them with the findings from the two IEs of the Program undertaken during the course of the Project—the first in 2013 and the second in 2017 — both financed by IADB but with TA provided by the World Bank. The IEs are representative of the impacts of the Program, as supported by the Bank and co-financing partners including the IADB and the Central American Integration Bank (BCIE for its acronyms in Spanish). IADB in particular worked closely with the World Bank and provided financial and technical support to the Program throughout Project preparation and implementation. 3. Coverage. At appraisal, the Program was expected to incorporate 60,000 households in 2010, adding 150,000 new households each year for four years, thus reaching 600,000 households by 2013. Due, in part, to fiscal and operational constraints, the Program reached a maximum coverage level of 250,000 households, achieved in 2014, with 210,343 beneficiary households at Project closing.33 As such, relative to the most recent poverty and extreme poverty headcounts of 619,680 and 525,922 rural households,34 respectively, assuming perfect targeting, the Program would have covered 34 and 40 percent of the poor and extreme poor, respectively. However, relative to other CCT programs, coverage of extremely poor households by the Program is low (World Bank, 2018).35 4. Benefit size. At the time the Project was approved, the maximum transfer amount of 10,000 Lempiras per year (US$44 per month), represented 25 percent of median household income for the extreme poor 33 Coverage levels between 2014 and 2016 were as follows: 250,000, 146,000, and 212,546 households, respectively. 34 From the 2017 Encuesta Permanente de Hogares de Propósitos Múltiples (EPHPM). 35 An analysis of the share of the poorest quintile that receives CCT payments demonstrates that Honduras falls short of an average of 40.3 percent, at roughly half of that amount in 2013. Page 46 of 55 The World Bank Social Protection (P115592) (US$176), and roughly 46.7 percent of the per capita income of those in the poorest quintile. That transfer amount in 2017 represents almost 28 percent of median household income of the extreme poor (36,000 Lempiras or US$132 per month). Using the 2017 data, a HNL 10,000 transfer would represent approximately 12 percent of the food staples basket in rural areas. The actual average transfer size in 2016 was HNL 5,967 (roughly US$ 20 per month), representing15 percent of median household income of the extreme poor, before the transfer, and 25 percent of the average per capita income of those among the poorest quintile. The value of the transfer as a share of post-transfer welfare would be 13.3 percent, just shy of a worldwide average of 15.6 percent (World Bank, 2018). In 2015, at the time of the AF, the team had estimated an annual transfer size of HNL 5,500, representing 24.4 percent of the income of the poorest quintile, suggesting that the team’s ex-ante estimation of the benefit size was accurate. 5. Program Effects. The Project Appraisal Document (PAD) analyzed the potential effects of the Program by: i) simulating the short-term effects of transfers on extreme poverty, and ii) analyzing potential long-term effects of an increase in years of schooling on future income. Evidence from IEs demonstrates that the Program had important impacts on poverty reduction (in a context of increasing poverty at the national level during the time period covered by the first IE), increasing the utilization of health and education services, and, in the case of the second IE, on infant nutrition and health outcomes (Table 3). The two IEs conducted at two different time periods also measure the program impacts of the program with the different benefit sizes. The first IE in 2013 measured the program impact when the benefit size was a lumpsum amount of HNL 10,000 per household, while the 2015 IE measures the program impact of differentiated benefit calculations based on household structure. Since the program impacts of the different benefit sizes were not measured in one IE, the two IE results are not directly comparable,36 however, the comparison in terms of areas in which the two IEs detected program impacts are a measure of efficiency gains in the Conditional Cash Transfers under Component 2. Short term objective: extreme poverty reduction 6. The initial economic analysis projected that the Program would result in a modest reduction in the extreme poverty headcount: 3percent in 2010 and 10 percent by the fourth year of the intervention. However, given that extreme poverty depth is so high in Honduras, the real impact of the Program was projected to be a reduction in the extreme poverty gap, falling by 7 percent in 2010 and 22 percent in 2013. Both IEs found that the Program led to decreases in both the poverty headcount and gap. The first IE found a reduction in the poverty headcount by a magnitude of 3.1 percent (from 82.4 to 79.3) while headcount in the second fell from 12.2 percent (from 71.8 to 59.6).37 The second IE also investigated Program impacts on the poverty headcount according to the Multidimensional Poverty Index (MPI), finding a reduction from 34.7 to 33.2 percent.38 With respect to income/consumption, the first IE documented an increase in consumption of 7.8 percent and the second an increase in income of 18 percent.39 36 The ideal IE design to compare different benefit sizes would have been to randomly assign communities with differentiated benefit sizes in one IE study. 37 Importantly, the PAD calculated poverty reduction impacts at the national level. IEs documented poverty impacts among treated and control households, only. 38 The IE also found significant reductions in poverty prevalence in the housing and health dimensions, falling from 11.7 to 9.5 percent and 52.8 to 49.9 percent, respectively. 39 The authors of the second IE state that the mechanisms and economic behaviors, probably linked to the labor supply, that make the income increase more than the size of the program are not obvious and deserve more investigation. One hypothesis considered in this case was that in scattered rural communities, the presence of a donation would guarantee a minimum income that would allow some households to try new (possibly risky) activities in nearby urban centers that would result in higher Page 47 of 55 The World Bank Social Protection (P115592) Long term objective: human capital investment 7. Education. School drop-outs in Honduras spike when children reach 12 years of age and transition from segundo to tercer ciclo, with many children opting to work as a means of generating extra income for their households. Accordingly, the PAD identified as one of the objectives of the Program to increase attendance and reduce dropouts. The PAD: i) described international evidence on CCT Program impacts on school enrollment and attendance; and ii) simulated the impact on income of an additional two years of schooling, finding that such an increase would increase beneficiaries’ average labor income by 18.8 percent by 2025. Given the limited time frame of both IEs (just one year in both cases), it is difficult to make any conclusions regarding the validity, in retrospect, of the original simulation. In order to do so, it would be necessary to undertake a long-term IE investigating the impacts on years of schooling. 8. Both IEs found that children 6-1740 year of age in the treatment group enrolled in and attended school more frequently than children in the control group, with mean differences significant for both years for which data is available (Table x).41 Both IEs also looked at heterogeneity of impacts across age groups, finding increased impacts the further along a child is in school. In the context of the first IE, the largest impacts on attendance were found for children registered in grades 4-7 at baseline (i.e., 9-12 years of age) (Figure 2). However, mean differences are not significant for all age groups and for all years.42 Enrollment impacts (not shown) were only found for children 11 and 12 years old at baseline in 2012; effects were no longer significant one year later. Figure [2]. Heterogenous effects on school attendance at the end of the 2012 school year and at follow-up data collection in 2013 Source: NORC (2013) incomes. 40 Because of a change in eligibility criteria following the first IE, the second IE measured impacts on children 7-11 years old. 41 2012 and 2013 for the first IE and 2016 and 2017 for the second. The second IE found slightly larger effect sizes than the first one. 42 For example, even though children 11 years old at baseline (e.g., grade 6) were significantly more likely to attend school at ages 12 and 13 (by 10 and 7 percent, respectively), for children 10 years old at baseline, results were only significant in 2012 at age 11; results are no longer significant one year later. Page 48 of 55 The World Bank Social Protection (P115592) 9. With respect to the second IE, relative to pre-básica (pre-primary), impacts were higher the further along a child was in his or her schooling. Results, however, were only significant for children in segundo ciclo (grades 4-6) and media (grades 10-12), with students 0.4 and 10.3 percent more likely to be enrolled relative to students in pre-básica, respectively. Results reflect the higher relative benefits paid in the context of households with older children. Likewise, with respect to child labor, the second IE found a reduction of 2.4 percent vis-à-vis the control group, mostly attributable to a reduction of 4.0 percent in the context of older children in tercer ciclo (grades 7-9).43 10. Health. The Program was also aimed at increasing use of public health services by mothers and children (i.e., prenatal controls, periodic controls for newborns, and vaccination rates), which was believed to positively impact future health outcomes. The original economic analysis cited international evidence that CCT programs had been shown to increase the use of health services; however, no simulation was undertaken with respect to the economic impact of an increase in the use of health services. 11. Both IEs document an increase in the prevalence of height and weight monitoring and visits to health centers with infants. No such impacts were found on use of prenatal attention44 or vaccination of infants.45,46 However, with respect to health, growth, and nutritional outcomes, several positive impacts were found among children 0-5 years old, including reductions in the prevalence of Acute Diarrheal Disease, Acute Respiratory Infections, and malnutrition in the context of the second IE. Impacts may have been a consequence of increased food and healthcare expenditures, prevalence of height and weight monitoring, and quality of food consumed. Few health impacts were detected by the first IE, likely due to a combination of i) the short time frame of the IE, which may not have left enough time to pass to detect meaningful impacts, ii) supply side constraints, and iii) the fact that, on average, treatment households received 1.9 payments over the last twelve months, when they should have received three (NORC, 2013).47 Doubts with respect to the amount and frequency of payments may have diminished compliance with co-responsibilities (i.e., school attendance, prenatal controls). Neither IE detected impacts on acute malnutrition or stunting among children 0-5 years old. 12. Program impacts with different benefit sizes: The two IEs conducted in 2013 and 2017 both reported reduction in poverty headcount, poverty gap, as well as increase in school registration and school attendance among 6 to 17-year-olds. Although both IEs reported increased attendance in weight and height checkups among infants (under-one), the program impacts on health outcomes such as acute diarrheal diseases, acute respiratory infection, childhood malnutrition, as well as reduction in childhood labor were only found in the 2017 IE, while the 2013 IE did not find statistically significant impacts. This may suggest that the improvements in the program operation allowed for the program to show impacts in a more comprehensive set of indicators in 2017, while the program continued to show positive program impacts in poverty-related 43 The first IE found no impact. 44 There were no significant effects on the number of prenatal controls, immunization rate of mothers against tetanus, levels of advice on the birth plan, institutional delivery rate nor of postnatal controls. 45 One explanation for the lack of impacts with respect to vaccination rates is that coverage was already high to begin with. For example, in the control group, the percentage of children with the BCG vaccine increased from 87 to 93%. 46 Indicators for i) visits to health centers in the context of infants, ii) prenatal checkups, and iii) vaccinations were reported in the context of the first IE, but not in the second. 47 There were no effects on indicators of child health, such as: frequency of diarrhea, fever or respiratory problems, Z-scores of the size-age, weight-height and weight-age relationship, and levels of hemoglobin in blood and incidence of anemia. Page 49 of 55 The World Bank Social Protection (P115592) indicators and education-related indicators. This is despite the reduced benefit sizes in 2015 and may suggest an improved program efficiency over time. Table 3: Selected indicators from the two impact evaluations Expected results 201348 201749 Poverty Headcount: poor 0.7% reduction4 3.1% reduction 8.2% reduction Poverty gap: poor 3.6% reduction4 3.0% reduction 27% reduction 4 Headcount: extreme poor 3.0% reduction n.s. n.a. Poverty gap: extreme poor 7.0% reduction4 3.4% reduction n.a. Household income n.a. n.a. 18% increase Consumption n.a. 7.80% increase n.a. Education Registration (first year), n.a. 2.0% increase 5.1% increase children 6-17 years old Attendance (first year), n.a. 2.5% increase 5.2% increase children 6-17 years old Registration (second year), n.a. 3.1% increase 4.6% increase children 6-17 years old Attendance (second year), n.a. 3.5% increase 5.1% increase children 6-17 years old Infant and Prenatal checkups n.a. n.s. n.a. maternal Check-ups, children 0-3 n.a. 4.1% increase n.a. healthcare years old2 utilization Height and weight n.a. 15.9 % increase 11.3% increase monitoring, children younger than one year1,2 Vaccination, children 0-3 n.a. n.s. n.a. years old2 Infant Incidence of Acute diarrheal n.a. n.s. 3.8% reduction nutrition disease (ADD), children 0-5 and health years old Acute Respiratory Infection n.a. n.s. 4.6% reduction (ARI), children 0-5 years old Z score high for age, n.a. n.s. n.s. children 0-5 years old Malnutrition, children 0-5 n.a. n.s. 1.8% reduction years old Anemia, children 0-5 years n.a. 4.0% reduction old 48 Informe Final, Primera Evaluación de Impacto del Programa Presidencial Bono de Educación, Salud y Nutrición, “Bono 10,000” en zonas rurales de la República de Honduras, 30 de octubre de 2013, NORC at the University of Chicago y ESA Consultores 49 Evaluación de Impacto del Programa de Transferencias Monetarias Bono Vida Mejor, Producto 15. Informe Final de la Evaluación de Resultados de la Evaluación de Impacto, diciembre 12 2017, Econometría Consultores. Page 50 of 55 The World Bank Social Protection (P115592) Other Labor force participation, n.a. n.s. 2.4% reduction indicators children 6-17 years old of interest Source: NORC (2013) and Econometría (2017) Note: n.s. means not significant, n.a. means not reported; all results are significant at at least the 10 percent level 1 weighed during the last 30 days 2 less than 3 years old in the context of the first IE, less than 5 years old in the context of the second 3 expected results indicate poverty impacts at the national level; IE results indicate poverty impacts of control group relative to the treatment group 4 first-year poverty impacts per the PAD Targeting efficiency 13. Using the Government’s poverty lines, roughly 90 percent of and 94 percent of the program beneficiaries were extremely poor and relatively poor respectively (see figure 3) in both 2014 and 2015, suggesting impressive targeting accuracy of the extreme poor households, a further evidence of efficiency in the Conditional Cash Transfers under Component 2. The Project provided TA to fine-tune the Proxy Means Test (PMT)50 to more accurately identify the extreme poor. Following the GoH’s decision to only target the extreme poor (rather than both extreme and moderate poor51), and the subsequent 2014 Project restructuring, the Project supported the refining of the PMT and updated the Unique Socioeconomic Form (Ficha Socioeconomica Unica, FSU) data collection tool. From thereafter, the Program collected data on approximately 120,000 new households (including IPs in agreement with the Indigenous and Afro-Honduran People’s Plan, IAPP).52 It also updated the RUP using the new FSU to validate existing beneficiary profiles and re-applied the PMT, contributing to a fully updated eligibility information on Program beneficiary households, far surpassing the target of 60 percent.53 As a result, by Project closing, the Program had, arguably, become one of the best targeted CCT programs in LCR.54 50 The PMT is a statistical formula to predict a household’s welfare level based on a set of characteristics, such as location, ownership of durable goods, and demographics. The PMT aims to proxy income or consumption to identify the chronic poor. 51 The moderate poor those who live with household income more than the Basic Food Basket but less than the Basic Consumption Basket (housing, education, health, transportation), according to GoH’s definition. 52 Data collection for 19,000 households was financed by the Project, including for hard-to-reach indigenous peoples in the Gracias a Dios department. The Government focused on the 141 most vulnerable municipalities identified by the government’s poverty maps as having a high rate of poverty, as well as at high risk of draught, migration, and violence. 53 Prior to closing, CENISS had begun an operative to update the FSUs of the 90% of beneficiaries for whom it was necessary to update eligibility conditions, given that such an updating is required every five years. 54 Inter-american Development Bank (2019). Presentation entitled “Programa Presidencial de Transferencias Monetarias Condicionadas “Bono Vida Mejor”. Page 51 of 55 The World Bank Social Protection (P115592) Figure [3]. Bono Vida Mejor Benefit Incidence in the rural areas 35% 30% % of rural Bono beneficiaries 25% 20% 15% 10% 5% 0% 1 2 3 4 5 6 7 8 9 10 Income deciles 2014 2015 Fiscal cost 14. At project preparation, it was estimated that by 2013 the Bono 10,000 Program will cost US$ 210 million (1.29 percent of GDP) for a differentiated benefit for 240,000 beneficiary households. Although the program reached over the coverage of 240,000 beneficiary households, the cost of the Program never reached the expected levels. The highest cost was in 2017, USD 66.29 million (0.75 percent of GDP). 2014 2015 2016 2017 Number of beneficiaries 247,546 60,796 212,260 234,001 Total in benefits (HNL millions) 1,284 805 933 1,551 Total in benefits (USD millions) 61.59 37.04 41.49 66.29 % GDP for benefits 0.70% 0.42% 0.47% 0.75% Page 52 of 55 The World Bank Social Protection (P115592) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS COMMENTS ON THE BANK’S IMPLEMENTATION COMPLETION REPORT ON IDA CREDITS 4774-HN, 5294-HN AND 5603-HN FOR THE SOCIAL PROTECTION PROJECT (P115592) (unofficial translation) 1. Paragraph 3. The benefits of the CCT Program, "Bono 10,000," were conditioned to school enrollment and school attendance, as well as the use of the Health Unit’s preventive services by children under 5 and pregnant or postpartum women. 2. Paragraph 12. Time necessary to disburse the funds requested by the Government for an eligible emergency (Weeks), however the category created to request emergency funds is included in the Credit Agreement of the First Additional Financing (IDA 5294-HN) and is suggested to be detailed in the second column of the table (table 1). 3. Paragraph 15. The Program also modified the scope of health co-responsibilities, moving from only compliance with the health protocol for children under 5 and pregnant and puerperal women, to two visits to the Health Unit by minors of the age of 5 required for compliance. However, participating households continue to be encouraged to use preventive services, although payment transfers are not enabled. 4. Table 1. Summary of Significant Changes to the Project. The institution that implements the project was also changed. The responsibility of sole executor of the Conditional Cash Transfers Program was moved from the Secretariat of the Presidency to the Family Allowance Program. 5. Paragraph 16. The Strategy for Productive Inclusion is referred to in the footnote as one of the changes during 2013-2014, though it is not reflected in detail. It is understood that this strategy concept is integrated into the project design. Although the Strategy for Productive Inclusion is not mentioned in the rest of the document, it is suggested to include it in the text. 6. US $10.4 million out of US$12.8 million were used to finance the institutional strengthening (Component 1), equivalent to 14% of the total cost of the Project; and US$61.6 out of US US$61.3 million USD, equivalent to 86% of the total funds were used to finance cash transfers (Component 2). It is suggested to review the amount per component as it differs from the total funds referred to. 7. Paragraph 32. It is recommended to mention the Impact Evaluation of 2010, as it is mentioned in the following paragraph. 8. Section C. Regarding the delays in operational efficiency, which states that "the rotation of the Project personnel involved specifically in the IP strategy was high, which delayed the pilot implementation until the final year of the Project”. The efficiency section focuses more on efficiency constraints than on Page 53 of 55 The World Bank Social Protection (P115592) addressing efficiency levels. It is suggested to incorporate the pilot achievements despite this staff turnover 9. SESAL did not create SINOVA for the Program. SESAL decided to use it for the co-responsibilities of the Program given its operational availability. 10. Paragraph 38. In 2018, the Government of Honduras reallocated resources for cash transfers to provide relief to those affected by the floods. Nonetheless, the reallocations of funds were made from the Institutional Strengthening Component to Conditional Cash Transfers (CCT). The Emergency category was not activated. 11.Paragraph 35. The improvement is not due to the reduction in the number of participants, but rather due to the Program’s impact based on its focus on extreme poverty. That without doubt can be considered as improved efficiency. some efficiency improvement. 12. Subsection IV. The correct year is 2014. It is recommended to refer to the PCM -001-2014 published in the Official Gazette on February 22, 2014. 13. Paragraph 47. It is recommended to modify the wording around the socio-political situation resulting from the General Elections of 2017. 14. Paragraph 62. The auditors qualified the opinion on the Financial Statements for two Loans (5294-HN and 4774-HN) because the beneficiaries did not meet the eligibility criteria. The final audit for loan 5603- HN is in process and includes an additional analysis to verify compliance with eligibility criteria for those identified as not meeting concurrent audits. This is related to the response submitted to the World Bank on the Follow-up Report on Findings of previous years of the three credits and is reflected as in process, because the Bank is still to respond on the recognition of substitutions for ineligible expenses. Certain ineligible expenditures that were non-compliant with the co-responsibilities of beneficiaries contributed to deficiencies in financial management. These have been partially reimbursed to the World Bank. However, these expenses have likewise been partially justified with substitution of eligible expenditures, financed with national counterpart funds. Additionally, we confirm our agreement with the figures detailed in the table on page No.2 of 55 of the ICR report in English. 15. Paragraph 64. It is recommended to update the new institution. Page 54 of 55 The World Bank Social Protection (P115592) ANNEX 6. CHANGES TO PROJECT DEVELOPMENT INDICATORS Original Credit May 2010 First AF (July 2013) Second AF (February By Project Closing 2015) % of conditional cash transfers % of households in the Program Idem received by households in the receiving CCTs in rural areas 1 poorest quintile that are extreme poor # of families whose compliance Moved as an Intermediate Dropped. Redundancy 2 with co-responsibilities is Results Indicator (IRI) with co-responsibility verified and reported indicator % of students in grades 1 to 6 % of students in grades 1 to 6 Idem participating in the Program participating in the Program who comply with the co- who comply with the co- 3 responsibility of 80 percent responsibility of 80 percent school attendance school attendance; New target: Target 80% 85% % of students in grades 7 Idem to 9 participating in the 6 Program who comply with the co-responsibility of 80% school attendance % of Program beneficiary Idem children aged 13-15 years that 4 completed primary education (sixth grade) % of female Program Idem beneficiary children aged 13-15 5 years that completed primary education (6th grade) % of Program beneficiary Idem children aged 16-18 years 7 that completed lower secondary education (9th grade) % of female Program Idem beneficiary children aged 16-18 years that 8 completed lower secondary education (9th grade) % of children, aged 0-5 years % of Program beneficiary % of Program participating in the Program children aged 12-23 months beneficiary children with complete vaccination with complete vaccination aged 0-23 months with 9 scheme scheme. complete vaccination Target 98% New target: 80% scheme. New target: 70% % of Program beneficiary % of Program beneficiary Dropped. Unable to pregnant women who receive pregnant women who measure with 10 prenatal controls in the first receive at least two administrative data trimester of pregnancy prenatal controls during (moved from IRI to PDO-I) pregnancy Time taken to disburse funds Idem 11 requested by Government for an eligible emergency (weeks) Page 55 of 55